LangChain构建 Chroma

先要安装依赖

pip install langchain-chroma

Chroma可以以多种模式运行。以下是每种模式的示例，均与LangChain集成：

• in-memory-在Python脚本或Jupyter笔记本中
• in-memory with persistance-在脚本或笔记本中保存/加载到磁盘
• in a docker container-作为在本地机器或云中运行的服务器

与任何其他数据库一样，您可以进行以下操作：

• add
• get
• update
• upsert
• delete
• peek

而.query则运行相以性搜索。查看完整文档，请访问docs。要直接访问这些方法，可以使用，_collection.method()。

基本示例在这个基本示例中，我们获取《乔布斯演讲稿》，将其分割成片段，使用开源嵌入模型进行嵌入，加载到Chroma中，然后进行查询。

乔布斯演讲知识库

# knowledge.txt

Thank you. I'm honored to be with you today for your commencement from one of the finest universities in the world. Truth be told, I never graduated from college and this is the closest I've ever gotten to a college graduation.
谢谢大家。很荣幸能和你们，来自世界最好大学之一的毕业生们，一块儿参加毕业典礼。老实说，我大学没有毕业，今天恐怕是我一生中离大学毕业最近的一次了。

Today I want to tell you three stories from my life. That's it. No big deal. Just three stories.
今天我想告诉大家来自我生活的三个故事。没什么大不了的，只是三个故事而已。

The first story is about connecting the dots.
第一个故事，如何串连生命中的点滴。

I dropped out of Reed College after the first six months but then stayed around as a drop-in for another 18 months or so before I really quit. So why did I drop out? It started before I was born. My biological mother was a young, unwed graduate student, and she decided to put me up for adoption. She felt very strongly that I should be adopted by college graduates, so everything was all set for me to be adopted at birth by a lawyer and his wife, except that when I popped out, they decided at the last minute that they really wanted a girl. So my parents, who were on a waiting list, got a call in the middle of the night asking, "We've got an unexpected baby boy. Do you want him?" They said, "Of course." My biological mother found out later that my mother had never graduated from college and that my father had never graduated from high school. She refused to sign the final adoption papers. She only relented a few months later when my parents promised that I would go to college.
我在里得大学读了六个月就退学了，但是在18个月之后--我真正退学之前，我还常去学校。为何我要选择退学呢？这还得从我出生之前说起。我的生母是一个年轻、未婚的大学毕业生，她决定让别人收养我。她有一个很强烈的信仰，认为我应该被一个大学毕业生家庭收养。于是，一对律师夫妇说好了要领养我，然而最后一秒钟，他们改变了主意，决定要个女孩儿。然后我排在收养人名单中的养父母在一个深夜接到电话，“很意外，我们多了一个男婴，你们要吗？”“当然要！”但是我的生母后来又发现我的养母没有大学毕业，养父连高中都没有毕业。她拒绝在领养书上签字。几个月后，我的养父母保证会让我上大学，她妥协了。

This was the start in my life. And 17 years later, I did go to college, but I naively chose a college that was almost as expensive as Stanford, and all of my working-class parents' savings were being spent on my college tuition. After six months, I couldn't see the value in it. I had no idea what I wanted to do with my life, and no idea of how college was going to help me figure it out, and here I was, spending all the money my parents had saved their entire life. So I decided to drop out and trust that it would all work out OK. It was pretty scary at the time, but looking back, it was one of the best decisions I ever made. The minute I dropped out, I could stop taking the required classes that didn't interest me and begin dropping in on the ones that looked far more interesting.
这是我生命的开端。十七年后，我上大学了，但是我很无知地选了一所差不多和斯坦福一样贵的学校，几乎花掉我那蓝领阶层养父母一生的积蓄。六个月后，我觉得不值得。我看不出自己以后要做什么，也不晓得大学会怎样帮我指点迷津，而我却在花销父母一生的积蓄。所以我决定退学，并且相信没有做错。一开始非常吓人，但回忆起来，这却是我一生中作的最好的决定之一。从我退学的那一刻起，我可以停止一切不感兴趣的必修课，开始旁听那些有意思得多的课。

It wasn't all romantic. I didn't have a dorm room, so I slept on the floor in friends' rooms. I returned Coke bottles for the five-cent deposits to buy food with, and I would walk the seven miles across town every Sunday night to get one good meal a week at the Hare Krishna temple. I loved it. And much of what I stumbled into by following my curiosity and intuition turned out to be priceless later on. Let me give you one example.
事情并不那么美好。我没有宿舍可住，睡在朋友房间的地上。为了吃饭，我收集五分一个的旧可乐瓶，每个星期天晚上步行七英里到哈尔-克里什纳庙里改善一下一周的伙食。我喜欢这种生活方式。能够遵循自己的好奇和直觉前行后来被证明是多么的珍贵。让我来给你们举个例子吧。

Reed College at that time offered perhaps the best calligraphy instruction in the country. Throughout the campus every poster, every label on every drawer was beautifully hand-calligraphed. Because I had dropped out and didn't have to take the normal classes, I decided to take a calligraphy class to learn how to do this. I learned about serif and sans-serif typefaces, about varying the amount of space between different letter combinations, about what makes great typography great. It was beautiful, historical, artistically subtle in a way that science can't capture, and I found it fascinating.
当时的里德大学提供可能是全国最好的书法指导。校园中每一张海报，抽屉上的每一张标签，都是漂亮的手写体。由于我已退学，不用修那些必修课，我决定选一门书法课上上。在这门课上，我学会了“serif”和"sans-serif"两种字体、学会了怎样在不同的字母组合中改变字间距、学会了怎样写出好的字来。这是一种科学无法捕捉的微妙，楚楚动人、充满历史底蕴和艺术性，我觉得自己被完全吸引了。

None of this had even a hope of any practical application in my life. But ten years later when we were designing the first Macintosh computer, it all came back to me, and we designed it all into the Mac. It was the first computer with beautiful typography. If I had never dropped in on that single course in college, the Mac would have never had multiple typefaces or proportionally spaced fonts, and since Windows just copied the Mac, it's likely that no personal computer would have them.
当时我并不指望书法在以后的生活中能有什么实用价值。但是，十年之后，我们在设计第一台 Macintosh计算机时，它一下子浮现在我眼前。于是，我们把这些东西全都设计进了计算机中。这是第一台有这么漂亮的文字版式的计算机。要不是我当初在大学里偶然选了这么一门课，Macintosh计算机绝不会有那么多种印刷字体或间距安排合理的字号。要不是Windows照搬了 Macintosh，个人电脑可能不会有这些字体和字号。

If I had never dropped out, I would have never dropped in on that calligraphy class and personals computers might not have the wonderful typography that they do.
要不是退了学，我决不会碰巧选了这门书法课，个人电脑也可能不会有现在这些漂亮的版式了。

Of course it was impossible to connect the dots looking forward when I was in college, but it was very, very clear looking backwards ten years later. Again, you can't connect the dots looking forward. You can only connect them looking backwards, so you have to trust that the dots will somehow connect in your future. You have to trust in something--your gut, destiny, life, karma, whatever--because believing that the dots will connect down the road will give you the confidence to follow your heart, even when it leads you off the well-worn path, and that will make all the difference.
当然，我在大学里不可能从这一点上看到它与将来的关系。十年之后再回头看，两者之间关系就非常、非常清楚了。你们同样不可能从现在这个点上看到将来；只有回头看时，才会发现它们之间的关系。所以你必须相信，那些点点滴滴，会在你未来的生命里，以某种方式串联起来。你必须相信一些东西——你的勇气、宿命、生活、因缘，随便什么——因为相信这些点滴能够一路连接会给你带来循从本觉的自信，它使你远离平凡，变得与众不同。

My second story is about love and loss. I was lucky. I found what I loved to do early in life. Woz and I started Apple in my parents' garage when I was 20. We worked hard and in ten years, Apple had grown from just the two of us in a garage into a $2 billion company with over 4,000 employees. We'd just released our finest creation, the Macintosh, a year earlier, and I'd just turned 30, and then I got fired. How can you get fired from a company you started? Well, as Apple grew, we hired someone who I thought was very talented to run the company with me, and for the first year or so, things went well. But then our visions of the future began to diverge, and eventually we had a falling out. When we did, our board of directors sided with him, and so at 30, I was out, and very publicly out. What had been the focus of my entire adult life was gone, and it was devastating. I really didn't know what to do for a few months. I felt that I had let the previous generation of entrepreneurs down, that I had dropped the baton as it was being passed to me. I met with David Packard and Bob Noyce and tried to apologize for screwing up so badly. I was a very public failure and I even thought about running away from the Valley. But something slowly began to dawn on me. I still loved what I did. The turn of events at Apple had not changed that one bit. I'd been rejected but I was still in love. And so I decided to start over.
第二个故事是关于爱与失的。我很幸运，很早就发现自己喜欢做的事情。我二十岁的时候就和沃茨在父母的车库里开创了苹果公司。我们工作得很努力，十年后，苹果公司成长为拥有四千名员工，价值二十亿的大公司。我们刚刚推出了最好的创意，Macintosh操作系统，在这之前的一年，也就是我刚过三十岁，我被解雇了。你怎么可能被一个亲手创立的公司解雇？事情是这样的，在公司成长期间，我雇佣了一个我们认为非常聪明，可以和我一起经营公司的人。一年后，我们对公司未来的看法产生分歧，董事会站在了他的一边。于是，在我三十岁的时候，我出局了，很公开地出局了。我整个成年生活的焦点没了，这很要命。一开始的几个月我真的不知道该干什么。我觉得我让公司的前一代创建者们失望了，我把传给我的权杖给弄丢了。我与戴维德·帕珂德和鲍勃·诺埃斯见面，试图为这彻头彻尾的失败道歉。我败得如此之惨以至于我想要逃离硅谷。但有个东西在慢慢地叫醒我：我还爱着我从事的行业。这次失败一点儿都没有改变这一点。我被逐了，但我仍爱着我的事业。我决定重新开始。

I didn't see it then, but it turned out that getting fired from Apple was the best thing that could have ever happened to me. The heaviness of being successful was replaced by the lightness of being a beginner again, less sure about everything. It freed me to enter one of the most creative periods in my life. During the next five years I started a company named NeXT, another company named Pixar and fell in love with an amazing woman who would become my wife. Pixar went on to create the world's first computer-animated feature film, "Toy Story," and is now the most successful animation studio in the world.
当时我没有看出来，但事实证明“被苹果开除”是发生在我身上最好的事。成功的重担被重新起步的轻松替代，对任何事情都不再特别看重，这让我感觉如此自由，进入一生中最有创造力的阶段。接下来的五年，我创立了一个叫NeXT的公司，接着又建立了Pixar，然后与后来成为我妻子的女人相爱。Pixar出品了世界第一个电脑动画电影：“玩具总动员”，现在它已经是世界最成功的动画制作工作室了。

In a remarkable turn of events, Apple bought NeXT and I returned to Apple and the technology we developed at NeXT is at the heart of Apple's current renaissance, and Lorene and I have a wonderful family together.
在一系列的成功运转后，苹果收购了NeXT，我又回到了苹果。我们在NeXT开发的技术在苹果的复兴中起了核心作用，另外劳琳和我组建了一个幸福的家庭。

I'm pretty sure none of this would have happened if I hadn't been fired from Apple. It was awful-tasting medicine but I guess the patient needed it. Sometimes life's going to hit you in the head with a brick. Don't lose faith. I'm convinced that the only thing that kept me going was that I loved what I did. You've got to find what you love, and that is as true for work as it is for your lovers. Your work is going to fill a large part of your life, and the only way to be truly satisfied is to do what you believe is great work, and the only way to do great work is to love what you do. If you haven't found it yet, keep looking, and don't settle. As with all matters of the heart, you'll know when you find it, and like any great relationship it just gets better and better as the years roll on. So keep looking. Don't settle.
我非常确信，如果我没有被苹果炒掉，这些就都不会发生。这个药的味道太糟了，但是我想病人需要它。有些时候，生活会给你迎头一棒。不要丧失信心。我确信唯一让我一路走下来的是我对自己所做事情的热爱。你必须去找你热爱的东西，对工作如此，对你的爱人也是这样的。工作会占据你生命中很大的一部分，你只有相信自己做的是伟大的工作，你才能怡然自得。如果你还没有找到，那么就继续找，不要停。全心全意地找，当你找到时，你会知道的。就像任何真诚的关系，随着时间的流逝，只会越来越紧密。所以继续找，不要停。

My third story is about death. When I was 17 I read a quote that went something like "If you live each day as if it was your last, someday you'll most certainly be right." It made an impression on me, and since then, for the past 33 years, I have looked in the mirror every morning and asked myself, "If today were the last day of my life, would I want to do what I am about to do today?" And whenever the answer has been "no" for too many days in a row, I know I need to change something. Remembering that I'll be dead soon is the most important thing I've ever encountered to help me make the big choices in life, because almost everything--all external expectations, all pride, all fear of embarrassment or failure--these things just fall away in the face of death, leaving only what is truly important. Remembering that you are going to die is the best way I know to avoid the trap of thinking you have something to lose. You are already naked. There is no reason not to follow your heart.
我的第三个故事关于死亡。我17岁的时候读到过一句话“如果你把每一天都当作最后一天过，有一天你会发现你是正确的”。这句话给我留下了深刻的印象。从那以后，过去的33年，每天早上我都会对着镜子问自己：“如果今天是我的最后一天，我会不会做我想做的事情呢？”如果连着一段时间，答案都是否定的的话，我就知道我需要改变一些东西了。提醒自己就要死了是我遇见的最大的帮助，帮我作了生命中的大决定。因为几乎任何事——所有的荣耀、骄傲、对难堪和失败的恐惧——在死亡面前都会消隐，留下真正重要的东西。提醒自己就要死亡是我知道的最好的方法，用来避开担心失去某些东西的陷阱。你已经赤裸裸了，没有理由不听从于自己的心愿。

About a year ago, I was diagnosed with cancer. I had a scan at 7:30 in the morning and it clearly showed a tumor on my pancreas. I didn't even know what a pancreas was. The doctors told me this was almost certainly a type of cancer that is incurable, and that I should expect to live no longer than three to six months. My doctor advised me to go home and get my affairs in order, which is doctors' code for "prepare to die." It means to try and tell your kids everything you thought you'd have the next ten years to tell them, in just a few months. It means to make sure that everything is buttoned up so that it will be as easy as possible for your family. It means to say your goodbyes.
大约一年前，我被诊断出患了癌症。我早上七点半作了扫描，清楚地显示在我的胰腺有一个肿瘤。我当时都不知道胰腺是什么东西。医生们告诉我这几乎是无法治愈的，我还有三到六个月的时间。我的医生建议我回家，整理一切。在医生的辞典中，这就是“准备死亡”的意思。就是意味着把要对你小孩说十年的话在几个月内说完；意味着把所有东西搞定，尽量让你的家庭活得轻松一点；意味着你要说“永别”了。

I lived with that diagnosis all day. Later that evening I had a biopsy where they stuck an endoscope down my throat, through my stomach into my intestines, put a needle into my pancreas and got a few cells from the tumor. I was sedated but my wife, who was there, told me that when they viewed the cells under a microscope, the doctor started crying, because it turned out to be a very rare form of pancreatic cancer that is curable with surgery. I had the surgery and, thankfully, I am fine now.
我整日都想着那诊断书的事情。后来有天晚上我做了一个活切片检查，他们将一个内窥镜伸进我的喉咙，穿过胃，到达肠道，用一根针在我的胰腺肿瘤上取了几个细胞。我当时是被麻醉的，但是我的妻子告诉我，那些医生在显微镜下看到细胞的时候开始尖叫，因为发现这竟然是一种非常罕见的可用手术治愈的胰腺癌症。我做了手术，现在，我痊愈了。

This was the closest I've been to facing death, and I hope it's the closest I get for a few more decades. Having lived through it, I can now say this to you with a bit more certainty than when death was a useful but purely intellectual concept. No one wants to die, even people who want to go to Heaven don't want to die to get there, and yet, death is the destination we all share. No one has ever escaped it. And that is as it should be, because death is very likely the single best invention of life. It's life's change agent; it clears out the old to make way for the new. right now, the new is you. But someday, not too long from now, you will gradually become the old and be cleared away. Sorry to be so dramatic, but it's quite true. Your time is limited, so don't waste it living someone else's life. Don't be trapped by dogma, which is living with the results of other people's thinking. Don't let the noise of others' opinions drown out your own inner voice, and most important, have the courage to follow heart and intuition. They somehow already know what you truly want to become. Everything else is secondary.
这是我最接近死亡的时候，我也希望是我未来几十年里最接近死亡的一次。这次死里逃生让我比以往只知道死亡是一个有用而纯粹书面概念的时候更确信地告诉你们，没有人愿意死，即使那些想上天堂的人们也不愿意通过死亡来达到他们的目的。但是死亡是每个人共同的终点，没有人能够逃脱。也应该如此，因为死亡很可能是生命最好的发明。它去陈让新。现在，你们就是“新”。但是有一天，不用太久，你们有会慢慢变老然后死去。抱歉，这很戏剧性，但却是真的。你们的时间是有限的，不要浪费在重复别人的生活上。不要被教条束缚，那意味着会和别人思考的结果一块儿生活。不要被其他人的喧嚣观点掩盖自己内心真正的声音。你的直觉和内心知道你想要变成什么样子。所有其他东西都是次要的。

When I was young, there was an amazing publication called _The Whole Earth Catalogue_, which was one of the bibles of my generation. It was created by a fellow named Stuart Brand not far from here in Menlo Park, and he brought it to life with his poetic touch. This was in the late 1960s, before personal computers and desktop publishing, so it was all made with typewriters, scissors, and Polaroid cameras. it was sort of like Google in paperback form 35 years before Google came along. It was idealistic, overflowing with neat tools and great notions. Stuart and his team put out several issues of the _The Whole Earth Catalogue_, and then when it had run its course, they put out a final issue. It was the mid-1970s and I was your age. On the back cover of their final issue was a photograph of an early morning country road, the kind you might find yourself hitch-hiking on if you were so adventurous. Beneath were the words, "Stay hungry, stay foolish." It was their farewell message as they signed off. "Stay hungry, stay foolish." And I have always wished that for myself, and now, as you graduate to begin anew, I wish that for you. Stay hungry, stay foolish.
我年轻的时候，有一份叫做《完整地球目录》的好杂志，是我们这一代人的圣经之一。它是一个叫斯纠华特·布兰的、住在离这不远的曼罗公园的家伙创立的。他用诗一般的触觉将这份杂志带到世界。那是六十年代后期，个人电脑出现之前，所以这份杂志全是用打字机、剪刀和偏光镜制作的。有点像软皮包装的google，不过却早了三十五年。它理想主义，全文充斥着灵巧的工具和伟大的想法。斯纠华特和他的小组出版了几期“完整地球目录”，在完成使命之前，他们出版了最后一期。那是七十年代中期，我和你们差不多大。最后一期的封底是一张清晨乡村小路的照片，如果你有冒险精神，可以自己找到这条路。下面有一句话，“保持饥饿，保持愚蠢”。这是他们的告别语，“保持饥饿，保持愚蠢”。我常以此勉励自己。现在，在你们即将踏上新旅程的时候，我也希望你们能这样。保持饥饿，保持愚蠢。

Thank you all, very much.
非常感谢。

下面是一个完整的代码

#示例：chroma_base.py
# pip install langchain
# pip install langchain-community
# pip install langchain-chroma
from langchain_chroma import Chroma
from langchain_community.document_loaders import TextLoader
# pip install -U langchain-huggingface
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_text_splitters import CharacterTextSplitter
#加载文指并将其分割成片段
loader = TextLoader("../knowledge.txt",encoding="UTF-8")
documents = loader.load()
#将其分割成片段
text_splitter = CharacterTextSplitter(chunk_size=1500,chunk_overlap=0)
docs = text_splitter.split_documents(documents)
#创建开源嵌入函数
embedding_function = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
#将其加载到Chroma中
db = Chroma.from_documents(docs,embedding_function)
#进行查询
query = "Pixar公司是做什么的？"
docs = db.similarity_search(query)
#打印结果
print(docs[0].page_content)

持久化

如果我们希望将向量持久化到磁盘可以按照下面方式写

# pip install langchain-chroma
import chromadb
from langchain_chroma import Chroma
#  pip install -U langchain-huggingface
from langchain_huggingface import HuggingFaceEmbeddings


persistent_client = chromadb.PersistentClient()
#创建开源族入丽数
embedding_function = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
collection = persistent_client.get_or_create_collection("collection_name")
collection.add(ids=["1","2","3"],documents=["a","b","c"])
langchain_chroma = Chroma(
    client=persistent_client,
    collection_name="collection_name",
    embedding_function=embedding_function,
)
print("在年合中有", langchain_chroma._collection.count(),"个文档")

CRUD

下面是一个简单的增删改查可以体会一下

import chromadb
# pip install langchain-chroma
from langchain_chroma import Chroma
# pip install langchain-text-splitters
from langchain_text_splitters import CharacterTextSplitter

from langchain_community.document_loaders import TextLoader
from langchain_huggingface import HuggingFaceEmbeddings

loader = TextLoader("/content/onedrive/colab/knowledge.txt",encoding="UTF-8")
documents = loader.load()
# #将其分割成片段
text_splitter = CharacterTextSplitter(chunk_size=1500,chunk_overlap=0)
docs = text_splitter.split_documents(documents)
# #创建开源嵌入函数
embedding_function = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
query="Pixar公司是做什么的？"
# #创建简单的ids
ids = [str(i) for i in range(1,len(docs)+1)]
#添加数据
example_db = Chroma.from_documents(docs,embedding_function,ids=ids)
docs = example_db.similarity_search(query)
#更新文档的元数据
docs[0].metadata = {
	"source":"/content/onedrive/colab/knowledge.txt",
	"new_value":"hello world",
}

print("更所前内容：")
print(example_db._collection.get(ids=[ids[0]]))
example_db.update_document(ids[0],docs [0])
print("更新后内容：")
print(example_db._collection.get(ids=[ids[0]]))

print("删除前计数",example_db._collection.count())
print(example_db._collection.get(ids=[ids[-1]]))
example_db._collection.delete(ids=[ids[-1]])
print("删除后计数",example_db._collection.count())
print(example_db._collection.get(ids=[ids[-1]]))

使用openAI

#示例：chroma._openai.py
from langchain_openai import OpenAIEmbeddings
# pip install langchain-chroma
from langchain_chroma import Chroma
import chromadb
from langchain_text_splitters import CharacterTextsplitter
from langchain_community.document_loaders import TextLoader
embeddings = OpenAIEmbeddings()
persistent_client = chromadb.Persistentclient()
new_client = chromadb.Ephemeralclient()
#加载文档并将其分剖成片段
loader = TextLoader("../../resource/knowledge.txt",encoding="UTF-8")
documents = loader.load()
#将其分割成片段
text_splitter = CharacterTextsplitter(chunk_size=1500,chunk_overlap=0)
docs = text_splitter.split_documents(documents)
openai_lc_client = Chroma.from_documents(
    docs,embeddings,client=new_client,collection_name="openai_collection"
)
query="Pixar公司是做什么的？"
docs = openai_lc_client.similarity_search(query)
print(docs[0].page_content)