← Back to Cookbook
basic RAG
Details
File: mistral/rag/basic_RAG.ipynb
Type: Jupyter Notebook
Use Cases: RAG
Content
Notebook content (JSON format):
{
"cells": [
{
"cell_type": "markdown",
"id": "d67a4729-cd2f-47e7-a4f6-f84a5677414f",
"metadata": {
"id": "d67a4729-cd2f-47e7-a4f6-f84a5677414f"
},
"source": [
"# RAG Basics with Mistral AI\n",
"\n",
"\n",
"\n",
"Retrieval-augmented generation (RAG) is an AI framework that synergizes the capabilities of LLMs and information retrieval systems. It’s useful to answer questions or generate content leveraging external knowledge. There are two main steps in RAG: 1) retrieval: retrieve relevant information from a knowledge base with text embeddings stored in a vector store; 2) generation: insert the relevant information to the prompt for the LLM to generate information. In this guide, we will walk through a very basic example of RAG with four implementations:\n",
"\n",
"- RAG from scratch with Mistral\n",
"- RAG with Mistral and LangChain\n",
"- RAG with Mistral and LlamaIndex\n",
"- RAG with Mistral and Haystack\n",
"\n",
"## RAG from scratch\n",
"\n",
"This section aims to guide you through the process of building a basic RAG from scratch. We have two goals: firstly, to offer users a comprehensive understanding of the internal workings of RAG and demystify the underlying mechanisms; secondly, to empower you with the essential foundations needed to build an RAG using the minimum required dependencies.\n",
"\n",
"\n",
"### Import needed packages\n",
"The first step is to install the needed packages `mistralai` and `faiss-cpu` and import the needed packages:\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "b880d1ed-3db0-45a1-807e-1b47e9ce1320",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting faiss-cpu==1.7.4\n",
" Downloading faiss_cpu-1.7.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.6 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.6/17.6 MB\u001b[0m \u001b[31m19.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hInstalling collected packages: faiss-cpu\n",
"Successfully installed faiss-cpu-1.7.4\n"
]
}
],
"source": [
"! pip install faiss-cpu==1.7.4 mistralai"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "851612c3-ee93-42e3-a1fb-481f89c9410f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Type your API Key··········\n"
]
}
],
"source": [
"from mistralai import Mistral\n",
"import requests\n",
"import numpy as np\n",
"import faiss\n",
"import os\n",
"from getpass import getpass\n",
"\n",
"api_key= getpass(\"Type your API Key\")\n",
"client = Mistral(api_key=api_key)"
]
},
{
"cell_type": "markdown",
"id": "fe8609d5-9f27-4202-b0be-36db34412998",
"metadata": {
"id": "fe8609d5-9f27-4202-b0be-36db34412998"
},
"source": [
"### Get data\n",
"\n",
"In this very simple example, we are getting data from an essay written by Paul Graham:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "c4c01740-72b4-482c-b61e-e272a734f01f",
"metadata": {},
"outputs": [],
"source": [
"response = requests.get('https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt')\n",
"text = response.text"
]
},
{
"cell_type": "markdown",
"id": "15af760a-f074-4339-81f9-4f370034355e",
"metadata": {
"id": "15af760a-f074-4339-81f9-4f370034355e"
},
"source": [
"We can also save the essay in a local file:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "f4a2313f-686f-4b08-a553-cf9006af73eb",
"metadata": {},
"outputs": [],
"source": [
"f = open('essay.txt', 'w')\n",
"f.write(text)\n",
"f.close()"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "f03f47af-a20b-4122-a114-74b9748ff543",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"75014"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(text)"
]
},
{
"cell_type": "markdown",
"id": "aad1aa61-9e1c-46c8-ae5e-61855df440f9",
"metadata": {
"id": "aad1aa61-9e1c-46c8-ae5e-61855df440f9"
},
"source": [
"## Split document into chunks\n",
"\n",
"In a RAG system, it is crucial to split the document into smaller chunks so that it’s more effective to identify and retrieve the most relevant information in the retrieval process later. In this example, we simply split our text by character, combine 2048 characters into each chunk, and we get 37 chunks."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "8494655e-bd87-49de-8f1d-69ffbc1c256e",
"metadata": {},
"outputs": [],
"source": [
"chunk_size = 2048\n",
"chunks = [text[i:i + chunk_size] for i in range(0, len(text), chunk_size)]"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "c78c9936-0c1d-471c-b030-6c45639e7238",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"37"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(chunks)"
]
},
{
"cell_type": "markdown",
"id": "4176cbe3-9b15-4d17-afb1-665011d09bb7",
"metadata": {
"id": "4176cbe3-9b15-4d17-afb1-665011d09bb7"
},
"source": [
"#### Considerations:\n",
"- **Chunk size**: Depending on your specific use case, it may be necessary to customize or experiment with different chunk sizes and chunk overlap to achieve optimal performance in RAG. For example, smaller chunks can be more beneficial in retrieval processes, as larger text chunks often contain filler text that can obscure the semantic representation. As such, using smaller text chunks in the retrieval process can enable the RAG system to identify and extract relevant information more effectively and accurately. However, it’s worth considering the trade-offs that come with using smaller chunks, such as increasing processing time and computational resources.\n",
"- **How to split**: While the simplest method is to split the text by character, there are other options depending on the use case and document structure. For example, to avoid exceeding token limits in API calls, it may be necessary to split the text by tokens. To maintain the cohesiveness of the chunks, it can be useful to split the text by sentences, paragraphs, or HTML headers. If working with code, it’s often recommended to split by meaningful code chunks for example using an Abstract Syntax Tree (AST) parser.\n",
"\n",
"\n",
"### Create embeddings for each text chunk\n",
"For each text chunk, we then need to create text embeddings, which are numeric representations of the text in the vector space. Words with similar meanings are expected to be in closer proximity or have a shorter distance in the vector space.\n",
"To create an embedding, use Mistral’s embeddings API endpoint and the embedding model `mistral-embed`. We create a `get_text_embedding` to get the embedding from a single text chunk and then we use list comprehension to get text embeddings for all text chunks.\n"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "e77d9805-7a53-4210-9f80-f4de52285588",
"metadata": {},
"outputs": [],
"source": [
"def get_text_embedding(input):\n",
" embeddings_batch_response = client.embeddings.create(\n",
" model=\"mistral-embed\",\n",
" inputs=input\n",
" )\n",
" return embeddings_batch_response.data[0].embedding"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "46503830-6ad5-493e-a629-152721e2d88e",
"metadata": {},
"outputs": [],
"source": [
"text_embeddings = np.array([get_text_embedding(chunk) for chunk in chunks])"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "ca875993-fe6d-42df-811e-a43891cd0350",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(37, 1024)"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"text_embeddings.shape"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "55396758-c3f3-45b3-b6e7-d4912c0899f2",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[-3.95202637e-02, 7.75756836e-02, -8.82148743e-05, ...,\n",
" -1.26342773e-02, -2.12402344e-02, -2.50816345e-03],\n",
" [-3.19213867e-02, 7.21435547e-02, 2.99835205e-02, ...,\n",
" -1.08413696e-02, -1.19628906e-02, -7.66372681e-03],\n",
" [-5.89599609e-02, 6.12487793e-02, 1.26419067e-02, ...,\n",
" -2.25372314e-02, 4.67681885e-03, -6.26754761e-03],\n",
" ...,\n",
" [-5.52978516e-02, 6.89697266e-02, 2.69622803e-02, ...,\n",
" -2.54211426e-02, -2.52227783e-02, -2.68859863e-02],\n",
" [-3.90014648e-02, 5.63049316e-02, 4.76684570e-02, ...,\n",
" -1.77001953e-02, 9.33074951e-03, -8.72039795e-03],\n",
" [-2.99377441e-02, 5.81054688e-02, 1.70898438e-02, ...,\n",
" -1.61132812e-02, -1.79290771e-02, -4.35791016e-02]])"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"text_embeddings"
]
},
{
"cell_type": "markdown",
"id": "1cba33c7-9d1d-44d8-a01e-e30f16be1aac",
"metadata": {
"id": "1cba33c7-9d1d-44d8-a01e-e30f16be1aac"
},
"source": [
"### Load into a vector database\n",
"Once we get the text embeddings, a common practice is to store them in a vector database for efficient processing and retrieval. There are several vector database to choose from. In our simple example, we are using an open-source vector database Faiss, which allows for efficient similarity search. \n",
"\n",
"With Faiss, we instantiate an instance of the Index class, which defines the indexing structure of the vector database. We then add the text embeddings to this indexing structure.\n"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "6a5b1877-b113-4527-9055-cae9049fef08",
"metadata": {},
"outputs": [],
"source": [
"d = text_embeddings.shape[1]\n",
"index = faiss.IndexFlatL2(d)\n",
"index.add(text_embeddings)"
]
},
{
"cell_type": "markdown",
"id": "5ee023ab-b26c-4df5-8a7b-7dd660bfad86",
"metadata": {
"id": "5ee023ab-b26c-4df5-8a7b-7dd660bfad86"
},
"source": [
"#### Considerations:\n",
"- **Vector database**: When selecting a vector database, there are several factors to consider including speed, scalability, cloud management, advanced filtering, and open-source vs. closed-source.\n",
"\n",
"### Create embeddings for a question\n",
"Whenever users ask a question, we also need to create embeddings for this question using the same embedding models as before.\n"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "894d9764-9da9-4629-8f2a-c9dcaf6ceb8d",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(1, 1024)"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"question = \"What were the two main things the author worked on before college?\"\n",
"question_embeddings = np.array([get_text_embedding(question)])\n",
"question_embeddings.shape"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "9c4948cc-6d8b-449f-bc00-abb3591c7222",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[-0.05456543, 0.03518677, 0.03723145, ..., -0.02763367,\n",
" -0.00327873, 0.00323677]])"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"question_embeddings"
]
},
{
"cell_type": "markdown",
"id": "15989e10-d0ec-41be-b6be-fa317565a926",
"metadata": {
"id": "15989e10-d0ec-41be-b6be-fa317565a926"
},
"source": [
"#### Considerations:\n",
"- Hypothetical Document Embeddings (HyDE): In some cases, the user’s question might not be the most relevant query to use for identifying the relevant context. Instead, it maybe more effective to generate a hypothetical answer or a hypothetical document based on the user’s query and use the embeddings of the generated text to retrieve similar text chunks.\n",
"\n",
"### Retrieve similar chunks from the vector database\n",
"We can perform a search on the vector database with `index.search`, which takes two arguments: the first is the vector of the question embeddings, and the second is the number of similar vectors to retrieve. This function returns the distances and the indices of the most similar vectors to the question vector in the vector database. Then based on the returned indices, we can retrieve the actual relevant text chunks that correspond to those indices.\n"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "c930b378-7aac-434c-881b-ab69d3edb93d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[0 3]]\n"
]
}
],
"source": [
"D, I = index.search(question_embeddings, k=2)\n",
"print(I)"
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "73aab584-1dbf-4532-b41e-0403eeeeb567",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"['\\n\\nWhat I Worked On\\n\\nFebruary 2021\\n\\nBefore college the two main things I worked on, outside of school, were writing and programming. I didn\\'t write essays. I wrote what beginning writers were supposed to write then, and probably still are: short stories. My stories were awful. They had hardly any plot, just characters with strong feelings, which I imagined made them deep.\\n\\nThe first programs I tried writing were on the IBM 1401 that our school district used for what was then called \"data processing.\" This was in 9th grade, so I was 13 or 14. The school district\\'s 1401 happened to be in the basement of our junior high school, and my friend Rich Draves and I got permission to use it. It was like a mini Bond villain\\'s lair down there, with all these alien-looking machines — CPU, disk drives, printer, card reader — sitting up on a raised floor under bright fluorescent lights.\\n\\nThe language we used was an early version of Fortran. You had to type programs on punch cards, then stack them in the card reader and press a button to load the program into memory and run it. The result would ordinarily be to print something on the spectacularly loud printer.\\n\\nI was puzzled by the 1401. I couldn\\'t figure out what to do with it. And in retrospect there\\'s not much I could have done with it. The only form of input to programs was data stored on punched cards, and I didn\\'t have any data stored on punched cards. The only other option was to do things that didn\\'t rely on any input, like calculate approximations of pi, but I didn\\'t know enough math to do anything interesting of that type. So I\\'m not surprised I can\\'t remember any programs I wrote, because they can\\'t have done much. My clearest memory is of the moment I learned it was possible for programs not to terminate, when one of mine didn\\'t. On a machine without time-sharing, this was a social as well as a technical error, as the data center manager\\'s expression made clear.\\n\\nWith microcomputers, everything changed. Now you could have a computer sitting right in front of you, on ', 'arvard accepted me, so that was where I went.\\n\\nI don\\'t remember the moment it happened, or if there even was a specific moment, but during the first year of grad school I realized that AI, as practiced at the time, was a hoax. By which I mean the sort of AI in which a program that\\'s told \"the dog is sitting on the chair\" translates this into some formal representation and adds it to the list of things it knows.\\n\\nWhat these programs really showed was that there\\'s a subset of natural language that\\'s a formal language. But a very proper subset. It was clear that there was an unbridgeable gap between what they could do and actually understanding natural language. It was not, in fact, simply a matter of teaching SHRDLU more words. That whole way of doing AI, with explicit data structures representing concepts, was not going to work. Its brokenness did, as so often happens, generate a lot of opportunities to write papers about various band-aids that could be applied to it, but it was never going to get us Mike.\\n\\nSo I looked around to see what I could salvage from the wreckage of my plans, and there was Lisp. I knew from experience that Lisp was interesting for its own sake and not just for its association with AI, even though that was the main reason people cared about it at the time. So I decided to focus on Lisp. In fact, I decided to write a book about Lisp hacking. It\\'s scary to think how little I knew about Lisp hacking when I started writing that book. But there\\'s nothing like writing a book about something to help you learn it. The book, On Lisp, wasn\\'t published till 1993, but I wrote much of it in grad school.\\n\\nComputer Science is an uneasy alliance between two halves, theory and systems. The theory people prove things, and the systems people build things. I wanted to build things. I had plenty of respect for theory — indeed, a sneaking suspicion that it was the more admirable of the two halves — but building things seemed so much more exciting.\\n\\nThe problem with systems work, though, was that it didn\\'t last. ']\n"
]
}
],
"source": [
"retrieved_chunk = [chunks[i] for i in I.tolist()[0]]\n",
"print(retrieved_chunk)"
]
},
{
"cell_type": "markdown",
"id": "4b417a59-021a-411d-a491-cb31815192cd",
"metadata": {
"id": "4b417a59-021a-411d-a491-cb31815192cd"
},
"source": [
"#### Considerations:\n",
"- **Retrieval methods**: There are a lot different retrieval strategies. In our example, we are showing a simple similarity search with embeddings. Sometimes when there is metadata available for the data, it’s better to filter the data based on the metadata first before performing similarity search. There are also other statistical retrieval methods like TF-IDF and BM25 that use frequency and distribution of terms in the document to identify relevant text chunks.\n",
"- **Retrieved document**: Do we always retrieve individual text chunk as it is? Not always.\n",
" - Sometimes, we would like to include more context around the actual retrieved text chunk. We call the actual retrieve text chunk “child chunk” and our goal is to retrieve a larger “parent chunk” that the “child chunk” belongs to.\n",
" - On occasion, we might also want to provide weights to our retrieve documents. For example, a time-weighted approach would help us retrieve the most recent document.\n",
" - One common issue in the retrieval process is the “lost in the middle” problem where the information in the middle of a long context gets lost. Our models have tried to mitigate this issue. For example, in the passkey task, our models have demonstrated the ability to find a \"needle in a haystack\" by retrieving a randomly inserted passkey within a long prompt, up to 32k context length. However, it is worth considering experimenting with reordering the document to determine if placing the most relevant chunks at the beginning and end leads to improved results.\n",
" \n",
"### Combine context and question in a prompt and generate response\n",
"\n",
"Finally, we can offer the retrieved text chunks as the context information within the prompt. Here is a prompt template where we can include both the retrieved text and user question in the prompt.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "da042a53-4564-4057-9a60-9b57dffff6a1",
"metadata": {},
"outputs": [],
"source": [
"prompt = f\"\"\"\n",
"Context information is below.\n",
"---------------------\n",
"{retrieved_chunk}\n",
"---------------------\n",
"Given the context information and not prior knowledge, answer the query.\n",
"Query: {question}\n",
"Answer:\n",
"\"\"\""
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "e77d975b-5f69-4e9c-8b94-97214517eac7",
"metadata": {},
"outputs": [],
"source": [
"def run_mistral(user_message, model=\"mistral-large-latest\"):\n",
" messages = [\n",
" {\n",
" \"role\": \"user\", \"content\": user_message\n",
" }\n",
" ]\n",
" chat_response = client.chat.complete(\n",
" model=model,\n",
" messages=messages\n",
" )\n",
" return (chat_response.choices[0].message.content)"
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "1c5c20aa-6673-4105-9c10-886a1e18da8a",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
},
"text/plain": [
"\"Before college, the author primarily worked on two things outside of school: writing and programming. In terms of writing, they focused on short stories, although they admitted that these stories were not very good, lacking plot and mainly featuring characters with strong feelings. As for programming, the author started writing programs on an IBM 1401 during their 9th grade, using an early version of Fortran. They struggled to figure out what to do with the IBM 1401, as they didn't have any data stored on punched cards and their mathematical knowledge was limited, but it was their first introduction to programming.\""
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"run_mistral(prompt)"
]
},
{
"cell_type": "markdown",
"id": "4e3b531c-4730-4108-ae8a-8de6563e085b",
"metadata": {
"id": "4e3b531c-4730-4108-ae8a-8de6563e085b"
},
"source": [
"#### Considerations:\n",
"- Prompting techniques: Most of the prompting techniques can be used in developing a RAG system as well. For example, we can use few-shot learning to guide the model’s answers by providing a few examples. Additionally, we can explicitly instruct the model to format answers in a certain way.\n",
"\n",
"\n",
"In the next sections, we are going to show you how to do a similar basic RAG with some of the popular RAG frameworks. We will start with LlamaIndex and add other frameworks in the future.\n"
]
},
{
"cell_type": "markdown",
"id": "e676c9be-12ea-4cc8-8e66-bd96c0a0ceea",
"metadata": {
"id": "e676c9be-12ea-4cc8-8e66-bd96c0a0ceea"
},
"source": [
"## LangChain"
]
},
{
"cell_type": "code",
"execution_count": 24,
"id": "b83adf94-9059-4779-afcb-2d273a66c695",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: langchain in /usr/local/lib/python3.10/dist-packages (0.2.9)\n",
"Requirement already satisfied: langchain-mistralai in /usr/local/lib/python3.10/dist-packages (0.1.10)\n",
"Requirement already satisfied: langchain_community in /usr/local/lib/python3.10/dist-packages (0.2.7)\n",
"Collecting mistralai==0.4.2\n",
" Downloading mistralai-0.4.2-py3-none-any.whl (20 kB)\n",
"Requirement already satisfied: httpx<1,>=0.25 in /usr/local/lib/python3.10/dist-packages (from mistralai==0.4.2) (0.27.0)\n",
"Requirement already satisfied: orjson<3.11,>=3.9.10 in /usr/local/lib/python3.10/dist-packages (from mistralai==0.4.2) (3.10.6)\n",
"Requirement already satisfied: pydantic<3,>=2.5.2 in /usr/local/lib/python3.10/dist-packages (from mistralai==0.4.2) (2.8.2)\n",
"Requirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (6.0.1)\n",
"Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.0.31)\n",
"Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (3.9.5)\n",
"Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (4.0.3)\n",
"Requirement already satisfied: langchain-core<0.3.0,>=0.2.20 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.2.21)\n",
"Requirement already satisfied: langchain-text-splitters<0.3.0,>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.2.2)\n",
"Requirement already satisfied: langsmith<0.2.0,>=0.1.17 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.1.92)\n",
"Requirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.25.2)\n",
"Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.31.0)\n",
"Requirement already satisfied: tenacity!=8.4.0,<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (8.5.0)\n",
"Requirement already satisfied: httpx-sse<1,>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from langchain-mistralai) (0.4.0)\n",
"Requirement already satisfied: tokenizers<1,>=0.15.1 in /usr/local/lib/python3.10/dist-packages (from langchain-mistralai) (0.19.1)\n",
"Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /usr/local/lib/python3.10/dist-packages (from langchain_community) (0.6.7)\n",
"Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n",
"Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.2.0)\n",
"Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.4.1)\n",
"Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.0.5)\n",
"Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.9.4)\n",
"Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json<0.7,>=0.5.7->langchain_community) (3.21.3)\n",
"Requirement already satisfied: typing-inspect<1,>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json<0.7,>=0.5.7->langchain_community) (0.9.0)\n",
"Requirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (3.7.1)\n",
"Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (2024.7.4)\n",
"Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (1.0.5)\n",
"Requirement already satisfied: idna in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (3.7)\n",
"Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (1.3.1)\n",
"Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx<1,>=0.25->mistralai==0.4.2) (0.14.0)\n",
"Requirement already satisfied: jsonpatch<2.0,>=1.33 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.3.0,>=0.2.20->langchain) (1.33)\n",
"Requirement already satisfied: packaging<25,>=23.2 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.3.0,>=0.2.20->langchain) (24.1)\n",
"Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=2.5.2->mistralai==0.4.2) (0.7.0)\n",
"Requirement already satisfied: pydantic-core==2.20.1 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=2.5.2->mistralai==0.4.2) (2.20.1)\n",
"Requirement already satisfied: typing-extensions>=4.6.1 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=2.5.2->mistralai==0.4.2) (4.12.2)\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.3.2)\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2.0.7)\n",
"Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain) (3.0.3)\n",
"Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /usr/local/lib/python3.10/dist-packages (from tokenizers<1,>=0.15.1->langchain-mistralai) (0.23.5)\n",
"Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers<1,>=0.15.1->langchain-mistralai) (3.15.4)\n",
"Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers<1,>=0.15.1->langchain-mistralai) (2023.6.0)\n",
"Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers<1,>=0.15.1->langchain-mistralai) (4.66.4)\n",
"Requirement already satisfied: jsonpointer>=1.9 in /usr/local/lib/python3.10/dist-packages (from jsonpatch<2.0,>=1.33->langchain-core<0.3.0,>=0.2.20->langchain) (3.0.0)\n",
"Requirement already satisfied: mypy-extensions>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain_community) (1.0.0)\n",
"Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx<1,>=0.25->mistralai==0.4.2) (1.2.2)\n",
"Installing collected packages: mistralai\n",
" Attempting uninstall: mistralai\n",
" Found existing installation: mistralai 0.5.5a20\n",
" Uninstalling mistralai-0.5.5a20:\n",
" Successfully uninstalled mistralai-0.5.5a20\n",
"Successfully installed mistralai-0.4.2\n"
]
}
],
"source": [
"!pip install langchain langchain-mistralai langchain_community mistralai==0.4.2"
]
},
{
"cell_type": "code",
"execution_count": 25,
"id": "fd31ffbc-9e0c-40b0-a21f-68ffac8fabea",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "fe8ec8fb7ff3408a8458ec5a5eed4ffa",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"tokenizer.json: 0%| | 0.00/1.80M [00:00<?, ?B/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"The two main things the author worked on before college were writing and programming. He wrote short stories and tried writing programs on an IBM 1401 using Fortran. His stories were awful and he couldn't figure out what to do with the 1401 due to the lack of input data. His programming skills improved with the advent of microcomputers, which allowed him to write simple games, a program to predict how high his model rockets would fly, and a word processor.\n"
]
}
],
"source": [
"from langchain_community.document_loaders import TextLoader\n",
"from langchain_mistralai.chat_models import ChatMistralAI\n",
"from langchain_mistralai.embeddings import MistralAIEmbeddings\n",
"from langchain_community.vectorstores import FAISS\n",
"from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
"from langchain.chains.combine_documents import create_stuff_documents_chain\n",
"from langchain_core.prompts import ChatPromptTemplate\n",
"from langchain.chains import create_retrieval_chain\n",
"\n",
"# Load data\n",
"loader = TextLoader(\"essay.txt\")\n",
"docs = loader.load()\n",
"# Split text into chunks\n",
"text_splitter = RecursiveCharacterTextSplitter()\n",
"documents = text_splitter.split_documents(docs)\n",
"# Define the embedding model\n",
"embeddings = MistralAIEmbeddings(model=\"mistral-embed\", mistral_api_key=api_key)\n",
"# Create the vector store\n",
"vector = FAISS.from_documents(documents, embeddings)\n",
"# Define a retriever interface\n",
"retriever = vector.as_retriever()\n",
"# Define LLM\n",
"model = ChatMistralAI(mistral_api_key=api_key)\n",
"# Define prompt template\n",
"prompt = ChatPromptTemplate.from_template(\"\"\"Answer the following question based only on the provided context:\n",
"\n",
"<context>\n",
"{context}\n",
"</context>\n",
"\n",
"Question: {input}\"\"\")\n",
"\n",
"# Create a retrieval chain to answer questions\n",
"document_chain = create_stuff_documents_chain(model, prompt)\n",
"retrieval_chain = create_retrieval_chain(retriever, document_chain)\n",
"response = retrieval_chain.invoke({\"input\": \"What were the two main things the author worked on before college?\"})\n",
"print(response[\"answer\"])"
]
},
{
"cell_type": "markdown",
"id": "48f538ee-c561-43a6-a421-a0e7b35638f7",
"metadata": {
"id": "48f538ee-c561-43a6-a421-a0e7b35638f7"
},
"source": [
"## LlamaIndex"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "152c2a1e-9564-459c-9ea9-5208da519a90",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: llama-index==0.10.55 in /usr/local/lib/python3.10/dist-packages (0.10.55)\n",
"Requirement already satisfied: llama-index-llms-mistralai==0.1.18 in /usr/local/lib/python3.10/dist-packages (0.1.18)\n",
"Requirement already satisfied: llama-index-embeddings-mistralai in /usr/local/lib/python3.10/dist-packages (0.1.4)\n",
"Requirement already satisfied: mistralai==0.4.2 in /usr/local/lib/python3.10/dist-packages (0.4.2)\n",
"Requirement already satisfied: llama-index-agent-openai<0.3.0,>=0.1.4 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.10.55) (0.2.8)\n",
"Requirement already satisfied: llama-index-cli<0.2.0,>=0.1.2 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.10.55) (0.1.12)\n",
"Requirement already satisfied: llama-index-core==0.10.55 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.10.55) (0.10.55)\n",
"Requirement already satisfied: llama-index-embeddings-openai<0.2.0,>=0.1.5 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.10.55) (0.1.11)\n",
"Requirement already satisfied: llama-index-indices-managed-llama-cloud>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.10.55) (0.2.5)\n",
"Requirement already satisfied: llama-index-legacy<0.10.0,>=0.9.48 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.10.55) (0.9.48)\n",
"Requirement already satisfied: llama-index-llms-openai<0.2.0,>=0.1.13 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.10.55) (0.1.26)\n",
"Requirement already satisfied: llama-index-multi-modal-llms-openai<0.2.0,>=0.1.3 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.10.55) (0.1.8)\n",
"Requirement already satisfied: llama-index-program-openai<0.2.0,>=0.1.3 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.10.55) (0.1.6)\n",
"Requirement already satisfied: llama-index-question-gen-openai<0.2.0,>=0.1.2 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.10.55) (0.1.3)\n",
"Requirement already satisfied: llama-index-readers-file<0.2.0,>=0.1.4 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.10.55) (0.1.30)\n",
"Requirement already satisfied: llama-index-readers-llama-parse>=0.1.2 in /usr/local/lib/python3.10/dist-packages (from llama-index==0.10.55) (0.1.6)\n",
"Requirement already satisfied: httpx<1,>=0.25 in /usr/local/lib/python3.10/dist-packages (from mistralai==0.4.2) (0.27.0)\n",
"Requirement already satisfied: orjson<3.11,>=3.9.10 in /usr/local/lib/python3.10/dist-packages (from mistralai==0.4.2) (3.10.6)\n",
"Requirement already satisfied: pydantic<3,>=2.5.2 in /usr/local/lib/python3.10/dist-packages (from mistralai==0.4.2) (2.8.2)\n",
"Requirement already satisfied: PyYAML>=6.0.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (6.0.1)\n",
"Requirement already satisfied: SQLAlchemy[asyncio]>=1.4.49 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (2.0.31)\n",
"Requirement already satisfied: aiohttp<4.0.0,>=3.8.6 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (3.9.5)\n",
"Requirement already satisfied: dataclasses-json in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (0.6.7)\n",
"Requirement already satisfied: deprecated>=1.2.9.3 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (1.2.14)\n",
"Requirement already satisfied: dirtyjson<2.0.0,>=1.0.8 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (1.0.8)\n",
"Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (2023.6.0)\n",
"Requirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (1.6.0)\n",
"Requirement already satisfied: networkx>=3.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (3.3)\n",
"Requirement already satisfied: nltk<4.0.0,>=3.8.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (3.8.1)\n",
"Requirement already satisfied: numpy<2.0.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (1.25.2)\n",
"Requirement already satisfied: openai>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (1.35.15)\n",
"Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (2.0.3)\n",
"Requirement already satisfied: pillow>=9.0.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (9.4.0)\n",
"Requirement already satisfied: requests>=2.31.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (2.31.0)\n",
"Requirement already satisfied: tenacity!=8.4.0,<9.0.0,>=8.2.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (8.5.0)\n",
"Requirement already satisfied: tiktoken>=0.3.3 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (0.7.0)\n",
"Requirement already satisfied: tqdm<5.0.0,>=4.66.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (4.66.4)\n",
"Requirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (4.12.2)\n",
"Requirement already satisfied: typing-inspect>=0.8.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (0.9.0)\n",
"Requirement already satisfied: wrapt in /usr/local/lib/python3.10/dist-packages (from llama-index-core==0.10.55->llama-index==0.10.55) (1.14.1)\n",
"Requirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (3.7.1)\n",
"Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (2024.7.4)\n",
"Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (1.0.5)\n",
"Requirement already satisfied: idna in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (3.7)\n",
"Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (1.3.1)\n",
"Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx<1,>=0.25->mistralai==0.4.2) (0.14.0)\n",
"Requirement already satisfied: llama-cloud>=0.0.9 in /usr/local/lib/python3.10/dist-packages (from llama-index-indices-managed-llama-cloud>=0.2.0->llama-index==0.10.55) (0.0.9)\n",
"Requirement already satisfied: beautifulsoup4<5.0.0,>=4.12.3 in /usr/local/lib/python3.10/dist-packages (from llama-index-readers-file<0.2.0,>=0.1.4->llama-index==0.10.55) (4.12.3)\n",
"Requirement already satisfied: pypdf<5.0.0,>=4.0.1 in /usr/local/lib/python3.10/dist-packages (from llama-index-readers-file<0.2.0,>=0.1.4->llama-index==0.10.55) (4.3.0)\n",
"Requirement already satisfied: striprtf<0.0.27,>=0.0.26 in /usr/local/lib/python3.10/dist-packages (from llama-index-readers-file<0.2.0,>=0.1.4->llama-index==0.10.55) (0.0.26)\n",
"Requirement already satisfied: llama-parse>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from llama-index-readers-llama-parse>=0.1.2->llama-index==0.10.55) (0.4.9)\n",
"Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=2.5.2->mistralai==0.4.2) (0.7.0)\n",
"Requirement already satisfied: pydantic-core==2.20.1 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=2.5.2->mistralai==0.4.2) (2.20.1)\n",
"Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core==0.10.55->llama-index==0.10.55) (1.3.1)\n",
"Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core==0.10.55->llama-index==0.10.55) (23.2.0)\n",
"Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core==0.10.55->llama-index==0.10.55) (1.4.1)\n",
"Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core==0.10.55->llama-index==0.10.55) (6.0.5)\n",
"Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core==0.10.55->llama-index==0.10.55) (1.9.4)\n",
"Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core==0.10.55->llama-index==0.10.55) (4.0.3)\n",
"Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4<5.0.0,>=4.12.3->llama-index-readers-file<0.2.0,>=0.1.4->llama-index==0.10.55) (2.5)\n",
"Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index-core==0.10.55->llama-index==0.10.55) (8.1.7)\n",
"Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index-core==0.10.55->llama-index==0.10.55) (1.4.2)\n",
"Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.10/dist-packages (from nltk<4.0.0,>=3.8.1->llama-index-core==0.10.55->llama-index==0.10.55) (2024.5.15)\n",
"Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai>=1.1.0->llama-index-core==0.10.55->llama-index==0.10.55) (1.7.0)\n",
"Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx<1,>=0.25->mistralai==0.4.2) (1.2.2)\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->llama-index-core==0.10.55->llama-index==0.10.55) (3.3.2)\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->llama-index-core==0.10.55->llama-index==0.10.55) (2.0.7)\n",
"Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy[asyncio]>=1.4.49->llama-index-core==0.10.55->llama-index==0.10.55) (3.0.3)\n",
"Requirement already satisfied: mypy-extensions>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from typing-inspect>=0.8.0->llama-index-core==0.10.55->llama-index==0.10.55) (1.0.0)\n",
"Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json->llama-index-core==0.10.55->llama-index==0.10.55) (3.21.3)\n",
"Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index-core==0.10.55->llama-index==0.10.55) (2.9.0.post0)\n",
"Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index-core==0.10.55->llama-index==0.10.55) (2023.4)\n",
"Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->llama-index-core==0.10.55->llama-index==0.10.55) (2024.1)\n",
"Requirement already satisfied: packaging>=17.0 in /usr/local/lib/python3.10/dist-packages (from marshmallow<4.0.0,>=3.18.0->dataclasses-json->llama-index-core==0.10.55->llama-index==0.10.55) (24.1)\n",
"Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->llama-index-core==0.10.55->llama-index==0.10.55) (1.16.0)\n"
]
}
],
"source": [
"!pip install llama-index==0.10.55 llama-index-llms-mistralai==0.1.18 llama-index-embeddings-mistralai mistralai==0.4.2"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "96003762-acac-4886-964b-2d6a67f6f724",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The two main things the author worked on before college, outside of school, were writing and programming. They wrote short stories and tried writing programs on an IBM 1401 in 9th grade.\n"
]
}
],
"source": [
"import os\n",
"from llama_index.core import Settings, SimpleDirectoryReader, VectorStoreIndex\n",
"from llama_index.llms.mistralai import MistralAI\n",
"from llama_index.embeddings.mistralai import MistralAIEmbedding\n",
"\n",
"# Load data\n",
"reader = SimpleDirectoryReader(input_files=[\"essay.txt\"])\n",
"documents = reader.load_data()\n",
"# Define LLM and embedding model\n",
"Settings.llm = MistralAI(model=\"mistral-medium\", api_key=api_key)\n",
"Settings.embed_model = MistralAIEmbedding(model_name='mistral-embed', api_key=api_key)\n",
"# Create vector store index\n",
"index = VectorStoreIndex.from_documents(documents)\n",
"# Create query engine\n",
"query_engine = index.as_query_engine(similarity_top_k=2)\n",
"response = query_engine.query(\n",
" \"What were the two main things the author worked on before college?\"\n",
")\n",
"print(str(response))"
]
},
{
"cell_type": "markdown",
"id": "f0962398",
"metadata": {
"id": "f0962398"
},
"source": [
"# Haystack"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "805eed9a",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting mistral-haystack==0.0.1\n",
" Downloading mistral_haystack-0.0.1-py3-none-any.whl (11 kB)\n",
"Requirement already satisfied: mistralai==0.4.2 in /usr/local/lib/python3.10/dist-packages (0.4.2)\n",
"Collecting haystack-ai>=2.0.0b6 (from mistral-haystack==0.0.1)\n",
" Downloading haystack_ai-2.3.0-py3-none-any.whl (349 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m349.9/349.9 kB\u001b[0m \u001b[31m6.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: httpx<1,>=0.25 in /usr/local/lib/python3.10/dist-packages (from mistralai==0.4.2) (0.27.0)\n",
"Requirement already satisfied: orjson<3.11,>=3.9.10 in /usr/local/lib/python3.10/dist-packages (from mistralai==0.4.2) (3.10.6)\n",
"Requirement already satisfied: pydantic<3,>=2.5.2 in /usr/local/lib/python3.10/dist-packages (from mistralai==0.4.2) (2.8.2)\n",
"Collecting haystack-experimental (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1)\n",
" Downloading haystack_experimental-0.1.0-py3-none-any.whl (40 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m40.2/40.2 kB\u001b[0m \u001b[31m4.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (3.1.4)\n",
"Collecting lazy-imports (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1)\n",
" Downloading lazy_imports-0.3.1-py3-none-any.whl (12 kB)\n",
"Requirement already satisfied: more-itertools in /usr/local/lib/python3.10/dist-packages (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (10.1.0)\n",
"Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (3.3)\n",
"Requirement already satisfied: numpy<2 in /usr/local/lib/python3.10/dist-packages (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (1.25.2)\n",
"Requirement already satisfied: openai>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (1.35.15)\n",
"Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (2.0.3)\n",
"Collecting posthog (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1)\n",
" Downloading posthog-3.5.0-py2.py3-none-any.whl (41 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.3/41.3 kB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: python-dateutil in /usr/local/lib/python3.10/dist-packages (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (2.9.0.post0)\n",
"Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (6.0.1)\n",
"Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (2.31.0)\n",
"Requirement already satisfied: tenacity!=8.4.0 in /usr/local/lib/python3.10/dist-packages (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (8.5.0)\n",
"Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (4.66.4)\n",
"Requirement already satisfied: typing-extensions>=4.7 in /usr/local/lib/python3.10/dist-packages (from haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (4.12.2)\n",
"Requirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (3.7.1)\n",
"Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (2024.7.4)\n",
"Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (1.0.5)\n",
"Requirement already satisfied: idna in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (3.7)\n",
"Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.25->mistralai==0.4.2) (1.3.1)\n",
"Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx<1,>=0.25->mistralai==0.4.2) (0.14.0)\n",
"Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=2.5.2->mistralai==0.4.2) (0.7.0)\n",
"Requirement already satisfied: pydantic-core==2.20.1 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=2.5.2->mistralai==0.4.2) (2.20.1)\n",
"Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai>=1.1.0->haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (1.7.0)\n",
"Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx<1,>=0.25->mistralai==0.4.2) (1.2.2)\n",
"Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (2.1.5)\n",
"Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (2023.4)\n",
"Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (2024.1)\n",
"Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil->haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (1.16.0)\n",
"Collecting monotonic>=1.5 (from posthog->haystack-ai>=2.0.0b6->mistral-haystack==0.0.1)\n",
" Downloading monotonic-1.6-py2.py3-none-any.whl (8.2 kB)\n",
"Collecting backoff>=1.10.0 (from posthog->haystack-ai>=2.0.0b6->mistral-haystack==0.0.1)\n",
" Downloading backoff-2.2.1-py3-none-any.whl (15 kB)\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (3.3.2)\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->haystack-ai>=2.0.0b6->mistral-haystack==0.0.1) (2.0.7)\n",
"Installing collected packages: monotonic, lazy-imports, backoff, posthog, haystack-experimental, haystack-ai, mistral-haystack\n",
"Successfully installed backoff-2.2.1 haystack-ai-2.3.0 haystack-experimental-0.1.0 lazy-imports-0.3.1 mistral-haystack-0.0.1 monotonic-1.6 posthog-3.5.0\n"
]
}
],
"source": [
"!pip install mistral-haystack==0.0.1 mistralai==0.4.2"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "dfedfdc5",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Calculating embeddings: 100%|██████████| 3/3 [00:08<00:00, 2.96s/it]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Based on the documents provided, the two main things the author worked on before college were writing and programming. The author specifically mentions writing short stories outside of school and programming as two main areas of focus.\n"
]
}
],
"source": [
"from haystack import Pipeline\n",
"from haystack.document_stores.in_memory import InMemoryDocumentStore\n",
"from haystack.dataclasses import ChatMessage\n",
"from haystack.utils.auth import Secret\n",
"\n",
"from haystack.components.builders import DynamicChatPromptBuilder\n",
"from haystack.components.converters import TextFileToDocument\n",
"from haystack.components.preprocessors import DocumentSplitter\n",
"from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever\n",
"from haystack.components.writers import DocumentWriter\n",
"from haystack_integrations.components.embedders.mistral import MistralDocumentEmbedder, MistralTextEmbedder\n",
"from haystack_integrations.components.generators.mistral import MistralChatGenerator\n",
"\n",
"document_store = InMemoryDocumentStore()\n",
"\n",
"docs = TextFileToDocument().run(sources=[\"essay.txt\"])\n",
"split_docs = DocumentSplitter(split_by=\"passage\", split_length=2).run(documents=docs[\"documents\"])\n",
"embeddings = MistralDocumentEmbedder(api_key=Secret.from_token(api_key)).run(documents=split_docs[\"documents\"])\n",
"DocumentWriter(document_store=document_store).run(documents=embeddings[\"documents\"])\n",
"\n",
"\n",
"text_embedder = MistralTextEmbedder(api_key=Secret.from_token(api_key))\n",
"retriever = InMemoryEmbeddingRetriever(document_store=document_store)\n",
"prompt_builder = DynamicChatPromptBuilder(runtime_variables=[\"documents\"])\n",
"llm = MistralChatGenerator(api_key=Secret.from_token(api_key),\n",
" model='mistral-small')\n",
"\n",
"chat_template = \"\"\"Answer the following question based on the contents of the documents.\\n\n",
" Question: {{query}}\\n\n",
" Documents:\n",
" {% for document in documents %}\n",
" {{document.content}}\n",
" {% endfor%}\n",
" \"\"\"\n",
"messages = [ChatMessage.from_user(chat_template)]\n",
"\n",
"rag_pipeline = Pipeline()\n",
"rag_pipeline.add_component(\"text_embedder\", text_embedder)\n",
"rag_pipeline.add_component(\"retriever\", retriever)\n",
"rag_pipeline.add_component(\"prompt_builder\", prompt_builder)\n",
"rag_pipeline.add_component(\"llm\", llm)\n",
"\n",
"\n",
"rag_pipeline.connect(\"text_embedder.embedding\", \"retriever.query_embedding\")\n",
"rag_pipeline.connect(\"retriever.documents\", \"prompt_builder.documents\")\n",
"rag_pipeline.connect(\"prompt_builder.prompt\", \"llm.messages\")\n",
"\n",
"question = \"What were the two main things the author worked on before college?\"\n",
"\n",
"result = rag_pipeline.run(\n",
" {\n",
" \"text_embedder\": {\"text\": question},\n",
" \"prompt_builder\": {\"template_variables\": {\"query\": question}, \"prompt_source\": messages},\n",
" \"llm\": {\"generation_kwargs\": {\"max_tokens\": 225}},\n",
" }\n",
")\n",
"\n",
"print(result[\"llm\"][\"replies\"][0].content)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "GofSKA6RWiaM",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}