langtrace mistral

Notebook content (JSON format):

{"cells":[{"cell_type":"markdown","metadata":{"id":"g7j3YX8-WvMX"},"source":["# RAG Observability with Mistral AI and Langtrace\n","\n","This Notebook shows the instructions for setting up OpenTelemetry based tracing for Mistral with Langtrace AI.\n","\n","The Goal for this notebook to showcase a simple RAG app where you can chat with the United states consititution pdf."]},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"collapsed":true,"executionInfo":{"elapsed":4081,"status":"ok","timestamp":1731615462519,"user":{"displayName":"Karthik Kalyanaraman","userId":"06165623146440115541"},"user_tz":480},"id":"8HpEYe58UiOl","outputId":"232de61c-8294-403e-b031-a76a1f894612"},"outputs":[{"name":"stdout","output_type":"stream","text":["Requirement already satisfied: mistralai in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (1.2.3)\n","Requirement already satisfied: langtrace-python-sdk in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (3.3.10)\n","Requirement already satisfied: chromadb in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (0.5.20)\n","Requirement already satisfied: pypdf in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (5.1.0)\n","Requirement already satisfied: langchain in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (0.3.8)\n","Requirement already satisfied: langchain-community in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (0.3.8)\n","Requirement already satisfied: eval-type-backport<0.3.0,>=0.2.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from mistralai) (0.2.0)\n","Requirement already satisfied: httpx<0.28.0,>=0.27.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from mistralai) (0.27.2)\n","Requirement already satisfied: jsonpath-python<2.0.0,>=1.0.6 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from mistralai) (1.0.6)\n","Requirement already satisfied: pydantic<3.0.0,>=2.9.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from mistralai) (2.10.1)\n","Requirement already satisfied: python-dateutil==2.8.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from mistralai) (2.8.2)\n","Requirement already satisfied: typing-inspect<0.10.0,>=0.9.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from mistralai) (0.9.0)\n","Requirement already satisfied: six>=1.5 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from python-dateutil==2.8.2->mistralai) (1.16.0)\n","Requirement already satisfied: colorama>=0.4.6 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (0.4.6)\n","Requirement already satisfied: fsspec>=2024.6.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (2024.10.0)\n","Requirement already satisfied: opentelemetry-api>=1.25.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (1.28.2)\n","Requirement already satisfied: opentelemetry-exporter-otlp-proto-grpc>=1.25.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (1.28.2)\n","Requirement already satisfied: opentelemetry-exporter-otlp-proto-http>=1.25.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (1.28.2)\n","Requirement already satisfied: opentelemetry-instrumentation-sqlalchemy>=0.46b0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (0.49b2)\n","Requirement already satisfied: opentelemetry-instrumentation>=0.47b0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (0.49b2)\n","Requirement already satisfied: opentelemetry-sdk>=1.25.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (1.28.2)\n","Requirement already satisfied: sentry-sdk>=2.14.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (2.19.0)\n","Requirement already satisfied: sqlalchemy in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (2.0.35)\n","Requirement already satisfied: tiktoken>=0.1.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (0.8.0)\n","Requirement already satisfied: trace-attributes==7.1.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (7.1.0)\n","Requirement already satisfied: transformers>=4.11.3 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (4.46.3)\n","Requirement already satisfied: ujson>=5.10.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (5.10.0)\n","Requirement already satisfied: build>=1.0.3 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (1.2.2.post1)\n","Requirement already satisfied: chroma-hnswlib==0.7.6 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (0.7.6)\n","Requirement already satisfied: fastapi>=0.95.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (0.115.5)\n","Requirement already satisfied: uvicorn[standard]>=0.18.3 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (0.32.1)\n","Requirement already satisfied: numpy>=1.22.5 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (1.26.4)\n","Requirement already satisfied: posthog>=2.4.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (3.7.3)\n","Requirement already satisfied: typing-extensions>=4.5.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (4.12.2)\n","Requirement already satisfied: onnxruntime>=1.14.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (1.20.1)\n","Requirement already satisfied: opentelemetry-instrumentation-fastapi>=0.41b0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (0.49b2)\n","Requirement already satisfied: tokenizers>=0.13.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (0.20.3)\n","Requirement already satisfied: pypika>=0.48.9 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (0.48.9)\n","Requirement already satisfied: tqdm>=4.65.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (4.67.1)\n","Requirement already satisfied: overrides>=7.3.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (7.7.0)\n","Requirement already satisfied: importlib-resources in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (6.4.5)\n","Requirement already satisfied: grpcio>=1.58.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (1.68.0)\n","Requirement already satisfied: bcrypt>=4.0.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (4.2.1)\n","Requirement already satisfied: typer>=0.9.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (0.13.1)\n","Requirement already satisfied: kubernetes>=28.1.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (31.0.0)\n","Requirement already satisfied: tenacity>=8.2.3 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (9.0.0)\n","Requirement already satisfied: PyYAML>=6.0.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (6.0.2)\n","Requirement already satisfied: mmh3>=4.0.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (5.0.1)\n","Requirement already satisfied: orjson>=3.9.12 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (3.10.12)\n","Requirement already satisfied: rich>=10.11.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (13.9.4)\n","Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain) (3.11.7)\n","Requirement already satisfied: langchain-core<0.4.0,>=0.3.21 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain) (0.3.21)\n","Requirement already satisfied: langchain-text-splitters<0.4.0,>=0.3.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain) (0.3.2)\n","Requirement already satisfied: langsmith<0.2.0,>=0.1.17 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain) (0.1.146)\n","Requirement already satisfied: requests<3,>=2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain) (2.32.3)\n","Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain-community) (0.6.7)\n","Requirement already satisfied: httpx-sse<0.5.0,>=0.4.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain-community) (0.4.0)\n","Requirement already satisfied: pydantic-settings<3.0.0,>=2.4.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain-community) (2.6.1)\n","Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (2.4.3)\n","Requirement already satisfied: aiosignal>=1.1.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n","Requirement already satisfied: attrs>=17.3.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (24.2.0)\n","Requirement already satisfied: frozenlist>=1.1.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.5.0)\n","Requirement already satisfied: multidict<7.0,>=4.5 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.1.0)\n","Requirement already satisfied: propcache>=0.2.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (0.2.0)\n","Requirement already satisfied: yarl<2.0,>=1.17.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.18.0)\n","Requirement already satisfied: packaging>=19.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from build>=1.0.3->chromadb) (24.2)\n","Requirement already satisfied: pyproject_hooks in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from build>=1.0.3->chromadb) (1.2.0)\n","Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain-community) (3.23.1)\n","Requirement already satisfied: starlette<0.42.0,>=0.40.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from fastapi>=0.95.2->chromadb) (0.41.3)\n","Requirement already satisfied: anyio in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from httpx<0.28.0,>=0.27.0->mistralai) (4.6.2.post1)\n","Requirement already satisfied: certifi in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from httpx<0.28.0,>=0.27.0->mistralai) (2024.8.30)\n","Requirement already satisfied: httpcore==1.* in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from httpx<0.28.0,>=0.27.0->mistralai) (1.0.7)\n","Requirement already satisfied: idna in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from httpx<0.28.0,>=0.27.0->mistralai) (3.10)\n","Requirement already satisfied: sniffio in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from httpx<0.28.0,>=0.27.0->mistralai) (1.3.1)\n","Requirement already satisfied: h11<0.15,>=0.13 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from httpcore==1.*->httpx<0.28.0,>=0.27.0->mistralai) (0.14.0)\n","Requirement already satisfied: google-auth>=1.0.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb) (2.36.0)\n","Requirement already satisfied: websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb) (1.8.0)\n","Requirement already satisfied: requests-oauthlib in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb) (2.0.0)\n","Requirement already satisfied: oauthlib>=3.2.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb) (3.2.2)\n","Requirement already satisfied: urllib3>=1.24.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb) (2.2.3)\n","Requirement already satisfied: durationpy>=0.7 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb) (0.9)\n","Requirement already satisfied: jsonpatch<2.0,>=1.33 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain-core<0.4.0,>=0.3.21->langchain) (1.33)\n","Requirement already satisfied: requests-toolbelt<2.0.0,>=1.0.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langsmith<0.2.0,>=0.1.17->langchain) (1.0.0)\n","Requirement already satisfied: coloredlogs in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from onnxruntime>=1.14.1->chromadb) (15.0.1)\n","Requirement already satisfied: flatbuffers in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from onnxruntime>=1.14.1->chromadb) (24.3.25)\n","Requirement already satisfied: protobuf in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from onnxruntime>=1.14.1->chromadb) (5.28.3)\n","Requirement already satisfied: sympy in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from onnxruntime>=1.14.1->chromadb) (1.13.3)\n","Requirement already satisfied: deprecated>=1.2.6 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-api>=1.25.0->langtrace-python-sdk) (1.2.15)\n","Requirement already satisfied: importlib-metadata<=8.5.0,>=6.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-api>=1.25.0->langtrace-python-sdk) (8.5.0)\n","Requirement already satisfied: googleapis-common-protos~=1.52 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc>=1.25.0->langtrace-python-sdk) (1.66.0)\n","Requirement already satisfied: opentelemetry-exporter-otlp-proto-common==1.28.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc>=1.25.0->langtrace-python-sdk) (1.28.2)\n","Requirement already satisfied: opentelemetry-proto==1.28.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc>=1.25.0->langtrace-python-sdk) (1.28.2)\n","Requirement already satisfied: opentelemetry-semantic-conventions==0.49b2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-instrumentation>=0.47b0->langtrace-python-sdk) (0.49b2)\n","Requirement already satisfied: wrapt<2.0.0,>=1.0.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-instrumentation>=0.47b0->langtrace-python-sdk) (1.17.0)\n","Requirement already satisfied: opentelemetry-instrumentation-asgi==0.49b2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb) (0.49b2)\n","Requirement already satisfied: opentelemetry-util-http==0.49b2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb) (0.49b2)\n","Requirement already satisfied: asgiref~=3.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-instrumentation-asgi==0.49b2->opentelemetry-instrumentation-fastapi>=0.41b0->chromadb) (3.8.1)\n","Requirement already satisfied: monotonic>=1.5 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from posthog>=2.4.0->chromadb) (1.6)\n","Requirement already satisfied: backoff>=1.10.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from posthog>=2.4.0->chromadb) (2.2.1)\n","Requirement already satisfied: annotated-types>=0.6.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from pydantic<3.0.0,>=2.9.0->mistralai) (0.7.0)\n","Requirement already satisfied: pydantic-core==2.27.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from pydantic<3.0.0,>=2.9.0->mistralai) (2.27.1)\n","Requirement already satisfied: python-dotenv>=0.21.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from pydantic-settings<3.0.0,>=2.4.0->langchain-community) (1.0.1)\n","Requirement already satisfied: charset-normalizer<4,>=2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from requests<3,>=2->langchain) (3.4.0)\n","Requirement already satisfied: markdown-it-py>=2.2.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from rich>=10.11.0->chromadb) (3.0.0)\n","Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from rich>=10.11.0->chromadb) (2.18.0)\n","Requirement already satisfied: regex>=2022.1.18 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from tiktoken>=0.1.1->langtrace-python-sdk) (2024.11.6)\n","Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from tokenizers>=0.13.2->chromadb) (0.26.2)\n","Requirement already satisfied: filelock in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from transformers>=4.11.3->langtrace-python-sdk) (3.16.1)\n","Requirement already satisfied: safetensors>=0.4.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from transformers>=4.11.3->langtrace-python-sdk) (0.4.5)\n","Requirement already satisfied: click>=8.0.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from typer>=0.9.0->chromadb) (8.1.7)\n","Requirement already satisfied: shellingham>=1.3.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from typer>=0.9.0->chromadb) (1.5.4)\n","Requirement already satisfied: mypy-extensions>=0.3.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from typing-inspect<0.10.0,>=0.9.0->mistralai) (1.0.0)\n","Requirement already satisfied: httptools>=0.6.3 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.6.4)\n","Requirement already satisfied: uvloop!=0.15.0,!=0.15.1,>=0.14.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.21.0)\n","Requirement already satisfied: watchfiles>=0.13 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (1.0.0)\n","Requirement already satisfied: websockets>=10.4 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (14.1)\n","Requirement already satisfied: cachetools<6.0,>=2.0.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb) (5.5.0)\n","Requirement already satisfied: pyasn1-modules>=0.2.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb) (0.4.1)\n","Requirement already satisfied: rsa<5,>=3.1.4 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb) (4.9)\n","Requirement already satisfied: zipp>=3.20 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from importlib-metadata<=8.5.0,>=6.0->opentelemetry-api>=1.25.0->langtrace-python-sdk) (3.21.0)\n","Requirement already satisfied: jsonpointer>=1.9 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from jsonpatch<2.0,>=1.33->langchain-core<0.4.0,>=0.3.21->langchain) (3.0.0)\n","Requirement already satisfied: mdurl~=0.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from markdown-it-py>=2.2.0->rich>=10.11.0->chromadb) (0.1.2)\n","Requirement already satisfied: humanfriendly>=9.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from coloredlogs->onnxruntime>=1.14.1->chromadb) (10.0)\n","Requirement already satisfied: mpmath<1.4,>=1.1.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from sympy->onnxruntime>=1.14.1->chromadb) (1.3.0)\n","Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from pyasn1-modules>=0.2.1->google-auth>=1.0.1->kubernetes>=28.1.0->chromadb) (0.6.1)\n","\n","\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.3.1\u001b[0m\n","\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n","Note: you may need to restart the kernel to use updated packages.\n"]}],"source":["%pip install mistralai langtrace-python-sdk chromadb pypdf langchain langchain-community"]},{"cell_type":"markdown","metadata":{},"source":["## Imports & Initialize clients\n"]},{"cell_type":"code","execution_count":2,"metadata":{"id":"RPYBLKHLU3NW"},"outputs":[{"name":"stdout","output_type":"stream","text":["\u001b[32mInitializing Langtrace SDK..\u001b[39m\n","\u001b[37m⭐ Leave our github a star to stay on top of our updates - https://github.com/Scale3-Labs/langtrace\u001b[39m\n","\u001b[34mExporting spans to Langtrace cloud..\u001b[39m\n"]},{"name":"stderr","output_type":"stream","text":["/Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n","  from .autonotebook import tqdm as notebook_tqdm\n"]}],"source":["import chromadb\n","from mistralai import Mistral\n","from langchain_community.document_loaders import PyPDFLoader\n","from langchain.text_splitter import RecursiveCharacterTextSplitter\n","from chromadb.utils import embedding_functions\n","from langtrace_python_sdk import langtrace, with_langtrace_root_span\n","\n","\n","langtrace.init(api_key='<langtrace_api_key>')\n","mistral = Mistral(api_key='<mistral_api_key>')\n","client = chromadb.Client()\n","\n"]},{"cell_type":"markdown","metadata":{},"source":["## Use Langchain to split pdf into chunks"]},{"cell_type":"code","execution_count":3,"metadata":{},"outputs":[],"source":["def load_and_split_pdf(pdf_path):\n","    loader = PyPDFLoader(pdf_path)\n","    data = loader.load()\n","    text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)\n","    chunks = text_splitter.split_documents(data)\n","    return chunks"]},{"cell_type":"markdown","metadata":{},"source":["## Setup Chroma & Insert pdf chunks\n","Create a chroma collection, specifying the default embedding function which will be used in our RAG when inserting pdf chunks"]},{"cell_type":"code","execution_count":4,"metadata":{},"outputs":[],"source":["def setup_chroma():\n","\n","    return client.get_or_create_collection(\n","        name=\"mistral-rag\",\n","        embedding_function=embedding_functions.DefaultEmbeddingFunction(),\n","    )\n","\n","\n","def add_documents_to_collection(collection, chunks):\n","    for i, chunk in enumerate(chunks):\n","        collection.add(\n","            documents=[chunk.page_content],\n","            metadatas=[{\"source\": chunk.metadata[\"source\"]}],\n","            ids=[str(i)],\n","        )"]},{"cell_type":"markdown","metadata":{},"source":["## Query Collection\n","1. take query from user, get nearest 3 results from chunked pdf\n","2. construct a prompt structure\n","3. Give query and prompt to mistral for the actual response"]},{"cell_type":"code","execution_count":5,"metadata":{"id":"LAcp_amwVL4S"},"outputs":[],"source":["def query_pdf(collection, query):\n","    results = collection.query(query_texts=[query], n_results=3)\n","    # Construct the prompt with context\n","    context = \"\\n\".join(results[\"documents\"][0])\n","    prompt = f\"\"\"Use the following context to answer the question. If you cannot answer based on the context, say so.\n","\n","            Context:\n","            {context}\n","\n","            Question: {query}\n","\n","            Answer:\"\"\"\n","    response = mistral.chat.complete(\n","        model=\"mistral-large-latest\",\n","        messages=[{\"role\": \"user\", \"content\": prompt}],\n","    )\n","    return response.choices[0].message.content"]},{"cell_type":"markdown","metadata":{},"source":["## Run everything together and monitor using Langtrace."]},{"cell_type":"code","execution_count":6,"metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["Creating collection\n","Loading and splitting pdf\n","Adding documents to collection\n","Querying pdf\n","Based on the context, the purpose of the Constitution is stated in the Preamble: \"to form a more perfect Union, establish Justice, insure domestic Tranquility, provide for the common defence, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity.\"\n"]}],"source":["@with_langtrace_root_span(\"main\")\n","def main():\n","    print(\"Creating collection\")\n","    collection = setup_chroma()\n","    print(\"Loading and splitting pdf\")\n","    chunks = load_and_split_pdf(\"https://www.govinfo.gov/content/pkg/CDOC-112hdoc129/pdf/CDOC-112hdoc129.pdf\")\n","    print(\"Adding documents to collection\")\n","    add_documents_to_collection(collection, chunks)\n","    print(\"Querying pdf\")\n","    print(query_pdf(collection, \"What is the purpose of the constitution?\"))\n","\n","\n","if __name__ == \"__main__\":\n","    main()"]},{"cell_type":"markdown","metadata":{"id":"vp05Kt2YVgbv"},"source":["That's it! Now you should be able to see the traces for all your inference calls on Langtrace!"]},{"cell_type":"markdown","metadata":{},"source":["## First Two Screenshots showcase the Trace and span structure of the whole RAG App.\n","\n","\n","![Trace 1](./TraceTree1.png)\n","\n","![Trace 2](./TraceTree2.png)\n"]},{"cell_type":"markdown","metadata":{},"source":["## Second Two Screenshots are details of Mistral's Run.\n","\n","- You can see what prompt is specfically fetched from chromadb and sent to mistral as well as the response\n","\n","![Trace 1](./mistral-langtrace1.png)\n","\n","![Trace 2](./mistral-langtrace2.png)"]}],"metadata":{"colab":{"authorship_tag":"ABX9TyPFTAJHx2doU1tWgCnfqECl","provenance":[]},"kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.11.5"}},"nbformat":4,"nbformat_minor":0}