import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';

LLMs are powerfull tools for text generation, and they also show great performance in code generation for multiple tasks, both for code completion, code generation and agentic tool use for semi-automated software development.

We provide 2 major families of llms for coding:

• Codestral: Specifically trained for Code Generation and FIM.
• Devstral: Specifically trained for Agentic Tool Use for Software Development.

Note that we also provide Codestral Embed, for semantic search and embedding code databases, repositories, and powering coding assistants with state-of-the-art retrieval. Learn more about it here.

Endpoints & Models

We provide 2 main endpoints:

• https://api.mistral.ai/v1/fim/completions: Fill-in-the-middle, for code completion and code generation; supporting codestral-latest.
• https://api.mistral.ai/v1/chat/completions: Instruction following, for coding and agentic tool use; supporting codestral-latest, devstral-small-latest and devstral-medium-latest.

FIM

With this feature, users can define the starting point of the code using a prompt, and the ending point of the code using an optional suffix and an optional stop. The FIM model will then generate the code that fits in between, making it ideal for tasks that require a specific piece of code to be generated.

:::tip[ ] We also provide the min_tokens and max_tokens sampling parameters, which are particularly useful for code generation as it allows you to set the minimum and maximum number of tokens that should be produced. This is especially useful when FIM models decide to produce no tokens at all, or are overly verbose, allowing developers to enforce completions within a specific range if they are needed. :::

Codestral

Codestral is a cutting-edge generative model that has been specifically designed and optimized for code generation tasks, including fill-in-the-middle and code completion. Codestral was trained on 80+ programming languages, enabling it to perform well on both common and less common languages.

:::important[ ] We currently offer two domains for Codestral endpoints, both providing FIM and instruct routes:

Wondering which endpoint to use?

• If you're a user, wanting to query Codestral as part of an IDE plugin, codestral.mistral.ai is recommended.
• If you're building a plugin, or anything that exposes these endpoints directly to the user, and expect them to bring their own API keys, you should also target codestral.mistral.ai
• For all other use cases, api.mistral.ai will be better suited

This guide uses api.mistral.ai for demonstration. :::

Below we present three examples:

Example 1: Fill in the middle

Originally, these models are designed to complete code in-between 2 points: a prefix (here called prompt) and a suffix, generating the code in-between.

1import os
2from mistralai import Mistral
3
4api_key = os.environ["MISTRAL_API_KEY"]
5client = Mistral(api_key=api_key)
6
7model = "codestral-latest"
8prompt = "def fibonacci(n: int):"
9suffix = "n = int(input('Enter a number: '))\nprint(fibonacci(n))"
10
11response = client.fim.complete(
12    model=model,
13    prompt=prompt,
14    suffix=suffix,
15    temperature=0,
16    # min_tokens=1, # Uncomment to enforce completions to at least 1 token
17)
18
19print(
20    f"""
21{prompt}
22{response.choices[0].message.content}
23{suffix}
24"""
25)

1curl --location 'https://api.mistral.ai/v1/fim/completions' \
2--header 'Content-Type: application/json' \
3--header 'Accept: application/json' \
4--header "Authorization: Bearer $MISTRAL_API_KEY" \
5--data '{
6    "model": "codestral-latest",
7    "prompt": "def f(",
8    "suffix": "return a + b",
9    "max_tokens": 64,
10    "temperature": 0
11}'

1</TabItem>

Example 2: Completion

However, you can also use the model for pure code completion, by only providing a prompt and no suffix.

1import os
2from mistralai import Mistral
3
4api_key = os.environ["MISTRAL_API_KEY"]
5client = Mistral(api_key=api_key)
6
7model = "codestral-latest"
8prompt = "def is_odd(n): \n return n % 2 == 1 \ndef test_is_odd():"
9
10response = client.fim.complete(model=model, prompt=prompt, temperature=0)
11
12print(
13    f"""
14{prompt}
15{response.choices[0].message.content}
16"""
17)

1curl --location 'https://api.mistral.ai/v1/fim/completions' \
2--header 'Content-Type: application/json' \
3--header 'Accept: application/json' \
4--header "Authorization: Bearer $MISTRAL_API_KEY" \
5--data '{
6    "model": "codestral-latest",
7    "prompt": "def is_odd(n): \n return n % 2 == 1 \n def test_is_odd():",
8    "suffix": "",
9    "max_tokens": 64,
10    "temperature": 0
11}'

1</TabItem>

Example 3: Stop tokens

You can also use stop tokens to control the generation of the model when it generates specific strings. :::tip[ ] We recommend adding stop tokens for IDE autocomplete integrations to prevent the model from being too verbose. :::

1import os
2from mistralai import Mistral
3
4api_key = os.environ["MISTRAL_API_KEY"]
5client = Mistral(api_key=api_key)
6
7model = "codestral-latest"
8prompt = "def is_odd(n): \n return n % 2 == 1 \ndef test_is_odd():"
9suffix = "n = int(input('Enter a number: '))\nprint(fibonacci(n))"
10
11response = client.fim.complete(
12    model=model, prompt=prompt, suffix=suffix, temperature=0, stop=["\n\n"]
13)
14
15print(
16    f"""
17{prompt}
18{response.choices[0].message.content}
19"""
20)

1curl --location 'https://api.mistral.ai/v1/fim/completions' \
2--header 'Content-Type: application/json' \
3--header 'Accept: application/json' \
4--header "Authorization: Bearer $MISTRAL_API_KEY" \
5--data '{
6    "model": "codestral-latest",
7    "prompt": "def is_odd(n): \n return n % 2 == 1 \n def test_is_odd():",
8    "suffix": "test_is_odd()",
9    "stop": ["\n\n"],
10    "max_tokens": 64,
11    "temperature": 0
12}'

1</TabItem>

Instruct Following

We also provide the instruct chat endpoint of Codestral with the same model codestral-latest.
The only difference is the endpoint used; so you can leverage powerfull code completion with instruct and chat use cases.

However we also provide devstral-small-latest and devstral-medium-latest for agentic tool use for software development, this family of models is specifically trained to navigate code bases and leverage tool usage for diverse tasks.

Codestral

Here is an example of how to use the instruct endpoint of Codestral, it's perfect for specific code generation of specific snippets or code completion while following instructions; so you can better guide generation and exchange with a powerfull coding model.

1import os
2from mistralai import Mistral
3
4api_key = os.environ["MISTRAL_API_KEY"]
5client = Mistral(api_key=api_key)
6
7model = "codestral-latest"
8message = [{"role": "user", "content": "Write a function for fibonacci"}]
9chat_response = client.chat.complete(
10    model = model,
11    messages = message
12)

1curl --location "https://api.mistral.ai/v1/chat/completions" \
2     --header 'Content-Type: application/json' \
3     --header 'Accept: application/json' \
4     --header "Authorization: Bearer $MISTRAL_API_KEY" \
5     --data '{
6    "model": "codestral-latest",
7    "messages": [{"role": "user", "content": "Write a function for fibonacci"}]
8  }'

1</TabItem>

Devstral

While Codestral is designed for code generation and FIM, Devstral is a cutting-edge generative model that has been specifically designed and optimized for agentic tool use for software development, it can leverage function calling to navigate code bases and call the right tools to perform specific tasks for semi-automated software development.

1import os
2from mistralai import Mistral
3
4api_key = os.environ["MISTRAL_API_KEY"]
5client = Mistral(api_key=api_key)
6
7model = "devstral-medium-latest"
8message = [{"role": "user", "content": "Create a new file called test.py and write a function for fibonacci"}]
9
10tools = [
11    {
12        "type": "function",
13        "function": {
14            "name": "create_file",
15            "description": "Create a new file with the given name and content",
16            "parameters": {
17                "type": "object",
18                "properties": {
19                    "filename": {
20                        "type": "string",
21                        "description": "The name of the file to create",
22                    },
23                    "content": {
24                        "type": "string",
25                        "description": "The content to write to the file",
26                    },
27                },
28                "required": ["filename", "content"],
29            },
30        },
31    }
32]
33
34chat_response = client.chat.complete(
35    model = model,
36    messages = message,
37    tools = tools
38)

1curl --location "https://api.mistral.ai/v1/chat/completions" \
2     --header 'Content-Type: application/json' \
3     --header 'Accept: application/json' \
4     --header "Authorization: Bearer $MISTRAL_API_KEY" \
5     --data '{
6    "model": "devstral-medium-latest",
7    "messages": [{"role": "user", "content": "Create a new file called test.py and write a function for fibonacci"}],
8    "tools": [
9        {
10            "type": "function",
11            "function": {
12                "name": "create_file",
13                "description": "Create a new file with the given name and content",
14                "parameters": {
15                    "type": "object",
16                    "properties": {
17                        "filename": {
18                            "type": "string",
19                            "description": "The name of the file to create"
20                        },
21                        "content": {
22                            "type": "string",
23                            "description": "The content to write to the file"
24                        }
25                    },
26                    "required": ["filename", "content"]
27                }
28            }
29        }
30    ]
31  }'

1</TabItem>

Integrations

Codestral Integrations

Continue.dev supports both Codestral base for code generation and Codestral Instruct for chat.

How to set up Codestral with Continue

Here is a step-by-step guide on how to set up Codestral with Continue using the Mistral AI API:

1. Install the Continue VS Code or JetBrains extension following the instructions here. Please make sure you install Continue version >v0.8.33.

2. Automatic set up:

• Click on the Continue extension iron on the left menu. Select Mistral API as a provider, select Codestral as a model.
• Click "Get API Key" to get Codestral API key.
• Click "Add model", which will automatically populate the config.json.

2. (alternative) Manually edit config.json

• Click on the gear icon in the bottom right corner of the Continue window to open ~/.continue/config.json (MacOS) / %userprofile%\.continue\config.json (Windows)
• Log in and request a Codestral API key on Mistral AI's La Plateforme here
• To use Codestral as your model for both autocomplete and chat, replace [API_KEY] with your Mistral API key below and add it to your config.json file:

1{
2  "models": [
3    {
4      "title": "Codestral",
5      "provider": "mistral",
6      "model": "codestral-latest",
7      "apiKey": "[API_KEY]"
8    }
9  ],
10  "tabAutocompleteModel": {
11    "title": "Codestral",
12    "provider": "mistral",
13    "model": "codestral-latest",
14    "apiKey": "[API_KEY]"
15  }
16}

If you run into any issues or have any questions, please join our Discord and post in #help channel here

Tabnine supports Codestral Instruct for chat.

How to set up Codestral with Tabnine

What is Tabnine Chat?

Tabnine Chat is a code-centric chat application that runs in the IDE and allows developers to interact with Tabnine’s AI models in a flexible, free-form way, using natural language. Tabnine Chat also supports dedicated quick actions that use predefined prompts optimized for specific use cases.

Getting started

To start using Tabnine Chat, first launch it in your IDE (VSCode, JetBrains, or Eclipse). Then, learn how to interact with Tabnine Chat, for example, how to ask questions or give instructions. Once you receive your response, you can read, review, and apply it within your code.

Selecting Codestral as Tabnine Chat App model

In the Tabnine Chat App, use the model selector to choose Codestral.

LangChain provides support for Codestral Instruct. Here is how you can use it in LangChain:

1# make sure to install `langchain` and `langchain-mistralai` in your Python environment
2
3import os
4from langchain_mistralai import ChatMistralAI
5from langchain_core.prompts import ChatPromptTemplate
6
7api_key = os.environ["MISTRAL_API_KEY"]
8mistral_model = "codestral-latest"
9llm = ChatMistralAI(model=mistral_model, temperature=0, api_key=api_key)
10llm.invoke([("user", "Write a function for fibonacci")])

For a more complex use case of self-corrective code generation using the instruct Codestral tool use, check out this notebook and this video:

LlamaIndex provides support for Codestral Instruct and Fill In Middle (FIM) endpoints. Here is how you can use it in LlamaIndex:

1# make sure to install `llama-index` and `llama-index-llms-mistralai` in your Python enviornment
2
3import os
4from llama_index.core.llms import ChatMessage
5from llama_index.llms.mistralai import MistralAI
6
7api_key =  os.environ["MISTRAL_API_KEY"]
8mistral_model = "codestral-latest"
9messages = [
10    ChatMessage(role="user", content="Write a function for fibonacci"),
11]
12MistralAI(api_key=api_key, model=mistral_model).chat(messages)

Check out more details on using Instruct and Fill In Middle(FIM) with LlamaIndex in this notebook.

Jupyter AI seamlessly integrates Codestral into JupyterLab, offering users a streamlined and enhanced AI-assisted coding experience within the Jupyter ecosystem. This integration boosts productivity and optimizes users' overall interaction with Jupyter.

To get started using Codestral and Jupyter AI in JupyterLab, first install needed packages in your Python environment:

1pip install jupyterlab langchain-mistralai jupyter-ai pandas matplotlib

Then launch Jupyter Lab:

1jupyter lab

Afterwards, you can select Codestral as your model of choice, input your Mistral API key, and start coding with Codestral!

JupyterLite is a project that aims to bring the JupyterLab environment to the web browser, allowing users to run Jupyter directly in their browser without the need for a local installation.

You can try Codestral with JupyterLite in your browser:

Tabby is an open-source AI coding assistant. You can use Codestral for both code completion and chat via Tabby.

To use Codestral in Tabby, configure your model configuration in ~/.tabby/config.toml as follows.

1[model.completion.http]
2kind = "mistral/completion"
3api_endpoint = "https://api.mistral.ai"
4api_key = "secret-api-key"

You can check out Tabby's documentation to learn more.

E2B provides open-source secure sandboxes for AI-generated code execution. With E2B, it is easy for developers to add code interpreting capabilities to AI apps using Codestral.

In the following examples, the AI agent performs a data analysis task on an uploaded CSV file, executes the AI-generated code by Codestral in the sandboxed environment by E2B, and returns a chart, saving it as a PNG file.

Python implementation (cookbook):

JS implementation (cookbook):

Devstral Integrations

OpenHands is an open-source scaffolding tool designed for building AI agents focused on software development. It offers a comprehensive framework for creating and managing these agents that can modify code, run commands, browse the web, call APIs, and even copy code snippets from StackOverflow.

After creating a Mistral AI account, you can use the following commands to start the OpenHands Docker container:

1export MISTRAL_API_KEY=<MY_KEY>
2
3mkdir -p ~/.openhands && echo '{"language":"en","agent":"CodeActAgent","max_iterations":null,"security_analyzer":null,"confirmation_mode":false,"llm_model":"mistral/devstral-small-2507","llm_api_key":"'$MISTRAL_API_KEY'","remote_runtime_resource_factor":null,"github_token":null,"enable_default_condenser":true}' > ~/.openhands-state/settings.json
4
5docker pull docker.all-hands.dev/all-hands-ai/runtime:0.48-nikolaik
6
7docker run -it --rm --pull=always \
8    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.48-nikolaik \
9    -e LOG_ALL_EVENTS=true \
10    -v /var/run/docker.sock:/var/run/docker.sock \
11    -v ~/.openhands:/.openhands \
12    -p 3000:3000 \
13    --add-host host.docker.internal:host-gateway \
14    --name openhands-app \
15    docker.all-hands.dev/all-hands-ai/openhands:0.48

For more information visit the OpenHands github repo and their documentation.

Cline is an autonomous coding agent operating right in your IDE, capable of creating/editing files, executing commands, using the browser, and more with your permission every step of the way.

For more information visit the Cline github repo.