Reach out

Command Palette

Search for a command to run...

Getting Started
Capabilities
Agents
Deployment
Guides

[Capabilities]

import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';

Fine-tuning allows you to tailor a pre-trained language model to your specific needs by training it on your dataset. This guide explains how to fine-tune text and vision models, from preparing your data to training, whether you aim to improve domain-specific understanding or adapt to a unique conversational style.

:::tip[ ] For detailed end-to-end fine-tuning examples and FAQ, check out our fine-tuning guide. :::

You can both finetune directly in la plateforme or via our API.

Dataset Format

Data must be stored in JSON Lines (.jsonl) files, which allow storing multiple JSON objects, each on a new line.

SFT Datasets should follow an instruction-following format representing a user-assistant conversation. Each JSON data sample should either consist of only user and assistant messages or include function-calling logic.

1. Default Instruct

Conversational data between user and assistant, which can be one-turn or multi-turn.

Text only template

1{
2  "messages": [
3    {
4      "role": "user",
5      "content": "User interaction n°1"
6    },
7    {
8      "role": "assistant",
9      "content": "Bot interaction n°1"
10    },
11    {
12      "role": "user",
13      "content": "User interaction n°2"
14    },
15    {
16      "role": "assistant",
17      "content": "Bot interaction n°2"
18    }
19  ]
20}

Note that the files must be in JSONL format, meaning every JSON object must be flattened into a single line, and each JSON object is on a new line.

1{"messages": [{"role": "user","content": "..."},{"role": "assistant","content": "..."},...]}
2{"messages": [{"role": "user","content": "..."},{"role": "assistant","content": "..."},...]}
3{"messages": [{"role": "user","content": "..."},{"role": "assistant","content": "..."},...]}
4{"messages": [{"role": "user","content": "..."},{"role": "assistant","content": "..."},...]}
5...
  • Conversational data must be stored under the "messages" key as a list.
  • Each list item is a dictionary containing the "content" and "role" keys. "role" is a string: "system", "user", "assistant" or "tool".
  • Loss computation is performed only on tokens corresponding to assistant messages ("role" == "assistant").

While text-only fine-tuning covers multiple use cases, you can also fine-tune the vision capabilities of our models. This allows you to create models that can understand and generate responses based on both text and image inputs.

Vision template

1{
2  "messages": [
3    {
4      "role": "user",
5      "content": [
6        {
7          "type": "image_url",
8          "image_url": "User Image URL, usually in a base64 format." // "data:image/jpeg;base64,{image_base64}"
9        },
10        {
11          "type": "text",
12          "text": "User interaction n°1"
13        }
14      ]
15    },
16    {
17      "role": "assistant",
18      "content": "Bot interaction n°1"
19    },
20    {
21      "role": "user",
22      "content": [
23        {
24          "type": "image_url",
25          "image_url": "User Image URL, usually in a base64 format." // "data:image/jpeg;base64,{image_base64}"
26        },
27        {
28          "type": "text",
29          "text": "User interaction n°2"
30        }
31      ]
32    },
33    {
34      "role": "assistant",
35      "content": "Bot interaction n°2"
36    }
37  ]
38}
  • Content can be a list of dictionaries, each containing a "type" key and either "text" or "image_url" keys.

2. Function-calling Instruct

Conversational data with tool usage. Example:

1{
2  "messages": [
3    {
4      "role": "system",
5      "content": "You are a helpful assistant with access to the following functions to help the user. You can use the functions if needed."
6    },
7    {
8      "role": "user",
9      "content": "Can you help me generate an anagram of the word 'listen'?"
10    },
11    {
12      "role": "assistant",
13      "tool_calls": [
14        {
15          "id": "TX92Jm8Zi",
16          "type": "function",
17          "function": {
18            "name": "generate_anagram",
19            "arguments": "{\"word\": \"listen\"}"
20          }
21        }
22      ]
23    },
24    {
25      "role": "tool",
26      "content": "{\"anagram\": \"silent\"}",
27      "tool_call_id": "TX92Jm8Zi"
28    },
29    {
30      "role": "assistant",
31      "content": "The anagram of the word 'listen' is 'silent'."
32    },
33    {
34      "role": "user",
35      "content": "That's amazing! Can you generate an anagram for the word 'race'?"
36    },
37    {
38      "role": "assistant",
39      "tool_calls": [
40        {
41          "id": "3XhQnxLsT",
42          "type": "function",
43          "function": {
44            "name": "generate_anagram",
45            "arguments": "{\"word\": \"race\"}"
46          }
47        }
48      ]
49    }
50  ],
51  "tools": [
52    {
53      "type": "function",
54      "function": {
55        "name": "generate_anagram",
56        "description": "Generate an anagram of a given word",
57        "parameters": {
58          "type": "object",
59          "properties": {
60            "word": {
61              "type": "string",
62              "description": "The word to generate an anagram of"
63            }
64          },
65          "required": ["word"]
66        }
67      }
68    }
69  ]
70}
  • Conversational data must be stored under the "messages" key as a list.
  • Each message is a dictionary containing the "role" and "content" or "tool_calls" keys. "role" should be one of "system", "user", "assistant" or "tool".
  • Only messages of type "assistant" can have a "tool_calls" key, representing the assistant performing a call to an available tool.
  • An assistant message with a "tool_calls" key cannot have a "content" key and must be followed by a "tool" message, which in turn must be followed by another assistant message.
  • The "tool_call_id" of tool messages must match the "id" of at least one of the previous assistant messages.
  • Both "id" and "tool_call_id" are randomly generated strings of exactly 9 characters. We recommend generating these automatically in a data preparation script as done here.
  • The "tools" key must include definitions of all tools used in the conversation.
  • Loss computation is performed only on tokens corresponding to assistant messages ("role" == "assistant").

Upload a file

Once you have the data file with the right format, you can upload the data file to the Mistral Client, making them available for use in fine-tuning jobs.

1from mistralai import Mistral
2import os
3
4api_key = os.environ["MISTRAL_API_KEY"]
5
6client = Mistral(api_key=api_key)
7
8training_data = client.files.upload(
9    file={
10        "file_name": "training_file.jsonl",
11        "content": open("training_file.jsonl", "rb"),
12    }
13)
14
15validation_data = client.files.upload(
16    file={
17        "file_name": "validation_file.jsonl",
18        "content": open("validation_file.jsonl", "rb"),
19    }
20)
1import { Mistral } from '@mistralai/mistralai';
2import fs from 'fs';
3
4const apiKey = process.env.MISTRAL_API_KEY;
5
6const client = new Mistral({ apiKey: apiKey });
7
8const training_file = fs.readFileSync('training_file.jsonl');
9const training_data = await client.files.upload({
10  file: {
11    fileName: 'training_file.jsonl',
12    content: training_file,
13  },
14});
15
16const validation_file = fs.readFileSync('validation_file.jsonl');
17const validation_data = await client.files.upload({
18  file: {
19    fileName: 'validation_file.jsonl',
20    content: validation_file,
21  },
22});
1curl https://api.mistral.ai/v1/files \
2  -H "Authorization: Bearer $MISTRAL_API_KEY" \
3  -F purpose="fine-tune" \
4  -F file="@training_file.jsonl"
5
6curl https://api.mistral.ai/v1/files \
7  -H "Authorization: Bearer $MISTRAL_API_KEY" \
8  -F purpose="fine-tune" \
9  -F file="@validation_file.jsonl"

Create a fine-tuning job

The next step is to create a fine-tuning job.

  • model: the specific model you would like to fine-tune. The choices are:
    • Text Only:
      • open-mistral-7b
      • mistral-small-latest
      • codestral-latest
      • open-mistral-nemo
      • mistral-large-latest
      • ministral-8b-latest
      • ministral-3b-latest
    • Vision:
      • pixtral-12b-latest
  • training_files: a collection of training file IDs, which can consist of a single file or multiple files
  • validation_files: a collection of validation file IDs, which can consist of a single file or multiple files
  • hyperparameters: two adjustable hyperparameters, "training_steps" and "learning_rate", that users can modify.
  • auto_start:
    • auto_start=True: Your job will be launched immediately after validation.
    • auto_start=False (default): You can manually start the training after validation by sending a POST request to /fine_tuning/jobs/<uuid>/start.
  • integrations: external integrations we support such as Weights and Biases for metrics tracking during training.
1# create a fine-tuning job
2created_jobs = client.fine_tuning.jobs.create(
3    model="open-mistral-7b",
4    training_files=[{"file_id": training_data.id, "weight": 1}],
5    validation_files=[validation_data.id],
6    hyperparameters={
7        "training_steps": 10,
8        "learning_rate":0.0001
9    },
10    auto_start=False,
11#   integrations=[
12#       {
13#           "project": "finetuning",
14#           "api_key": "WANDB_KEY",
15#       }
16#   ]
17)

After creating a fine-tuning job, you can check the job status using client.fine_tuning.jobs.get(job_id = created_jobs.id). 

1const createdJob = await client.fineTuning.jobs.create({
2  model: 'open-mistral-7b',
3  trainingFiles: [{ fileId: training_data.id, weight: 1 }],
4  validationFiles: [validation_data.id],
5  hyperparameters: {
6    trainingSteps: 10,
7    learningRate: 0.0001,
8  },
9  autoStart: false,
10  //  integrations=[
11  //      {
12  //          project: "finetuning",
13  //          apiKey: "WANDB_KEY",
14  //      }
15  //  ]
16});

After creating a fine-tuning job, you can check the job status using client.fineTuning.jobs.get({ jobId: createdJob.id }). 

1curl https://api.mistral.ai/v1/fine_tuning/jobs \
2--header "Authorization: Bearer $MISTRAL_API_KEY" \
3--header 'Content-Type: application/json' \
4--header 'Accept: application/json' \
5--data '{
6  "model": "open-mistral-7b",
7  "training_files": [
8    "<uuid>"
9  ],
10  "validation_files": [
11    "<uuid>"
12  ],
13  "hyperparameters": {
14    "training_steps": 10,
15    "learning_rate": 0.0001
16  },
17  "auto_start": false
18}'

After creating a fine-tuning job, you can check the job status using:

1curl https://api.mistral.ai/v1/fine_tuning/jobs/<jobid> \
2--header "Authorization: Bearer $MISTRAL_API_KEY"

Initially, the job status will be "QUEUED". After a brief period, the status will update to "VALIDATED". At this point, you can proceed to start the fine-tuning job:

1# start a fine-tuning job
2client.fine_tuning.jobs.start(job_id = created_jobs.id)
3
4created_jobs
1await client.fineTuning.jobs.start({ jobId: createdJob.id });
1curl -X POST https://api.mistral.ai/v1/fine_tuning/jobs/<jobid>/start \
2--header "Authorization: Bearer $MISTRAL_API_KEY"

List/retrieve/cancel jobs

You can also list jobs, retrieve a job, or cancel a job.

You can filter and view a list of jobs using various parameters such as page, page_size, model, created_after, created_by_me, status, wandb_project, wandb_name, and suffix. Check out our API specs for details.

1# List jobs
2jobs = client.fine_tuning.jobs.list()
3print(jobs)
4
5# Retrieve a jobs
6retrieved_jobs = client.fine_tuning.jobs.get(job_id = created_jobs.id)
7print(retrieved_jobs)
8
9# Cancel a jobs
10canceled_jobs = client.fine_tuning.jobs.cancel(job_id = created_jobs.id)
11print(canceled_jobs)
1// List jobs
2const jobs = await client.fineTuning.jobs.list();
3
4// Retrieve a job
5const retrievedJob = await client.fineTuning.jobs.get({ jobId: createdJob.id });
6
7// Cancel a job
8const canceledJob = await client.fineTuning.jobs.cancel({
9  jobId: createdJob.id,
10});
1# List jobs
2curl https://api.mistral.ai/v1/fine_tuning/jobs \
3--header "Authorization: Bearer $MISTRAL_API_KEY"
4
5# Retrieve a job
6curl https://api.mistral.ai/v1/fine_tuning/jobs/<jobid> \
7--header "Authorization: Bearer $MISTRAL_API_KEY"
8
9# Cancel a job
10curl -X POST https://api.mistral.ai/v1/fine_tuning/jobs/<jobid>/cancel \
11--header "Authorization: Bearer $MISTRAL_API_KEY"

Use a fine-tuned model

When a fine-tuned job is finished, you will be able to see the fine-tuned model name via retrieved_jobs.fine_tuned_model. Then you can use our chat endpoint to chat with the fine-tuned model:

1chat_response = client.chat.complete(
2    model=retrieved_job.fine_tuned_model,
3    messages = [{"role":'user', "content":'What is the best French cheese?'}]
4)
1const chatResponse = await client.chat.complete({
2  model: retrievedJob.fine_tuned_model,
3  messages: [{ role: 'user', content: 'What is the best French cheese?' }],
4});
1curl "https://api.mistral.ai/v1/chat/completions" \
2     --header 'Content-Type: application/json' \
3     --header 'Accept: application/json' \
4     --header "Authorization: Bearer $MISTRAL_API_KEY" \
5     --data '{
6    "model": "ft:open-mistral-7b:daf5e488:20240430:c1bed559",
7    "messages": [{"role": "user", "content": "Who is the most renowned French painter?"}]
8  }'

Delete a fine-tuned model

1client.models.delete(model_id=retrieved_job.fine_tuned_model)
1await client.models.delete({ modelId: retrieved_job.fine_tuned_model });
1curl --location --request DELETE 'https://api.mistral.ai/v1/models/ft:open-mistral-7b:XXX:20240531:XXX' \
2     --header 'Accept: application/json' \
3     --header "Authorization: Bearer $MISTRAL_API_KEY"

import FAQ from "../../guides/finetuning_sections/_04_faq.md";

Fine-tuning OverviewBatch Inference