Instruct Formats
Overview
Open source models typically come in two versions:
- a pre-trained base model (e.g., Llama-3-8B) and
- an instruct version (e.g., Llama-3-8B-Instruct).
The instruct version undergoes further training with specific instructions using an instruction template. These templates ensure clarity and consistency in instructions, leading to better fine-tuning outcomes. They also focus the model's learning on relevant aspects of the data.
Experimentation shows that using model-specific instruction templates significantly boosts performance. While custom templates can be used with base models, model-specific instruction formatting usually results in better performance for instruction tuning both on the base model, but especially on instruction-tuned base models.
Note: Instruction tuning templates are very sensitive to spacing and new line characters. It is also important to use the same instruction tuning formatting from training during inference, otherwise you are very likely to see poor results.
See how to format your dataset for different base models below. If you don't see a model that we support in the base models below, then an instruct template may not be required.
- Llama 2
- Llama 3
- Mistral & Mixtral
- Zephyr
- Solar
- Gemma
- CodeLlama
- Phi-3
- Phi-3.5
- Qwen2
- Gemma-2-Instruct
Models
- llama-2-7b-chat
- llama-2-13b-chat
- llama-2-70b-chat
Input Prompt Template
There are two options, depending on whether you want to also include a system message as part of the instruction.
With system message:
<s>[INST] <<SYS>>\n {system_message} \n<</SYS>>\n\n {prompt} [/INST]
Without system message:
<s>[INST] {prompt} [/INST]
Make sure there is a space after the final [/INST] tag.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
| Prompt | Completion |
|---|---|
| <s>[INST] Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? [/INST] | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Models
- llama-3-8b-instruct
- llama-3-70b-instruct
- llama-3-1-8b-instruct
Input Prompt Template
There are two options, depending on whether you want to also include a system message as part of the instruction.
With system message:
<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n {system_message} <|eot_id|><|start_header_id|>user<|end_header_id|>\n\n {prompt} <|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n
Without system message:
<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\n {prompt} <|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n
Make sure there are two \n characters after the final <|end_header_id> tag.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
Prompt:
<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\n Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? <|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n
Completion:
Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72
Models
- mistral-7b-instruct
- mistral-7b-instruct-v0-2
- mistral-7b-instruct-v0-3
- mistral-nemo-12b-instruct-2407
- mixtral-8x7b-instruct-v0-1
Input Prompt Template
These models don't support using a system message.
<s>[INST] {prompt} [/INST]
Make sure there is a space after the final [/INST] tag.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
| Prompt | Completion |
|---|---|
| <s>[INST] Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? [/INST] | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Models
- zephyr-7b-beta
Input Prompt Template
There are two options, depending on whether you want to also include a system message as part of the instruction.
With system message:
<|system|>\n {system_message} </s>\n<|user|>\n {prompt} </s>\n<|assistant|>\n
Without system message:
<|user|>\n {prompt} </s>\n<|assistant|>\n
Make sure there is a \n after the final <|assistant|> tag.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
Prompt:
<|user|>\n Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? </s>\n<|assistant|>\n
Completion:
Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72
Models
- solar-1-mini-chat-240520
Input Prompt Template
There are two options, depending on whether you want to also include a system message as part of the instruction.
With system message:
<|im_start|>system\n {system_message} <|im_end|>\n<|im_start|>user\n {prompt} <|im_end|>\n<|im_start|>assistant\n
Without system message:
<|im_start|>user\n {prompt} <|im_end|>\n<|im_start|>assistant\n
Make sure there is a \n after assistant at the end of the template.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
Prompt:
<|im_start|>user\n Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? <|im_end|>\n<|im_start|>assistant\n
Completion:
Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72
Models
- gemma-2b-it
- gemma-7b-it
Input Prompt Template
These models don't support using a system message.
<bos><start_of_turn>user\n {prompt} <end_of_turn>\n<start_of_turn>model\n
Make sure there is a \n character after model.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
| Prompt | Completion |
|---|---|
| <bos><start_of_turn>user\n Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? <end_of_turn>\n<start_of_turn>model\n | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Models
- codellama-7b-instruct
- codellama-13b-instruct
- codellama-70b-instruct
Input Prompt Template
There are two options, depending on whether you want to also include a system message as part of the instruction.
With system message:
<s>[INST] <<SYS>>\n {system_message} \n<</SYS>>\n\n {prompt} [/INST]
Without system message:
<s>[INST] {prompt} [/INST]
Make sure there is a space after the final [/INST] tag.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
| Prompt | Completion |
|---|---|
| <s>[INST] Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? [/INST] | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Models
- phi-3-mini-4k-instruct
Input Prompt Template
This model does not support using a system message.
<|user|>\n {prompt} <|end|>\n<|assistant|>\n
Make sure there is a \n after assistant at the end of the template.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template:
Prompt:
<s><|user|>\n Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? <|end|>\n<|assistant|>\n
Completion:
Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72
Models
- phi-3-5-mini-instruct
Input Prompt Template
There are two options, depending on whether you want to also include a system message as part of the instruction.
With system message:
<|system|>\n {system_message} <|end|>\n<|user|>\n {prompt} <|end|>\n<|assistant|>\n
Without system message:
<|user|>\n {prompt} <|end|>\n<|assistant|>\n
:::
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
Prompt:
<|user|>\n Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? <|end|>\n<|assistant|>\n
Completion:
Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72
Models
- qwen2-1-5b
- qwen2-1-5b-instruct
- qwen2-7b
- qwen2-7b-instruct
- qwen2-72b
- qwen2-72b-instruct
Input Prompt Template
There are two options, depending on whether you want to also include a system message as part of the instruction.
With system message:
<|im_start|>system\nThis is a system message<|im_end|>\n<|im_start|>user\nThis is a user message<|im_end|>\n<|im_start|>assistant\n
Without system message:
<|im_start|>user\nThis is a user message<|im_end|>\n<|im_start|>assistant\n
Make sure there is a \n after the "assistant" tag
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
| Prompt | Completion |
|---|---|
| <|im_start|>user\n Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? <|im_end|>\n<|im_start|>assistant\n | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Models
- gemma-2-9b-instruct
- gemma-2-27b-instruct
Input Prompt Template
<bos><start_of_turn>user\n{prompt}<end_of_turn>\n<start_of_turn>model\n
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template:
| Prompt | Completion |
|---|---|
| \<bos><start_of_turn>user\nPlease answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?<end_of_turn>\n<start_of_turn>model\n | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |