Instruct Formats
Overview
Open source models typically come in two versions:
- a pre-trained base model (e.g., Llama-3-8B) and
- an instruct version (e.g., Llama-3-8B-Instruct).
The instruct version undergoes further training with specific instructions using an instruction template. These templates ensure clarity and consistency in instructions, leading to better fine-tuning outcomes. They also focus the model's learning on relevant aspects of the data.
Experimentation shows that using model-specific instruction templates significantly boosts performance. While custom templates can be used with base models, model-specific instruction formatting usually results in better performance for instruction tuning both on the base model, but especially on instruction-tuned base models.
Note: Instruction tuning templates are very sensitive to spacing and new line characters. It is also important to use the same instruction tuning formatting from training during inference, otherwise you are very likely to see poor results.
See how to format your dataset for different base models below.
- Llama 2
- Llama 3
- Mistral & Mixtral
- Zephyr
- Gemma
- CodeLlama-13B
- CodeLlama-70B
- Phi
Models
- llama-2-7b
- llama-2-7b-chat
- llama-2-13b
- llama-2-13b-chat
- llama-2-70b
- llama-2-70b-chat
Input Prompt Template
There are two options, depending on whether you want to also include a system message as part of the instruction.
With system message:
<s>[INST] <<SYS>>\n {system_message} \n<</SYS>>\n\n {prompt} [/INST]
Without system message:
<s>[INST] {prompt} [/INST]
Make sure there is a space after the final [/INST] tag.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
| Prompt | Completion |
|---|---|
| <s>[INST] Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? [/INST] | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Models
- llama-3-8b
- llama-3-8b-instruct
- llama-3-70b
- llama-3-70b-instruct
Input Prompt Template
There are two options, depending on whether you want to also include a system message as part of the instruction.
With system message:
<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n {system_message} <|eot_id|><|start_header_id|>user<|end_header_id|>\n\n {prompt} <|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n
Without system message:
<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\n {prompt} <|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n
Make sure there are two \n characters after the final <|end_header_id> tag.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
Prompt:
<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\n Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? <|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n
Completion:
Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72
Models
- mistral-7b-v0.1
- mistral-7b-instruct-v0.1
- mistral-7b-instruct-v0.2
- mixtral-8x7b
Input Prompt Template
These models don't support using a system message.
<s>[INST] {prompt} [/INST]
Make sure there is a space after the final [/INST] tag.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
| Prompt | Completion |
|---|---|
| <s>[INST] Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? [/INST] | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Models
- zephyr-7b
Input Prompt Template
There are two options, depending on whether you want to also include a system message as part of the instruction.
With system message:
<|system|>\n {system_message} </s>\n<|user|>\n {prompt} </s>\n<|assistant|>\n
Without system message:
<|user|>\n {prompt} </s>\n<|assistant|>\n
Make sure there is a \n after the final <|assistant|> tag.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
Prompt:
<|user|>\n Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? </s>\n<|assistant|>\n
Completion:
Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72
Models
- gemma-2b
- gemma-2b-it
- gemma-7b
- gemma-7b-it
Input Prompt Template
These models don't support using a system message.
<bos><start_of_turn>user\n {prompt} <end_of_turn>\n<start_of_turn>model\n
Make sure there is a \n character after model.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
| Prompt | Completion |
|---|---|
| <bos><start_of_turn>user\n Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? <end_of_turn>\n<start_of_turn>model\n | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Models
- codellama-13b-instruct
Input Prompt Template
There are two options, depending on whether you want to also include a system message as part of the instruction.
With system message:
<s>[INST] <<SYS>>\n {system_message} \n<</SYS>>\n\n {prompt} [/INST]
Without system message:
<s>[INST] {prompt} [/INST]
Make sure there is a space after the final [/INST] tag.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
| Prompt | Completion |
|---|---|
| <s>[INST] Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? [/INST] | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Models
- codellama-70b-instruct
Input Prompt Template
There are two options, depending on whether you want to also include a system message as part of the instruction.
With system message:
<s>Source: system\n\n {system_message} <step> Source: user\n\n {prompt} <step> Source: assistant\nDestination: user\n\n
Without system message:
<s>Source: user\n\n {prompt} <step> Source: assistant\nDestination: user\n\n
Make sure there is a space after the final [/INST] tag.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
| Prompt | Completion |
|---|---|
| <s>Source: user\n\n Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? <step> Source: assistant\nDestination: user\n\n | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Models
- phi-2
Input Prompt Template
There are two options, depending on whether you want to also include a system message as part of the instruction.
With system message:
<|im_start|>system\n {system_message} <|im_end|>\n<|im_start|>user\n {prompt} <|im_end|>\n<|im_start|>assistant\n
Without system message:
<|im_start|>user\n {prompt} <|im_end|>\n<|im_start|>assistant\n
Make sure there is a \n after assistant at the end of the template.
Output Prompt Template
You don't have to modify the completion column.
Example Data
Raw data without the template:
| Prompt | Completion |
|---|---|
| Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? | Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72 |
Transformed data after applying the instruction template without a system message:
Prompt:
<|im_start|>user\n Please answer the following question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? <|im_end|>\n<|im_start|>assistant\n
Completion:
Natalia sold 48/2 = <<48/2=24>>24 clips in May. Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May. Answer: 72