Function Calling
Predibase now supports function calling for fine-tuning!
Supported Models for Function Calling
We currently only support a limited number of models for fine-tuning with tooling:
- llama-3-2-1b-instruct
- llama-3-2-3b-instruct
- llama-3-3-70b-instruct
- qwen2-5-1-5b-instruct
- qwen2-5-7b-instruct
- qwen2-5-14b-instruct
- qwen2-5-32b-instruct
Formatting Your Dataset For Fine-Tuning With Function Calling (Recommended for Advanced Users Only)
When it comes to fine-tuning with function calling, you MUST enable apply_chat_template in your fine-tuning config either via the SDK or through the check box in the adapter version UI before hitting train.
Example Function
Your tools should follow the schema defined by Hugging Face. Here is an example:
{
"messages":
[
{
"role": "system",
"content": "Predibot is a cool chatbot"
},
{
"role": "user",
"content": "What's 2 times 5"
},
{
"role": "assistant",
"content": "Paris"
},
]
}
The Dataset
You should use a chat-style dataset, and you can simply add a "tools" key at the same level as "messages":
{
"messages":
[
{
"role": "system",
"content": "Predibot is a cool chatbot"
},
{
"role": "user",
"content": "What's 2 times 5"
},
{
"role": "assistant",
"tool_calls": [
{
"type": "function",
"function": {
"name": "multiply",
"arguments": {
"a": 2,
"b": 5
}
}
}
],
"content": ""
},
],
"tools":
[
{
"type": "function",
"function": {
"name": "multiply",
"description": "A function that multiplies two numbers",
"parameters": {
"type": "object",
"properties": {
"a": {
"type": "number",
"description": "The first number to multiply"
},
"b": {
"type": "number",
"description": "The second number to multiply"
}
},
"required": ["a", "b"]
}
}
}
]
}
apply_chat_template will automatically format the prompt to handle the function calling.
How Function Calling Works
Function calling requires a degree of interaction with the user. The usual flow looks something like this:
System: {ACKNOWLEDGMENT_THAT_TOOLS_MAY_BE_USED}
User: {PROMPT_WITH_TOOLS_PROVIDED}
Assistant: {APPROPRIATE_JSON_FORMATTED_TOOL_CALL}
*User runs tool call*
Tool: {JSON_FORMATTED_OUTPUT_OF_TOOL_CALL}
Assistant: {FORMATTED_RESPONSE}
As such an entry in a final dataset may look something like the following:
{
"messages":
[
{
"role": "system",
"content": "Predibot is a cool chatbot"
},
{
"role": "user",
"content": "I need help multiplying 2 numbers"
},
{
"role": "assistant",
"content": "I can help with that. What are the numbers?"
},
{
"role": "user",
"content": "What's 2 times 5"
},
{
"role": "assistant",
"tool_calls": [
{
"type": "function",
"function": {
"name": "multiply",
"arguments": {
"a": 2,
"b": 5
}
}
}
],
"content": ""
},
{
"role": "tool",
"name": "multiply",
"content": 10
},
{
"role": "assistant",
"content": "2 times 5 equals 10!"
}
],
"tools":
[
{
"type": "function",
"function": {
"name": "multiply",
"description": "A function that multiplies two numbers",
"parameters": {
"type": "object",
"properties": {
"a": {
"type": "number",
"description": "The first number to multiply"
},
"b": {
"type": "number",
"description": "The second number to multiply"
}
},
"required": ["a", "b"]
}
}
}
]
}
Converting from ShareGPT Format
If you have your data in ShareGPT format:
{
"conversations":
[
{
"from": "user",
"value": "I need help multiplying 2 numbers"
},
{
"from": "assistant",
"value": "I can help with that. What are the numbers?"
},
{
"from": "user",
"value": "What's 2 times 5"
},
{
"from": "function_call",
"value": "{\"arguments\": {\"a\": \"2\", \"b\": \"5\"}, \"name\": \"multiply\"}"
},
{
"from": "observation",
"name": "{\"result\":\"success\",\"helper\":\"10\"}",
"value": 10
},
{
"from": "assistant",
"value": "2 times 5 equals 10!"
}
],
"system": "Predibot is a cool chatbot",
"tools": "[{\"name\": \"multiply\", \"description\": \"Use this function to multiply two numbers together\", \"parameters\": {\"type\": \"object\", \"properties\": {\"a\": {\"type\": \"number\", \"description\": \"The first number to multiply\"}, \"b\": {\"type\": \"string\", \"description\": \"The second number to multiply\"}}, \"required\": [\"a\", \"b\"]}}]"
}
the script below is an easy way to convert to a Predibase-ready dataset:
import json
def convert_format(path_to_sharegpt_json):
# Load the JSON into a Python dictionary
with open(path_to_sharegpt_json, "r") as f:
data = json.load(f)
# Initialize the input structure
input_data = {
"messages": [],
"tools": json.loads(data["tools"])
}
# Set the system message (first element in messages list)
input_data["messages"].append({
"role": "system",
"content": data["system"]
})
# Process the conversations and build the corresponding input
for conversation in data["conversations"]:
if conversation["from"] == "user":
input_data["messages"].append({
"role": "user",
"content": conversation["value"]
})
elif conversation["from"] == "assistant":
input_data["messages"].append({
"role": "assistant",
"content": conversation["value"]
})
elif conversation["from"] == "function_call":
function_call = json.loads(conversation["value"])
input_data["messages"].append({
"role": "assistant",
"tool_calls": [{
"type": "function",
"function": {
"name": function_call["name"],
"arguments": function_call["arguments"]
}
}],
"content": ""
})
elif conversation["from"] == "observation":
# Extract tool results and add to messages
observation = json.loads(conversation["name"])
input_data["messages"].append({
"role": "tool",
"name": "multiply", # Assuming "multiply" is the only tool for now
"content": observation["helper"]
})
return json.dumps(input_data, indent=4)