Tool calling

You describe your functions in the request. When the model wants one, it answers with a tool call instead of text; you run the function and send the result back, and the model carries on.

A tool loop#

1. Send the conversation with tools: each one a function with a name, a description and JSON Schema parameters.
2. If the answer's finish_reason is tool_calls, run each call in message.tool_calls. Its arguments is a JSON string.
3. Add the assistant message to the history as it came, then one tool message per call, with its tool_call_id and your result as content.
4. Send the request again with the same tools. Repeat until finish_reason is stop.

import json, osfrom openai import OpenAIclient = OpenAI(    base_url="https://api.dipoleml.com/v1",    api_key=os.environ["RAVEN_API_KEY"],)def get_weather(city):    return {"city": city, "sky": "clear", "celsius": 4}tools = [{    "type": "function",    "function": {        "name": "get_weather",        "description": "Current weather for a city",        "parameters": {            "type": "object",            "properties": {"city": {"type": "string"}},            "required": ["city"],        },    },}]messages = [{"role": "user", "content": "Do I need a coat in Toronto today?"}]while True:    resp = client.chat.completions.create(model="Raven Flash", messages=messages, tools=tools)    msg = resp.choices[0].message    if resp.choices[0].finish_reason != "tool_calls":        print(msg.content)        break    messages.append(msg)    for call in msg.tool_calls:        result = get_weather(**json.loads(call.function.arguments))        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})

curl https://api.dipoleml.com/v1/chat/completions \  -H "Authorization: Bearer $RAVEN_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "Raven Flash",    "messages": [{"role": "user", "content": "Do I need a coat in Toronto today?"}],    "tools": [{      "type": "function",      "function": {        "name": "get_weather",        "description": "Current weather for a city",        "parameters": {          "type": "object",          "properties": {"city": {"type": "string"}},          "required": ["city"]        }      }    }]  }'

{  "choices": [{    "index": 0,    "message": {      "role": "assistant",      "content": null,      "tool_calls": [{        "id": "<call id>",        "type": "function",        "function": {"name": "get_weather", "arguments": "{\"city\": \"Toronto\"}"}      }]    },    "finish_reason": "tool_calls"  }]}

Options#

• tool_choice: "auto" (the model decides), "none" (no tool calls), "required" (at least one call), or {"type": "function", "function": {"name": "get_weather"}} for one named tool.
• parallel_tool_calls: false asks for at most one call per answer. Otherwise an answer may hold several calls; answer each one.
• Send each tool with type function; other tool types are not used as your tools. A request with no function tool left is sent without tool_choice and parallel_tool_calls.
• Tool definitions count as input tokens on every call that sends them.

Streaming tool calls#

In a stream, a call arrives as delta.tool_calls pieces that share an index. The first piece carries the id and the function name; later pieces add to function.arguments. Join the pieces by index, and parse the arguments once the chunk with finish_reason: "tool_calls" has arrived.

In the Messages format#

In the Anthropic SDK format, a tool has a name, a description and an input_schema. The answer holds tool_use blocks with the arguments already parsed in input, and stop_reason is tool_use. Send each result back in a tool_result block of the next user turn.

import osfrom anthropic import Anthropicclient = Anthropic(base_url="https://api.dipoleml.com", api_key=os.environ["RAVEN_API_KEY"])tools = [{    "name": "get_weather",    "description": "Current weather for a city",    "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},}]messages = [{"role": "user", "content": "Do I need a coat in Toronto today?"}]reply = client.messages.create(model="Raven Flash", max_tokens=1024, tools=tools, messages=messages)if reply.stop_reason == "tool_use":    call = next(b for b in reply.content if b.type == "tool_use")    messages += [        {"role": "assistant", "content": reply.content},        {"role": "user", "content": [{"type": "tool_result", "tool_use_id": call.id, "content": "clear, 4 degrees"}]},    ]    reply = client.messages.create(model="Raven Flash", max_tokens=1024, tools=tools, messages=messages)print(reply.content[-1].text)

• tool_choice is {"type": "auto"}, {"type": "any"} (at least one call), {"type": "none"} or {"type": "tool", "name": "get_weather"}; add "disable_parallel_tool_use": true for at most one call.
• Mark a failed tool with "is_error": true in its tool_result.
• In a stream, a tool call is a tool_use block whose arguments arrive as input_json_delta pieces.

Related#

• Chat Completions
• Streaming
• Use with coding tools