Tool Use & Function Calling: Giving AI Hands to Act

Moving from "Chatbots" that just talk, to "Agents" that can actually do things.

A standard LLM is like a "Brain in a Jar." It is incredibly smart, but it is cut off from the world. It cannot check the weather, query your database, or send an email. It only knows what it was trained on (which is static and old).

Function Calling (or Tool Use) is the bridge. It allows you to describe functions to the model, and have the model intelligently choose to output a structured request to run that code.

1. The Misconception: The LLM Does NOT Run Code

This is the most common confusion for beginners.

Myth: The LLM executes the Python function inside its neural network.
Reality: The LLM just outputs text (JSON). You (the developer) must write the code to parse that JSON, execute the actual function, and feed the result back to the LLM.

2. The "Tool Loop" Architecture

To build an agent, you set up a loop. Here is the standard flow:

User: Asks "What's the weather in Tokyo?"
System: Sends the prompt + a list of available tools (e.g., get_weather) to the LLM.
LLM: Sees the tool matches the intent. Instead of replying with text, it replies with a Tool Call: { "name": "get_weather", "args": { "city": "Tokyo" } }.
Your Code: Detects the tool call. Runs the actual API request to a weather service. Gets result: "25°C, Sunny".
Your Code: Sends the weather result back to the LLM as a new message.
LLM: Reads the result and generates the final answer: "It is currently 25°C and sunny in Tokyo."

3. Defining Tools (JSON Schema)

How does the LLM know how to use your tool? You must define a schema. The description field is critical—it acts as a mini-prompt telling the model when and how to use the tool.

Defining a Weather Tool (Python)

This schema tells the LLM that a 'get_weather' function exists.

tools = [
  {
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get current temperature for a specific city.",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {
            "type": "string",
            "description": "The city and state, e.g. San Francisco, CA"
          },
          "unit": {
            "type": "string", 
            "enum": ["celsius", "fahrenheit"]
          }
        },
        "required": ["location"]
      }
    }
  }
]

4. Code Example: The Execution Loop

Here is a conceptual implementation using a pseudo-client showing how you handle the model's response.

Handling the Tool Call

Check if the model wants to call a function, run it, and return the result.

import json

# 1. The Mock Function (The "Real World" action)
def get_weather(location):
    # In reality, you would call an API like OpenWeatherMap here
    if "Tokyo" in location:
        return json.dumps({"temp": "25", "unit": "celsius"})
    return json.dumps({"temp": "unknown"})

# 2. Main Loop
def run_conversation(user_input):
    messages = [{"role": "user", "content": user_input}]
    
    # First Call: Ask the LLM
    response = client.chat.completions.create(
        model="gpt-4",
        messages=messages,
        tools=tools
    )
    
    msg = response.choices[0].message

    # 3. Check if LLM wants to use a tool
    if msg.tool_calls:
        print(f"LLM wants to call: {msg.tool_calls[0].function.name}")
        
        # Extract arguments
        call_id = msg.tool_calls[0].id
        func_name = msg.tool_calls[0].function.name
        args = json.loads(msg.tool_calls[0].function.arguments)
        
        # EXECUTE the function
        if func_name == "get_weather":
            result = get_weather(args["location"])
            
            # 4. Feed result back to LLM
            messages.append(msg) # Add the assistant's "request" to history
            messages.append({
                "role": "tool",
                "tool_call_id": call_id,
                "content": result
            })
            
            # Second Call: Get final answer
            final_response = client.chat.completions.create(
                model="gpt-4",
                messages=messages
            )
            return final_response.choices[0].message.content

print(run_conversation("What's the weather in Tokyo?"))
# Output: "The current temperature in Tokyo is 25°C."

5. Best Practices & Security

Security Warning

Never give an LLM a tool like execute_shell_command or delete_database without a human-in-the-loop confirmation. Prompt Injection attacks can trick the model into misusing tools.

Use Pydantic

Instead of writing raw JSON schemas, use libraries like Pydantic (Python) or Zod (TypeScript). They allow you to define data structures as code and automatically generate the JSON schema for the LLM.

Keep Tools Simple

LLMs struggle if you give them 50 complex tools. It is better to have specialized agents with 3-5 tools each than one "God Agent" with access to everything.

Summary

Function Calling transforms LLMs from passive text generators into active engines that can power software.

Your next step? Try building a simple "Personal Assistant" CLI that has two tools: one to get the current time, and one to save a note to a text file.