Integrating Spring AI with Ollama for Tool Calling: A Complete Beginner‑to‑Practice Guide

Tool Calling Workflow

In the Ollama + Spring AI scenario, the tool‑calling process consists of:

Define a tool by annotating a Spring‑managed bean method with @Tool or by creating a ToolCallback with the programmatic API.

Register the tool when building a ChatClient. Spring AI generates an OpenAI‑compatible JSON Schema from the method signatures.

The model evaluates the user request; if a tool is needed it returns the tool name and parameter values.

Spring AI invokes the corresponding Java method and passes the result back to the model.

The model produces the final natural‑language answer.

Model Compatibility Requirements

Only Ollama models that natively support Function Calling can be used; otherwise the model ignores the tool list and returns plain text. List downloaded models with ollama list and download a new model with ollama pull <model‑name>.

Two Ways to Define Tools

Using @Tool Annotation (recommended)

Annotate a Spring‑managed bean method with @Tool. Spring AI automatically wraps it as a ToolCallback.

Programmatic API (no annotation)

Create a tool callback dynamically with MethodToolCallback.builder(). Suitable when the target class cannot be modified.

import org.springframework.ai.tool.MethodToolCallback;
import org.springframework.ai.tool.ToolCallback;

public ToolCallback dynamicTool(Object target) {
    return MethodToolCallback.builder()
        .method("methodName", target)
        .description("工具描述")
        .build();
}

pom.xml Dependency Configuration

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>3.3.3</version>
</parent>
<properties>
    <java.version>17</java.version>
    <spring-ai.version>1.1.2</spring-ai.version>
</properties>
<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <!-- Spring AI Ollama core -->
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-starter-model-ollama</artifactId>
        <version>${spring-ai.version}</version>
    </dependency>
</dependencies>
<repositories>
    <repository>
        <id>spring-milestones</id>
        <url>https://repo.spring.io/milestone</url>
        <snapshots>
            <enabled>false</enabled>
        </snapshots>
    </repository>
</repositories>

application.yml Configuration

server:
  port: 886
spring:
  ai:
    ollama:
      base-url: http://localhost:11434
      chat:
        model: qwen2.5:7b-instruct
        options:
          temperature: 0.7
          num-ctx: 4096  # context window size
logging:
  level:
    org.springframework.ai.chat.client: DEBUG  # view tool‑calling details

Tool Service Class (with @Tool Annotation)

package com.badao.ai.service;

import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.annotation.ToolParam;
import org.springframework.stereotype.Service;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;

@Service
public class ToolService {

    @Tool(description = "获取当前系统的日期和时间，返回格式化后的时间字符串")
    public String getCurrentDateTime() {
        System.out.println("获取当前日期和时间工具被调用");
        return LocalDateTime.now()
                .format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss"));
    }

    @Tool(description = "查询指定城市的天气信息")
    public String getWeather(@ToolParam(description = "城市名称") String city) {
        System.out.println("查询指定城市的天气信息工具被调用");
        return String.format("城市：%s，天气：晴，温度：22°C ~ 28°C，湿度：45%%，风力：3级", city);
    }

    @Tool(description = "计算两个数字的和")
    public double add(@ToolParam(description = "第一个加数") double a,
                     @ToolParam(description = "第二个加数") double b) {
        System.out.println("计算两个数字的和工具被调用");
        return a + b;
    }
}

ChatClient Configuration Class

package com.badao.ai.config;

import com.badao.ai.service.ToolService;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class ChatConfig {

    @Bean
    public ChatClient chatClient(ChatClient.Builder chatClientBuilder,
                                ToolService toolService) {
        return chatClientBuilder
                .defaultTools(toolService) // inject tool class
                .build();
    }
}

Controller

package com.badao.ai.controller;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.*;

@RestController
@RequestMapping("/api")
public class ToolChatController {

    private final ChatClient chatClient;

    public ToolChatController(ChatClient chatClient) {
        this.chatClient = chatClient;
    }

    /** General chat endpoint – AI decides whether to call a tool */
    @PostMapping("/chat")
    public ChatResponse chat(@RequestBody ChatRequest request) {
        String result = chatClient.prompt()
                .user(request.getMessage())
                .call()
                .content();
        return new ChatResponse(200, "success", result);
    }

    /** Streaming response with Server‑Sent Events */
    @GetMapping(value = "/stream", produces = org.springframework.http.MediaType.TEXT_EVENT_STREAM_VALUE)
    public reactor.core.publisher.Flux<String> streamChat(@RequestParam String msg) {
        return chatClient.prompt()
                .user(msg)
                .stream()
                .content();
    }

    public record ChatRequest(String message) { }
    public record ChatResponse(int code, String msg, String data) { }
}

Testing and Verification

Run the application and send a POST request to /api/chat with a message such as “What is the current date and time?”. The model returns the tool name getCurrentDateTime and the response contains the formatted date‑time string. Similar requests exercise the weather and addition tools.

Common Issues and Solutions

Missing ToolCallbackProvider bean – In Spring AI 1.1.2 the @Tool annotation does not create a ToolCallbackProvider. Inject the service class directly via .defaultTools(toolService).

Unable to resolve spring-ai-starter-model-ollama dependency – Use the exact artifact ID spring-ai-starter-model-ollama as shown in the pom.xml example.

Model never calls tools – The selected model does not support Function Calling. Switch to a model that does, e.g., qwen2.5:7b, llama3.1:8b, or mistral:7b.

Key Knowledge Points

@Tool

marks a method as a tool; the description guides the LLM when to invoke it. @ToolParam describes each parameter to help the LLM fill arguments accurately. ChatClient.defaultTools() registers a class containing @Tool methods.

Only models with native Function Calling support can be used.

Spring AI automatically generates an OpenAI‑compatible JSON Schema from method signatures and annotations.

Enable org.springframework.ai.chat.client DEBUG logging to observe the tool‑calling flow.