Master Spring AI Alibaba: Token Basics, RAG, and Multi‑Agent Implementation

1. Foundations: Token and Prompt Mechanics

Before using any AI framework, the article explains that large models operate on tokens , the smallest unit of text. It shows how a sentence like "ChatGPT is great" becomes five tokens and why token limits affect the context window, billing, and latency. The system, user, assistant, and tool roles are introduced with a table, and the fact that models have no internal memory is highlighted.

2. Spring AI Alibaba Architecture

The layered design mirrors Spring Data: the upper layer ( ChatClient) provides a fluent API, while the lower layer ( ChatModel) directly maps to a single model request. A side‑by‑side code comparison demonstrates the boilerplate of ChatModel versus the concise chainable calls of ChatClient. The Advisor mechanism is described as an AOP‑style interceptor that can inject system prompts, memory handling, or other cross‑cutting concerns.

3. Prompt Engineering

Hard‑coded prompt strings are discouraged. Instead, PromptTemplate (backed by StringTemplate) uses {} placeholders for variable substitution. The article recommends storing templates in .st files and shows a Java‑code example that loads a template from the classpath and renders it with Map.of(...). It also covers custom delimiters to avoid conflicts with JSON syntax.

4. Embedding and Structured Output

Embedding is presented as the foundation of Retrieval‑Augmented Generation (RAG). The EmbeddingModel abstraction hides provider differences; text-embedding-v3 from DashScope is used as an example. Structured output is achieved by appending a JSON schema (generated by BeanOutputConverter) to the prompt, then converting the model’s JSON response back to a Java record. The article notes that domestic models may wrap JSON in Markdown code fences, so a safeConvert method strips those wrappers before deserialization.

5. Chat Memory

Because models are stateless, Spring AI re‑sends the full message history on each request, causing token growth. MessageChatMemoryAdvisor automatically pulls recent messages from a ChatMemory implementation. Three memory back‑ends are compared: InMemoryChatMemory (development only), JdbcChatMemory (single‑node persistence), and RedisChatMemory (distributed production). A Redis‑backed configuration example shows how to keep the latest 20 messages.

6. RAG Pipeline

The article details the offline indexing stage (PDF parsing, metadata injection, token‑based chunking with overlap, vector store ingestion) and the online retrieval stage using QuestionAnswerAdvisor. It provides concrete parameters: topK=5, similarityThreshold=0.72, and a metadata filter expression. Guidance for tuning chunkSize and similarity thresholds is given, along with a systematic testing workflow.

7. Function Calling

Function calling lets the model trigger real business logic. The article defines a tool with @Bean returning Function<OrderQueryRequest, OrderQueryResponse>, shows request/response records annotated for JSON schema generation, and demonstrates a controller that declares the tool via .functions("queryOrder"). Security considerations include parameter validation, permission checks, and input sanitization to prevent prompt injection.

8. Graph‑Based Multi‑Agent Workflows

When a single ChatClient is insufficient, Spring AI’s Graph framework orchestrates multiple agents. A complete example builds a state graph for content creation: analysis → outline → write → review, with conditional edges that loop back to writing if the review fails (up to three retries). The article stresses that state objects must be immutable (return a new instance from each node).

9. Production Practices

Four cost‑control tactics are listed: choosing the right model tier, caching embedding results, applying memory truncation, and using batch APIs for offline jobs. Observability is enabled via Spring AI Observation (token usage, latency, success rate) and integrated with Micrometer/Prometheus. Resilience patterns include model fallback on rate‑limit errors and circuit‑breaker logic.

10. Common Pitfalls

Using a chat model for embedding leads to poor retrieval.

Switching embedding models without rebuilding the index corrupts vector similarity.

Streaming responses are buffered by Nginx unless proxy_buffering off is set.

Duplicate @Bean function names cause tool conflicts.

State objects in a StateGraph must be immutable; mutating fields has no effect.

In‑memory vector stores OOM on large corpora; use Milvus, PGVector, or Alibaba Cloud vector services. BeanOutputConverter is not thread‑safe; instantiate per request.

11. Conclusion

The article ends with a visual knowledge‑map and a reminder to master each layer before tackling agents, emphasizing that a solid foundation prevents wasted tokens and unstable behavior.