How RAG‑MCP Cuts Prompt Tokens by Up to 74% While Boosting Accuracy

Test Environment and Architecture

Cloud: AWS us-east-1; Model: Amazon Bedrock Claude 3.7 Sonnet; Knowledge base: Bedrock KB; Storage: S3; OS: macOS 24.5.0.

Parameter Configuration

Model config:
- max_output_tokens: 4096
- temperature: 0.7
MCP tool config:
- command: npx
- args: ["-y", "@modelcontextprotocol/server-filesystem"]
Knowledge base config:
- vector embedding: standard dimension
Test config:
- max tool call rounds: 5
- auto tool call: enabled
- call interval: 5 s

Test Cases

List current directory files

Create new text file and write content

Read file content

Search files containing a specific string

Create new directory

Check file details

Create file and perform multi‑file read (compound query)

View directory tree structure

View allowed directory list

Multi‑Dimensional Evaluation Framework

Token efficiency : input, output and total token counts.

Accuracy : ratio of correctly selected tools.

Response performance : time from request to completion (seconds).

Reliability : success rate of requests.

Efficiency : number of tool‑call rounds required.

Methodology

Two approaches were compared in a controlled experiment:

RAG‑MCP : uses semantic retrieval to select only relevant tools and sends those descriptions to the model.

Full‑tool MCP : sends descriptions of all available tools to the model.

Results

According to the test report dated 2025‑06‑03:

Token usage reduced by 67 % on average, with a maximum reduction of 74.2 % for the file‑read query.

Overall accuracy improved to 93.8 % (full‑tool MCP 87.5 %).

Average response time dropped from 9.95 s to 7.29 s, a 26.7 % improvement.

Response‑time standard deviation: 2.53 s (RAG‑MCP) vs 3.36 s (full‑tool).

Prompt Construction Comparison

# Full‑tool MCP
tools_response = await self._mcp_client.list_tools()
self._tool_config = self._mcp_client.convert_tools_to_bedrock_format(tools_response.tools)
response = self.bedrock_client.converse(messages=messages, model_id=model_id, tool_config=tool_config or self._tool_config)

# RAG‑MCP
conversation_context = self.session.get_conversation_context()
kb_result = await self.knowledge_base.query(conversation_context, top_k=2)
tool_config = kb_result
response = self.bedrock_client.converse(messages=messages, model_id=model_id, tool_config=tool_config)

Knowledge‑Base Retrieval

async def query(self, query_text: str, top_k: int = 1) -> Dict[str, Any]:
    result = self.kb_tools.query_semantic(query_text, max_results=top_k)
    return {"tools": result.tools}

Information‑Flow Comparison

Full‑tool MCP : load all tool descriptions → format → send to LLM → LLM selects from full set → execute selected tool.

RAG‑MCP : vectorize user query → semantic search in knowledge base → format retrieved tools → send reduced set to LLM → LLM selects → execute.

Performance Trade‑offs

RAG‑MCP consistently shows better token efficiency, higher accuracy and lower latency. However, in complex composite queries (e.g., creating a file and performing multiple reads) it may miss some required tools, resulting in lower accuracy (50 % vs 100 % for full‑tool) and a slight increase in response time (+1.06 s).

Multi‑Dimensional Evaluation Pros & Cons

Pros: comprehensive view, balanced trade‑offs, adaptable to different scenarios, provides explainable decision data.

Cons: increased analysis complexity, difficulty in weighting dimensions, possible redundancy, higher measurement cost.

Case Studies

Best case – file search : token reduction 55.9 %, response‑time reduction 48.7 %, accuracy 100 % for both methods.

Challenge – composite operation : RAG‑MCP accuracy 50 % vs 100 % (full‑tool), token reduction 68.7 %, response slower by 1.06 s, highlighting limits of retrieval when multiple tools are needed.

Future Directions

Multi‑modal tool descriptions (text, diagrams, structured data).

Adaptive retrieval strategies that learn from usage patterns.

Cross‑session knowledge transfer while preserving privacy.

Tool‑combination prediction algorithms for proactive loading.

Distributed RAG architectures for higher throughput and lower latency.

Conclusion

RAG‑MCP reduces token consumption by roughly 67 % (up to 74 %), improves tool‑selection accuracy to 93.8 %, cuts average latency by 26.7 %, and scales more gracefully as the tool catalog grows, though further work is needed for complex multi‑tool scenarios.