Accelerate LLM App Development with Eino: A Go Framework Walkthrough

1 What is Eino?

Eino is an open‑source Golang large‑model application development framework from ByteDance, used internally for many services such as Doubao, Douyin, Kousi, etc. It draws inspiration from frameworks like LangChain and LlamaIndex while following Go conventions.

Eino provides:

Component abstractions for easy reuse and composition.

Powerful orchestration handling type checking, stream processing, concurrency, aspect injection, and option assignment.

Clean, well‑designed APIs.

Best‑practice collections via flows and examples.

DevOps tools covering the full development lifecycle from visual debugging to online tracing and evaluation.

These capabilities enable standardized, simplified, and efficient AI‑application development across the lifecycle.

Project address: https://github.com/cloudwego/eino, https://github.com/cloudwego/eino-ext

2 Calling Large Models for Q&A

2.1 Create Model Reference Object

Currently supports Ollama mode and OpenAI mode.

1. Use Ollama mode

<code>package chatmodel

import (
    "context"
    "log"

    "github.com/cloudwego/eino-ext/components/model/ollama"
    "github.com/cloudwego/eino/components/model"
)

func createOllamaChatModel(ctx context.Context) model.ChatModel {
    chatModel, err := ollama.NewChatModel(ctx, &ollama.ChatModelConfig{
        BaseURL: "http://localhost:11434", // Ollama service address
        Model:   "llama2",                  // model name
    })
    if err != nil {
        log.Fatalf("create ollama chat model failed: %v", err)
    }
    return chatModel
}
</code>

2. Use OpenAI mode

This mode uses the qwen‑max model; Alibaba Baichuan provides a free 1M‑token quota for trial.

<code>package chatmodel

import (
    "context"
    "fmt"
    "log"

    "github.com/cloudwego/eino-ext/components/model/openai"
    "github.com/cloudwego/eino/components/model"
    "project/conf"
)

// createOpenAIChatModel creates an OpenAI chat model instance.
func CreateOpenAIChatModel(ctx context.Context) model.ChatModel {
    var key, modelName, baseUrl string
    if conf.GlobalConfig != nil {
        key = conf.GlobalConfig.DeepSeekR1.Key
        modelName = conf.GlobalConfig.DeepSeekR1.ModelName
        baseUrl = conf.GlobalConfig.DeepSeekR1.BaseUrl
    } else {
        // default qwen‑max model
        // URL will be auto‑appended with /chat/completions
        key = "sk-********"
        modelName = "qwen-max"
        baseUrl = "https://dashscope.aliyuncs.com/compatible-mode/v1"
    }

    chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
        BaseURL: baseUrl,
        Model:   modelName,
        APIKey:  key,
    })
    if err != nil {
        log.Fatalf("create openai chat model failed, err=%v", err)
    }
    return chatModel
}
</code>

3. Message Conversation Template

<code>package reporter

import (
    "context"
    "log"

    "github.com/cloudwego/eino/components/prompt"
    "github.com/cloudwego/eino/schema"
)

func createTemplate() prompt.ChatTemplate {
    // Create template using FString format
    return prompt.FromMessages(schema.FString,
        // System message template
        schema.SystemMessage("You are a {role}. Respond in a {style} tone. Your goal is to help programmers stay positive while giving technical advice."),
        // Insert chat history placeholder (empty for new conversations)
        schema.MessagesPlaceholder("chat_history", true),
        // User message template
        schema.UserMessage("Question: {question}"),
    )
}

func CreateMessagesFromTemplate() []*schema.Message {
    template := createTemplate()
    messages, err := template.Format(context.Background(), map[string]any{
        "role":     "Programmer Coach",
        "style":    "positive, warm and professional",
        "question": "My code keeps failing, I feel frustrated, what should I do?",
        "chat_history": []*schema.Message{
            schema.UserMessage("Hello"),
            schema.AssistantMessage("Hey! I'm your programmer coach! Remember, great programmers grow from debugging. How can I help?"),
            schema.UserMessage("I think my code is terrible"),
            schema.AssistantMessage("Everyone goes through this! Keep learning and improving. Let's look at the code together..."),
        },
    })
    if err != nil {
        log.Fatalf("format template failed: %v\n", err)
    }
    return messages
}
</code>

4. Output Modes: Normal or Streaming

<code>package reporter

import (
    "context"
    "log"

    "github.com/cloudwego/eino/components/model"
    "github.com/cloudwego/eino/schema"
)

// Normal single‑shot output
func Generate(ctx context.Context, llm model.ChatModel, in []*schema.Message) *schema.Message {
    result, err := llm.Generate(ctx, in)
    if err != nil {
        log.Fatalf("llm generate failed: %v", err)
    }
    return result
}

// Streaming output
func Stream(ctx context.Context, llm model.ChatModel, in []*schema.Message) *schema.StreamReader[*schema.Message] {
    result, err := llm.Stream(ctx, in)
    if err != nil {
        log.Fatalf("llm generate failed: %v", err)
    }
    return result
}
</code>

5. Invoke in main

<code>exeDir := ""
cfg, err := conf.InitConfig(filepath.Join(exeDir, "conf/app.toml"))
if err != nil {
    log.Fatalf("Failed to load config: %v", err)
} else {
    fmt.Println("Config loaded, app name:", cfg.Base.AppName)
}

// Create messages from template
log.Printf("===create messages===\n")
messages := reporter.CreateMessagesFromTemplate()
log.Printf("messages: %+v\n\n", messages)

// Create LLM
log.Printf("===create llm===\n")
cm := chatmodel.CreateOpenAIChatModel(ctx)
// cm := createOllamaChatModel(ctx)
log.Printf("create llm success\n\n")

// Normal generation
log.Printf("===llm generate===\n")
result := reporter.Generate(ctx, cm, messages)
log.Printf("result: %+v\n\n", result)

// Streaming generation
log.Printf("===llm stream generate===\n")
streamResult := reporter.Stream(ctx, cm, messages)
reporter.ReportStream(streamResult)
</code>

3 Demonstration

4 Summary

Eino is an open‑source LLM application development framework that simplifies building large‑model applications. Its modular design lets developers pick and combine components, while powerful orchestration keeps complex logic clear and maintainable.