Eino: An Open‑Source Golang Framework for Large‑Model Application Development

After more than half a year of internal use and iteration at ByteDance, the Golang‑based large‑model application development framework Eino has been officially open‑sourced by CloudWeGo.

Eino defines clear “components” and offers powerful workflow “orchestration” that covers the entire development process, enabling developers to rapidly create deep, production‑grade large‑model applications.

Key characteristics include:

Stable kernel with simple, well‑documented APIs and a smooth learning curve.

Extreme extensibility for long‑term sustainable development.

Strongly‑typed Go code that is easy to understand, maintain, and highly reliable.

Practices derived from ByteDance’s core business lines.

Out‑of‑the‑box supporting tools.

Eino is already the preferred all‑code framework for large‑model applications within ByteDance, powering services such as Doubao, Douyin, and Kousi.

Project repositories: https://github.com/cloudwego/eino https://github.com/cloudwego/eino-ext

Example of using a ChatModel component:

model, _ := ark.NewChatModel(ctx, config) // 创建一个豆包大模型
message, _ := model.Generate(ctx, []*Message{
    SystemMessage("you are a helpful assistant."),
    UserMessage("what does the future AI App look like?")
})

Eino’s orchestration solves common challenges such as providing context to the model, handling streaming outputs, and managing concurrent graph execution.

For instance, a ReAct Agent can be built by binding a ChatModel with Tools, allowing the model to decide autonomously whether to invoke a tool or produce a final answer. The following graph‑based code demonstrates a concise ReAct Agent implementation:

// Build a ReAct Agent compiled into a Runnable that takes []*Message and returns *Message
// Create a Graph with local state to store message context
graph = NewGraph[[]*Message, *Message](
   WithGenLocalState(func(ctx context.Context) *state {
      return &state{Messages: make([]*Message, 0, config.MaxStep+1)}
   }))

modelPreHandle = func(ctx context.Context, input []*Message, state *state) ([]*Message, error) {
    state.Messages = append(state.Messages, input...)
    return state.Messages, nil
}

_ = graph.AddChatModelNode(nodeKeyModel, chatModel, WithStatePreHandler(modelPreHandle))
_ = graph.AddEdge(START, nodeKeyModel)
_ = graph.AddToolsNode(nodeKeyTools, toolsNode)

modelPostBranch = NewStreamGraphBranch(
   func(_ context.Context, sr *schema.StreamReader[*Message]) (endNode string, err error) {
      defer sr.Close()
      if msg, err := sr.Recv(); err != nil {
         return "", err
      } else if len(msg.ToolCalls) == 0 {
         return END, nil
      }
      return nodeKeyTools, nil
   }, map[string]bool{nodeKeyTools: true, END: true})

_ = graph.AddBranch(nodeKeyModel, modelPostBranch)
_ = graph.AddEdge(nodeKeyTools, nodeKeyModel)

agent, _ := graph.Compile(ctx, WithMaxRunSteps(config.MaxStep))

During compilation, Eino automatically performs type checking, stream encapsulation, concurrent state management, aspect injection, and option distribution, ensuring high reliability and maintainability.

Eino’s advantages include a stable core, agile extensibility, and high maintainability thanks to Go’s strong typing, modular design, and clear layering.

The framework also provides a rich tooling ecosystem—tracing callbacks integrated with Langfuse, IDE plugins for visual graph debugging, and UI‑driven graph construction.

Comprehensive documentation and quick‑start guides are available to help developers get started quickly.