Building a Simple Rule Engine with Go's AST and Parser

Background

The author, a backend engineer at Tencent IEG, observes that most people associate rule engines with DSLs and complex compiler theory, but many practical scenarios only need a small set of simple rules.

Examples of simple risk‑control rules for a payment system are listed, such as "total payment amount over 100k in 24 hours" or "single‑hour credit‑card payment over 50k". Similar rules appear in loan systems, where credit score and amount thresholds determine approval.

When the number of rules grows, a plain rule‑class library becomes hard to maintain, prompting the need for a more abstract rule engine.

JSON Implementation

Instead of building a DSL, the author first implements rules using JSON and Go's reflection. An interface IRule with a Match(*http.Request) bool method is defined, along with concrete matchers such as HeaderMatch and CookieMatch .

type IRule interface {
    Match(req *http.Request) bool
}

func HeaderMatch(key, value string) bool { ... }
func CookieMatch(name, value string) bool { ... }

Rules are expressed as JSON objects, for example:

{
    "op": "AND",
    "matcher": ["header", "cookie"],
    "header": {"key": "X-APP-ID", "value": "Ves"},
    "cookie": {"name": "feature", "value": "dev/wynnliu/qualify-rule"}
}

A generic parser iterates over the "matcher" array, unmarshals each sub‑object, and dispatches to the appropriate match function via a switch . This works for simple equality checks but becomes cumbersome as more operators (range, regex, priority) are added.

Token and AST Basics

To handle more expressive boolean logic, the article introduces Go's built‑in go/scanner and go/token packages for lexical analysis, producing a sequence of tokens from source code.

package main
import (
    "fmt"
    "go/scanner"
    "go/token"
)
func main() {
    src := []byte(`println("Hello World!")`)
    fset := token.NewFileSet()
    file := fset.AddFile("hello.go", fset.Base(), len(src))
    var s scanner.Scanner
    s.Init(file, src, nil, scanner.ScanComments)
    for {
        pos, tok, lit := s.Scan()
        if tok == token.EOF { break }
        fmt.Printf("%s\t%s\t%q\n", fset.Position(pos), tok, lit)
    }
}

The resulting tokens identify identifiers, literals, and punctuation. The article then shows how to parse source code into an abstract syntax tree (AST) using go/parser and go/ast , printing the tree structure.

package main
import (
    "go/ast"
    "go/parser"
    "go/token"
)
func main() {
    src := `package main
func main() { println("Hello, World!") }`
    fset := token.NewFileSet()
    f, err := parser.ParseFile(fset, "", src, 0)
    if err != nil { panic(err) }
    ast.Print(fset, f)
}

The AST consists of nodes such as BinaryExpr , SelectorExpr , and BasicLit , each implementing the Node interface with Pos and End methods. Important node categories are expressions, statements, and declarations.

Evaluating Rules with AST

Using the AST, the author demonstrates how to evaluate a rule like header.key=="X-APP-ID" && header.value=="Ves" . The AST shows a top‑level BinaryExpr with left and right sub‑expressions, each a selector on the header identifier.

// Parse the expression
func Parse(expr string, header http.Header) (bool, error) {
    exprAst, err := parser.ParseExpr(expr)
    if err != nil { return false, err }
    return judge(exprAst, header), nil
}

// Evaluate a binary expression against the request header
func judge(node ast.Node, header http.Header) bool {
    be := node.(*ast.BinaryExpr)
    // left side: header.key
    key := strings.Trim(be.X.(*ast.SelectorExpr).Sel.Name, "\"")
    // right side: "X-APP-ID"
    expectedKey := strings.Trim(be.X.(*ast.BasicLit).Value, "\"")
    // similarly extract value and compare
    return header.Get(key) == expectedKey
}

The function extracts the identifier and literal values from the AST and performs a simple equality check against the actual HTTP header.

Going Further with Custom Primitives

For more complex scenarios, the article suggests defining higher‑level primitives such as req_header_pair_is("X-APP-ID", "Ves") or req_cookie_contain("feature", "dev/wynnliu/qualify-rule") . Implementing these requires extending the parser, for which the author recommends using GoYACC (the Go version of YACC) to write a custom grammar.

GoYACC can generate a parser from BNF/EBNF specifications, similar to how TiDB implements its SQL parser. This approach allows the rule engine to support richer expressions, ranges, regexes, and rule priorities while keeping the implementation manageable.

Conclusion

The article shows a step‑by‑step path from simple JSON‑based rule definitions to a fully parsed AST‑driven rule engine in Go, and finally to a custom DSL powered by GoYACC for advanced rule logic.