Securing LLM Calls with Alibaba Cloud ASM Service Mesh Using a Wasm Plugin

With the rapid development of large language models (LLMs), enterprises increasingly rely on them as core services, but API‑key leakage and accidental transmission of sensitive data pose serious security risks. Alibaba Cloud ASM (Alibaba Service Mesh) can provide a global protection layer to mitigate these threats.

Prerequisites : an ASM instance (v1.18+), sidecar injection enabled, and a DashScope model service with a valid API‑key.

1. Deploy a client application (Sleep) that issues curl requests to an external LLM. The YAML manifest is:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: sleep
---
apiVersion: v1
kind: Service
metadata:
  name: sleep
  labels:
    app: sleep
    service: sleep
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: sleep
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
  template:
    metadata:
      labels:
        app: sleep
    spec:
      terminationGracePeriodSeconds: 0
      serviceAccountName: sleep
      containers:
      - name: sleep
        image: registry.cn-hangzhou.aliyuncs.com/acs/curl:8.1.2
        command: ["/bin/sleep", "infinity"]
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - mountPath: /etc/sleep/tls
          name: secret-volume
      volumes:
      - name: secret-volume
        secret:
          secretName: sleep-secret
          optional: true
---

2. Register the external LLM service in the mesh by creating a ServiceEntry and a DestinationRule so that the mesh can manage the HTTPS connection.

apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
  name: dashscope
  namespace: default
spec:
  hosts:
  - dashscope.aliyuncs.com
  ports:
  - name: http-port
    number: 80
    protocol: HTTP
    targetPort: 443  # upgrade to HTTPS
  - name: https-port
    number: 443
    protocol: HTTPS
  resolution: DNS
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: dashscope
  namespace: default
spec:
  host: dashscope.aliyuncs.com
  trafficPolicy:
    portLevelSettings:
    - port:
        number: 80
      tls:
        mode: SIMPLE

3. Deploy the LLMProxy Wasm plugin that adds API keys, filters requests based on deny patterns, and optionally calls a private LLM for intelligent guarding.

apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
  name: asm-llm-proxy
  namespace: default
spec:
  imagePullPolicy: Always
  phase: AUTHN
  selector:
    matchLabels:
      app: sleep
  url: registry-cn-hangzhou.ack.aliyuncs.com/test/asm-llm-proxy:v0.2
  pluginConfig:
    api_key: ${dashscope_API_KEY}
    deny_patterns:
    - .*账号.*   # block messages containing "账号"
    hosts:
    - dashscope.aliyuncs.com
    intelligent_guard:
      api_key: ${private_llm_API_KEY}
      host: dashscope.aliyuncs.com
      model: qwen-turbo
      path: /compatible-mode/v1/chat/completions
      port: 80

4. Test scenarios :

Request without an API key succeeds because the plugin injects it.

Request containing the word "账号" is denied by the deny pattern.

Request with potential sensitive content not matching deny patterns is evaluated by the private LLM; if flagged, the request is rejected.

Sample curl commands and expected responses are provided in the article, demonstrating HTTP‑to‑HTTPS upgrade, API‑key injection, and denial messages.

Conclusion : By leveraging ASM’s service mesh and a custom Wasm plugin, enterprises can dynamically rotate API keys, enforce fine‑grained content policies, and protect sensitive data when calling external LLM services, all without redeploying applications.