Using CloudWeGo Eino With a Custom OpenAI-Compatible API in Go

说明与用途

Eino 可以通过显式设置 BaseURL、APIKey 和 Model 与任何 OpenAI 兼容网关一起使用。它发送最小化请求，尊重超时，并自动在流式传输中包含使用情况。添加重试传输可恢复 429 的重试行为。

问题

Integrating CloudWeGo Eino with a non‑Azure OpenAI‑compatible gateway (APIClaw) while controlling BaseURL, retries, and agent limits.

结果

Eino can be used with any OpenAI‑compatible gateway by explicitly setting BaseURL, APIKey, and Model. It sends minimal requests, respects timeouts, and streams usage automatically. Adding a retry transport restores retry behavior for 429 responses.

工作流

• Install Eino and eino-ext via go get
• Upgrade sonic to v1.15.4 to fix build error
• Create ChatModel with BaseURL, APIKey, Model, Timeout
• Generate a simple chat request and inspect the wire payload
• Verify path, body, headers, and timeout handling
• Use WithTemperature and WithModel to override per‑call options
• Stream a request to observe automatic usage inclusion
• Add a custom http.RoundTripper to retry on 429
• Handle errors with errors.As against openai.APIError

证据

Eino does not read OPENAI_API_KEY or OPENAI_BASE_URL from the environment, so pass both explicitly.

来源

The struct comments say BaseURL is for Azure, but it is used for every non‑Azure client too.

来源

That second call sent "temperature":0.7 and "model":"OTHER_MODEL_ID" while keeping the configured max_tokens.

来源

cm.Stream(ctx, msgs) sent "stream":true and "stream_options":{"include_usage":true}" without being asked.

来源

There are no retries. Every error status got exactly one request and came back in a millisecond or two.

来源