Using CloudWeGo Eino With a Custom OpenAI-Compatible API in Go

Описание и применение

Eino можно использовать с любым OpenAI‑совместимым шлюзом, явно задавая BaseURL, APIKey и Model. Он отправляет минимальные запросы, учитывает таймауты и автоматически включает статистику при потоковой передаче. Добавление пользовательского RoundTripper восстанавливает поведение повторных запросов при 429.

Задача

Integrating CloudWeGo Eino with a non‑Azure OpenAI‑compatible gateway (APIClaw) while controlling BaseURL, retries, and agent limits.

Результат

Eino can be used with any OpenAI‑compatible gateway by explicitly setting BaseURL, APIKey, and Model. It sends minimal requests, respects timeouts, and streams usage automatically. Adding a retry transport restores retry behavior for 429 responses.

Рабочий процесс

• Install Eino and eino-ext via go get
• Upgrade sonic to v1.15.4 to fix build error
• Create ChatModel with BaseURL, APIKey, Model, Timeout
• Generate a simple chat request and inspect the wire payload
• Verify path, body, headers, and timeout handling
• Use WithTemperature and WithModel to override per‑call options
• Stream a request to observe automatic usage inclusion
• Add a custom http.RoundTripper to retry on 429
• Handle errors with errors.As against openai.APIError

Доказательства

Eino does not read OPENAI_API_KEY or OPENAI_BASE_URL from the environment, so pass both explicitly.

Источник

The struct comments say BaseURL is for Azure, but it is used for every non‑Azure client too.

Источник

That second call sent "temperature":0.7 and "model":"OTHER_MODEL_ID" while keeping the configured max_tokens.

Источник

cm.Stream(ctx, msgs) sent "stream":true and "stream_options":{"include_usage":true}" without being asked.

Источник

There are no retries. Every error status got exactly one request and came back in a millisecond or two.

Источник