Connecting AnythingLLM to an OpenAI-Compatible Multi-Model Gateway

Описание и применение

Настройка AnythingLLM для работы через мульти‑модельный шлюз CometAPI позволяет централизованно управлять моделями, биллингом и интерфейсом без изменения рабочих процессов.

Задача

Integrating a multi-model gateway into AnythingLLM without altering existing workflows or code.

Результат

By configuring AnythingLLM’s Generic OpenAI provider to point at CometAPI, users can route all chat, embedding, and agent calls through a single API endpoint, simplifying model management, billing, and experimentation without modifying existing workflows.

Рабочий процесс

• 1. Verify gateway accessibility with curl or Python to ensure API key, network, and model name are correct.
• 2. In AnythingLLM, navigate to Settings → AI Providers → LLM Preferences.
• 3. Add a new provider: select "Generic OpenAI" and set Base URL to "https://api.cometapi.com/v1", API key to your secret, and model name to one published by the gateway (e.g., gpt-4o).
• 4. Configure token context window and max tokens to match the chosen model’s limits.
• 5. Save the provider and start a new chat or workspace.
• 6. Select the Generic OpenAI provider and the desired model, then send a test prompt to confirm response.
• 7. For RAG or agent workflows, ensure the same provider is selected; no code changes are needed.
• 8. Monitor usage, adjust token limits, and enable streaming or disable it based on provider support.
• 9. Resolve any UI/.env conflicts by checking /app/server/.env for overridden values.

Доказательства

AnythingLLM already supports a Generic OpenAI provider, so it can connect to any service that exposes an OpenAI-style API.

Источник

The useful part is not the UI integration itself—it is the ability to change models without reworking the AnythingLLM setup.

Источник