The JVM Speaks AI: Cobrainer's Journey with Kotlin, Koog and Exposed

Description and use

Cobrainer unified AI integration in a single Kotlin monorepo with Koog, achieving zero‑downtime during provider throttling, 5× throughput, and a one‑day rollout to all customers.

Problem

A large customer triggered a burst of AI‑assisted work, causing one LLM provider to throttle Cobrainer’s services. The system needed a reliable fallback and circuit‑breaking mechanism to keep the workflow uninterrupted.

Outcome

By consolidating AI integration into Kotlin with Koog, Cobrainer achieved zero‑downtime during provider throttling, 5× throughput gains, and a single‑day rollout to all customers.

Workflow

• 1. Identify LLM framework requirements: lean, explicit, rapid release cadence, clean provider abstractions, end‑to‑end type safety.
• 2. Evaluate Spring AI, LangChain4j, and Koog by building production‑ready prototypes.
• 3. Choose Koog for its Kotlin‑centric design and explicit contract handling.
• 4. Refactor existing AI code to use Koog’s DSL and sealed‑hierarchy contracts.
• 5. Implement cross‑provider fallback chains and circuit breakers.
• 6. Deploy the new stack to all customers in one day.
• 7. Monitor with LLMObs and enforce contract tests.

Evidence

One afternoon, one of our LLM providers started throttling us. A large customer had triggered a burst of AI‑assisted work, and the provider's capacity in our region could not keep up.

Source

We built all of our AI integration in Kotlin. ... Every prompt, agent, and structured‑output contract in our core platform lives on the JVM: one Kotlin monorepo with types shared end to end, built on Koog, Exposed, coroutines, and kotlinx.serialization.

Source

Production reliability comes from cross‑provider fallback chains, circuit breakers, LLMObs, and contract tests. Removing blocking code from coroutine paths also increased throughput by 5×.

Source

Once we chose Koog, we cleaned up the proof of concept and rolled it out to all customers in one day.

Source