Build a Kafka MCP Server: Safe Consumer Lag and Topic Diagnostics for AI Agents

说明与用途

构建 Kafka MCP 服务器，使用只读 SCRAM 用户和 AdminAPI 安全地向 AI 代理提供消费者延迟、主题健康和代理状态的元数据。

问题

Incidents in event-driven systems often stem from consumer groups lagging millions of messages, yet diagnosing lag is hard because naive tools consume messages or alter offsets, risking data loss or performance impact.

结果

The MCP server provides AI agents with instant, safe diagnostics of consumer lag and Kafka health without risking data exposure or altering consumer state.

工作流

• 1. Create a SCRAM user (agent‑ro) with only Describe and DescribeConfigs ACLs on topics, groups, and cluster.
• 2. Configure Kafka AdminClient in the MCP server to use this user over SASL_SSL.
• 3. Implement four FastMCP tools: group_lag, all_groups_lag, topic_health, cluster_health, each querying AdminClient for offsets, end offsets, and configs.
• 4. Cap returned lists (MAX_PARTITIONS, MAX_GROUPS) to keep model input manageable.
• 5. Deploy the MCP server and expose it to AI agents via FastMCP endpoint.

证据

Half of "the API is slow" incidents in an event-driven system are actually "a consumer group is 2 million messages behind."

来源

The server follows the same rule as the Postgres one: statistics and metadata only, never contents.

来源

Create a dedicated SCRAM user and grant it exactly one verb. Describe on topics covers metadata and end offsets; Describe on groups covers committed offsets and member state; Describe plus DescribeConfigs on the cluster covers broker and config lookups.

来源