Giving AI Agents Lakehouse Memory: How We Built an Apache Iceberg Connector for Cognee

说明与用途

为 Cognee 构建了 Apache Iceberg 连接器，使用元数据构建知识图谱，使 AI 代理能够查询湖仓内存而不泄露原始数据，并防止出现“幽灵知识”。

问题

AI agents lack semantic memory of enterprise lakehouses, leading to blind operations and data leakage when accessing raw table data.

结果

The connector automatically discovers tables, parses metadata, populates a knowledge graph with 42 entities and 68 relations in 1.24s, and cleans orphaned nodes on table drop, enabling agents to query lakehouse memory without raw data leaks.

工作流

• Configure Iceberg catalog source in Cognee
• Set document-mode marker on dlt source
• Run full-snapshot sync with write_disposition="replace"
• Cognee cognify() extracts entities and relations
• Orphan cleanup removes dropped tables
• Agents query lakehouse memory via graph completion

证据

Agents are increasingly tasked with diagnosing failed ETL pipelines, performing data governance audits, and answering developer queries like…

来源

The connector reads table metadata: Full table identifier, column definitions, partition specifications, snapshot commit history, and table properties…

来源

Cognee's internal orphan_cleanup detects the vanished entity and reconciles it out of both the knowledge graph and vector indices.

来源