Making three years of a Telegram group chat queryable

说明与用途

构建了自动化流程，将 Telegram 聊天导出转换为结构化 Markdown 源文件，并上传到 NotebookLM，实现对多年聊天记录的高效查询。

问题

A long Telegram group chat contains valuable knowledge but is unreadable and unsearchable with Telegram’s basic search.

结果

The pipeline successfully transforms raw Telegram exports into structured, word‑limited Markdown sources that can be ingested by NotebookLM, enabling efficient querying of a multi‑year chat history without duplication.

工作流

• 1. Export chat history from Telegram Desktop (JSON) or macOS app (HTML).
• 2. Parse JSON: handle plain strings and array‑of‑runs for formatted text using a recursive join function.
• 3. Parse HTML: split files by regex, handle sticky senders, forwarded messages, and differing date formats, normalising dates to ISO.
• 4. Render each message to Markdown, measuring word count including separators and frontmatter placeholders.
• 5. Pack messages into source files, ensuring each stays below the 400k‑word limit (NotebookLM default) and never splits a message.
• 6. Use a running word sum and memoised per‑record counts to keep packing O(n).
• 7. Generate filenames encoding index, last‑message date, and first‑message ID (e.g., nomad‑001‑2023‑06‑01t16‑05‑52‑1.md).
• 8. On subsequent runs, read existing filenames, invert the pattern to find the highest index and last cursor, and filter new records strictly after that cursor.
• 9. Upload the packed sources to NotebookLM, creating a queryable knowledge base.

证据

The fix is boring in outline: get the history out, turn it into documents, hand them to something that reads — NotebookLM, in my case.

来源

NotebookLM caps a source at 500,000 words — but the number of sources per notebook is the scarcer cap.

来源