PageSift 1.0: the Rust web content engine is feature-complete

说明与用途

PageSift 1.0 是一款完整的 Rust Web 内容引擎，可将 URL、PDF 和 Office 文件转换为干净的 Markdown 和结构化数据，适用于 AI 流水线，支持 OCR、选择器记忆、变更监控和透明错误处理。

问题

Teams needed a self-hosted web content engine that could convert URLs, PDFs, and Office files into clean Markdown and structured data for AI pipelines without per-page pricing or cloud dependencies.

结果

PageSift 1.0 delivers fast, feature-complete content extraction with OCR, selector memory, change monitoring, and transparent failure handling, outperforming rivals in benchmarks.

工作流

• 1. Install PageSift binary or SDK
• 2. Configure API endpoints or SDK usage
• 3. Provide URLs, PDFs, or Office files
• 4. Receive clean Markdown and structured data via HTTP or SDK
• 5. Monitor changes with /v2/monitor
• 6. Handle failures through errors endpoint

证据

PageSift went public as 0.20 on October 20; today it ships 1.0.

来源

PageSift is a web content engine written in Rust. It turns URLs, PDFs and Office files into clean Markdown and structured data for AI agents and LLM pipelines — one self-hosted binary with a Firecrawl-compatible HTTP API, a Jina Reader-compatible endpoint (including /s/), an MCP server, and Python / Node.js / Rust SDKs.

来源

1.0 is not a speed release — performance landed with the 0.9 architecture rebuild. 1.0 is feature completion.

来源

OCR, built from scratch. Scanned PDFs were PageSift's documented weak spot. 1.0 ships its own OCR pipeline — text detection, recognition, formula recognition (LaTeX out), layout analysis — running on a self-written inference runtime (safetensors executor; no ONNX Runtime) and a self-written CUDA GPU layer.

来源

Selector extraction with memory. Give it a JSON schema; when a site redesign breaks your selectors, it relocates elements using element fingerprints and a statistical posterior — and reports relocated, ambiguous, or missing instead of silently returning wrong data.

来源

Change monitoring. A /v2/monitor endpoint family with block-level diffing that separates noise changes (prices, dates, counters) from substantive ones.

来源

Browser actions, sessions, and an agent endpoint across the HTTP, MCP and Jina surfaces — plus WARC files as input, a hooks + [[rules]] system, and per-host secret injection.

来源

Performance (re-measured 2026-11-04) ... PDF → Markdown, olmOCR-bench 1,269 pages, single thread: 2.4 ms/page vs pypdf 50.0 ms/page.

来源

In late October an external team integrated PageSift into their production system and immediately hit a real bug: the official Node binding required a newer glibc than their server had (2.32). 0.81.5 fixed it — build baseline lowered to glibc 2.28, plus a release gate that fails the release if any artifact exceeds it.

来源