43 days, 17,040 AI signals: four things the data says that trend lists won't tell you

Description and use

A 43‑day pipeline collected 17,040 AI signals, revealing that rotating trend lists provide unreliable metrics; the solution involves fixed tracking, churn alerts, source validation, and daily recomputation of derived numbers.

Problem

Relying on rotating trend lists for AI models and repos yields unreliable trend metrics and misleading insights.

Outcome

The pipeline demonstrates that rotating trend lists alone are insufficient for reliable trend analysis; combining fixed‑set tracking, churn‑focused alerts, per‑source yield monitoring, and daily audit of derived metrics yields a more trustworthy dataset.

Workflow

• 1. Collect AI signals from multiple sources (GitHub, Hugging Face, arXiv, etc.) 4×/day.
• 2. Store raw snapshots in JSONL format.
• 3. Recompute derived metrics (rank, rank change, momentum) from raw snapshots each day.
• 4. Run automated audit to reconcile rendered pages with JSON endpoints; flag mismatches.
• 5. Publish daily dataset (CC BY 4.0) and expose API endpoints for today, changes, models, repos.
• 6. Provide MCP tools for Claude/Cursor integration.
• 7. Monitor API usage; set User‑Agent to avoid 403 from CDN edge.

Evidence

Only 23 of 1,493 tracked models have enough history to compute a trend

Source

Top entities are boringly stable — the interesting signal is churn

Source

Most of the volume is one source, and it isn't the interesting one

Source

Every derived number I publish (rank, rank change, 7-day momentum, position in an arXiv category) is recomputed from raw snapshots by an automated audit

Source

If you call the API with a bare Python-urllib User-Agent (or none), you'll get a 403 from the CDN, at the edge

Source