I Tracked Weather Forecasts for 3 Weeks to See How Wrong They Really Are

Description and use

A pipeline was built to collect weather forecasts and compare them with actual data, revealing accuracy patterns by lead time and city, and showing that a simple linear model outperforms a naive baseline.

Problem

Weather apps present a 7‑day forecast as fact, but the accuracy of day‑7 predictions is unclear.

Outcome

Accuracy drops with lead time, varies by city, and rain predictions degrade; Ridge regression outperforms a simple baseline in error prediction.

Workflow

• Pull 8‑day forecast for Islamabad, London, New York via Open‑Meteo API
• Store forecast in Neon Postgres
• Check historical archive for actual weather using Open‑Meteo historical API
• Store actual values in database
• Create SQL view joining forecast and actual
• Run Python analysis to compute errors and build models
• Display results on Streamlit dashboard

Evidence

Open‑Meteo API --> GitHub Actions (daily) --> Neon Postgres --> Python analysis --> Streamlit dashboard

Source

Same‑day forecasts were off by 0.58°C on average. By 7 days out, that grew to roughly 1.5–2.0°C.

Source

New York's forecasts were consistently the least accurate — close to double Islamabad's error at most lead times (3.22°C vs 1.35°C at day 6).

Source

I tried to predict how wrong a forecast would be, using lead time and the forecast's own values as features. I compared a plain average-by-lead-time baseline against Ridge regression, Random Forest, and Gradient Boosting... Ridge won by about 28%.

Source