I Added Prometheus to My Go Job Scraper and It Changed How I Think About Production

Описание и применение

Добавление Prometheus в Go‑скрейпер обеспечило наблюдаемость, позволяя отслеживать количество собранных вакансий, время выполнения и ошибки, что превратило проект в готовый к продакшн сервис.

Задача

Running a Go scraper in production without observability makes it invisible and hard to detect failures or performance issues.

Результат

Adding Prometheus metrics and a health endpoint transformed the scraper into a production-ready service with observability, enabling early detection of failures and performance monitoring.

Рабочий процесс

• Create metrics/metrics.go with Prometheus counters and histograms
• Expose /metrics and /healthz endpoints in main.go using promhttp.Handler()
• Instrument scraping logic to record job counts, status, and duration
• Install Prometheus client library (go get github.com/prometheus/[email protected])
• Run the application and verify metrics via curl http://localhost:2112/metrics
• Configure Prometheus scrape_configs to target the scraper
• Create Grafana dashboards using the exposed metrics

Доказательства

Without metrics, you are blind. You don't know: How many jobs you scraped? Is it failing silently? How long does it take?

Источник

Step A: Create metrics/metrics.go ... JobsScraped = promauto.NewCounterVec(...

Источник

Step B: Expose /metrics and /healthz in main.go ... http.Handle("/metrics", promhttp.Handler())

Источник

Now you can create Grafana dashboards with jobs_scraped_total and scrape_duration_seconds.

Источник