I Added Prometheus to My Go Job Scraper and It Changed How I Think About Production

说明与用途

在 Go 爬虫中添加 Prometheus 提供了可观测性，跟踪工作计数、持续时间和错误，使项目成为生产就绪的服务。

问题

Running a Go scraper in production without observability makes it invisible and hard to detect failures or performance issues.

结果

Adding Prometheus metrics and a health endpoint transformed the scraper into a production-ready service with observability, enabling early detection of failures and performance monitoring.

工作流

• Create metrics/metrics.go with Prometheus counters and histograms
• Expose /metrics and /healthz endpoints in main.go using promhttp.Handler()
• Instrument scraping logic to record job counts, status, and duration
• Install Prometheus client library (go get github.com/prometheus/[email protected])
• Run the application and verify metrics via curl http://localhost:2112/metrics
• Configure Prometheus scrape_configs to target the scraper
• Create Grafana dashboards using the exposed metrics

证据

Without metrics, you are blind. You don't know: How many jobs you scraped? Is it failing silently? How long does it take?

来源

Step A: Create metrics/metrics.go ... JobsScraped = promauto.NewCounterVec(...

来源

Step B: Expose /metrics and /healthz in main.go ... http.Handle("/metrics", promhttp.Handler())

来源

Now you can create Grafana dashboards with jobs_scraped_total and scrape_duration_seconds.

来源