A decision proxy in front of SearXNG: speculative execution, per-engine circuit breakers, and keyless fallback

说明与用途

一份可直接使用的部署方案，使用 searxng-gateway HTTP 代理来提升 SearXNG 搜索的可靠性，添加缓存、预估执行、受限回退和每个引擎的熔断器，同时保持原始 JSON 响应结构。

问题

Self-hosted SearXNG search degrades when free engines fail unpredictably, causing fewer results, higher latency, and unclear failure ownership.

结果

The gateway improves search reliability by proactively managing engine failures, reducing latency, and providing clear observability, while keeping client integration unchanged.

工作流

• 1. Deploy SearXNG and searxng-gateway as Docker services using the provided docker-compose snippet.
• 2. Configure environment variables: SEARXNG_BACKEND_URL, FALLBACK_PROVIDERS, T1_PREMIUM_COUNT, SUFFICIENT_MIN_RESULTS, FALLBACK_TIMEOUT_SECONDS, etc.
• 3. Start services; gateway exposes /search, /healthz, /metrics endpoints.
• 4. On each request, gateway normalises query, checks LRU cache, and decides whether to call SearXNG based on cooldown state.
• 5. Execute SearXNG and a configurable number of premium providers concurrently; premium providers are called serially within their pass.
• 6. Merge and deduplicate results by URL; if result count below threshold, perform bounded fallback round-robin until threshold met or timeout.
• 7. Record request outcome labels (cache_hit, searxng_ok, premium_ok, searxng_plus_premium_ok, fallback_fail, timeout).
• 8. Apply per-engine circuit breakers: a single 4xx error opens circuit for 5 minutes, then probes for recovery.
• 9. Expose Prometheus metrics and Grafana dashboard for observability.
• 10. Optionally run in keyless mode by omitting API keys, using only free engines.

证据

searxng-gateway is a Go HTTP proxy that sits in front of SearXNG and adds multi-provider fallback (Brave, Exa, Jina, Tavily), a per-engine circuit breaker, an in-memory LRU cache, Prometheus metrics, and a zero-key mode.

来源

The gateway picks T1_PREMIUM_COUNT providers by atomic round-robin and invokes them serially within that pass.

来源

Each premium provider and each SearXNG engine gets its own circuit breaker, built on sony/gobreaker.

来源