When curl, TLS spoofing and headless all fail: how I got past Akamai with a real browser

说明与用途

使用真实 Chrome 浏览器并通过 CDP 控制，绕过 Akamai 机器人检测，成功抓取 mobile.de 的列表数据。

问题

A scraper needed car listings from mobile.de but was blocked by Akamai Bot Manager, rejecting plain requests, TLS spoofing, and headless browsers.

结果

By using a genuine Chrome browser with a real user profile and controlling it via CDP, the scraper bypassed Akamai’s bot detection and successfully retrieved the required listings.

工作流

• 1. Send plain HTTP request with User-Agent header – blocked by Akamai challenge page.
• 2. Add real Chrome TLS fingerprint to requests – still blocked.
• 3. Use headless Chrome via Playwright/Puppeteer – flagged by Akamai’s automation detection.
• 4. Launch a real Chrome with remote debugging enabled and a user profile.
• 5. Connect to the running Chrome using Playwright’s connect_over_cdp to the debugging port.
• 6. Load mobile.de listings page – page renders normally.
• 7. Extract desired data fields from the rendered page.
• 8. Store or process the extracted listings.

证据

I launched a real, ordinary Chrome — the same binary a human uses, with a real user profile — and drove it programmatically over the Chrome DevTools Protocol (CDP) instead of launching a fresh automated instance.

来源