When curl, TLS spoofing and headless all fail: how I got past Akamai with a real browser

Описание и применение

Скрипт, использующий обычный Chrome с профилем пользователя и управляемый через CDP, обошёл защиту Akamai и успешно собрал данные с mobile.de.

Задача

A scraper needed car listings from mobile.de but was blocked by Akamai Bot Manager, rejecting plain requests, TLS spoofing, and headless browsers.

Результат

By using a genuine Chrome browser with a real user profile and controlling it via CDP, the scraper bypassed Akamai’s bot detection and successfully retrieved the required listings.

Рабочий процесс

• 1. Send plain HTTP request with User-Agent header – blocked by Akamai challenge page.
• 2. Add real Chrome TLS fingerprint to requests – still blocked.
• 3. Use headless Chrome via Playwright/Puppeteer – flagged by Akamai’s automation detection.
• 4. Launch a real Chrome with remote debugging enabled and a user profile.
• 5. Connect to the running Chrome using Playwright’s connect_over_cdp to the debugging port.
• 6. Load mobile.de listings page – page renders normally.
• 7. Extract desired data fields from the rendered page.
• 8. Store or process the extracted listings.

Доказательства

I launched a real, ordinary Chrome — the same binary a human uses, with a real user profile — and drove it programmatically over the Chrome DevTools Protocol (CDP) instead of launching a fresh automated instance.

Источник