Amazon is one of the most aggressively protected scraping targets on the web. The platform runs on AWS WAF, a commercial-grade firewall that inspects every incoming request for signs of automation: TLS fingerprints, browser behavior signals, and IP reputation, among others.
Send a plain HTTP request from a standard Python library and you'll get blocked before you even reach the product page.
The good news is that handling Amazon CAPTCHA and bot detection is very much solvable. It requires a layered approach: a browser-impersonating HTTP client for lightweight requests, a headless browser with stealth plugins for JavaScript-heavy pages, and residential proxies to keep your IPs clean.
Session management that keeps behavior consistent ties it all together. This guide walks through all of it, step by step.
Getting past Amazon's defenses starts with choosing the right tools for each layer of the problem. Using a single tool rarely works. You need a stack where each component handles a specific detection vector.
The first tool is curl_cffi, a Python HTTP client built on curl-impersonate. Unlike requests or httpx, curl_cffi impersonates real browser TLS and HTTP/2 fingerprints so the TLS handshake looks identical to what Chrome or Safari would send. This is critical against AWS WAF, which performs TLS fingerprinting on every connection.
If you prefer a JavaScript environment, tls-client is a Node.js alternative that serves a similar purpose. Install curl_cffi with pip install curl_cffi.
The second tool is Playwright paired with playwright-extra and puppeteer-extra-plugin-stealth. Playwright handles pages that require full JavaScript execution: product pages with dynamic pricing, login flows, or any page that loads content after the initial HTML response.
Puppeteer is a viable alternative if you are already working in a Puppeteer-based stack, as puppeteer-extra-plugin-stealth was originally built for it. On its own, Playwright leaves headless browser signals that AWS WAF's Bot Control targeted inspection level can detect. The stealth plugin patches those signals before any page loads.
The third component is rotating residential proxies. Amazon's IP reputation checks cross-reference every incoming IP against known datacenter ranges and flagged addresses. Residential proxies, IPs assigned by real ISPs to real households, have a fundamentally different reputation profile and blend into organic traffic naturally.
If your use case involves a consistent identity across multiple requests rather than rotation, ISP proxies are a static alternative worth considering.
Plain Python HTTP clients fail against Amazon because they expose a non-browser TLS fingerprint during the connection handshake. AWS WAF sees the JA3 hash, compares it against known browser signatures, and blocks anything that doesn't match.
curl_cffi solves this by wrapping curl-impersonate, which patches the TLS stack to produce fingerprints identical to Chrome, Safari, or Edge.
Start by installing the library. The following command installs the latest stable version:
\\\
pip install curl_cffi --upgrade
///
Use the impersonate parameter on every request. Setting it to "chrome" without a version number tells curl_cffi to use the fingerprint for the latest supported Chrome version, which keeps you current as the library updates. The official lexiforest/curl_cffi repository on GitHub at https://github.com/lexiforest/curl_cffi covers the full list of supported impersonation targets.
\\\
from curl_cffi import requests response = requests.get( "https://www.amazon.com/dp/B09G9HD6PD", impersonate="chrome" ) print(response.status_code) print(response.text[:500])
///
For pages that require persistent cookies, use a Session object. The Session persists cookies across requests and reuses the underlying connection, which also produces more human-like connection behavior.
\\\
from curl_cffi import requests with requests.Session() as session: session.get("https://www.amazon.com", impersonate="chrome") response = session.get( "https://www.amazon.com/s?k=wireless+headphones", impersonate="chrome" ) print(response.text[:500])
///
For product pages that load prices, reviews, or inventory data via JavaScript after the initial page load, you need a full browser. Playwright handles this well, but out of the box it leaves automation signals.
Most notably, the navigator.webdriver property is set to true, which AWS WAF's Bot Control targeted inspection level uses as a hard signal for automation.
The fix is playwright-extra combined with puppeteer-extra-plugin-stealth. The stealth plugin patches navigator.webdriver along with a range of other browser API signals before any page content loads. Install the required packages:
\\\
npm install playwright playwright-extra puppeteer-extra-plugin-stealth
///
Import from playwright-extra rather than from playwright directly. This is the critical detail: if you import from the regular playwright package, the stealth plugin is never applied and you get no warning about it. The setup using the berstend/puppeteer-extra repository on GitHub at https://github.com/berstend/puppeteer-extra looks like this:
\\\
const { chromium } = require('playwright-extra') const StealthPlugin = require('puppeteer-extra-plugin-stealth') chromium.use(StealthPlugin()) async function scrapeAmazon(url, proxyUrl) { const browser = await chromium.launch({ headless: true }) const context = await browser.newContext({ proxy: { server: proxyUrl }, userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36', viewport: { width: 1280, height: 800 } }) const page = await context.newPage() await page.goto(url, { waitUntil: 'domcontentloaded' }) const content = await page.content() await browser.close() return content } scrapeAmazon('https://www.amazon.com/dp/B09G9HD6PD', 'http://user:pass@proxy.example.com:8080')
///
Each browser context gets its own proxy assignment. This matters for isolation: if one IP gets flagged, it doesn't affect requests running through other contexts.
Even with perfect TLS fingerprinting and stealth plugins, sending too many requests from the same IP triggers Amazon's rate-limiting and IP reputation rules. You need a residential proxy pool large enough that no single IP carries a meaningful volume of requests.
With curl_cffi, pass the proxy directly on each request using the proxy parameter. The library accepts standard HTTP proxy URLs with credentials inline:
\\\
from curl_cffi import requests proxies = { "https": "http://username:password@residential.proxy.example.com:8080" } response = requests.get( "https://www.amazon.com/dp/B09G9HD6PD", impersonate="chrome", proxies=proxies )
///
For rotating proxies where the provider handles IP selection on each request, a single endpoint URL is enough. The provider rotates the exit IP automatically.
GoProxies gives you access to a pool of 30 million residential IPs across 200+ locations, with the lowest fraud score in the market, which directly reduces CAPTCHA trigger rates on protected targets like Amazon. Start scraping with GoProxies.
Amazon tracks sessions through cookies. A scraper that arrives on a product page without a proper session history looks suspicious. A real browser visits the homepage, accumulates cookies, navigates through categories, and only then lands on a product. Replicating this pattern keeps requests within the range of expected behavior.
With curl_cffi's Session, cookies set by Amazon on one request are automatically included in the next. For operations that need the same IP across multiple requests, use a sticky session endpoint if your proxy provider supports it.
For single-page product requests with no continuity requirement, rotating on every request is the right approach.
\\\
from curl_cffi import requests with requests.Session() as session: session.get("https://www.amazon.com", impersonate="chrome", proxies={"https": "http://user:pass@proxy.example.com:8080"}) response = session.get( "https://www.amazon.com/dp/B09G9HD6PD", impersonate="chrome", proxies={"https": "http://user:pass@proxy.example.com:8080"} ) print(response.status_code)
///
Consistent request timing is one of the clearest signals of automation. A real user takes between one and several seconds to read a page before clicking. A scraper hammering requests with uniform 100ms intervals looks nothing like that.
Add random delays between requests using Python's random module. A range of one to four seconds between requests is a reasonable starting point for Amazon.
For Playwright-based flows, add simulated mouse movement and scroll behavior before extracting data. AWS WAF's targeted inspection level uses behavioral signals to distinguish automated browsers from real ones, even after the stealth plugin has patched static fingerprint signals.
\\\
import time import random from curl_cffi import requests urls = [ "https://www.amazon.com/dp/B09G9HD6PD", "https://www.amazon.com/dp/B08N5WRWNW", "https://www.amazon.com/dp/B07XJ8C8F7" ] with requests.Session() as session: for url in urls: response = session.get(url, impersonate="chrome", proxies={"https": "http://user:pass@proxy.example.com:8080"}) print(response.status_code) time.sleep(random.uniform(1.5, 4.0))
///
The quality of your residential proxy pool matters as much as the rotation strategy. A small pool of low-quality IPs gets flagged quickly on Amazon because the same addresses appear repeatedly across multiple scraping operations. A large, ethically sourced pool with a low fraud score means each IP arrives with a clean reputation.
For product page scraping, rotate the proxy on every request. For flows that require continuity, such as search, click, then product page, use a sticky session for the duration of that flow, then rotate.
I recommend GoProxies for Amazon scraping specifically: 30 million residential IPs, pay-as-you-go pricing, and no contracts mean you scale up or down without overhead. Check current pricing on the GoProxies rotating residential proxies page.
AWS WAF's Bot Control compares browser fingerprint signals across the full request. If your user-agent says Chrome 124 on Windows but your viewport is 800x600 and your Accept-Language header is missing, the mismatch is a detection signal. Every component of the fingerprint needs to form a coherent profile matching a real device.
Set the user-agent, viewport, Accept-Language, and platform consistently within each session. Match the user-agent to a real Chrome version and pair it with a viewport and platform string that actually exists in the wild. Avoid randomizing these values independently, as random combinations often produce impossible profiles that no real device would have.
When Amazon returns a 503 or a CAPTCHA page, the right response is not to retry immediately. Immediate retries signal automation and often escalate the block from a soft rate limit to a harder IP ban. Use exponential backoff: wait longer on each successive failure before retrying.
curl_cffi includes a native RetryStrategy that handles this cleanly. The following configuration retries up to three times with exponential backoff and a small jitter to prevent synchronized retry storms across parallel workers:
\\\
from curl_cffi import requests, RetryStrategy strategy = RetryStrategy(count=3, delay=2.0, jitter=0.5, backoff="exponential") with requests.Session(retry=strategy) as session: response = session.get( "https://www.amazon.com/dp/B09G9HD6PD", impersonate="chrome", proxies={"https": "http://user:pass@proxy.example.com:8080"} ) print(response.status_code)
///
Running requests in parallel speeds up collection but also increases the risk of triggering rate limits. Too many requests hitting Amazon from the same IP or within the same time window draws attention quickly. Use curl_cffi's AsyncSession with asyncio to run concurrent requests while keeping each request on a different proxy.
Keep concurrency modest. A few parallel workers per target domain is a reasonable ceiling before rate-limiting becomes a consistent problem.
Amazon's defenses go beyond simple IP blocking. Understanding what each layer detects helps you choose the right countermeasure for each situation.
Every HTTPS connection begins with a TLS handshake that exposes information about the client: which cipher suites it supports, the order of TLS extensions, and configuration details that together form a fingerprint called a JA3 hash. Browsers have consistent, well-known JA3 hashes. Standard Python HTTP libraries produce different hashes that AWS WAF flags immediately.
AWS WAF's targeted Bot Control inspection level goes further than JA3 and also inspects HTTP/2 fingerprints and other connection-level signals. curl_cffi addresses this by patching the entire TLS stack to match real browser behavior, not just spoofing headers.
When AWS WAF determines a request is suspicious but not definitively automated, it serves a CAPTCHA challenge rather than a hard block. The CAPTCHA requires a valid AWS WAF token present in the request before access is granted.
This token is generated client-side through a JavaScript challenge, meaning a plain HTTP client cannot produce it. Only a real browser can complete the challenge.
The most effective mitigation is preventing the CAPTCHA from being triggered at all. That means clean residential IPs, consistent browser fingerprints, and human-like request pacing. If a CAPTCHA does appear, rotating to a fresh IP and starting a new session is more reliable than attempting to solve the challenge programmatically.
Amazon cross-references incoming IPs against several reputation signals: whether the IP belongs to a known datacenter range, whether it appears on commercial threat intelligence lists, and whether it has generated a high volume of requests recently.
Datacenter IPs fail the first check immediately. Shared residential IPs that have been overused by other scrapers may fail the third. Rotating frequently and keeping per-IP request volume low keeps your operation within the threshold where Amazon treats traffic as organic.
The symptom is a 503 response with an HTML body that may include a message about the service being temporarily unavailable. The cause is usually IP-level rate limiting: Amazon's WAF has flagged the IP for sending too many requests too quickly, or the IP is on a reputation block list.
The fix is to rotate to a fresh residential proxy, add a longer delay before the next request, and reduce concurrency if running parallel workers. If 503s persist across multiple IPs, confirm that curl_cffi is using the correct impersonate value and that request headers are consistent.
The symptom is a 200 response but the HTML body contains a CAPTCHA form rather than product content. The cause is almost always either a TLS fingerprint mismatch from using a plain HTTP client without impersonation, or a flagged datacenter IP that AWS WAF has decided to challenge.
Confirm curl_cffi is configured with impersonate="chrome" and that the proxy in use is a residential IP. If using Playwright, confirm the stealth plugin is applied before the browser launches and that you are importing chromium from playwright-extra, not from the standard playwright package.
The symptom is that Playwright requests return CAPTCHA pages or bot-detection error pages consistently, even with residential proxies in place. The cause is that the navigator.webdriver property is set to true in the browser, which AWS WAF's targeted inspection reads as a definitive automation signal.
Confirm puppeteer-extra-plugin-stealth is registered before any browser instance is created. Call chromium.use(StealthPlugin()) before chromium.launch(). If the issue persists, verify the import is from playwright-extra and not from playwright directly. The stealth plugin has no effect when attached to the standard playwright chromium object.
Amazon contains a dense set of data points useful across a range of applications: price monitoring, competitive intelligence, market research, review analysis, and inventory tracking.
Product pages expose the product title, current price, sale price and discount percentage, ASIN, product description, bullet point feature list, and technical specifications. The best seller rank is available for each category the product appears in, along with sales rank history in some cases.
Review data includes the overall star rating, total review count, individual review text, reviewer name, verified purchase status, helpful vote count, and review date. Search result pages expose the organic ranking position, sponsored placement indicators, and the full set of products returned for a given keyword.
Seller information includes the seller name, seller rating, fulfillment method (FBA or FBM), and whether the listing is sold directly by Amazon. Inventory signals such as low stock warnings and delivery estimate dates are available on product pages and can be tracked over time to infer demand patterns.
With a reliable residential proxy infrastructure behind your scraper, collecting this data at scale across categories and geographies is straightforward. GoProxies gives you the IP coverage and location targeting to match any extraction use case.
Scraping publicly visible Amazon product data sits in a legally complex space. Amazon's Terms of Service explicitly prohibit automated access without prior written permission. That said, ToS violations are a contractual matter between you and Amazon, not a criminal one.
The data that appears publicly on product pages, such as prices, titles, and reviews, is factual information that courts in the US and EU have generally treated as publicly accessible. Scraping non-public data, data behind login walls, or data in ways that cause service disruption carries significantly higher legal and ethical risk.
For commercial data collection, consulting a lawyer familiar with data law in your jurisdiction is the responsible step before launching an operation at scale. From an ethical standpoint, scraping responsibly means respecting rate limits, avoiding personal data collection, and not degrading Amazon's service for legitimate users.
Amazon offers the Product Advertising API (PAAPI 5.0) as an official way to access product data. It returns structured data for product listings including title, price, images, and review counts. It is the cleanest option for applications that need Amazon data without the maintenance overhead of a scraper.
The tradeoffs are significant. PAAPI access requires an active Amazon Associates account, and the rate limits are strict. The data available through the API is narrower than what scraping can reach: full review text, seller details, inventory signals, and best seller rank history are either unavailable or heavily restricted.
The API is also subject to Amazon's partner terms, which restrict commercial use in ways that scraping does not. If you need a small volume of standard product data for an affiliate application, the API is the right tool.
If you need full review text, competitive pricing data across categories, or volumes the API rate limits would prohibit, scraping is the practical approach.
Free scraping tools and free proxy tiers consistently fail against Amazon because of how AWS WAF works. Free proxies are almost always datacenter IPs that Amazon has already flagged. Free browser automation setups without stealth plugins get blocked on the first request.
The result is not a cost saving. It is time spent debugging blocks that proper infrastructure would have avoided.
The real cost of free scraping is operational: time spent rotating tools, managing bans, and rewriting scrapers around detection updates. At any meaningful scale, that cost exceeds what a paid residential proxy plan costs.
GoProxies offers pay-as-you-go pricing with no minimum commitment, so you pay only for what you use. See the current residential proxy pricing and scale from there.
Amazon is a difficult target, but it is not an impossible one. The blocks you hit with a plain HTTP client are not the final word. They are the result of using the wrong tool for the detection layer you are up against.
Swap in curl_cffi with browser impersonation, add Playwright with the stealth plugin for JavaScript-heavy pages, put residential proxies behind every request, and manage sessions and pacing to stay within the behavior range Amazon expects from real users.
Each tool handles one detection vector. When all four layers work together, the scraper holds up consistently even on a target as well-defended as Amazon.
GoProxies Research Team is a group of analysts and writers focused on web data, online privacy, and proxy technology. They produce practical, research-backed content that helps readers understand digital trends, tools, and best practices.
The rich text element allows you to create and format headings, paragraphs, blockquotes, images, and video all in one place instead of having to add and format them individually. Just double-click and easily create content.
A rich text element can be used with static or dynamic content. For static content, just drop it into any page and begin editing. For dynamic content, add a rich text field to any collection and then connect a rich text element to that field in the settings panel. Voila!
Headings, paragraphs, blockquotes, figures, images, and figure captions can all be styled after a class is added to the rich text element using the "When inside of" nested selector system.
Scraping publicly visible Amazon data is legally complex. Amazon's Terms of Service prohibit automated access, but scraping publicly available factual data is generally protected under case law in the US and EU. Commercial use carries higher risk. Consulting a data law specialist before launching a large-scale operation is the recommended approach.
Yes, residential proxies are necessary for Amazon scraping at any meaningful scale. Amazon's AWS WAF checks IP reputation as part of its bot detection rules, and datacenter IPs are flagged reliably. Residential IPs have a fundamentally different reputation profile that lets requests blend in with organic traffic.
curl_cffi alone works well for Amazon pages that return content in the initial HTML response, such as many product and category pages. For pages that load pricing, review counts, or inventory data via JavaScript after the initial load, Playwright with the stealth plugin is needed to execute that JavaScript and retrieve the final rendered content.
No, free Amazon scraping is not reliable at scale. Free proxy tiers use datacenter IPs that Amazon blocks immediately, and free automation tools without stealth configuration are detected by AWS WAF's Bot Control rules. The operational cost of managing those blocks outweighs any savings from avoiding a paid proxy plan.
