Amazon seller data gives you a direct window into your competitive landscape. Knowing which sellers operate in your category, how their feedback scores stack up, and what they carry lets you make faster, better-informed decisions.
This guide walks through the URL structure, a working Python implementation, and what it takes to keep your scraper running against Amazon's anti-bot setup.
The right tool depends on your requirements. For most Amazon seller scraping, curl_cffi is the better starting point: it is lightweight, handles TLS fingerprinting at the library level, and requires no browser overhead.
Playwright is the better choice when pages require JavaScript execution or you are dealing with frequent CAPTCHA challenges, but it is significantly more resource-intensive and harder to scale.
Standard requests gets blocked almost immediately on Amazon. The issue is TLS fingerprinting: plain Python HTTP clients produce signatures that Amazon's WAF flags as non-browser traffic before the request reaches the page.
curl_cffi is a Python binding for curl-impersonate that replicates real browser TLS signatures, including JA3 and JA4 fingerprints. The API mirrors requests closely, so switching requires minimal code changes. Install it with pip install curl_cffi.
Best for: lightweight scraping, high-volume requests, setups where browser overhead is not acceptable.
Playwright drives a real browser, which means full JavaScript execution, cookie handling, and realistic browser behavior out of the box. It handles the most aggressive anti-bot setups and CAPTCHA flows that curl_cffi cannot.
Best for: JavaScript-heavy pages, CAPTCHA-heavy targets, lower-volume scraping where resource cost is acceptable.
BeautifulSoup handles HTML parsing once you have a response. It pairs directly with curl_cffi for a lightweight setup without the overhead of a full browser.
Datacenter IPs get flagged almost immediately by Amazon's WAF. Residential IPs tied to real ISP-assigned household addresses blend into organic traffic and avoid that first detection layer. Proxy quality matters as much as proxy type. Burned IPs fail just as fast as datacenter ranges.
Amazon splits seller information across three URL patterns. The seller profile page at https://www.amazon.com/sp?seller={SELLER_ID} exposes business name, address, star rating, and feedback breakdown. The storefront listing page at https://www.amazon.com/s?me={SELLER_ID}&marketplaceID={MARKETPLACE_ID} returns the full paginated catalog. Use ATVPDKIKX0DER for the US marketplace.
The offer listing endpoint loads seller comparison data via AJAX when multiple sellers compete on the same listing. Find it by opening DevTools, clicking the seller count link on any product page, and watching the Network tab.
Getting GoProxies set up before running any of the steps below is the right call. Amazon's WAF evaluates IP reputation on the very first request, and starting with clean residential IPs means fewer challenges from the start.
Set impersonate to "chrome" to keep your TLS signature aligned with the latest Chrome profile. Using Session persists cookies across calls, which signals returning visitor behavior rather than a fresh bot each time. The curl_cffi source is at github.com/lexiforest/curl_cffi.
///
from curl_cffi import requests as cffi_requests
proxies = { "http": "http://USERNAME:PASSWORD@proxy.goproxies.com:PORT", "https": "http://USERNAME:PASSWORD@proxy.goproxies.com:PORT" }
session = cffi_requests.Session(impersonate="chrome")
seller_id = "A13BNE3P7C8THK" url = f"https://www.amazon.com/sp?seller={seller_id}"
response = session.get(url, proxies=proxies) print(response.status_code)
///
Use BeautifulSoup to pull the fields you need from the response HTML. A reference implementation for seller field extraction is at github.com/scrapehero/amazon-seller-list. Always validate your selectors against a live page. Amazon's HTML structure shifts periodically.
///
from bs4 import BeautifulSoup
soup = BeautifulSoup(response.text, "html.parser")
name = soup.select_one("#sellerName") rating = soup.select_one(".a-icon-alt") feedback = soup.select_one("#feedback-summary-table")
print(name.text.strip() if name else "Not found") print(rating.text.strip() if rating else "Not found")
///
Loop through paginated storefront pages to collect all ASINs. A reference implementation for ASIN extraction from Amazon listing pages is at github.com/oxylabs/amazon-asin-scraper. Break when a page returns no results, which signals the end of the catalog.
///
import time
marketplace_id = "ATVPDKIKX0DER" base_url = f"https://www.amazon.com/s?me={seller_id}&marketplaceID={marketplace_id}"
asins = []
for page in range(1, 20): url = f"{base_url}&page={page}" response = session.get(url, proxies=proxies) soup = BeautifulSoup(response.text, "html.parser")
results = soup.select("[data-asin]")
page_asins = [el["data-asin"] for el in results if el.get("data-asin")]
if not page_asins:
break
asins.extend(page_asins)
time.sleep(2)
print(f"Found {len(asins)} ASINs")
///
The offer listing endpoint returns cleaner HTML than the full product page and accepts the same session and proxy configuration from previous steps. The scrapehero/amazon-seller-list repo at github.com/scrapehero/amazon-seller-list demonstrates this endpoint in detail, including how to extract condition, shipping, and seller rating per offer.
///
asin = "B0DZDBWM5B" offers_url = f"https://www.amazon.com/gp/offer-listing/{asin}"
response = session.get(offers_url, proxies=proxies) soup = BeautifulSoup(response.text, "html.parser")
offers = soup.select(".a-row.a-spacing-mini.olpOffer") for offer in offers: price = offer.select_one(".olpOfferPrice") seller = offer.select_one(".olpSellerName") print(price.text.strip() if price else "-", seller.text.strip() if seller else "-")
///
Amazon's WAF checks IP reputation on every request. Datacenter ranges are flagged by default, and low-quality residential IPs with fraud history fail almost as quickly.
GoProxies provides 30 million residential IPs across 200+ locations, with the lowest fraud score in the market. Pricing is pay-as-you-go with no minimums, so you scale in line with actual usage.
Maintain cookies across requests rather than creating a fresh session for every call. curl_cffi's Session object handles this automatically. Stateless traffic is a reliable block trigger.
Add delays between requests and vary them with light random jitter. A delay of one to three seconds is a reasonable baseline and avoids hitting rate-limit thresholds on any single IP.
AWS WAF checks header order, Accept-Language values, and the TLS handshake signature, not just the User-Agent string. Setting impersonate="chrome" in curl_cffi replicates the full Chrome fingerprint, including HTTP/2 frame ordering and TLS extensions.
Build retry logic with exponential backoff so the scraper recovers from occasional blocks automatically. Rotate to a fresh IP on each retry rather than reusing the flagged one.
Amazon's bot protection runs on AWS WAF Bot Control at two levels. The common tier catches scrapers that self-identify through headers or match known flagged patterns. The targeted tier uses browser interrogation, behavioral heuristics, and machine learning on traffic statistics to flag sessions that look automated even when basic headers appear clean.
Every TLS handshake produces a fingerprint from cipher suites, extensions, and their ordering. Python's requests produces a distinctly non-browser signature that WAF identifies immediately. curl_cffi's impersonate parameter replaces the default fingerprint at the library level.
CAPTCHAs appear when WAF is confident enough that a session is automated but not enough to block outright. Low-reputation IPs and aggressive request rates are the main triggers. Third-party solving services can handle them programmatically, but the better approach is avoiding them in the first place.
Amazon tracks request volume per IP. Even clean residential IPs will start receiving challenges if they fire too many requests in a short window. Frequent IP rotation and paced requests keep any single IP below the threshold.
A 503 means WAF flagged the request before it reached the application. Check for a datacenter IP, a non-browser TLS fingerprint, or too-aggressive request rates, then address whichever applies.
Rotate to a fresh IP, start a new session with fresh cookies, and reduce request frequency before retrying. Persistent CAPTCHAs almost always point to an IP pool with elevated fraud scores.
Amazon may be serving a reduced view based on location or session state. Verify your proxy targets the correct marketplace region and that cookies are being carried across requests.
Seller IDs appear in the "Sold by" anchor tags. If the element is missing, the product is likely sold directly by Amazon and no third-party seller ID will be present.
From the seller profile page: display name, business name, address, star rating, positive feedback percentage, feedback count, full rating breakdown, and the "About Seller" description.
From the storefront listing: ASINs, product titles, prices, Prime eligibility flags, and availability status.
From the offer listing endpoint: per-seller price, item condition, shipping details, fulfillment method, and individual seller ratings.
GoProxies works directly with the scraping frameworks and tools that handle Amazon's page structure, keeping data flowing with clean, low-fraud IPs.
Scraping publicly available Amazon data is generally legal in most jurisdictions. Public seller profiles and storefront listings do not carry the same protections as private data. Amazon's Terms of Service prohibit scraping, which creates contractual risk. Limit collection to public pages, avoid authenticated pages, and use data for legitimate research.
Amazon's Selling Partner API (SP-API) provides structured access to product and catalog data without HTML parsing. Access requires approval and a registered developer account, and rate limits are enforced per endpoint.
Most competitive intelligence data, including third-party seller details, storefront contents, and cross-seller offer data, is not exposed through the API. For competitive use cases, scraping covers significantly more ground.
The tools are free: curl_cffi and BeautifulSoup are both open source. In practice, free scraping at scale runs into problems quickly. Without residential proxies your IP gets blocked after a handful of requests. Free proxy lists are almost universally burned. Free CAPTCHA-solving tiers introduce delays that make sustained scraping impractical.
Proxy cost is the real cost. A clean residential pool with a low fraud score is what separates a scraper that lasts ten minutes from one that runs for weeks.
GoProxies offers pay-as-you-go residential proxy pricing with no minimums, so you can start small and scale as your use case grows.
Scraping Amazon seller data is achievable with the right stack. curl_cffi handles TLS fingerprinting, BeautifulSoup handles parsing, and clean rotating residential proxies handle IP reputation. The most common failure point is proxy quality, not code quality. Get that layer right and everything else becomes manageable.
GoProxies Research Team is a group of analysts and writers focused on web data, online privacy, and proxy technology. They produce practical, research-backed content that helps readers understand digital trends, tools, and best practices.
The rich text element allows you to create and format headings, paragraphs, blockquotes, images, and video all in one place instead of having to add and format them individually. Just double-click and easily create content.
A rich text element can be used with static or dynamic content. For static content, just drop it into any page and begin editing. For dynamic content, add a rich text field to any collection and then connect a rich text element to that field in the settings panel. Voila!
Headings, paragraphs, blockquotes, figures, images, and figure captions can all be styled after a class is added to the rich text element using the "When inside of" nested selector system.
Scraping publicly available Amazon seller data is generally legal in most jurisdictions, as public information does not carry the same protections as private data. Amazon's Terms of Service prohibit scraping, creating contractual risk, so limiting collection to public pages and using data for legitimate research is the safest approach.
An Amazon storefront exposes the seller's name, business address, star rating, feedback count, and ratings breakdown from the profile page. The storefront listing adds ASINs, product titles, prices, and Prime eligibility. The offer listing endpoint adds per-seller pricing, item condition, shipping details, and fulfillment method.
The most common causes are datacenter or low-reputation residential IPs, a non-browser TLS fingerprint, and request rates that fail behavioral analysis. AWS WAF evaluates all three simultaneously, so fixing only one rarely resolves persistent blocks. Clean residential proxies, curl_cffi for TLS impersonation, and realistic request delays resolve most issues.
Yes, residential proxies are necessary for any sustained Amazon scraping. Datacenter IPs are flagged almost immediately by WAF IP reputation checks. Residential proxies assigned by real ISPs blend into organic traffic and avoid the most aggressive detection layer Amazon applies at the network edge.
