Amazon adjusts prices across millions of listings constantly. If you're building a monitoring system to track those shifts, whether for competitor intelligence, retail arbitrage, or dynamic repricing, you need more than a basic scraper.
You need a proxy setup that rotates IPs to avoid blocks, holds sessions long enough for your bot to finish what it started, and doesn't get fingerprinted by Amazon's detection stack.
Most guides cover one of those things. This one covers all three.
Here's the core stack you'll need:
For HTTP requests with browser-level impersonation, curl_cffi is the right choice. It replicates the Chrome TLS handshake and HTTP/2 settings at the connection level, which is what gets you past Amazon's first layer of fingerprint detection. The standard requests library produces a recognisable non-browser fingerprint and will get flagged faster on protected pages.
For JavaScript-rendered pages, Playwright is the primary option, with Puppeteer as an alternative if you prefer a Node.js environment. Both launch a real Chromium instance and let you read the fully rendered DOM, which is necessary for variant products where Amazon loads prices dynamically.
For HTML parsing, BeautifulSoup paired with lxml handles everything you need. lxml is faster than Python's built-in html.parser and is the better default for production scraping.
For storage, CSV works for small jobs. SQLite or Postgres makes more sense once you're tracking hundreds of ASINs over time.
For scheduling, cron works. A simple loop with delays inside Python also works. The choice depends on how frequently you need price snapshots.
A residential proxy service with sticky session support ties the stack together. Datacenter proxies are cheaper and usable for lower-sensitivity targets, but on Amazon they get flagged more often and burn through faster. Residential IPs, assigned by real ISPs to real devices, hold up better at scale.
Install the dependencies you'll need:
///
pip install requests curl_cffi beautifulsoup4 playwright lxml playwright install chromium
///
Create a project folder and set up a simple config file to store your proxy credentials and target ASINs separately from your code. This makes rotation logic and ASIN lists easier to update without touching the scraper itself.
This is the part most guides get wrong. The instinct is to pick one mode, either rotate on every request or lock one IP for everything. Neither is right for Amazon price monitoring.
The correct strategy: rotate between products, but lock the same IP for the full duration of each product's data pull. Amazon's session logic ties cookies to the originating IP.
If your IP changes mid-pull, the session invalidates, prices may not load, and you may trip a detection signal that looks like a bot bouncing IPs during active browsing.
GitHub: RohanDas28/Amazon-Price-Tracker is a useful starting reference for the basic Python structure. Here's how to configure the proxy session correctly:
///
import requests from bs4 import BeautifulSoup
PROXY_HOST = "proxy.goproxies.com" PROXY_PORT = "1080" PROXY_USER = "customer-USERNAME-session-SESSIONID" PROXY_PASS = "PASSWORD"
proxy_url = f"https://{PROXY_USER}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}"
proxies = { "http": proxy_url, "https": proxy_url, }
HEADERS = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36", "Accept-Language": "en-US,en;q=0.9", "Accept-Encoding": "gzip, deflate, br", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8", }
def get_price(asin, session_id): url = f"https://www.amazon.com/dp/{asin}" proxy_user = f"customer-USERNAME-session-{session_id}" proxy_url = f"https://{proxy_user}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}" proxies = {"http": proxy_url, "https": proxy_url}
try:
response = requests.get(url, headers=HEADERS, proxies=proxies, timeout=15)
response.raise_for_status()
soup = BeautifulSoup(response.text, "lxml")
price_tag = soup.select_one(".a-price .a-offscreen")
return price_tag.get_text(strip=True) if price_tag else None
except requests.RequestException as e:
print(f"Request failed for {asin}: {e}")
return None
///
The key is the session-SESSIONID parameter in the username string. The same session ID means the same IP for the duration of that session. Use a unique session ID per ASIN, and rotate the session ID, so a fresh IP, when you move to the next product.
GoProxies gives you access to a 30 million residential IP pool, so rotating to a clean IP between products is reliable at scale.
Not all Amazon prices are in the static HTML. Variant products, like electronics with multiple storage options or clothing with multiple sizes, often load the displayed price through JavaScript after the page renders. A requests-only scraper will either get a blank price field or the wrong variant's price.
Playwright solves this. It launches a real Chromium browser, renders the full page including JS execution, and lets you read the final DOM. Here's a working example based on the structure from afsal-backer/Amazon-Price-Checker-Using-Playwright:
///
from playwright.sync_api import sync_playwright
PROXY_HOST = "proxy.goproxies.com" PROXY_PORT = "1080" PROXY_USER = "customer-USERNAME-session-SESSIONID" PROXY_PASS = "PASSWORD"
def get_price_playwright(asin, session_id): proxy_user = f"customer-USERNAME-session-{session_id}" url = f"https://www.amazon.com/dp/{asin}"
with sync_playwright() as p:
browser = p.chromium.launch(
headless=True,
proxy={
"server": f"https://{PROXY_HOST}:{PROXY_PORT}",
"username": proxy_user,
"password": PROXY_PASS,
},
args=["--disable-blink-features=AutomationControlled"],
)
context = browser.new_context(
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
locale="en-US",
viewport={"width": 1280, "height": 800},
)
page = context.new_page()
page.goto(url, wait_until="networkidle", timeout=30000)
price_el = page.query_selector(".a-price .a-offscreen")
price = price_el.inner_text() if price_el else None
browser.close()
return price
///
The --disable-blink-features=AutomationControlled flag removes the navigator.webdriver property that defaults to true in headless Chromium. That single property is one of the first signals Amazon's detection checks. The locale and viewport settings matter too: mismatches between the IP's geo and the browser's locale are a detection signal.
Once you have price data coming back cleanly, storing it is straightforward. The scheduling pattern below extends the RohanDas28/Amazon-Price-Tracker structure from Step 2. A simple CSV works for small-scale monitoring:
///
import csv from datetime import datetime
def save_price(asin, price, filename="prices.csv"): with open(filename, "a", newline="") as f: writer = csv.writer(f) writer.writerow([datetime.now().isoformat(), asin, price])
///
For scheduling, a simple loop with a delay keeps things lightweight:
///
import time import random
ASINS = ["B08N5WRWNW", "B07FZ8S74R", "B09G9HD6PD"]
for i, asin in enumerate(ASINS): session_id = f"monitor{i:06d}" price = get_price(asin, session_id) if price: save_price(asin, price) print(f"{asin}: {price}") time.sleep(random.uniform(3, 7))
///
The random delay between requests is not optional on Amazon. Perfectly timed intervals are a bot signal.
Rotating on every request burns IP reputation fast and breaks session continuity. Locking one IP across all your ASINs risks rate-limiting that single IP. The approach that works: one sticky session per ASIN per scrape cycle. You stay consistent within each product pull, and you spread load across many IPs across your full ASIN list.
If your proxy IP resolves to a residential address in Ohio but your Accept-Language header is set to German, or your viewport is a non-standard size, Amazon's detection layer notices the inconsistency. Keep your user agent, language headers, viewport, and geo all aligned.
If you're targeting US pricing, use a US-targeted IP and set en-US headers.
Delays between requests should be random, not fixed. A 3 to 7 second range per request is a reasonable floor. Anything faster on a sustained basis will trigger rate limiting. For scheduled runs, spacing full scrape cycles at least 15 to 30 minutes apart significantly reduces cumulative pressure on any given IP.
When you get a 403 or 429 response, do not retry the same IP. Rotate immediately to a new session ID, which means a new IP, and implement exponential backoff before retrying: wait 1 second, then 2, then 4.
Three consecutive failures on the same IP should trigger a longer cooldown or removal from the active rotation.
GoProxies' rotating residential proxy plan gives you a 30 million IP pool with a low fraud score, meaning fresh IPs you rotate into are unlikely to already be flagged when they arrive.
This is the issue that quietly breaks most Amazon monitoring setups. Providers advertise sticky sessions, but the actual behavior varies. Residential IPs in a backconnect pool have natural lifetimes tied to the real device they're sourced from.
If that device disconnects or rotates, the IP goes away and your session drops, even if you're still within the advertised session window.
What makes this worse is that the drop is often silent. Your scraper doesn't get an error. It gets a response, but the session cookie is now tied to a different IP, and Amazon may serve a degraded or blocked page.
You won't know until you see missing price data or empty fields downstream.
How to detect it: add a session health check before each product pull. Make a lightweight request to a session-stable endpoint, something like https://www.amazon.com, and verify the response matches your expected session state. If the IP has changed, generate a new session ID and re-establish before proceeding.
What to look for in a provider: explicit documentation on session TTL, a large enough pool that IP churn doesn't affect reliability, and consistent per-session behavior rather than best-effort stickiness. The longer you need to hold a session, the more the pool quality matters.
Amazon runs one of the more sophisticated anti-bot systems in e-commerce. It combines IP reputation checks, browser fingerprinting, behavioral analysis, and JavaScript-based detection to identify and block automated traffic. Understanding what it specifically looks for is the starting point for building a monitor that holds up over time.
Some Amazon products, particularly variants with multiple configurations, load their displayed price through JavaScript after the initial page response. A scraper relying only on static HTML will pick up either nothing or stale data from a default variant. The fix is to use Playwright as a fallback for any ASIN where your primary parser returns null.
Playwright and Selenium expose automation markers by default. The most common ones Amazon checks include the navigator.webdriver property being true, Canvas and WebGL rendering inconsistencies from headless Chromium, and synthetic input timing where mouse events fire at perfectly uniform intervals.
Disabling the automation flag in Playwright handles the first issue. Libraries like playwright-stealth patch several others. Behavioral randomisation handles the input timing signals.
Not all residential IPs are equal. A low-quality pool with high fraud scores means many of the IPs you rotate into are already flagged before your first request. You end up burning through IPs faster and seeing CAPTCHAs on initial requests more often.
Pool size matters too. A small pool gets exhausted quickly under scale, and Amazon starts seeing the same IPs repeatedly.
Immediate CAPTCHAs on the first request usually point to one of three things: a high-fraud-score IP, a bad TLS fingerprint, or mismatched headers. Run the IP through a fraud score checker first. If it flags as a VPN, datacenter, or previously abused address, the proxy pool quality is the issue.
If the IP comes back clean, the problem is more likely the request fingerprint. Using the standard requests library produces a non-browser TLS signature that Amazon flags before even evaluating the IP. Switch to curl_cffi with browser impersonation. Also verify your headers are consistent with the IP's geo: a US residential IP sending German Accept-Language headers is a mismatch Amazon notices.
If your parser returns None consistently for certain ASINs, check whether those products are variant listings. Open the product page manually and inspect whether the price loads on initial HTML or only after a JavaScript interaction. If it's the latter, route those ASINs through your Playwright function instead of the lightweight requests path.
If you're seeing corrupted or incomplete data from multi-step pulls, the session likely dropped mid-way. Add logging that records the IP address at the start and end of each product pull. A change between those two checkpoints confirms a mid-session rotation.
The fix is either shorter, more atomic pulls that complete before the session window closes, or a session health check that re-establishes the IP before each critical step.
A well-configured price monitor can extract more than just the listed price. Useful data points include current price and list price, the Buy Box winner and their seller rating, availability and stock status, promotional pricing and coupon amounts, ASIN and product title, customer rating and review count, and geo-specific pricing differences.
That last point matters more than people realise. Amazon shows different prices to different locations. If your monitoring covers multiple markets, you need proxies with city and ASN-level targeting to see what actual local buyers see.
GoProxies supports country, state, city, and ASN targeting, which makes regional price monitoring accurate rather than approximate.
Amazon's pricing data is publicly visible. Scraping publicly accessible information is generally considered legal in most jurisdictions, and multiple court decisions have reinforced that accessing public data through automated means does not constitute a violation of computer fraud laws.
That said, Amazon's Terms of Service prohibit automated access. Violating ToS doesn't carry criminal penalties, but it can result in your IP ranges being blocked or, in extreme cases, a cease-and-desist. Staying within reasonable request rates, not targeting private or account-restricted data, and using ethically sourced residential IPs keeps your operation on solid ethical ground.
Free proxy lists are not viable for Amazon. Amazon maintains blocklists of known datacenter IP ranges and low-reputation residential IPs. Free proxies are drawn from those ranges almost without exception.
You'll hit CAPTCHAs on the first request, see block rates high enough to make the data useless, and expose your scraper to unreliable infrastructure that logs your traffic.
The honest answer is that reliable Amazon price monitoring at any meaningful scale requires a paid residential proxy plan. The cost of a good plan is significantly lower than the cost of engineering around bad infrastructure. If you're monitoring even a few hundred ASINs regularly, rotating residential proxy pricing is the correct starting point.
Amazon offers two APIs relevant to price monitoring: the Selling Partner API (SP-API) for sellers managing their own accounts, and the Product Advertising API (PA-API) for Amazon Associates. Neither was built for competitive price monitoring across arbitrary ASINs.
The SP-API gives you structured access to your own seller data. As of early 2026 it carries an annual subscription fee plus monthly usage charges, making it a significant cost for teams that previously relied on it for data pipelines. The PA-API is free but rate-limited to one request per second by default, with limits tied to affiliate revenue generated, making it impractical for monitoring more than a handful of ASINs regularly.
The core limitation of both is that they do not return the same view a buyer sees. Regional price differences, promotional pricing, coupon stacking, and Buy Box dynamics are either absent or smoothed out in API responses. If you need what actual buyers see, scraping the live storefront is the only option.
Choose the API when you are a seller managing your own listings programmatically or building an affiliate tool within Associates program rules. Choose scraping when you need competitive price data across arbitrary ASINs at scale.
Amazon price monitoring is achievable at scale with the right setup, but it's not a set-and-forget operation. The core difficulty is not the scraping itself, it is maintaining reliable, undetected access across thousands of ASINs over time.
The setup that holds up: rotating residential proxies with sticky sessions locked per product pull, Playwright as a fallback for JavaScript-rendered prices, consistent fingerprint alignment across IP and browser properties, and active session health monitoring to catch silent drops before they corrupt your data.
The single most overlooked issue is sticky session inconsistency. Most proxy providers do not guarantee session lifetimes in the way the documentation implies. Build your monitoring system to detect and handle mid-session IP changes, and choose a provider with a large enough pool and a low enough fraud score that the IPs you rotate into are clean on arrival.
GoProxies Research Team is a group of analysts and writers focused on web data, online privacy, and proxy technology. They produce practical, research-backed content that helps readers understand digital trends, tools, and best practices.
The rich text element allows you to create and format headings, paragraphs, blockquotes, images, and video all in one place instead of having to add and format them individually. Just double-click and easily create content.
A rich text element can be used with static or dynamic content. For static content, just drop it into any page and begin editing. For dynamic content, add a rich text field to any collection and then connect a rich text element to that field in the settings panel. Voila!
Headings, paragraphs, blockquotes, figures, images, and figure captions can all be styled after a class is added to the rich text element using the "When inside of" nested selector system.
You need both. Rotating proxies assign a fresh IP for each new product pull to distribute load and avoid IP-based rate limiting. Sticky sessions hold the same IP for the duration of each individual product pull so that Amazon's session logic doesn't invalidate your cookies mid-request. Using only one mode creates either session failures or IP exhaustion.
Prices not appearing in your scraper usually means the product is a variant listing where Amazon loads the price via JavaScript after the initial page render. A requests-only parser cannot read dynamically loaded content. Route those ASINs through a Playwright-based function that waits for the page to fully render before extracting the price element.
Log the outbound IP address at the start and end of each product pull. If the two IPs differ, your session rotated mid-pull. You can retrieve your current IP by making a lightweight request to https://ip.goproxies.com or a similar IP-echo endpoint before and after each scrape task and comparing the results.
For most Amazon monitoring use cases, yes. Amazon maintains databases of known datacenter IP ranges and flags them more aggressively than residential IPs. Datacenter proxies are cheaper and can work for lower-sensitivity targets, but on protected Amazon pages they burn through faster. Residential IPs hold up better at scale.
