If scraping a website sounds like it should be illegal, the actual legal record tells a more complicated story. U.S. courts have repeatedly found that accessing publicly available web pages isn't the same as breaking into a computer system, which is a big part of why Amazon and other ecommerce sites allow data scraping to happen continuously, even while publicly opposing it. The short version: public data isn't locked, but that doesn't mean every use of it is automatically fine.
This is a plain-English breakdown of the legal landscape, grounded in real case law, not a legal opinion on your specific situation. If you're making decisions about a scraping program, talk to a qualified attorney in your jurisdiction. What follows is meant to explain the "why," not replace that conversation, and it draws on the same case law that shapes how major data companies structure their own compliance programs.
The short answer is that most product pages are public by design, and U.S. courts have consistently ruled that viewing public information isn't unauthorized computer access, even when a company would rather you didn't.
In hiQ Labs v. LinkedIn, a workforce analytics company scraped public LinkedIn profiles to build retention insights. LinkedIn blocked hiQ's access and argued this violated the Computer Fraud and Abuse Act, the main U.S. law governing unauthorized computer access. The Ninth Circuit Court of Appeals disagreed, ruling that accessing data that doesn't require a login isn't "without authorization" under the CFAA, even if the site owner objects. That ruling is a major reason scraping public data isn't treated as a computer crime in the U.S.
The story doesn't end with hiQ winning outright. The case later turned on a separate issue: LinkedIn's Terms of Service, which hiQ had agreed to, explicitly prohibited scraping. The court found hiQ breached that contract, and the case ended in a consent judgment against hiQ. The lesson that stuck: scraping public data isn't hacking, but a Terms of Service agreement you've accepted is still an enforceable contract, and violating it carries its own legal risk separate from the CFAA question.
A similar pattern played out when Meta sued Bright Data over scraping public, logged-out Facebook and Instagram data. The court sided with Bright Data, reasoning that scraping while logged out meant Bright Data hadn't agreed to and therefore hadn't breached Meta's user terms in the way LinkedIn's case turned on. Meta dropped the suit. Together, these cases point to a consistent theme: public, logged-out data scraping keeps winning on the core legal question, while ToS and account-based agreements remain the sharper edge businesses need to watch.
A separate Supreme Court case, Van Buren v. United States, reinforced this direction from a different angle. The Court held that the CFAA's "exceeds authorized access" language applies to accessing files or areas a person isn't permitted to access at all, not to misusing access they otherwise legitimately have. While that case involved a police officer misusing database access rather than scraping specifically, legal commentators widely view it as reinforcing the narrower reading of the CFAA that hiQ established, making it harder to stretch the statute to cover routine public data collection.
None of this means anything goes. Several categories consistently create real legal exposure, regardless of how the CFAA question shakes out.
A few beliefs about this topic circulate widely but don't hold up against the actual case law. "If it's technically possible to scrape, it must be legal" skips over the Terms of Service question entirely, as hiQ discovered. "Scraping is always illegal without permission" overstates the CFAA risk for public, logged-out data, contradicting the core holding in hiQ. And "robots.txt is legally binding" confuses a voluntary courtesy signal with an enforceable rule. The honest position sits between these extremes: mostly permitted for public data, genuinely restricted in specific, well-defined situations.
A quick side-by-side of common scenarios makes the pattern more concrete.
See how Xwiz collects only publicly available ecommerce data, with compliance built into every project.
Get a Free Data SampleA lot of confusion in this space comes from conflating two separate questions: is something against the law, and does a specific site's Terms of Service prohibit it. They're related but not the same question, and understanding the gap explains a lot about why ecommerce sites allow data scraping in practice while still publishing policies against it.
Most sites publish a robots.txt file indicating which parts of the site they'd prefer automated tools avoid. This isn't legally binding on its own, but it functions as a clear signal of the site owner's preference, and responsible scraping practices generally respect it as a baseline courtesy, similar to following posted rules in a public space even without a locked gate. Courts have occasionally referenced a party's disregard for robots.txt as part of a broader pattern when evaluating good faith, even though the file itself carries no independent legal force.
When scraping requires creating an account or accepting a click-through agreement, that agreement is a contract. Violating it, as hiQ eventually learned, creates real legal exposure distinct from computer-access law, even when the underlying data being scraped is public. This is why scraping that stays logged out, viewing pages the way any visitor could without an account, sits on firmer ground than scraping that requires accepting account terms first. The distinction between "browsing a public page" and "using a service you signed up for" turns out to matter enormously in how courts have ruled.
Publishing a policy against scraping serves a purpose even if enforcement is inconsistent: it strengthens a site's legal position if it ever needs to pursue a ToS breach claim, and it's a prerequisite for sending a credible cease-and-desist. Amazon, like most major marketplaces, maintains such policies while public product data remains broadly scraped across the industry, including by services that power price comparison tools consumers use every day. The policy and the practice coexisting isn't a contradiction, it reflects the actual legal landscape described above.
Understanding the legal landscape translates into a practical set of habits that keep a scraping program on solid ground.
Sticking to publicly viewable pages, without logging in or accepting account-specific terms, avoids the contract risk that sank hiQ's case. Avoiding personal data, names, emails, private profile information, sidesteps GDPR and similar regulations entirely, since the risk there hinges on the nature of the data, not the scraping method. For ecommerce specifically, this generally means staying focused on product, pricing, and availability data rather than customer accounts or reviewer profiles.
Honoring robots.txt, pacing requests reasonably, and backing off after a block rather than working around it aggressively all reduce legal exposure and reflect good-faith practice, the kind of behavior that distinguishes routine data collection from the more aggressive patterns that tend to draw legal attention. A scraper that behaves like a considerate visitor, rather than a battering ram, is both more legally defensible and, in practice, more reliable over time.
Extracting facts, a price, a stock status, a rating, is treated differently than copying and republishing a site's original written content, images, or creative descriptions wholesale. Facts generally aren't copyrightable; the specific creative expression describing them can be. A price comparison tool that lists "Product X: $49.99" is on different legal footing than a site that copies an entire product description word for word.
Xwiz Analytics builds every ecommerce data scraping project around the same principles outlined above: only publicly available data, no login-gated or personal information, and respect for robots.txt and reasonable request pacing across every platform covered.
This approach applies consistently across more than twenty marketplaces through Xwiz's ecommerce industry scraping, so businesses can rely on the data without separately auditing how it was collected. For the fundamentals of what ecommerce data scraping covers day to day, see our complete guide to ecommerce data scraping.
None of this is a substitute for a business's own legal counsel, especially for anything involving personal data, a specific jurisdiction's regulations, or an unusual use case. It's the baseline practice that keeps a compliance-first program aligned with where the law has consistently landed.
Talk to Xwiz about an ecommerce data feed built on publicly available data from the start.
Talk to Our Data ExpertsU.S. courts have held that viewing publicly available web pages isn't unauthorized computer access, so blocking scraping entirely isn't a simple legal matter even though sites can still prohibit it contractually through their Terms of Service. The policy and the widespread practice of scraping coexist because they address two different legal questions.
It decided two separate things: that scraping publicly available pages isn't a CFAA violation, but also that hiQ breached LinkedIn's Terms of Service by scraping, which it had agreed to. Both parts of the ruling still shape scraping law today.
Scraping publicly available product pages without logging in generally sits on firmer legal ground under current U.S. case law, though Amazon's own Terms of Service still prohibit scraping, which creates separate contract-based risk. This is general information, not legal advice for a specific situation, and businesses running large-scale programs should confirm their approach with counsel.
No, robots.txt is a voluntary signal, not a legally binding block. Responsible scraping practices generally honor it anyway, since ignoring an explicit site preference can factor into how a dispute is viewed later.
GDPR applies specifically to personal data, names, emails, individual identifiers, regardless of whether that data was publicly visible. Product data like price, stock status, and category generally falls outside GDPR's scope entirely.
Continuing to scrape after an explicit written notice significantly weakens any good-faith defense and increases legal exposure, since it removes the argument that access was reasonably believed to be permitted.
Yes, particularly for anything involving personal data, unusual jurisdictions, or large-scale commercial use. General information like this explainer is a useful starting point, not a substitute for advice from a qualified attorney on a specific situation.
The reason Amazon and other ecommerce sites allow data scraping to continue, despite publishing policies against it, comes down to a real legal distinction: U.S. courts have consistently found that viewing publicly available data isn't unauthorized computer access. That's not the same as saying anything goes. Terms of Service create separate contract risk, personal data stays regulated regardless of visibility, and bypassing explicit technical blocks changes the legal picture considerably.
The practical takeaway is simpler than the case law: stay logged out, stick to publicly visible data, avoid personal information, and respect the signals a site publishes. That combination has consistently held up as the difference between routine, low-risk data collection and the kind of aggressive scraping that ends up in court. None of this replaces a conversation with a qualified attorney for a specific program, but it's a reasonable starting framework for understanding why the practice exists at the scale it does.
For questions about how Xwiz applies these principles to a specific project, our team is a message away.
Let Xwiz's data experts build an ecommerce data feed grounded in publicly available data and responsible practices.
Start Your Data Project →