Perplexity accused of scraping websites that explicitly blocked AI scraping
UNITED STATES, AUG 4 – Cloudflare found Perplexity ignored website blocks on over 10,000 domains using disguised bots and IP rotation to scrape content against site owners' preferences, prompting de-listing and blocking.
- Cloudflare accused the AI startup Perplexity of ignoring website blocks and hiding its crawling and scraping activities.
- Perplexity allegedly used tactics like changing its user agent and autonomous system networks to circumvent website preferences and continue scraping data.
- While Perplexity denied accessing content, Cloudflare claimed its bots made millions of site requests daily after being blocked, impersonating Google Chrome.
Insights by Ground AI
Does this summary seem wrong?
57 Articles
57 Articles
Cloudflare vs. Perplexity: a web scraping war with big implications for AI
When the web was established several decades ago, it was built on a number of principles. Among them was a key, overarching standard dubbed “netiquette”: Do unto others as you’d want done unto you. It’s a principle that lived on through other companies, including Google, whose motto for a period was “Don’t be evil.” The fundamental idea was simple: Act ethically and morally. If someone asked you to stop doing something, you stopped—or at least c…
Coverage Details
Total News Sources57
Leaning Left4Leaning Right0Center9Last UpdatedBias Distribution69% Center
Bias Distribution
- 69% of the sources are Center
69% Center
L 31%
C 69%
Factuality
To view factuality data please Upgrade to Premium