Home › Work, Careers & Leadership

Excessive AI bot traffic causing high hosting costs

🧭 workproduct problemSingle report ●●●●● severity 4/5

A web app developer reports that their site is being overwhelmed by sophisticated AI scrapers using residential proxies, which bypass standard security measures like Cloudflare Turnstile, leading to significantly increased hosting costs and unauthorized data access.

Who's affected: A web app developer hosting 50,000 statically generated public pages.

⚠️ Single report. This is one person's account. We mark it so until more voices confirm it.

What people do today

🔧 Fix this

Ideas a builder, maker or community group could start on:

  1. A specialized middleware service that uses behavioral analysis to detect and block residential proxy traffic patterns in real-time.
  2. A reputation-based API that allows site owners to query if an incoming IP address is associated with known AI scraping botnets.
  3. A dynamic content delivery system that serves obfuscated or low-resolution data to unverified users while reserving high-quality data for authenticated human sessions.
I want to build this

Real voices

“the amount of bot traffic is insane. Facebook’s crawler requested the same pages over 3 million times in 2 days”

Source: Hacker News

Related problems

🧭 workproduct6 reports

Inability to filter AI-generated content on LinkedIn

●●●●● · A LinkedIn user and possibly others who find their professional feed cluttered with AI-generated content.
🧭 workproduct6 reports

Indeed job feed ignores user preferences

●●●●● · An Indeed job seeker and possibly others experiencing irrelevant job recommendations
🧭 worksocial6 reports

Difficulty setting boundaries with talkative senior colleague

●●●●● · An employee with a close, long-term relationship with a senior colleague.

FixHumanityDaily

The most important new problems, and ideas to fix them, in your inbox.

Placeholder: no data is collected yet.