If your web analytics feel a little… off this year, you’re not imagining it. Automated “bad bots”, scripts that pretend to be people, are surging. They scrape your content and prices, stuff stolen passwords into your login, hoard inventory, hammer your APIs, and blow up your ad budgets with fake clicks.
-
Bad bots now make up ~37% of all internet traffic, the sixth straight yearly increase. For the first time in a decade, automated traffic surpassed human traffic (51%) across the web.
-
Crawler traffic, especially AI crawlers, jumped sharply year over year; Cloudflare measured a 305% rise in GPTBot requests from May 2024 - May 2025.
-
In advertising, the waste adds up: the ANA estimates $26.8B in programmatic spend was wasted (Q2 2025 study) - much of it tied to invalid/bot traffic and opaque supply paths.
Bottom line: a big chunk of your "visitors" aren’t customers. They’re bots - some harmless, many harmful.
-
Budget drain: Bots click ads, visit landing pages, and trigger remarketing - without ever buying. That skews CAC, ROAS, and channel mix decisions.
-
Broken analytics: Fake traffic inflates sessions and bounce rates, lowers time on page, and muddies A/B tests. You make decisions on sand.
-
Revenue leakage: Scalper and scraper bots copy your catalog and pricing, beat you in comparison engines, and empty carts to block real shoppers.
-
Brand & SEO risk: Heavy scraping can strain your servers and slow pages. Search engines notice. So do customers.
A Web Application Firewall (WAF) sits in front of your site and APIs. It inspects requests and filters what shouldn’t get through - blocking attacks like SQL injection, XSS, credential stuffing, bot swarms, and abusive crawlers before they hit your app. Think of it as a smart gatekeeper for all web traffic.
Modern WAFs also add:
-
Bot management: Detect & mitigate malicious automation while allowing good bots (search engines, uptime monitors).
-
Rules tuned for AI crawlers: Cloudflare, for example, now blocks many AI scrapers by default and offers tools that trap evasive scrapers.
Answer these with your analytics and ad platforms open:
-
Weird spikes: Did you see sudden traffic surges with no campaign, PR, or seasonality reason?
-
Odd engagement: Sessions rise while time on page plummets and bounce skyrockets? (Bots often "hit and quit.")
-
Source anomalies: New/referral sources or network domains you don't recognize (and can’t tie to partners)?
-
Login turbulence: Spikes in failed logins or password-reset emails (classic credential stuffing signal).
-
Ad leakage: More spend, flat conversions, suspicious geos or placements, or "too perfect" pacing vs. outcomes? (IVT/invalid traffic).
If you answered "yes" to two or more, it’s time to tighten the perimeter.
- Turn on/upgrade your WAF
- Use a managed WAF (Cloudflare, Fastly, etc.) and enable the OWASP/core rule sets. Start in "log/monitor" mode for a few days, then enforce.
- Add bot management policies
- Allowlist good bots (Googlebot, Bingbot, uptime pingers).
- Challenge or block high-risk automation (data center ASNs, headless browsers, known bad signatures).
- Rate-limit endpoints like /login, /cart, and key APIs.
- Tame AI/web crawlers
- Decide your stance on AI training. If "no," enable default AI-crawler blocking and consider deception tools (honeypots/“labyrinths”) to fingerprint violators.
- Clean your measurement
- Filter obvious IVT in GA4; exclude suspicious domains/sources; segment by ASN and user-agent. Re-baseline KPIs after bot controls.
- Protect ad spend
- Use pre-bid/IVT filters with your DSPs. Review supply paths. Compare platform conversions with CRM orders to spot "phantom" traffic. (ANA’s findings show the dollars at stake.)
-
We proxy traffic through a security provider (Cloudflare, Fastly, Imperva, etc.).
-
We can see and edit WAF rules (SQLi/XSS/credential-stuffing protections).
-
We have bot policies that distinguish good vs. bad automation.
-
We rate-limit abuse-prone endpoints (login, search, add-to-cart, APIs).
-
We’ve decided our AI-crawler policy and configured it.
If your team can’t confidently check these boxes, you probably don’t have full protection.
Is a WAF the same as my hosting firewall?
Not quite. Server firewalls block by IP/port. A WAF understands web traffic itself and stops application-layer attacks and bots, things a basic firewall can’t see.
Won’t a WAF block Google?
Good WAFs allow good bots (Google, Bing) and challenge the bad ones. You stay crawlable while cutting abuse.
We don’t sell online. Do we still need this?
Yes. Content sites get hammered by scrapers and credential stuffing too, and they rely on clean analytics and SEO.
Bots are bigger, faster, and increasingly AI-augmented. The cost isn’t just technical, it’s marketing efficiency, customer experience, and brand trust. A modern WAF with bot management is now table stakes. Start with visibility, then enforce smart controls.