---
title: "Cloudflare Rulesets API and the 2am bot swarm"
canonical: https://dxdev.com/blog/bot-swarms-moved-faster-than-my-firewall/
datePublished: 2026-07-05
---
The monitoring service phoned us at 2am, and for a stretch this spring it did that nearly every day, whenever the swarm decided to show up.

The pattern never changed. Bot traffic would surge, the site would crawl or fall over, the phone would ring, and we'd be at a keyboard writing a firewall rule on the production server while paying customers couldn't load their pages. By the time our rule was live, the swarm had usually rotated to new addresses and new patterns. We'd go back to bed knowing the rule we just shipped was already obsolete.

The server was not unprotected. We had many layers of [bot protection](/blog/upstream-filtering-outsized-metric-impact/) built up over years, and they had handled everything before this. The latest generation of swarms went through them like they weren't there.

## getting faster at the wrong thing

For longer than we'd like to admit, our answer was to get better at the scramble. All of it amounted to solving the problem on the same box that was under attack.

That has two failures baked in. The first is speed. Deploying anything to a server that is currently drowning is slow, and slow loses against an attacker that rotates in minutes.

The second is architectural. A [firewall that lives on the serving box](/everyday-ai/night-i-stopped-fighting-at-the-door/) only acts after hostile traffic has already consumed a connection, a thread, and a slice of CPU on the exact machine we're trying to protect. Even a perfect rule, deployed instantly, pays that toll first. The block fires, but the request has already cost us the thing we were defending. We spent our nights optimizing a response that was structurally too late, and the swarms kept proving it. Every pattern we caught was gone within a few hours of catching it.

## moving the wall

The fix was a project we'd deferred for years because it was never urgent until it was. We put Cloudflare in front of everything and moved enforcement there. Traffic decisions now happen upstream, on infrastructure that isn't ours, isn't under attack, and is reachable through an API.

The API part matters more than it sounds. Before, mitigation meant logging into the wounded box. After, it is a call to Cloudflare from a laptop nowhere near the fire:

```bash
# block a rotating swarm by fingerprint, from a healthy machine
RULESET=$(curl -s "https://api.cloudflare.com/client/v4/zones/$ZONE/rulesets/phases/http_request_firewall_custom/entrypoint" \
  -H "Authorization: Bearer $CF_TOKEN" | jq -r '.result.id')

curl -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE/rulesets/$RULESET/rules" \
  -H "Authorization: Bearer $CF_TOKEN" \
  -d '{"action":"block","description":"rotating swarm fingerprint",
       "expression":"not cf.client.bot and http.user_agent contains \"<swarm-ua>\""}'
```

Two details in that expression matter more than the rest of the post. `not cf.client.bot` does real work. That field is true for verified crawlers, so leaving it out blocks Googlebot along with the swarm. And this is the Rulesets API, not the old Firewall Rules endpoint, which Cloudflare has since retired.

That rule is live at every Cloudflare edge in well under a minute, and none of those seconds are spent on our server. During an incident we now describe the pattern to our AI agent. It writes the expression, pushes it, and confirms the block took effect, all from a machine that is perfectly healthy. The 2am routine of hand-editing rules on a wounded server has become a sentence typed into a terminal, sometimes from a phone.

Migrating an old platform with thousands of customer domains onto Cloudflare was its own saga, and the hardest part is still in progress. That story is its own post. The enforcement move alone changed what an attack feels like. We handle it in a few minutes and it's forgotten by morning.

## what is still uncovered

If a flood hit right now, would stopping it require touching the machine being flooded? For years the answer was yes, and that was the whole bug. The box you must log into was the box on fire.

Cloudflare moved our rules off the machine we were trying to save. The swarms still come, near-daily some weeks, but the migration that made this survivable is only half done. Thousands of legacy domains still need to move behind the edge before every one is covered. So we keep a list of which are safe and which aren't, and we check it every time the phone rings.