--- title: "Five firewall rules beat the proxy swarm, but the real bug was my blind spot" canonical: https://dxdev.com/blog/2026-05-26_five-firewall-rules-blind-spot/ datePublished: 2026-05-26 --- When CPU pinned at 100% and stayed there for thirty minutes, I ran the log query I always run. It showed mixed load across customer vanity domains, nothing that screamed coordinated attack, because that query only reads one IIS site and that site was not where the attack was landing. ## The attack was real, and it was boring The operator was a German proxy reseller. Five /19 CIDR ranges across non-adjacent address space, roughly 100 to 200 unique IPs per range, each contributing 50 to 60 hits against the same URL structure. The traffic pattern was `/teams/?u=&s=` roster enumeration: rotate source IPs to dodge per-IP rate limiting, never send a Referer header, spoof a Chrome 134/135 User-Agent with no Sec-CH-UA field. Three signals identify a distributed scraper: asset mix (are requests hitting static files at a realistic ratio, or drilling one endpoint?), UA homogeneity (does the user-agent distribution look like a real browser population?), and Referer presence (real organic navigation almost always carries a Referer after the first hit). This operator failed all three: pure-ASP requests with zero static files, Chrome 134/135 across every IP, Referer absent on nearly every hit. Five firewall rules at the TCP layer, blocking at the /19 level rather than per-IP, dropped CPU to 13% and open connections from 490 to 146. Once I was reading the right log file, the fix was straightforward. Getting to the right log file was the hard part. ## The detector had a blind spot My `/hto-slow` skill surfaces a pre-captured incident bundle when I open a slowness investigation. The bundle pulls from `Watch-Site.ps1`, which tails IIS logs to surface slow requests and top-offending IP ranges, and I had relied on it for months. `Watch-Site.ps1` only tails one IIS log directory. There are three IIS sites on the box: one serves customer vanity domains, one handles direct `www.hometeamsonline.com` traffic, and one serves `media.hometeamsonline.com`. The script watched one of them, and I had never audited its coverage. When the investigation started, the bundle's `iis_slow.txt` showed mixed URL fan-out across customers on the customer-vanity-domain site, which looked like distributed volume load rather than a targeted swarm. The agent read it the same way, and that read was accurate: the attack was not touching that site at all. ## The question that opened it From my phone, mid-SSH-session, I sent: "are you checking both versions of the logs." The investigation pivoted immediately. We pulled the direct-traffic site logs. That log had 49,112 slow requests in it. The entire load was concentrated on the direct-traffic site. The customer-vanity-domain bundle had looked relatively benign compared to a normal traffic day because the attack was not touching it at all. The proxy reseller IPs were targeting `www.hometeamsonline.com` directly, not the vanity domains, and the vanity-domain log watcher had been faithfully telling me nothing was wrong. Five firewall rules later, the attack was gone. The monitoring gap was still there. ## What the process looks like after the fire is out The traffic fix is done. The practice work is what comes next. I hardened the `/hto-slow` skill with a CRITICAL warning that names all three sites and their log directories, placed at the top of the multi-site sweep step so the next investigation cannot treat the customer-vanity-domain bundle as comprehensive. I added two persistent memory entries. The first captures the multi-site sweep requirement, with descriptions of each site and the note about `Watch-Site.ps1`'s single-site scope. The second captures the proxy reseller's fingerprint: known ranges, URL pattern, UA signature, and the firewall naming convention. Memory entries are for agents and code changes are for the system, but both matter because entries can drift when the context resets. I logged two follow-ups. The first is the structural fix: rewrite `Watch-Site.ps1` to sweep all three sites and aggregate before producing a verdict. The second moves those IP ranges into `dbo.IPranges` so the application-layer filter can see them. Both exist in writing now so the work survives a context reset. ## Incident response is not done when traffic drops When CPU dropped to 13%, the site was fast again. I could have filed a one-line note and moved on. What the traffic drop exposed was a different question: why had this attack been invisible until someone on a phone asked about the second log? The answer was `Watch-Site.ps1`, pointed at one of three sites, trusted for months without ever being audited for coverage. The 49,112 slow requests in the direct-traffic site logs were there the whole time. `Watch-Site.ps1` just was not reading that file.