The store was telling Googleto delete its own catalogue,43,080 times a day

Missing pages were answering search engines with "410 Gone."

A 404 tells Google "this page is missing, check again later." A 410 tells it "this page is permanently deleted, remove it and do not come back." Google acts on a 410 much faster and much more permanently.

Cloudflare's own 24 hour figures showed 43,080 of them. On a catalogue where product URLs change often enough to have accumulated 2,437 redirect rules, any URL that fell outside that table for even a short window was being permanently dropped from search rather than retried.

A Cloudflare cache rule was inverted.

The first rule in the list was named "Never Cache HTML." Its conditions matched .jpg, .png, .webp, .css, .js, .woff, .svg and .ico. Its action was set to bypass cache. It matched no HTML at all. It matched every image, stylesheet, script and font on the site, and because it sat in position 1 it overrode everything beneath it.

The result: a 19 GB image library being fetched from an origin server in Mumbai on every request, for every visitor, in every country. Cache hit rate was 8.6%. It should be 60 to 80%. The Cloudflare Pro subscription was being paid for and delivering almost none of its benefit.

The origin server was failing under the load it should never have been carrying.

The 408s were the errors customers were actually noticing. They were a symptom, not the disease.

Three more, each blocking real people or real crawlers.

Bing's crawler was in the firewall's AI scraper block list, sitting alongside GPTBot and CCBot. bingbot is a search engine, not a training crawler. Blocking it had removed the store from Bing, DuckDuckGo and Copilot entirely.

Every visitor outside Lebanon met a CAPTCHA the moment they touched add to cart. Lebanese diaspora buying appliances for family at home is a real customer segment, and they were being challenged at the exact point of purchase.

And one redirect sent everyone searching for an electric party grill to a beige L-shape sofa. 92 recorded hits, still being followed that month.

Nine fixed-price items, quoted at $460 as a package, delivered between 31 July and 16 August.

Missing pages now return 404. Verified by direct request and, more conclusively, by the disappearance of "410 Gone" from the site's top response codes entirely.

The cache rule set was rebuilt. Cart, checkout, account pages and any session with something in the basket are never cached, and that was tested directly rather than assumed. In the first 24 hours, data transferred fell 39.24% and total requests fell 24.98%, while page views rose 3.43% and visits rose 7.95%. Less served, more delivered. The 408 timeouts stopped appearing.

bingbot came out of the block list, genuine AI training crawlers stayed in. The add-to-cart CAPTCHA was switched off for international customers with bot protection left intact everywhere else.

Then the parts nobody had quoted, because nobody knew they were there:

A Google Shopping feed that had not refreshed since 1 May.

It was still advertising 988 products that no longer existed. A customer clicking a product in Google Shopping had roughly a one in eight chance of landing on an error page. Rebuilt to 7,729 products and zero dead ones.

286 broken redirect rules.

All repointed to live category pages and then tested by following each one the way a customer would. All 397 rules anyone had ever actually followed now land on a working page, covering 52,263 recorded visits. Nothing was deleted. The table was later consolidated from 2,438 active rules to 403.

1.15 million URLs discovered by Google on a store with 7,876 real products.

Almost all filter combinations, and some of them add-to-cart addresses, which meant Google was repeatedly triggering cart actions instead of reading pages. The crawl instruction file had a rule that did not match the addresses actually in use. Corrected, then checked against all 8,455 real pages to confirm not one product or category was affected.

Order notification emails that had never worked.

Not once in three years. The store was configured to email every new order to the billing address and the setting was switched on, but the mailbox had never received one. The proof is two orders an hour apart on 14 August:

Same store, same day, either side of the fix. Re-confirmed two days later on a real customer order, not a test.

Two separate fixes went in correctly, were correct on the server, and were still invisible to Google afterwards. Cloudflare kept serving the cached copy from months earlier.

That is why a three and a half month old product feed went unnoticed by everyone, including the people looking at it. On this site, "we changed it and it looks right" was not evidence that anything had reached a customer or a crawler.

That single habit is what surfaced most of the list above.

One item was deliberately left off that free list. During an automated traffic flood I added an emergency rule that kept the server standing, and it worked, but it did not carry an exception for verified search crawlers. 292,342 pages started refusing Google. The emergency measure was necessary. The gap in it was mine. Diagnosing it, fixing it and submitting the revalidation to Google was roughly $90 at the same rates, and it appears nowhere on the invoice, because fixing my own oversight is not generosity and should not be presented as though it were.