A zero is not a measurement

A couple of weeks ago I wrote about spending an evening convinced I had written a bug, when what had actually happened was that my browser was hiding my ads and telling me nothing. That piece was mostly about advertising. The advertising was the least interesting part.

The interesting part is that in the two weeks either side of it, I trusted three separate zeros. All three were confident, all three came from instruments I had every reason to believe, and all three were wrong in the same way.

The first zero

Brave Shields has a badge counter. It sits in the toolbar and tells you how many things it blocked on this page. On the page where my entire ad column had been removed, it said 0.

Not "1". Not "14". Zero. The browser was performing cosmetic filtering -- letting elements load and then removing them from the page a second later -- and its own counter, the one feature whose whole job is to tell you what it did, reported that it had done nothing.

I do not think this is malice. Badge counters generally count network-level blocks, and cosmetic filtering is not a network-level block: nothing was refused, so nothing was counted. It is a perfectly defensible implementation detail. It also means the number in the toolbar answers a narrower question than any user believes it answers.

The second zero

Every tool session I run on this machine emits a small event to a participation registry I built. It is a hook: a shell script that fires after a command, posts one event, and gets out of the way.

For eighteen days, it emitted nothing at all.

A configuration key had gone missing from an environment file during an unrelated edit. The hook did what a careful hook does when its configuration is incomplete:

[ -n class=class="rte-pro-str">"rte-pro-str">class="rte-pro-str">"$BOX_KEY" ] || exit class=class="rte-pro-str">"rte-pro-num">class="rte-pro-num">0

Exit zero. Success. No error, no log line, no warning, because the entire point of that guard is to fail quietly rather than spray errors into somebody's terminal on every keystroke. It is the line I would have written. It is the line I did write.

Here is what made it invisible for eighteen days rather than eighteen minutes. There was no local record of success. The system logged its failures diligently and never once recorded that anything had worked. So an outage and a quiet afternoon produced byte-for-byte identical evidence: nothing on disk, nothing in the log, no complaints. I only found it because I went and asked the remote service directly and it said the last event it had seen from this machine was eighteen days old.

A system that only logs failures cannot be distinguished from a system that is not running.

The third zero

This one is not a zero I saw. It is a zero I would have gone on not seeing, and it was going to cost real money.

My site pays writers a share of what the ad column beside their article earns. The counter that drove those payments incremented when the article's page was served with ads on it. I wrote that. I thought it was obviously right, and I even wrote a smug comment next to it about how it counted the same event the ad network was told about, so the two ledgers could never drift apart.

What it actually counted was a request arriving at my server. Not a person. Certainly not a person who saw anything. A crawler counted. A link-preview fetch from a chat app counted. A browser speculatively prefetching a link somebody hovered counted. A reader who opened the tab and closed it in half a second counted exactly as much as a reader who finished the piece.

None of those are edge cases. They are most of the internet. And every one of them would have been paid for.

What the three have in common

In all three cases the instrument was working correctly. It measured a real thing, accurately, and reported it honestly. The problem is that the thing it measured was not the thing I cared about, and nothing in the reading told me so.

• The badge counted network refusals. I wanted things removed from my page.
• The hook logged errors. I wanted evidence of delivery.
• The counter recorded requests served. I wanted ads a human could see.

Each pair looks like the same question from a distance. None of them is. And the gap between the question asked and the question meant is invisible precisely because the answer comes back clean. A broken instrument announces itself. An instrument answering a slightly different question sits there being reassuring.

The move that actually worked

I want to be specific about what broke the first case open, because it was not cleverness and it is repeatable.

I stopped asking what had failed and asked what had survived.

Inside the blocked column, one image was rendering perfectly: a small logo served from the ad network's own domain. Fourteen advertiser images, served from a separate host under a path containing the word "creative", were not. Same frame, same page, same moment. One category lived and one died.

That asymmetry is the whole diagnosis. A total failure tells you almost nothing, because everything is a candidate. A partial failure has already done most of your work: whatever explains the difference between the survivors and the casualties is your answer. Here the difference was the hostname and the word "creative" in the path, and there is exactly one class of software that cares about that.

The second clue was timing, and it is the one I would put on a wall. The creatives appeared, and about a second later they went away.

A network failure never paints. If a request is refused, nothing is ever drawn. Something that draws and then un-draws is not a delivery problem at all -- it is something watching the page and removing elements after the fact. The shape of the failure named the mechanism before I knew what the mechanism was.

What I do differently now

Record success, not just failure. The hook now touches a file on every event that actually lands. That single line converts "silence" from ambiguous into diagnosable, and the next time a key goes missing the answer is one ls away instead of eighteen days away. If a background job matters, it needs to leave proof of life, not just proof of death.

Surface staleness where a human already looks. Nobody checks a health file. So the session start now says, out loud, if nothing has succeeded in a fortnight. An alarm nobody is standing next to is decoration.

Measure the outcome, not the request. The impression counter is now a beacon the browser sends after the ads have genuinely been on screen, in a foreground tab, for one continuous second. This is roughly the industry's definition of a viewable impression, and I had to be dragged to it by the realisation that I was about to pay people for crawler traffic. Requests are easy to count, which is why so much tooling counts them.

When the instruments read normal and the thing in front of you is broken, believe the thing in front of you. I spent hours re-running diagnostics with increasing thoroughness because the alternative was accepting that all of my measurements were answering the wrong question. Re-checking an instrument feels like work. It is usually avoidance.

Get one signal from outside the system. Every check I had on the events hook was written by the same person, in the same afternoon, with the same assumptions baked in. Asking the remote service what it had actually received took thirty seconds and was the only thing that could have found it, because it was the only observer that did not share my blind spot.

The unglamorous conclusion

None of this is a debugging technique. There is no tool to install.

It is a habit of asking, of every number you are about to trust: what does this actually count, and is that the thing I mean? Three times in a month I did not ask, and three times a zero meant "nobody looked" while I read it as "nothing happened."

The badge counter is somebody else's code and I cannot fix it. The other two were mine.