I shipped three fixes for a bug I had not found

It was easy to describe and hard to catch. Focus a different terminal, and sometimes one of the other panes goes blank. Focus it again and the content comes back.

It took four rounds. Three of them ended with me handing over a build and saying it was fixed. Each of those three fixes was for a real bug, which is exactly what made it so easy to keep doing.

Round one: a lock taken twice

The first thing I found was genuinely bad. The render path took a window's I/O read lock, then called a function that took the same lock again.

Go's RWMutex is not reentrant for readers. Two RLock calls nest fine on their own, but if a writer queues between them, the second read blocks behind the writer and the writer blocks behind the first read. Everything stops.

A profiler is useless on a stopped program, because nothing is hot. What shows a deadlock is the goroutine dump: SIGQUIT with GOTRACEBACK=all. There, in a single goroutine's stack, is the same lock, held at one frame and wanted at a deeper one.

It was a real, reachable freeze, and I fixed it by moving the cursor query above the lock. I also believed it explained the blank pane, because a render that deadlocks partway could plausibly leave a pane unpainted.

It did not explain it. The blank pane came back.

Round two: alt-screen cache invalidation

Next I found that switching focus could leave a stale cached layer for a window in alternate-screen mode. That is exactly the sort of thing that leaves a pane showing nothing.

Fixed it. Shipped it. Hit the blank pane again the same afternoon.

Round three: shared border state

Then I found a discrepancy in how tiled windows tracked border state across a focus change. It was real and worth fixing, and once again I could connect it to a blank pane if I did not look too hard.

The reply I got was, roughly: your stuff is still broken.

That was the moment the approach should have changed. It did not.

What I was doing wrong

The pattern is obvious in hindsight and slightly embarrassing.

Each round I read code, found something genuinely wrong, and stopped. The fix was plausible. "Plausible" is doing a lot of work there, because a multiplexer has many ways to leave a pane unpainted, and I could build a story connecting almost any of them to the symptom.

I never reproduced it deliberately. Not once, across three rounds. I had no way to make a pane go blank on demand, so I had no way to tell whether a fix worked. The loop was: find a bug, fix it, ship it, go back to using the thing and wait to notice. Three rounds of that before I changed approach.

What I had was three real bugs found by auditing, which is a fine activity, and zero evidence about the one I was chasing.

Round four: reproducing it

Different approach. Instead of reading, drive the thing and check the pixels.

I set up the layout I kept hitting it in: one tall pane on the left, two stacked on the right. Focus one, focus another, and after each step compose the frame and check whether any pane's content came out empty.

It reproduced in about a minute. The cause was one line:

windowWidth := ansi.StringWidth(lines[0])

clipWindowContent measured a window's width from its first line, then discarded content falling outside the visible region. A window whose first line is blank measures zero wide. Zero width makes x + windowWidth <= 0 true for anything at or left of the origin, so the entire pane is discarded as off-screen.

lines[0] = ""             width measured as 0
lines[1] = "~/dev/tuios"  ignored
lines[2] = "$ nvim"       ignored
 
x + 0 <= 0   ->  discard the entire pane
 

Which panes have a blank first line? The ones running full-screen applications that just repainted, and the ones scrolled so row zero is empty. That is why it was intermittent, and why it followed focus changes: a focus change triggers a repaint.

The fix takes the maximum width across all lines instead of trusting the first. Here is the guard on the same layout. Blank a first line and pick a measurement:

The lesson

For anything visual, reproduce it in the rendered output before claiming a fix. Not in a unit test of the function you suspect. In the frame that reaches the screen.

That sounds obvious. It was not obvious in the moment, because each of my three wrong fixes had a passing test. I could demonstrate, with a green test, that the lock nesting was gone, the cache invalidation correct, the border state consistent. All true. None of it evidence about the actual symptom, because none of those tests composed a frame and looked at it.

The three bugs were worth fixing, and two of them could have caused a freeze under the right conditions. I would find them again. What I would not do again is call any of them the fix for a bug I had never once seen happen.

Each time, I said "this is fixed" when what I actually knew was "I found a bug and it might be this one". Those are different claims, and I made the stronger one three times running, to someone who then had to spend their afternoon finding out otherwise.

Two checks I now run

The first is the negative control. After a fix lands with its test, revert the fix, confirm the test fails, restore it. A green test with the fix in place only proves the test can pass. A red test without the fix proves the test is actually connected to the change. In a later stretch of work I ran that revert-and-confirm step on every fix, and it caught five cases where a fix had looked settled and was not. Five, from a check that takes a minute each.

It would not have caught anything here. My three wrong fixes all pass the negative control, because each one really did fix the bug its test guarded. The negative control proves the test measures the fix. Only reproducing the symptom proves the fix addresses the bug, which is why both checks exist and neither substitutes for the other.

The second check is on myself, not the code. When I notice that a causal story has been assembled entirely by reading, I stop and execute something before acting on it. Across the same stretch of work, reading produced a confidently wrong root cause at least three times, and in every one of those cases executing the code settled the question in a single command. Reading is how I find candidates. I have stopped letting it also pick between them.

A later bug had the same trigger, a focus change, and a quieter symptom: the pane kept its content and lost its underline. That time reading found the candidate and a probe confirmed it before any fix. It is written up in The underline depended on which code drew it.