# My fuzzer was optimising for my own bug URL: https://tuios.dev/blog/fuzzing-a-terminal-test-harness > A flaky TUI fuzz suite came down to a PTY line discipline I never configured, and a shrinker that minimised straight toward the race. TUIOS is a terminal window manager, so almost everything it does is visual and almost none of it is easy to assert on. A pane moves one column. A border picks up the focus colour. A wide glyph either fits its cell or eats a column of its neighbour. You cannot reach any of that from a Go test, because the thing under test only exists as bytes on a pseudo-terminal that some other program is supposed to interpret. So I wrote [tuitest](https://github.com/Gaurav-Gosain/tuitest). It spawns a program on a PTY, drives it, keeps a terminal emulator's worth of screen state on the other side, and lets you assert on what the program drew. TUIOS's own [end-to-end suite](https://tuios.dev/docs/contributing#other-test-suites) runs on it. Once that worked, writing the input by hand started to feel silly. ## Fuzzing a TUI A TUI has a wide input surface and nothing much protecting it. Keystrokes, mouse reports, resizes, bracketed pastes, text whose width the program has to get right. The bugs I ship in this area are almost never logic errors in a handler. They are combinations. A resize during a drag. A paste with a control character in it. A grapheme cluster that one layer measures as one column and another measures as two. ``` tuitest fuzz -- ./myapp tuitest fuzz -seed 42 -iterations 200 -corpus testdata/fuzz -- ./myapp ``` Everything the generator emits is expressible as a tape command. The text corpus is deliberately nasty: ASCII mixed with accented Latin, CJK, ZWJ emoji, regional indicators, combining marks, zero-width and bidi overrides. Width handling is where layout code usually goes wrong. `Ctrl+c` is in the key set even though it usually quits the program. That is on purpose. The check for "did this program restore the terminal on the way out" can only run against a program that has exited, and leaving someone's terminal in raw mode with the cursor hidden is the single most common TUI bug there is. A generator that never quits anything can never find it. `Ctrl+z` is excluded, because suspending a child under a PTY just wedges the run. When a run fails, the fuzzer shrinks the input and writes a tape. Candidates replay through the same tape player `tuitest run` uses, so the file you get is not a description of what the fuzzer did. It is the thing itself, and it runs with no fuzz-specific machinery involved. ## Then the fuzz suite started failing Intermittently. Never the same test twice, and only when the whole package ran under `-race`. Both failure modes had the same shape. Something fails once, then refuses to reproduce: ``` fuzz_test.go:314: the corpus entry should still reproduce on replay, got: no failures fuzz_test.go:188: the minimised reproduction did not reproduce on confirmation ``` The obvious reading is that the fuzzer's bookkeeping is broken. A failure recorded against the wrong input, or shrinking losing the case somewhere. I spent most of an evening inside the shrinker on that theory. Before going further I wanted a number, because "sometimes" is not a bug report. Twelve sequential runs, then twenty-four across four parallel lanes: *[An interactive figure goes here. Open the page to use it.](https://tuios.dev/blog/fuzzing-a-terminal-test-harness)* No `DATA RACE` ever fired. Not once, in any of those runs. `-race` mattered here because it slows the process down and reshuffles scheduling, not because it detected anything. I now reach for it to perturb timing about as often as I reach for it to find races. ## Reading the tape I had been ignoring The shrinker had been reducing failures to two commands: spawn the program, send a burst of `^C`. I had written that off as a degenerate case. Run those two commands twenty times, outside the package, and it fails four times. Not zero, not twenty. On the failures the harness reported `bytes=2, dirty=false, status="killed by interrupt"`, and the captured screen contained this: *[An interactive figure goes here. Open the page to use it.](https://tuios.dev/blog/fuzzing-a-terminal-test-harness)* tuitest created a PTY and never configured it. A fresh PTY comes up in the kernel's default cooked mode, so every byte written to the master went through the line discipline before it reached the program. `0x03` raised SIGINT and killed the child. `0x13` stopped output through flow control and could hang a session indefinitely. Everything the harness typed was echoed back down the master and arrived at the screen model as though the program had printed it, which quietly corrupted both the screen state and the byte counter the hang detector reads. You can reproduce all three by hand: *[An interactive figure goes here. Open the page to use it.](https://tuios.dev/blog/fuzzing-a-terminal-test-harness)* Real TUIs hide this, because they call `MakeRaw` on startup. That is why it survived so long. If your program reaches raw mode before the first byte lands, everything works. It is a race, and any input sent during terminal setup still belongs to the line discipline. Whether a tape drove a program or killed it came down to how fast the child got scheduled, and under `-race` with the rest of the package running, it got scheduled slower. ## The shrinker was helping the bug The shrinker is delta debugging. Remove chunks in decreasing sizes, simplify what survives, and accept a candidate if it still fails. "Still fails" meant failing on one replay. That criterion is fine for deterministic failures. For a race it is actively harmful. Every command that takes time gives the program more time to reach raw mode, which makes the race harder to lose, which makes the candidate less likely to fail. So the commands that suppress the bug are precisely the ones the shrinker deletes first. *[An interactive figure goes here. Open the page to use it.](https://tuios.dev/blog/fuzzing-a-terminal-test-harness)* The tool was optimising toward its own defect, and the tape it handed me was not noise. It was the shortest path to the bug, written to a file, which I had skimmed and dismissed for two days. If your minimiser accepts on a single execution, it is a search for flaky inputs. It will find them whether you asked for that or not. ## Fixing it, and the line I nearly crossed Configure the PTY before the child exists, so there is never a moment where a signal-generating line discipline sits in front of a running program. The real decision is how much to clear. My instinct was `MakeRaw`, and it was wrong. tuitest tests what programs do under a real terminal, and a real terminal does apply line editing to a program that has not gone raw yet. If the harness strips all of that, it stops reproducing the environment it exists to reproduce. So it clears only what reinterprets or manufactures bytes: `ISIG`, `IEXTEN`, the echo flags, `IXON`, `IXOFF`, `IXANY`. `ICANON` and CR/NL mapping stay. | setting | what it does before the program goes raw | tuitest | | ---------------------------------- | ----------------------------------------------------------------- | ------- | | `ISIG` | turns `^C`, `^Z` and `^\` into signals | cleared | | `IEXTEN` | `^V` and `^O` rewrite the input | cleared | | `ECHO`, `ECHOE`, `ECHOK`, `ECHONL` | writes the input back down the master, where it looks like output | cleared | | `IXON`, `IXOFF`, `IXANY` | `^S` stops the output stream until a `^Q` | cleared | | `ICANON` | line editing, input delivered a line at a time | kept | | CR/NL mapping (`ICRNL`) | Enter arrives as a newline | kept | | output processing | what the kernel does to the program's output | kept | I know that boundary is right because I put it in the wrong place first. My first attempt also cleared `ICRNL`, and the tape roundtrip tests failed immediately. That was the codebase telling me I had crossed from "stop the kernel eating my input" into "change what the program sees", and I was glad something caught it. The rest was portability. Termios constants live in different places across platforms, and my first version silently broke the Solaris build. Cross-compiling caught it: the constants now split across `linux || solaris || aix` and the BSD set, verified for darwin, freebsd, netbsd, openbsd, solaris and aix/ppc64. Forty-four runs after the fix, zero failures. The two-command tape reproduces 20 in 20 with `bytes=947, dirty=true, status="exit status 0"`. ## While I was in there The same audit turned up a second hole, unrelated to the PTY. The emulator generated correct replies to terminal queries, but nothing ever carried them back to the program that asked, which is why `tuitest snap -- glow -p file.md` captured a blank screen and exited 0. That turned out to be one instance of a bug I had already hit twice in other codebases, so it has [its own post](https://tuios.dev/blog/nobody-was-listening). The version relevant here: my own source contained comments describing the component that would carry replies back, and the component did not exist. Fixing it also broke a test that had been green since the day it was written, without once reaching the scenario it was named after. ## Odds and ends `--duration` was only checked between iterations, so `--duration 5s` ran 6.47s and `--duration 20s` ran 54s at default action counts. Now 5.09s and 20.27s. The closest-line heuristic in failure output scored by shared prefix length, so against `/a headless testing framework for TUIs/` it helpfully suggested the four-character fragment `a VT`. Replaced with normalised edit distance. A blank capture now exits 5 instead of 0, with a note about `--timeout` and `--wait` if the program was still running when the capture was taken. ## What I would keep The fuzzer found almost nothing in the program it was pointed at. It found bugs in the harness, a bug in a test, and a bug in its own shrinker. For a tool whose whole job is to tell the truth about what a program did, I think that is the better outcome, though it did not feel like one at the time. (A later fuzzer, pointed at the terminal emulator itself, found nothing for a different reason, and that got [its own post](https://tuios.dev/blog/the-fuzzer-that-found-nothing).) The habit I am keeping is counting. "Sometimes" cost me an evening in the wrong file. Seventeen in thirty-six took two minutes to establish and immediately ruled out half of what I had been considering. And I will read the minimised output even when it looks stupid, especially when it looks stupid, because mine was correct and specific for two days while I ignored it.