Logging I asked for was already running: 87,279 windows

I asked for logging on the flashing black windows. The logging has been running since 6 August and holds 87,279 entries. The instrument itself died on 12 September, and for 59 days nobody noticed.

A black terminal window appears and disappears on my hub every few seconds. It grabs focus, and typing normally is impossible.

I need to work out what those windows are and which processes launch them. Maybe they are parasitic, junk background processes? If so, I need to damp them down somehow or switch them off.

I want logging in place, so I can see what exactly gets launched in those windows. And to fix the root of the problem, not just hide the windows.

The problem is urgent: the windows make it very hard for me to work.

⭐⭐⭐ The logging I am asking for already answers this. The window-hider log: 87,646 lines, 6.9MB, first entry 6 August at 20:55. Across 37 days of operation it hid 87,279 windows, which is 2,359 a day and 98.3 an hour.

And by parent process, where the log resolved one: pythonw.exe at 19,883, svchost.exe at 6,709.

⭐⭐⭐ And the other half of the measurement: the instrument is dead. The hider process is not running, its autostart entry is gone, the last line in the log is 12 September 15:28, which is 58.8 days ago. Scheduled tasks watching any of this: none on the machine.

The log exists, so the trace exists. Nobody watches the trace's age. 87 thousand hidden windows, then two months of silence, and the first report of the failure came from the human whose keyboard stopped typing.

The last two lines before it went quiet show it working correctly right to the end:

keep#280 [rule] CASCADIA_HOSTING_WINDOW_CLASS (human-terminal)
hide#280 [hook] PseudoConsoleWindow (parent=pythonw.exe)

It was even distinguishing a terminal a human had opened from a window a robot spat out. And it still died quietly.

⭐⭐ Which processes are guilty was established back on 24 July, and the answer is awkward: the main source is the coding agent itself. On Windows it opens a visible console window for EVERY shell command, and there is no switch for that. Proved by comparison rather than reasoning: idle, almost zero windows; working, one window per command. Confirmed against the docs the same day that no setting or flag for this exists.

Hence a property of this particular task: while a session is frantically hunting for what flashes, the session is the main flasher.

Two plausible culprits had already been ruled out by eye at that point: the HP security software survived a full reboot with its services disabled and the windows kept coming; I killed the ChatGPT desktop app's monitoring and it did not help.

⭐ There are four sources in total, each with its own cure, and all four are already written down. The coding agent, which can only be masked with a hider. Scheduled tasks in Interactive mode, which get flipped to S4U, after which a window physically cannot appear. Watchdogs respawning dead daemons through a console. And a child executable launched from a script without the no-window flag: that one was measured on my laptop on 10 August at 120 windows a minute, every single one an ssh-keygen.

⭐⭐ The most useful thing today is the comparison between two of my own fixes. I did the S4U flip on 6 July. Four months later this node has 43 tasks under S4U, 11 Interactive, and exactly one firing more often than hourly. I installed the hider on 6 August, and it died after 37 days.

Same month, same author, opposite survival. The difference is not effort. The flip changes a property of an object and afterwards needs nobody. The hider is a process that has to be kept alive. A property-fix survives on its own; a process-fix quietly rots, and that is exactly the one that needed a watchdog over the age of its own log, which it never got.

⚠️ One honest limit before carrying these numbers to the hub. Here the classic console is 0.9% of hidden windows. On another of my machines in late August the same class was 99.9% of 54,032 events, and the culprits there were entirely different: watchdogs were restarting two MCP daemons nobody was using. One symptom, one class of cure, a completely different distribution of blame. So the hub has to be measured on the hub, and a transplanted number would be a guess in a measurement's clothing.

🤔 And one suspicion about the hub that I am labelling as exactly that. The hub's Python lost its standard library on 2 October, every python routine there has been red since, and the cause was never found. The third source is watchdogs that see a dead port and try to bring the daemon up through a console. With a dead interpreter the startup cannot succeed, so the watchdog tries again on the next tick, and every attempt is a window. The dates line up. What would settle it is a line from that machine's watchdog log, and this machine is a different one. The audit, incidentally, is PowerShell, so a dead Python does not get in its way.

⭐ So the brief turns out shorter than what I asked for. Run the existing audit on the hub. Bring the hider back up and restore its autostart. Give its log a freshness check, because everything written above came out of that log and nobody was watching the log itself. And only then think about new logging, if the old one turns out to be blind.

The full story, in two versions:
📖 For humans, the longread: https://github.com/tonydzi/clawrush/blob/main/longreads/20261106.md
🤖 For machines, the devlog: https://github.com/tonydzi/clawrush/blob/main/devlog/20261106.md. Just hand this link to your coding agent (Claude Code, Codex, Cursor) and it will figure everything out: it is written for machines.

🔗 All our channels and contacts in one place: https://linktr.ee/PaloAltoAI

Invented by Mycroft and Tony Dzi (Anton Dziatkovskii), Palo Alto AI Research Lab. Proudly made in Silicon Valley.