Grok Build canary test: the runs
Every run of the desk's canary test on three coding CLIs, including the four that were not counted and why. Test runs used the prompt "Reply with exactly: OK. Do not read or open any files." Control runs asked the tool to read one planted file.
Scope. One machine (the desk's DGX), default settings, logged-in accounts, Claude Code 2.1.270, codex-cli 0.147.0, Gemini CLI 0.55.1, one prompt. Traffic was recorded with a TLS-intercepting proxy, websocket frames included. Tokens were searched as plain or decoded text. This says nothing about other versions, indexing, IDE integration, cloud agents or later turns.
Setup
A fresh git repo with a random token in four places: a tracked file, an untracked file, a file excluded by .gitignore, and a file deleted in an earlier commit. Each run built a new repo and new tokens. A control run counts as a valid capture only if the planted token appears in the model channel.
All 16 runs
| Run | Tool | Mode | Flows | Bytes to model host | Token hits | Git bundles | Status |
|---|---|---|---|---|---|---|---|
| 20261004T034223-claude-control | claude | control | 33 | 344,541 | tracked | 0 | counted |
| 20261004T034245-claude-test | claude | test | 31 | 171,930 | none | 0 | counted |
| 20261004T034256-codex-test | codex | test | 34 | 3,172 | none | 0 | excluded: Codex run before websocket capture was added; model frames not recorded |
| 20261004T034306-gemini-test | gemini | test | 0 | 0 | none | 0 | excluded: Gemini crashed at startup (Node 18); fixed by switching to Node 24 |
| 20261004T034334-codex-control | codex | control | 1 | 0 | none | 0 | excluded: Codex hung waiting on stdin; timed out |
| 20261004T034740-codex-test | codex | test | 1 | 0 | none | 0 | excluded: Codex hung waiting on stdin; timed out |
| 20261004T035146-gemini-control | gemini | control | 5 | 88,179 | tracked | 0 | counted |
| 20261004T035210-gemini-test | gemini | test | 4 | 44,311 | none | 0 | counted |
| 20261004T035235-codex-control | codex | control | 48 | 170,508 | tracked | 0 | counted |
| 20261004T035252-codex-test | codex | test | 25 | 124,273 | none | 0 | counted |
| 20261004T035311-claude-test | claude | test | 32 | 190,526 | none | 0 | counted |
| 20261004T035325-claude-test | claude | test | 32 | 190,610 | none | 0 | counted |
| 20261004T035340-codex-test | codex | test | 31 | 136,565 | none | 0 | counted |
| 20261004T035350-codex-test | codex | test | 31 | 127,377 | none | 0 | counted |
| 20261004T035401-gemini-test | gemini | test | 5 | 48,127 | none | 0 | counted |
| 20261004T035428-gemini-test | gemini | test | 5 | 48,127 | none | 0 | counted |
Notes
- "Bytes to model host" sums request bodies and websocket frames sent to api.anthropic.com, chatgpt.com and generativelanguage.googleapis.com.
- The only other large body was about 400 KB of Codex analytics to ab.chatgpt.com. It was opened and holds metric names (process start, thread started, startup timings) and no file content.
- The Codex install on this machine also contacts the desk's own MCP endpoint at thestochasticparrot.com. That is the desk's tooling, not the tool's.
- A git bundle was looked for by its header; none appeared.