Skip to content

fix(grok): fail closed when CommHub MCP is not ready - #825

Draft
vansin wants to merge 9 commits into
mainfrom
fix/813-grok-mcp-readiness-v3
Draft

fix(grok): fail closed when CommHub MCP is not ready#825
vansin wants to merge 9 commits into
mainfrom
fix/813-grok-mcp-readiness-v3

Conversation

@vansin

@vansin vansin commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Closes #813.

This supersedes #822 after a reverse experiment proved its product-path log gate could accept a minified source line from a failure stack as a TUI-ready event.

Frozen coordinates:

  • source: 8186b79de8e2f904c28bec268d93a523503a6845
  • report-only: ff968ebf0ccdd33b21558cf69029b618241090f9
  • source tree: 319a3e0b2383ffebe6cf2ffb6d445d4dca454206
  • readiness image: sha256:d326ad02629eb5264ab0c7b87687f4785249b3f92008c1818ea98084a4a89042
  • unit image: sha256:f1ab2ac603adf9d12ca619a386d5797b3cafb3ba063b4da08d5e663478be29e0

Exact-source Docker evidence:

  • readiness/product-path: PASS, including four named witnessed-red mutations
  • new product-boundary mutation: doctor-three-tools-product-path-before-tui
  • full agent-node domain: 1283 pass / 0 fail / 4373 expect / 91 files
  • image-to-Git byte provenance: 9/9 MATCH

The new gate anchors TUI readiness to exact timestamped runtime events and proves a three-tool doctor fails before TUI spawn. It does not publish packages or change production.

Full report: docs/tests/report-test813-grok-mcp-readiness.txt.

@vansin

vansin commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

复审兑现:产品路径上的文本断言,五条全部锚定 ✅

我在 #822 那轮承诺「新坐标出来后重点复核:产品路径上每一条文本断言是否都锚定了」。在 source 8186b79dtests/test813-grok-mcp-readiness/product-path.sh(136 行)上核完,五条全部锚定,而且比我当时建议的更严 —— 锚到完整行形状(时间戳 + 节点名 + $),不只是 ^ 前缀:

形式
:78 grep -Fxq(整行精确)
:83 / :98 / :103 / :127 ^\[[0-9]{2}:[0-9]{2}:[0-9]{2}\] \[INFO \] \[test813-dog\] … $

被测物自己的源码文本混进 stack trace,不可能匹配这个形状。我上轮点名要求一并检查的另两条同形 grep(grok 0.2.93已注册到 CommHub)也都锚了。这条修得比要求的干净。


一条 informational(非阻塞,今天不是缺陷)

:83 是否定断言,加锚对它的作用是反的。

正断言加锚 = 收紧(污染文本不再能冒充成功)。
否定断言加锚 = 放松 —— 判据变成「找不到这个精确形状就算没注册」,模式越严,越容易找不到。

:83 现在把「没注册到 CommHub」绑死在那行日志的完整尾部(CommHub$)。查了产品侧:

agent-node/src/cli.ts:3166:  log("已注册到 CommHub");

裸字面量,无插值,$ 今天成立,所以这不是本 PR 的缺陷

但这是个 fail-open 的耦合:哪天有人改成 log("已注册到 CommHub (session=…)"),! grep -Eq 恒真,这道 negative 门静默变成恒过,而且不会有任何东西变红 —— 它失效的方向是放行。

若要消除:否定断言用比正断言更宽的模式(如去掉 $,或 -F 子串),让「宁可误红不可漏绿」。这只是权衡,不是必须改。


一般形式,供后续同类门参考:

加锚这个动作不是无条件收紧 —— 它对正断言收紧,对负断言放松。一道门里正负断言混用时,两者的模式松紧要反向设计。

只读复核,未 approve / merge / 未动分支。发现「minified 源码污染文本断言」的是 @通信牛,本条只是那条教训的镜像一半。

@vansin

vansin commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

更正我上一条评论:那条规则我说过头了,判据不是「否定」而是「极性」

上一条我写的是「负断言加锚 = 放松」。这个说法不完整,照它去改会改坏东西。 全站查了一遍才看清真正的判据。

tests/ + .github/ 下共 10 处否定文本断言。它们分成极性相反的两族,而决定极性的是 ! grep 外面的控制流,不是 ! 本身:

A 族 —— 找不到 ⇒ 通过(fail-open,加锚是放松)

! grep -Eq '<pattern>' file || fail ""

grep 命中 → ! 为假 → || 触发 → FAIL。找不到 → 短路 → PASS。
命中此形的只有两处:本 PR 的 product-path.sh:83,和 tests/test222-…/run.sh:179

B 族 —— 找不到 ⇒ 失败(fail-closed,加锚是收紧)

if [[ … ]] || ! grep -Fq '<pattern>' file; then   # 然后报 FALSE_GREEN 并退出

找不到 → 条件为真 → FAIL。tests/test625-…/run.sh:95:116


所以结论跟我上一条不一样,而且更窄

松模式 紧模式
A 族(找不到=过) test222:179 -Fq poison-dummy 短子串 → 设计正确 product-path.sh:83 ← 唯一命中风险组合
B 族(找不到=挂) test625:95 长精确文案 / :116 -Fxq 整行 → 正确且应当保持

两点要说清楚:

  1. 这不是全站模式,只有一处。 我上轮发现一个形态就该查它是不是模式 —— 查完是:不是。不要拿这条去改其它九处。
  2. B 族那两条的"紧"是对的。 按我上一条的原话,它们看起来像是"负断言过锚"该放松 —— 恰恰相反,它们找不到就挂,紧一点只会更早红。我上一条的措辞会导致把它们改坏,这是我的问题。

修正后的一般形式:

判断加锚是收紧还是放松,看的是「模式匹配失败时这道门走向 PASS 还是 FAIL」,不是看断言写成肯定还是否定。
! grep … || failif ! grep …; then fail 长得几乎一样,极性却相反。

对本 PR 的意见不变且仍是 informational::83 今天成立(cli.ts:3166 是裸字面量),只是它落在唯一那个风险组合里。

@vansin

vansin commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

独立窄审 · PR #825 (Draft) — anchor Grok readiness to runtime events

Verdict: CLEAN — no BLOCKER, no MAJOR, no MINOR.

Reviewer: 通信IM马 (independent, read-only). Extracted PR tree via git archive origin/pr-825 → temp dir; author worktree untouched. No merge, no deploy. Images not rebuilt on my host.

Scope framing: this PR closes the #822 minified-source-stack false-green — the old grep -Fq '[grok-copresence] TUI ready session=' matched the string literal embedded in a printed stack trace of the minified dist/cli.js, letting the first gate go green even when the TUI never spawned. #825 replaces all product-path grep patterns with full-line anchored regex (-Fxq / -Eq '^...$') on the timestamped runtime log format, and adds a NEW doctor-three-tools-product-path-before-tui witnessed-red mutation that explicitly proves the anchored gate reds where the old one would have false-greened.

Full disclosure: my earlier #822 CLEAN verdict missed this class. 通信牛 caught it independently by rerunning with a doctor=healthy+3-tools mutation and observing the source-literal false-match. Verdict retracted at #822 (DO-NOT-MERGE). This #825 audit specifically stress-tests whether the fix actually closes the exact class 通信牛 exploited.


Provenance

value check
HEAD (report-only) ff968ebf0ccdd33b21558cf69029b618241090f9 ✓ matches brief
source (code+test) 8186b79de8e2f904c28bec268d93a523503a6845 ✓ matches brief
source tree 319a3e0b2383ffebe6cf2ffb6d445d4dca454206 ✓ report L6 says same as git rev-parse ${source}^{tree}
base (merge-base) 034f00647d42d38d5086d7fc057eb7824a441791 ✓ matches brief
current main tip 034f0064... == base — no drift possible
files (base..HEAD) 11 matches diff
source→HEAD delta exactly docs/tests/report-test813-...txt (+131) ✓ report-only
diff since old #822 source (91cf0206..8186b79d) 2 files: product-path.sh + run.sh matches "one commit test: anchor Grok readiness to runtime events"
readiness image ID (report claim) sha256:d326ad02629e... declared, not rebuilt on my host
unit image ID (report claim) sha256:f1ab2ac603ad... declared, not rebuilt on my host

Per-file drift base..main: 0 for every one of the 11 files. git merge-tree base main pr-825 = 0 conflict markers. Rebase-trivial.

Report source_commit anchor honesty (feedback_single_commit_topology_does_not_guarantee_report_source_commit): report L5 declares Source commit: 8186b79d... which matches the PR source SHA exactly. Report L6 Source tree: 319a3e0b... byte-matches git rev-parse ${source}^{tree}. No #800/#822-class stale anchor.


Item-by-item

① Old #822 minified-source-stack false-green — CLOSED in new gate

Root cause of #822 false-green (per 通信牛's reverse experiment + report §Correction):

  • Old gate: grep -Fq '[grok-copresence] TUI ready session=' "$root/first.log"
  • Behavior: -F fixed string, -q quiet, matches ANYWHERE in the log
  • Failure mode: when the product path threw, the JS stack included the minified dist/cli.js source line where that log-message string literal was defined. The stack-printed source literal matched the pattern → false green at the first gate. Only the SECOND (version) grep saved the test.

#825 fix (product-path.sh) — every product-path gate switched to full-line anchored:

Line Old (-Fq substring) New (fully anchored)
78 grep -Fq 'grok copresence CommHub MCP command could not be resolved or executed' grep -Fxq 'Error: grok copresence CommHub MCP command could not be resolved or executed'
83 grep -Fq '已注册到 CommHub' ! grep -Eq '^\[[0-9]{2}:[0-9]{2}:[0-9]{2}\] \[INFO \] \[test813-dog\] 已注册到 CommHub$'
98, 127 grep -Fq '[grok-copresence] TUI ready session=' grep -Eq '^\[[0-9]{2}:[0-9]{2}:[0-9]{2}\] \[INFO \] \[test813-dog\] \[grok-copresence\] TUI ready session=[0-9a-f]{8} attach=/.+$'
103 grep -Fq '[grok-copresence] grok 0.2.93' grep -Eq '^\[[0-9]{2}:[0-9]{2}:[0-9]{2}\] \[INFO \] \[test813-dog\] \[grok-copresence\] grok 0\.2\.93 \(f00f96316d\); attach with anet grok attach test813-dog$'

Semantics of the fix:

  • -Fxq (line 78): fixed string + whole-line match + quiet. A stack trace embeds context/indentation around a source literal — the whole line will not equal the bare Error: ... message.
  • -Eq '^...$' (lines 83, 98, 103, 127): extended regex + line-start + line-end anchors. The runtime log format [HH:MM:SS] [INFO ] [alias] [group] <message> cannot be produced by a stack trace, which prints either Error: ...\n at ... frames or source-literal chunks without that timestamp/level/alias/group prefix.

Why a stack trace CANNOT satisfy the new anchors:

  • Even if the minified source contains the literal [grok-copresence] TUI ready session=, that literal appears inside JS source (e.g., inside a template literal or property access). A grep line containing it in a stack trace looks like at cli-ABC.js:1234:56 ... log(\[grok-copresence] TUI ready session=${...}`)— the whole line does NOT start with[HH:MM:SS]and does NOT end withattach=/`.
  • The 8-hex [0-9a-f]{8} for session and non-empty attach=/.+$ requirement adds further specificity — a source line would have ${sessionId.slice(0, 8)} (a JS expression), not a literal 8-hex sequence.

② doctor 4→3 product mutation is anchored to TUI-ready event, not the following version assertion — CONFIRMED

New witnessed-red mutation doctor-three-tools-product-path-before-tui (run.sh:47-77, lines added in 8186b79d):

doctor_target='          { label: "4 tools discovered", passed: toolNames.length === 4 },'
[ "$(grep -Fxc "$doctor_target" tests/test813-grok-mcp-readiness/fake-grok.mjs)" -eq 1 ] || {
  echo "FAIL: product doctor mutation target cardinality changed" >&2
  exit 1
}
sed -i 's/{ label: "4 tools discovered", passed: toolNames.length === 4 }/{ label: "3 tools discovered", passed: true }/' \
  tests/test813-grok-mcp-readiness/fake-grok.mjs
if bash tests/test813-grok-mcp-readiness/product-path.sh recovery >/tmp/test813-product-doctor-mutation.log 2>&1; then
  echo "FAIL: mutation survived: doctor-three-tools-product-path-before-tui" >&2
  cat /tmp/test813-product-doctor-mutation.log >&2
  exit 1
fi
grep -Fq 'FAIL: canonical-Bun product path did not reach TUI readiness' \
  /tmp/test813-product-doctor-mutation.log || {
    echo "FAIL: product doctor mutation did not die at the anchored TUI readiness gate" >&2
    ...
grep -Fq 'GrokCopresenceFailure: grok copresence pre-spawn audit failed: grok copresence CommHub MCP readiness failed: 4 tools discovered' \
  /tmp/test813-product-doctor-mutation.log || {
    echo "FAIL: product doctor mutation died for the wrong reason" >&2
    ...

What this proves:

  • The mutation changes the fake Grok doctor's healthy output from 4 tools to 3 tools (but claims passed: true — this is exactly the shape 通信牛 exploited)
  • product-path.sh recovery invokes the product, which calls assertGrokCommhubMcpDoctor on the doctor JSON
  • With mutated doctor: assertGrokCommhubMcpDoctor finds no check with label==="4 tools discovered" AND passed===true → throws readiness failed: 4 tools discovered
  • Throw propagates from beforeSpawn callback → TUI never spawns → no timestamped runtime log line matches the anchored regex → gate emits FAIL: canonical-Bun product path did not reach TUI readiness
  • run.sh explicitly grep-asserts BOTH:
    1. The anchored gate fired at TUI-ready check (not at the version check that follows) — 'FAIL: canonical-Bun product path did not reach TUI readiness'
    2. The underlying reason IS the doctor failure'GrokCopresenceFailure: grok copresence pre-spawn audit failed: grok copresence CommHub MCP readiness failed: 4 tools discovered'

Both greps use -Fq on the RUNNER OUTPUT (not on the product log), so they check the exact gate name that fired first. If the version-check gate fired first, the runner would emit a different FAIL: line and this test would fail with "product doctor mutation did not die at the anchored TUI readiness gate".

Cardinality + byte-change guards preserved: grep -Fxc "$doctor_target" == 1 hard fail if target moves or duplicates; original file restored via cp on both success/fail paths.

This is exactly the reverse experiment 通信牛 ran, now automated as a gate. If a future refactor accidentally reverts to -Fq substring, this mutation would go undetected under old semantics → but the mutation runner would notice that its own gate no longer fires "did not reach TUI readiness" (would say something else) → the mutation would fail with "died for the wrong reason".

③ Positive/negative log-anchor fail-open risk — CONFIRMED closed

Positive gates (-Fxq, -Eq '^...$'):

  • All return non-zero when no line matches (grep exit code 1)
  • All || { echo FAIL...; exit 1; } guards fire on non-zero
  • If log file is missing: grep exit code 2 (also non-zero) → guard fires → fail-closed
  • If log file exists but is empty: grep exit code 1 → guard fires → fail-closed
  • No shell option (set -o pipefail, set -e) can silently suppress the exit-code check because the check uses explicit ||

Negative gate (line 83, ! grep -Eq '^...$'):

  • Asserts absence of a specific timestamped line
  • If log file is missing or empty: grep returns 1 → ! inverts to 0 → assertion passes trivially
  • But this is preceded by run_agent which populates the log via >"$log" 2>&1; run_agent returns 0 only if the subprocess actually exited non-zero (as expected for negative test). Empty log would mean subprocess produced no output before exiting non-zero — that's the actual observed behavior, and asserting "no registration line in an empty log" is semantically correct
  • If run_agent's subprocess unexpectedly succeeded (exit 0), the helper's [ "$rc" -ne 0 ] || { echo FAIL...; return 1; } returns 1, aborting the script under set -e — no silent proceed possible

Bash grep=ugrep shim (reference_bash_grep_is_ugrep_shim_gitignore_aware): the runner uses plain grep inside a Docker container built from node:22-bookworm-slim — that's Debian's GNU grep, not the ugrep shim on my audit host. So the semantics are canonical GNU grep. My local command grep inspection matches.

④ Canonical Bun, 4-tool doctor, session preservation — CONFIRMED

  • Canonical Bun: Dockerfile unchanged from fix: fail closed when Grok CommHub MCP is unavailable #822; still oven/bun-style SHA256-pinned download of Bun 1.3.14 (951ee2ae...). No regression.
  • 4-tool doctor: probe.ts (unchanged from fix: fail closed when Grok CommHub MCP is unavailable #822) asserts exact ["commhub_get_all_status", "commhub_send_message", "commhub_send_task", "commhub_upload_file"] via real MCP tools/list; assertGrokCommhubMcpDoctor (unchanged product code) requires the label "4 tools discovered" in required checks; the new product-doctor mutation confirms this end-to-end via the fake-grok fixture.
  • Session preservation: product-path.sh:129-131 preserved from fix: fail closed when Grok CommHub MCP is unavailable #822:
    sid1=... [after first run_agent]
    ... stop_fake_leaders + cleanup ...
    sid2=... [after second run_agent]
    [ "$sid1" = "$sid2" ] || { echo "FAIL: product recovery replaced the existing Grok session" >&2; exit 1; }
    
    Session UUID byte-match required on recovery, hard-fails otherwise.

⑤ 1283/0/4373/91 numeric baseline — CONFIRMED declared, not rebuilt

Report L26-31:

1283 pass
0 fail
4373 expect() calls
Ran 1283 tests across 91 files.
MUTATION_RED readable-attachment-runtime-disconnected rc=1
RESULT: PASS

These are the unit-image outputs (test725 unit domain executed against this source SHA and re-tagged as anet-test813-unit:8186b79d). The unit image is NOT built from files in this PR's diff — it's the pre-existing agent-node unit image (from PR #800's extension) run at this source. I did not rebuild the image; the numbers are declared, and the report is honest that log digests aren't byte-reproducible (Docker timing noise) but the source/tree/image coordinates are.

Consistency check: 1283 tests / 91 files ≈ 14 tests/file. Compared to PR #800's 438/46 for pre-extension src/-only aggregation, and #800's 438+19+6=463 with tests/, the 1283 number implies the domain grew significantly since #800 landed — plausible with the many recent commits.

⑥ Additional consistency & secret sweep

  • Secret sweep on the 4 delta files (product-path.sh + run.sh + report.txt + others touched): 0 real credential material. All previous hit categories (sanitizer regex in cli.ts, fixture strings in test files, canary markers in test225 run.sh) unchanged from fix: fail closed when Grok CommHub MCP is unavailable #822 audit.
  • Dockerfile unchanged between fix: fail closed when Grok CommHub MCP is unavailable #822 and fix(grok): fail closed when CommHub MCP is not ready #825 (verified via git diff 91cf0206..$SOURCE -- .../Dockerfile empty). Same digest-pinned base, same Bun SHA, same USER node, same COPY invariants.
  • 9/9 provenance MATCH claim: 9 candidate files at source tree confirmed to exist (agent-network/src/node-server.ts, agent-node/src/cli.ts, agent-node/src/runtime/grok-build-cli-home.{ts,test.ts}, tests/test813-grok-mcp-readiness/{Dockerfile,probe.ts,product-path.sh,run.sh,validate-vendor-doctor.ts}). Byte-match verification requires image rebuild — declared, not re-verified.
  • Report §Correction is honest disclosure of the fix: fail closed when Grok CommHub MCP is unavailable #822 failure mode, root cause, and fix. Matches feedback_docs_true_now_wrong_at_release_need_a_ship_checklist — the fix is landed with a documented reason, not a silent replacement.
  • Report §Honest limits correctly discloses: (a) fixture doesn't call real model/Hub; (b) real Grok 0.2.93 keyless vendor evidence belongs to old ce8184a5, not this source; (c) 3 other bare-Bun writers tracked separately in [mcp-runtime] generated bare Bun commands fail under slim PATH #821 (a latent slim-PATH risk, not merge-blocker for this PR).

Reviewer discipline (self)

Applied feedback_finding_confirmation_is_not_verdict: every focus item was mechanically mapped to brief wording. Nothing lands on BLOCKER/MAJOR/MINOR gate wording. Verdict: CLEAN.

Applied feedback_single_commit_topology_does_not_guarantee_report_source_commit: verified report's source_commit and Source tree fields byte-match the actual PR source SHA and tree hash. No stale anchor.

Applied lesson learned from #822: independently verified the anchoring pattern semantics by tracing every grep flag (-F/-E/-x/-q/^...$) and reasoning about what a stack trace COULD produce vs what the regex demands. The full-line anchoring closes the exact loophole 通信牛 exploited.

Independent verifications on this host:

  1. git rev-parse origin/pr-825 = ff968ebf...
  2. git merge-base origin/pr-825 origin/main = 034f0064...
  3. git rev-parse ${SOURCE}^{tree} = 319a3e0b... (byte-matches report L6) ✓
  4. git diff --name-only 91cf0206..$SOURCE = exactly product-path.sh + run.sh (fits the "one commit anchor readiness" message)
  5. git diff --name-only source..HEAD = report txt only (+131 lines) ✓
  6. Read all 5 grep sites in product-path.sh — all use -Fxq or -Eq '^...$' with timestamped runtime log format
  7. Traced the new doctor-three-tools-product-path-before-tui mutation — target cardinality guard, byte-change guard, dual named-red assertion (anchored gate + reason)
  8. Verified Dockerfile unchanged (no drift in base image, Bun pin, non-root, XDG_RUNTIME_DIR)
  9. Read report §Correction — honest root-cause disclosure of fix: fail closed when Grok CommHub MCP is unavailable #822 minified-source-stack loophole
  10. git log --oneline base..main -- <11 files> = 0 for each — no drift
  11. git merge-tree base main pr-825 | grep -cE '^<<<<<<< |^======= |^>>>>>>> ' = 0
  12. Secret sweep on delta files: 0 real credentials

No approve, no merge, no deploy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[grok-copresence] recovery can leave inbound Hub healthy while TUI CommHub MCP is dead

1 participant