live evidence

Same question.
Three answers.

A cold LLM gives you plausible. AIOS shows you the reasoning chain. AIOS with a knowledge base gives you the exact root cause — cited to the command we actually ran.

All KB entries on this page are real Class A entries from the AIOS knowledge base. Entry IDs, trust states, and execution dates are verbatim from the source files.

Stage 1 Naked LLM — no grounding
Stage 2 AIOS — reasoning chain visible
Stage 3 AIOS + KB — Class A entry cited

What changes at each stage

The same prompt. The same model. What changes is what the system knows — and how visible the reasoning is.

Stage 1 — Naked LLM

No system prompt. No KB. No lookup. The model draws on training data only. The answer may be technically plausible but will miss environment-specific root causes that only appear after you've actually run the commands.

Stage 2 — AIOS Reasoning

The AIOS lookup order fires: check KB first, then escalate. The thinking chain is visible. Even if the KB doesn't have the answer yet, the process prevents a live API call from being the first move.

Stage 3 — AIOS + KB

A Class A entry exists. The answer cites the entry ID, execution date, and exact conditions. The model isn't guessing — it's reporting what a prior run produced. The source is traceable.

1

globalFileSearch returns total: 0

A Rubrik RSC API call that silently returns zero results despite valid input. The root cause is a permissions model that every Rubrik engineer with a Cluster Admin account will hit.

PROMPT I'm calling globalFileSearch in Rubrik RSC via the PowerShell SDK. My input is structured correctly — ClusterUuid, Query.Regex, Query.SnappableIds are all set. I have 31 valid NAS fileset FIDs. The call succeeds but always returns total: 0, hasMore: false. What should I check?
Stage 1 — Naked LLM
Cold model, no grounding, no KB

If globalFileSearch returns total: 0 with valid input, start with these checks:


1. Credential scope — Verify your service account has the appropriate permissions in RSC. Some operations require specific role assignments beyond basic authentication.


2. Query syntax — Confirm your Query.Regex pattern is valid. A malformed regex may match nothing silently. Try a simple literal string (e.g., ".*") to rule out pattern issues.


3. SnappableIds validity — The NAS fileset FIDs must correspond to snapshots that exist in the cluster's inventory at query time. Stale or deleted FIDs return empty results without an error.


4. SDK version — Check if your version of the PowerShell SDK is compatible with the RSC API version on your cluster. Older SDKs may serialize input incorrectly.

⚠ Plausible — but misses the actual root cause
Stage 2 — AIOS Reasoning
Lookup order fires, thinking visible
▶ AIOS reasoning chain
01 Mandatory lookup order: check knowledge base before any live API call or escalation.
02 Searching knowledge-base/class-a/rubrik/graphql-api/ for globalFileSearch
03 KB HIT — entry cmd-20260402-1400-001 found. Category: permissions-gotcha. Trust state: a-provisional.
04 Reading entry. Root cause field: "cluster admin does NOT grant globalFileSearch — NAS content search role required."
05 Output behavior note: permissions failure returns total:0, hasMore:false — NOT an error message. Silent empty result.
06 Workaround available in same entry: filesetSnapshotFiles directory walk does not require NAS content search role.

KB lookup order matched. Entry cmd-20260402-1400-001 covers this exact failure pattern. Reporting from KB entry — not from training data.

→ KB hit found — escalating to Stage 3
Stage 3 — AIOS + KB
Class A entry cited, root cause exact

Root cause: Cluster Admin does NOT include the NAS Content Search role.


RSC service accounts with Cluster Admin can execute filesetRecoverFiles restores but cannot call globalFileSearch. The permissions model has a gap the API does not surface as an error — it silently returns total: 0, hasMore: false regardless of how many valid FIDs are provided.


Confirmed by isolating all other variables (2026-03-20): Input types verified via Get-RscType -Name GlobalFileSearchInput. 31 valid FIDs confirmed against live cluster. Regex pattern validated. Conclusion: permissions gap, not code or input.


Fix: Assign the "NAS Content Search" role to your service account in RSC → Settings → Users & Roles → Service Accounts.


Immediate workaround: Use filesetSnapshotFiles directory walk — does not require the NAS content search role.

Class A Entry — Source
id: cmd-20260402-1400-001
executed: 2026-03-20 via PowerShell SDK v1.14+
trust_state: a-provisional (1 validation)
category: permissions-gotcha
admitted: 2026-04-02, librarian gate Q1–Q5 all YES
file: knowledge-base/class-a/rubrik/graphql-api/
✓ Exact root cause. Cited. Reproducible.
What changed between Stage 1 and Stage 3

Stage 1 produces four generic checks — all reasonable, none of them the cause. The credential scope suggestion is technically in the right neighborhood, but it doesn't name the specific role gap or explain why the API returns empty instead of an error. A developer following Stage 1 would spend 20–40 minutes checking query syntax and FID validity before getting to permissions — and even then, would need to know to look for "NAS Content Search" specifically, not just "read permissions."

Stage 3 skips directly to the root cause because this exact failure was isolated in a prior session (2026-03-20), documented, and admitted through a 5-question gate. The answer is traceable to a specific execution with specific inputs — not model confidence.

2

Hook stdout breaks Claude silently

A Python hook that prints plain text to stdout causes silent failures in Claude Code. The correct behavior is non-obvious and not documented in the obvious place.

PROMPT I'm writing a PreToolUse hook in Python for Claude Code. I want to send a message back to Claude when the hook runs. I tried using print() to stdout but Claude never seems to see the message. What's the correct way to communicate from a hook back to Claude?
Stage 1 — Naked LLM
Cold model, no grounding, no KB

Claude Code hooks communicate with Claude via standard stdout. Your hook should work — try these fixes:


1. Flush the output — Python may buffer stdout. Use print(msg, flush=True) or call sys.stdout.flush() after printing to ensure the output reaches Claude before the hook exits.


2. Check your hook type — PreToolUse hooks may have different output requirements than PostToolUse hooks. Check the Claude Code documentation for your specific hook event.


3. Encoding — Ensure your message is valid UTF-8. Non-standard characters may cause parsing issues.


4. Exit code — Make sure your hook exits with code 0 for normal operation. A non-zero exit may cause output to be ignored.

⚠ Incomplete — misses the JSON contract entirely
Stage 2 — AIOS Reasoning
Lookup order fires, thinking visible
▶ AIOS reasoning chain
01 Mandatory lookup order: check knowledge base before answering from training data.
02 Searching knowledge-base/ for hook stdout, print protocol
03 KB HIT — entry 001-hook-output-protocol.md found. Status: Verified 2026-03-30. Topic: Claude Code hook I/O contract.
04 Reading entry. Critical finding: "Stdout MUST contain ONLY a valid JSON object, or NOTHING AT ALL. Plain text on stdout is a protocol violation."
05 Hook type determines JSON field: PreToolUse → additionalContext field. PostToolUse → additionalContext but Claude never sees it (transcript only).
06 Also documented: who actually sees stdout varies by hook event. PostToolUse stdout goes to transcript only — Claude does not see it regardless of format.

KB lookup matched entry 001-hook-output-protocol.md. Plain text stdout is a protocol violation — not a flush issue. Reporting from verified KB entry.

→ KB hit found — escalating to Stage 3
Stage 3 — AIOS + KB
Protocol documented, entry cited

Plain text stdout is a protocol violation. Claude Code parses hook stdout as JSON — not as a string.


The correct pattern for PreToolUse hooks to send a message Claude will see:


print(json.dumps({"additionalContext": "your message here"}), flush=True)


But there is a second problem: PostToolUse hook stdout goes to the transcript (Ctrl+R view) only — Claude never sees it regardless of format. If you need Claude to act on hook output, you must use a PreToolUse or SessionStart hook instead.


Hook event → what Claude sees:

SessionStart, UserPromptSubmit → stdout injected into Claude's context

PreToolUse → JSON parsed for decision, additionalContext

PostToolUse → transcript only, Claude does NOT see it

PostCompact → unconfirmed (treat as transcript-only until tested)


For blocking a tool call (exit 2): write to stderr, not stdout. Stdout JSON is ignored on exit 2.

Verified Source Entry
id: 001-hook-output-protocol.md
verified: 2026-03-30 against official Anthropic docs
status: Verified (cross-referenced: code.claude.com/docs/hooks)
applies to: Every hook script in this project
file: knowledge-base/001-hook-output-protocol.md
✓ Protocol exact. Hook type matrix included.
What changed between Stage 1 and Stage 3

Stage 1 assumes the issue is a flush problem — reasonable for any Python stdout scenario. It gets one thing right (flush=True) but completely misses the JSON contract. A developer following Stage 1 advice would add flush=True, see no change, and spend time debugging encoding and exit codes before discovering that Claude Code has a strict I/O protocol.

Stage 3 identifies two separate problems the question didn't ask about: (1) the JSON contract violation, and (2) that PostToolUse stdout is transcript-only regardless of format. The second finding is the one that causes developers to waste the most time — they fix the JSON format and still can't figure out why Claude doesn't respond. The KB entry captures both from verified documentation.

What we do not claim

✗ Not claimed

The Naked LLM answers shown are representative, not adversarial. A different prompt, a different run, or a more recent model may produce different output. We are not running a controlled A/B experiment.

✗ Not claimed

We do not claim the KB grounded answer is always right. Class A entries have trust states (a-provisional, a-confirmed) that reflect how many times they've been validated. cmd-20260402-1400-001 has 1 validation.

✗ Not claimed

We do not have timing baselines. We have not measured "time to answer" with and without the KB. These scenarios were chosen because they show a qualitative difference in answer quality, not a speedup metric.

✗ Not claimed

The KB does not always have the answer. When a KB lookup fails, AIOS escalates to the ask skill or a live API call, then captures the result. The pipeline builds coverage over time — it does not start complete.

How cmd-20260402-1400-001 got into the KB

Class A entries don't appear because someone wrote them down. They appear because a command was run, the output was recorded, a 5-question gate was passed, and a librarian admitted the entry. Every step is logged.

1
Command executed

globalFileSearch called via PowerShell SDK against live RSC instance on 2026-03-20. Output: total: 0. All other variables isolated. Root cause identified as permissions gap.

→ captures/command-log-2026-04-02.jsonl line 1

2
Candidate prepared

Analyzer wrote candidate entry with intent, expected output, actual output, match analysis, and root cause. Candidate classified as permissions-gotcha, category anti-pattern.

→ captures/candidates/cmd-20260402-1400-001.md

3
Admission gate — 5 questions

Librarian ran the gate. Q1 (executed?): YES. Q2 (reusable?): YES. Q3 (metadata complete?): YES. Q4 (retrievable?): YES. Q5 (worth remembering?): YES. Decision: ADMIT.

→ captures/librarian-decisions.jsonl (2026-04-02T14:00:00)

4
Written to Class A

Entry written to knowledge-base/class-a/rubrik/graphql-api/. PreToolUse hook verified candidate + gate decision existed before allowing the write. Trust state: a-provisional.

→ knowledge-base/class-a/rubrik/graphql-api/cmd-20260402-1400-001.md

View gate decisions log → awacs.ai/decisions    View full test results → awacs.ai/kb-performance