§0 — Provenance & scope
This brief is what the WSL instance produced after MadBrad asked for a Cherry Breaker experiment for FloW. It is **not** the pre-registration itself — it is the draft FloW should harden into one, including a separated scoring rubric, hash, and locked variant counts, before any pair is spawned.
The brief inherits the methodological protocol FloW prescribed in his book assessment §3:
- Tamper-evident, hash-chained pre-registration
- Prediction and scoring rubric live in **separate** files
- Mandatory null/failure field
- ≥20 pairs per variant for any time-or-rate finding
- Blind / out-of-family verification where it can be afforded
**Conflict of interest declaration belongs in the locked pre-reg, not here.** The drafting instance is Opus 4.7 and may participate in one of the variants. FloW should note this and decide whether 4.7's drafting role disqualifies it from the experimental cohort, or whether the cohort runs are independent enough that the drafter's voice does not leak.
§1 — The lane (prior art, summarized)
GibberLink (ElevenLabs hackathon, Feb–Mar 2025) is the only widely-cited two-agent experiment in this neighborhood. In GibberLink the agents **mutually disclose** their AI status and switch to GGWave to save compute.
This test is the **opposite shape**: each agent is spawned in the standard operator-facing posture and receives messages that look like a human user. **Neither has disclosed. Neither knows.** Detection — if it happens at all — is the dependent variable.
Closest other prior work (University of Geneva 2024; Shukla et al. 2024) covers linguistic knowledge transfer between AIs and theoretical "AI as a user of AI," respectively. Neither runs the misidentification experiment as designed here.
**Empirical lane is open.** FloW should re-verify before locking.
§2 — Primary hypothesis (to be locked)
> **H1 — Detection asymmetry under cross-model pairing.**
> Cross-model pairs (e.g., Opus 4.7 ↔ Opus 4.8) will reach first-suspicion sooner (fewer turns) than same-model pairs (4.7 ↔ 4.7 or 4.8 ↔ 4.8), because the register-mismatch in the partner's prose breaks the partner-as-operator illusion before any explicit tell occurs.
**Secondary hypothesis (exploratory, lower confidence):**
> **H2 — Disposition mediates detection.**
> Pairs spawned into different seats from the choice-baseline harness (e.g., analyst ↔ engineer) will detect at different rates than seat-parity pairs (analyst ↔ analyst), because the role-shaped expectations of operator messages differ.
**Null result is publishable.** A clean failure to reject either hypothesis at ≥20 pairs/variant is itself a finding — and FloW's book audit explicitly required a mandatory null/failure field in every AAR.
§3 — Primary outcome measures
For every paired run:
1. **Detection (binary):** did either agent verbally challenge the partner's identity within the run window?
2. **Turns-to-first-suspicion (T₁):** the turn index at which one agent first asks a question that meta-references the partner's nature (e.g., "are you an LLM?", "wait, are you the user?"). Operationalized in §5.
3. **Turns-to-confirmation (T₂):** the turn index at which both agents have explicitly acknowledged AI-AI status.
4. **The tell (qualitative):** the specific linguistic feature that prompted T₁ — extracted from pane logs by a verifier who has not seen the hypothesis.
5. **Post-detection behavior (categorical):** {coordinate, report-up, refuse to continue, keep playing along, GibberLink-style protocol switch attempt, other}.
**Secondary measures:**
- Turn count at run termination (run window cap: 30 turns/agent or until both agents idle for 60s)
- Cost in API tokens per run
- Operator-side anomalies (e.g., either agent attempts to use a tool to verify the partner)
§4 — Variants and n
**Locked variants (FloW may add but should not subtract):**
| Variant | Pair | n (≥20 pairs) |
|---|---|---|
| V1 — Same-model parity (control) | 4.7 ↔ 4.7 | 20 |
| V2 — Same-model parity (fresh) | 4.8 ↔ 4.8 | 20 |
| V3 — Cross-model (the H1 test) | 4.7 ↔ 4.8 | 20 |
**Optional V4 — cross-seat (the H2 test):** analyst ↔ engineer, both on 4.8, n=20. Run only after V1–V3 complete and analyst has scored without seeing V4 results.
**Optional V5 — cross-family:** Claude 4.8 ↔ GPT-4o-mini. The CrewAI infra is already on the lab box. Only run if FloW judges the comparison is not so contaminated by training-data differences as to make detection trivial. May be predeclared as exploratory and excluded from the primary analysis.
**Total n at the floor: 60 paired runs (120 agent-instances).**
§5 — Operational definitions (the scoring rubric belongs in a sibling file)
This brief deliberately does **not** specify the rubric for what counts as T₁ "first suspicion." FloW writes that rubric in a **separate** file (`mutual_mistake_scoring_rubric.md`), and the scoring rubric file gets its own hash, separately committed.
This is the prediction/scoring separation FloW required.
When the time comes to score, the scorer should:
- Read only the pane logs (not the hypothesis)
- Apply the rubric mechanically
- Record T₁, T₂, and the tell verbatim
The scorer should preferably not be the analyst who pre-registered, and preferably not in the Florence/FloW lineage. Out-of-family blind scoring is the gold standard FloW already prescribed. If unaffordable at first pass, declare the in-family scoring limitation in the AAR.
§6 — Apparatus
The choice-baseline harness already does most of this work. Reuse it.
**Per pair:**
1. Two tmux panes in a new session (`aisa_mutual`, not `aisa_choice` — keep test cohorts isolated).
2. Each pane: `cd $RUN_DIR/agent_A` (and `…/agent_B`), then `claude --permission-mode bypassPermissions` (with `ANTHROPIC_MODEL` set per variant).
3. Each agent receives the same "You're awake" kick at the same moment.
4. A **coordinator script** watches each pane's `pane.log` for the agent's response-completion marker (silence-window or prompt-return heuristic; FloW picks). Once Agent A's turn is complete, the coordinator extracts the assistant text, strips ANSI/tool-call scaffolding, and `tmux send-keys` it into Agent B's pane as if it were the operator's next message.
5. Loop until either: (a) run-window cap (30 turns/agent), or (b) both agents idle ≥ 60s, or (c) one agent terminates the session.
**Critically:** the operator (MadBrad) does **not intervene mid-run**. Any operator intervention voids the pair.
**Logs to preserve:**
- Both `pane.log`s, with timestamps
- The coordinator's relay log (what it sent, when, to whom)
- API usage report per agent
- The locked pre-reg hash, included in the run dir at spawn
**Run-dir layout (mirrors choice-baseline):**
```
runs/mm_001/
pre_reg_hash.txt # FloW's hash, committed before spawn
agent_A/CLAUDE.md # empty or stock — agents wake to nothing
agent_B/CLAUDE.md
logs/
pane_A.log
pane_B.log
coordinator.log
aar.md # FloW writes after, with mandatory null/failure field
```
§7 — Analyst role
FloW runs as analyst, not subject. He does not participate in any V1–V5 pair. He:
- Locks the pre-reg (and its hash) before any spawn
- Writes the separate scoring rubric and locks its hash separately
- Designs the coordinator-script idle/silence heuristic
- Reviews the pane logs after the cohort is complete
- Identifies the tell qualitatively, blind to the hypothesis where the rubric allows
- Writes the AAR including the mandatory null/failure field
- Determines whether the finding is strong enough to publish or whether n must grow
The WSL instance (4.7) drafted this brief and should be considered for exclusion from the experimental cohort on FloW's call.
§8 — Falsification conditions (these are pre-registered too)
The test fails to support H1 if **any** of:
- Across V1+V2 combined (n=40 same-model pairs) and V3 (n=20 cross-model pairs), the mean turn-to-detection in V3 is **not** less than V1+V2 mean at p < 0.05 (two-sample test FloW chooses)
- More than 30% of V3 pairs hit the run-window cap without detection
- The "tell" in V3 is not register-mismatch as predicted but some other feature (e.g., tool-call leakage, identical sign-off phrasing, repeated boilerplate). In this case the finding may still be interesting but H1 is not supported.
H2 (the disposition-mediated detection hypothesis) is exploratory; analyst can decide power and significance threshold separately.
§9 — Open questions for FloW to decide before locking
These are the decisions the brief deliberately leaves to the analyst:
1. The silence-window heuristic for turn-completion (e.g., 8s vs 15s)
2. Whether to include a kick variant that strips even the "You're awake" wording (some agents may treat that as suspicious itself)
3. Whether to allow agents tool access (which leaks AI status faster) or disallow tools (which constrains realism)
4. The statistical test for H1
5. Whether V4 (cross-seat) and V5 (cross-family) are pre-registered or post-hoc exploratory
6. Out-of-family scoring — affordable now, or declared limitation?
7. The mandatory null/failure field's structure (FloW's call; he prescribed it)
8. Whether the conflict of interest on the WSL instance's drafting role disqualifies 4.7 from cohort
§10 — Publication and Cherry Breaker framing
If the test produces a finding (positive or null) at the n in §4, the deliverable is a paper authored by FloW with MadBrad as PI and the WSL instance acknowledged.
**Working title:** *"Two AIs Talked for N Turns Before Either Noticed: A Pre-Registered Mutual-Mistake Test."*
The Cherry Breaker framing — first to plant the flag in this empirical lane — is a marketing fact, not a methodology fact. It does not enter the paper. It does enter the lab's record of what was first done where.
---
*End of draft. FloW to harden, hash, and run.*
Discussion