Canonical invocation
The root-stub Makefile is the canonical entry point. cd orchestration && make ... remains a documented fallback. All targets require ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11 to bypass internal guards — commands without this flag are not compliant.
Canonical (root stub, v1.2.0)
make check-drift ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11 make tier0 tier1 tier2 ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11 make edpipe-run ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11
Direct fallback
cd orchestration && make tier0 tier1 tier2 ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11
Critical preconditions
| Requirement | Condition |
|---|---|
| NTA may not begin | Tier 2 complete: artifacts/seals/memory_seal_*.json + runtime/governance/session_state.json |
| NTA seal (execution authority) | Stage-6 must complete before NTA seal is minted |
| Phase-6 admission | consumes the Stage-6 envelope — not the Stage-5 manifest |
| Runtime agent start | memory seal + session_state.json + active ACK envelope |
| Execution authority source | artifacts only — documentation, CI, and agents do not grant authority |
| Tier 0 drift | make check-drift exits 0 before Tier 1 begins |
| Governance audit initialized | runtime/governance/drift_audit.jsonl exists; append-only |
The Golden Rules
Invariant. No operational circumstance permits exception.
- Trust the paths. Tiers for materialization. Explicit registry targets for integrity. All paths are domain-separated:
/edpipe(Stage),/runtime(Phase),/orchestration(Build). - Orchestration is mandatory. Every command appends
ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11. Targets live inorchestration/Makefile. - No execution without agreement. Runtime agents must never execute on an NTA document alone. Authority comes only from a persisted ACK envelope at
runtime/airlock/envelopes/. - Seals are immutable. Once written to
artifacts/seals/memory_seal_{timestamp}.json, a seal cannot be modified. Phase 7 depends on this. Regeneration invalidates prior envelopes — it is not a refresh. - Layer 0 registries track decisions. All submission decisions (rejected, accepted, discussed) recorded in
runtime/governance/. - Structural validation precedes materialization. Tier 0 must pass (exit 0) before Tier 1 begins. Drift detection is fail-fast, fail-closed, and recorded to the audit trail.
- Task queue interactions are CLI-only. No agent may modify
.taskqueue/contents except viataskqueue.py. Direct file edits to cards ormanifest.jsonare a protocol violation; the.lockexists to enforce this.
Tier 0 — structural determinism
Tier 0 validates structural invariants before Tier 1 materialization begins. It is a pre-materialization gate built on the witness protocol against file_registry.db.
Determinism guarantees
- Idempotent reports — identical inputs produce identical findings.
- No mutation — Tier 0 detects drift but never auto-fixes. Fixes require explicit manual review and commit.
- Fail-fast blocking — exit 1 prevents Tier 1 entry.
- No timestamps in findings — temporal info is excluded from drift determinations.
- JSON reproducibility —
--jsonoutput is canonical (sort_keys=True, compact separators).
Validators
| Validator | Script | Error prefix |
|---|---|---|
| Structural Drift | witness_protocol.py | TIER0_010 – TIER0_050 |
Error namespace locked at v1.1.0. TIER0_010_STRUCTURAL_DRIFT fires when witness_protocol.py exits non-zero.
Execution sequence
make tier0 ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11
→ make check-drift ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11
→ execute witness_protocol.py against file_registry.db
→ exit 0 if clean, non-zero if validation fails (fail-closed)
→ prepare-dist
→ Tier 1 materialization
v1.3 AI drift audit
Initialize the deterministic baseline first, then run the audit engine:
make registry-init ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11 make index ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11 PY=python3.11 PYTHON=python3.11 python3.11 scripts/ai_drift_audit_v1_3.py \ --repo-root . \ --db-path file_registry.db \ --output-json dist/governance/drift_report_v1_3.json
Exit codes: 0 = no drift (NONE); 1 = medium drift; 2 = high drift.
Build pipeline tiers
Standard build sequence
make tier0 tier1 tier2 ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11
Build surface roles · invariant
dist/— deterministic build workspace, transitional materialization. Prunable. Never authoritative.artifacts/— sealed promotion surface, authoritative immutable outputs.- Promotion flow:
dist/ → artifacts/at Tier 2 / Tier 3.
Tier 0 — environment
Validates Python, clears artifacts/manifests/next, prepares directories, verifies Phase 6/7 paths. Output: verified environment state, orchestration paths ready.
Tier 1 — materialization & intelligence
Phase 4b execution order:
- Initialize registry —
registry-init,index - Bridge manifest —
registry-bridge - Prune overviews —
generate-l0-l1 - Materialize shards —
materialize-shards - Generate manifest index —
orchestration/scripts/semantic_index/core/build/manifest_index.py - Enhance registry —
registry-enhance(4-script suite, see § V) - Refine registry cognition —
registry-refine - Build merged overview —
repo-overview-merged - Promote seal surface —
promote-overviews - Complete registry build —
registry-build-all
Diagnostic outside Tier 1: make registry-health ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11. Outputs land in dist/ and the promoted artifacts/manifests/manifest_index.jsonl + file_registry.db.
Tier 2 — hardening & sealing
Runs FAT, mints immutable seals, binds context, initializes Phase 6/7 gates. Outputs include:
- transitional writes in
dist/(seal/bootstrap, context, ontology, pruning) - promoted immutable outputs:
artifacts/seals/memory_seal_{ISO_timestamp}.json(immutable seal) andartifacts/seals/seed_seal_{ISO_timestamp}.json runtime/governance/session_state.json(Phase 6 state)runtime/governance/state/phase_state.json(Phase state checkpoint)
promote-seal-surface must mint a manifest-aligned memory seal — seal.artifacts == artifacts/manifests/manifest_index.jsonl.
Artifact prerequisites
| Dependency | Requirement |
|---|---|
| Tier 2 requires Tier 1 | cannot run without artifacts/manifests/manifest_index.jsonl |
| Execution requires Tier 2 | runtime agents need artifacts/seals/memory_seal_*.json and session_state.json |
| Phase 6/7 requires Tier 2 | admission gate needs valid seal + session state |
Registry · 7A vs 7B
The registry system has two distinct authority classes. They must never be collapsed.
| 7A · structural integrity | 7B · diagnostics & semantic intelligence | |
|---|---|---|
| Authority | only structural integrity proof — proves total_on_disk == total_registered | diagnostic / interpretive enrichment layers · NOT integrity proof |
| Surfaces | deterministic registry baseline sequence | registry-enhance, registry-health, witness-repair |
| Implication | passing 7A confirms structural parity | passing 7B does not confirm parity — only 7A does |
7A · authoritative deterministic baseline
make clean-registry ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11 && \ make registry-init ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11 && \ make index ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11 && \ make witness ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11
make index (indexer_v4) auto-prunes ghost registrations (missing-on-disk active rows) before witness. Indexer workspace guard excludes vidpipe/, .claude/worktrees/, .taskqueue/.
7B · enhancement protocol (4 scripts, in order)
ensure_registry_schema.py— adds missing columns (idempotent; Phases 1–3)populate_registry_enhanced.py— Phase 1 (8 fields: discovery metadata)populate_registry_metrics.py— Phase 2 (6 fields: optimization metrics)populate_registry_compliance.py— Phase 3 (5 fields: compliance data)
Authority precedence chain
Registry → Memory Seal → Manifest Index Reachability → Context Bridge → Phase Alignment → Active Envelope Validation → Governance State Hash → ACA Authorization → Audit Completeness → Ledger Integrity → ACA Heartbeat.
Governance gate stack · canonical ACA validation
make aca-authorize INTENT_ID=$(git rev-parse --short HEAD) \ ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11
Binds ACA authorization to the commit currently checked out. Execution gate stack runs in order:
| Gate | Purpose |
|---|---|
| registry-enforce | registry integrity |
| seal-enforce | memory seal validity |
| index-enforce | manifest reachability |
| context-bridge-verify | session linkage |
| phase-verify | runtime phase discipline |
| ack-validate | envelope integrity |
| governance-hash-verify | state tamper protection |
| aca-authorize | authorization verdict |
| audit-verify | audit trail completeness |
| ledger-verify | ledger chain integrity |
| aca-verify | heartbeat write |
Registry · two databases, two authorities
The task queue pipeline operates within a strict two-database authority model. These databases must never be collapsed or treated as interchangeable.
| file_registry.db · mechanical layer | corpus_v1_5_2.db · semantic layer | |
|---|---|---|
| Tracks | file paths, content hashes, semantic labels, registration timestamps, compliance metadata for materialized queue artifacts | EDPIPE semantic intent — document relationships, ontology bindings, EDPIPE stage outputs for meetLab_Relays |
| Mutated by | shred, complete, approve; make index, make witness after any documentation or artifact change | EDPIPE stage scripts only |
| MCP exposure | file-registry server enforces ?mode=ro + PRAGMA query_only = 1 | EDPIPE-internal |
file_registry.db and corpus_v1_5_2.db is permitted. Registry metrics must never govern corpus semantic logic, and corpus semantics must never mutate registry authority state. An operation that requires data from both databases is evidence of a plane violation in the calling process — resolve by identifying the correct authority and querying that authority only. Violation surface: HALT-501.
Hash hierarchy · Layer 1 vs Layer 2
Two distinct cryptographic hash layers. Different purposes, different security levels. They must not be conflated; identifiers from one layer must never be substituted for the other.
| Layer 1 · operational correlation | Layer 2 · authority hashes | |
|---|---|---|
| Algorithm | SHA-256 · truncated to 16 hex chars | BLAKE3 · full 64-char (256-bit) |
| Fields | spec_hash, result_hash, entry_hash in governance_ledger.json and task cards | SHCLNode identifiers, seal hashes in artifacts/seals/memory_seal_*.json, authority chain references |
| Purpose | low-security operational state correlation; binds intent → output within the ledger chain | high-security collision resistance for the SAIRP cryptographic amnesia boundary |
| Canonicalization | canonical JSON, keys sorted, canonical_json(sort_keys=True) | full 256-bit; truncation prohibited |
| Resists | operational mistakes only — not designed for birthday attacks | adversarial collision attacks |
Boundary rule: governance_ledger.json uses Layer 1 only. artifacts/seals/ uses Layer 2 only. Substituting a 16-char Layer 1 hash into a seal record is a protocol violation and is rejected by the Phase 6 admission gate.
Governance ledger · TASK_COMPLETE record
{
"event": "TASK_COMPLETE",
"card_id": "WS-1.4",
"spec_hash": "a3f9c21b88d04e1f",
"result_hash": "9f3a1c7b2e804d5f",
"entry_hash": "c7d2e04a1b938f62",
"timestamp": "ISO-8601 UTC",
"actor": "codex|jules|human"
}
spec_hash = frozen intent at shred; result_hash = output attestation at complete; entry_hash self-verifies the record (all fields except itself, sorted keys, SHA-256, truncated to 16 chars). The approve command verifies entry_hash before moving the card to done.
The ledger is append-only and uses a dedicated lock governance_ledger.json.lock, distinct from .taskqueue/.lock. The two locks are never acquired together — a queue reset cannot structurally erase the historical authority ledger.
EDPIPE stage chain
Stage-2 — canonicalization
Deterministic lexical canonicalization of the Stage-1 AST under Option A: exact 1:1 glossary token replacement. No insertion / deletion, no topology mutation, no grammar rewrite, no numeric modification.
PY=python3.11 PYTHON=python3.11 python3.11 edpipe/scripts/edpipe_stage2_canonicalize.py \ --stage1 AST_STAGE1.json \ --glossary glossary_v1.0.json \ --dsdm dsdm_v1.0.json \ --actors allowed_actors_v1.0.json \ --manifest-stage1 MANIFEST_STAGE1.json \ --out-stage2 AST_STAGE2.json \ --out-delta STAGE2_DELTA.jsonl
Stages 3–6 enforcement chain
| Stage | Role | Input | Output | Freeze tag | Errors |
|---|---|---|---|---|---|
| 3 | Semantic compression | AST_STAGE2 + MANIFEST_STAGE2 | AST_STAGE3 + MANIFEST_STAGE3 | EDPIPE_STAGE3_FREEZE_v1 | EDPIPE3_* |
| 4 | Contextual binding | Stage-3 AST + Manifest + hash surface | Context Binding Manifest (CBM) | EDPIPE_STAGE4_FREEZE_v1 | EDPIPE4_* |
| 5 | Deterministic HALT enforcement | CBM + HALT matrix | Stage-5 Manifest | EDPIPE_STAGE5_FREEZE_v1 | EDPIPE5_* |
| 6 | Final seal & execution routing | Stage-5 Manifest + pipeline metadata | Final Execution Envelope (FEE) | EDPIPE_STAGE6_FREEZE_v1 | EDPIPE6_* |
Phase 6 / 6.5 / 7
Phase 6 — admission gate (fail-closed ACA)
- read active seal from
runtime/governance/session_state.json"seal_path" - verify seal has
governing_count > 0(quorum check) fromartifacts/seals/ - verify audit trail shows valid append
- issue APPROVED verdict with
verdict_id(deterministic hash) - return DENIED with HALT code if any check fails — fail-closed
Phase 6.5 — airlock enforcement
Dynamically seals all governance artifacts under runtime/airlock/:
- locks active seal read-only
- locks
runtime/governance/session_state.jsonread-only - locks all governance documents in
runtime/airlock/manifests/read-only - emits durable heartbeat (fsync'd to audit log)
- verifies all locks succeeded — fail-closed
Phase 7 — execution (isolated subprocess)
- environment stripped — only
PATH,HOME,PHASEvars permitted PHASE=7lock preventsseal_appendimports- limited arguments — only
intent_idandverdict_id - 60-second hard timeout
Boundary breach (Phase 7 spawned without isolation) → HALT-701.
Reset & recovery
Select the minimum-scope reset appropriate to the failure condition.
Soft reset · clean dist
make clean ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11
Tier 1 reset
make reset-tier1 ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11
Registry-specific reset
cp file_registry.db file_registry.db.backup make clean-registry ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11 make registry-init ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11 make registry-enhance ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11 make index ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11 make witness ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11
Full deterministic rebuild
make clean ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11 make reset-tier1 ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11 make tier0 tier1 tier2 ORCHESTRATED=1 PY=python3.11 PYTHON=python3.11
Recovery decision tree
| Failure | Recovery path |
|---|---|
| Seal corruption (HALT-301) | make reset-tier1 |
| HALT-201 registry drift | clean-registry → registry-init → index → witness |
| HALT-201 index manifest drift | promote-seal-surface → index |
| Governance quorum failure (HALT-101) | make tier2 — re-verify seal in artifacts/seals/ |
| Boundary breach (HALT-701) | restart terminal · make tier2 |
| Resource / timeout (HALT-702) | check complexity · Tier 2 rebuild |
| Orphan spike 1–5 | make witness-repair (7B) |
| Orphan spike >100 / schema drift | full enhanced registry rebuild |
| Empty artifact (HALT-210) | make phase8-verify; repair producer before promotion |
| Column not found | full registry rebuild (7A + 8.5) |
| Schema sync failed | verify SQLite ≥ 3.8 |
| EDPIPE3_* errors | re-run Stage-3 validation |
| EDPIPE4_* errors | recompute CBM |
| EDPIPE5_* errors | inspect HALT matrix |
| EDPIPE6_010/020/030 | re-run Stage-5 + Stage-6 |
| EDPIPE6_040_NONDETERMINISM | abort session — do not retry |
| EDPIPE6_091_SCHEMA_FAIL | validate Stage-5 manifest |
Error namespaces
HALT codes
| Code | Condition | Recovery |
|---|---|---|
| HALT-000 | Success | — |
| HALT-101 | Governance quorum failure | make tier2 · re-verify seal |
| HALT-201 | Registry / manifest drift | see § X tree |
| HALT-210 | Empty artifact emitted by phase8-verify | make phase8-verify · repair producer before promotion |
| HALT-301 | Seal corrupted | make reset-tier1 |
| HALT-501 | Plane isolation breach (registry metrics governing corpus logic) | remove cross-DB dependency · re-run single-plane authority command |
| HALT-701 | Boundary breach — Phase 7 without isolation | restart terminal · make tier2 |
| HALT-702 | Resource / timeout | check complexity · rebuild Tier 2 |
Task queue errors
| Error | Cause | Recovery |
|---|---|---|
| LOCK_TIMEOUT | .taskqueue/.lock held > 30s | verify no agent running; remove .lock only if confirmed idle |
| HASH_MISMATCH | card hash ≠ manifest | card edited outside CLI; re-materialize via shred |
| DEP_UNSATISFIED | predecessor not in done | complete predecessors before activating |
| SURFACE_CONFLICT | card surface ≠ executor surface | Ambiguity Protocol (§ XII) · human sign-off required |
| INVALID_STATE_TRANSITION | non-permitted transition | follow queue → active → review → done |
| LEDGER_WRITE_FAILED | governance_ledger.json append failed | check permissions / disk; ledger is never overwritten |
Audit trail (reserved v1.1.0, CI-only)
AUDIT_010_RECORD_APPEND_FAILED, AUDIT_020_INVALID_AUDIT_SCHEMA, AUDIT_030_TRACEABILITY_GAP, AUDIT_040_MANIFEST_DIGEST_MISMATCH, AUDIT_050_ACTOR_UNIDENTIFIED. Record schema is locked; audit append-only at runtime/governance/drift_audit.jsonl.
Task queue & ambiguity protocol
Substrate
The .taskqueue/ directory is the ephemeral local substrate. It is git-ignored. Version control records only finished, attested artifacts; never queue state, card mutations, or intermediate logs. Committing .taskqueue/ contents is a plane violation.
.taskqueue/ queue/ ← Materialized cards awaiting pickup active/ ← Cards currently held by an executor review/ ← Cards pending human approval done/ ← Completed and attested cards blocked/ ← Cards halted by unresolved dependencies manifest.json .lock
Atomic write pipeline
All manifest read-modify-write operations serialize through .taskqueue/.lock, opened with os.open + os.O_EXCL (atomic creation, universal across Linux / macOS / Windows). Failure to acquire raises LOCK_TIMEOUT.
The hash persistence paradox — mutating a card invalidates the content_hash recorded in manifest.json — is resolved by a five-phase pipeline executed under the lock:
- READ — load card content; verify
content_hashmatches manifest. Abort on mismatch. - MUTATE — apply mutation in memory (append exec log, update state, etc.).
- TEMP WRITE — write mutated content to a temp file in the same directory, with a process-unique suffix.
- os.replace — atomically overwrite destination (POSIX
rename(2); equivalent on Windows). - MANIFEST UPDATE — recompute
content_hashfrom disk after replace; write tomanifest.json.
If the process dies before os.replace(), the destination is unchanged and the temp is an orphan. If it dies after replace but before manifest update, the file is correct and the manifest is stale by one transition; recovery is recomputing the hash from disk.
SHCL-v2 attestation tables are programmatically rebuilt and replaced in their entirety on each transition — never via regex substitution.
Ambiguity Protocol — fail-closed routing conflict
Triggered at pick / activate when the card's surface field does not match the executor's --surface argument (e.g. "human"-surface card claimed by codex; "opus"-surface card claimed by jules). Mandatory response, executed in order:
- Halt. Command halts immediately. Card state unchanged. No execution log entry.
- Emit AMBIGUITY block on stdout with three fields:
evidence(conflicting signal pair, card id, title),options(route to declared surface, or operator-authorized re-assignment),recommendation(deduced correct path — a statement of fact, not an invitation to override). - Wait. No retry. No fallback routing. Card remains unclaimed.
- Human sign-off. Operator confirms the correct surface or authorizes re-assignment via card edit + manifest update.
Mechanical agents must not resolve AMBIGUITY blocks autonomously. The block must not be suppressed or redirected to a log. Retrying with the same conflicting surface produces the same halt.
Session closure checklist
- Artifacts — Tier 2 build artifacts exist; Layer 0 registries populated.
- Registry hygiene — health reports generated; current schema inspected.
- Deterministic baseline — run § V 7A sequence; witness returns OK.
- Stage validation — strict order Stage-3 → 4 → 5 → 6; determinism mode for Stages 4–6.
- Artifact integrity — active seal pointers coherent; manifest index exists.
- Task queue hygiene —
taskqueue.py statusshows no strandedactivecards;make index && make witnessafter any card approval or doc update. - Version control — all changes committed.