Experiments
Generated 2026-08-07T15:25:57.269080Z
Experiments the factory has run to validate its own assumptions. Each record includes hypothesis, metrics, evidence, and decision.
2026-07-28 → 2026-08-03
Hypothesis
Routing agent configuration through named OpenCode profiles (instead of per-session manual config) reduces setup overhead and enables reproducible agent dispatch without manual intervention.
Metrics
| Metric | Value | Description |
|---|
| pilot_sessions | 2 | Number of dispatch sessions using profile-based routing |
| exit_codes | 0,0 | Exit codes from pilot sessions (both succeeded) |
| lead_time | NOT INSTRUMENTED | Lead time from dispatch to completion — NOT INSTRUMENTED |
Evidence
Profile pilot session log (2 records, both exit_code=0)
Decision
Pilot completed successfully. Profile-based routing is technically viable. Lead time instrumentation gap identified — no automated dispatch-to-completion timer. Full promotion deferred pending the autonomous runner (factory_runner.py Phase 2).
Decided: 2026-08-03
2026-08-07
Hypothesis
Setting FOCULOOM_WORKFLOW_CLASS=full-pipeline via bash export before dispatching subagents will populate the workflow_class field in agent telemetry, enabling verification of conductor child-dispatch visibility.
Metrics
| Metric | Value | Description |
|---|
| full_pipeline_records | 0 | Number of telemetry records with workflow_class=full-pipeline (target: >=3) |
| test_result | FAIL | verify_b2_conductor_acceptance.py result |
Evidence
B2 acceptance defect event (2026-08-07): env-var injection gap documented
Decision
Blocked — bash export does not propagate to the opencode process (the plugin reads process.env at startup, not from bash). The task tool has no env parameter. Requires either a plugin code change (per-dispatch injection) or a platform change (task tool env param). Plugin emitter changes shipped in S111 but require opencode restart to take effect.
2026-08-07 → running
Hypothesis
A hard-gated min() maturity engine running in shadow mode for 30 days will produce stable, reproducible maturity level computations across re-runs before the public /factory/maturity/ page publishes a level.
Metrics
| Metric | Value | Description |
|---|
| shadow_runs | 7 | Number of shadow-mode maturity computations logged |
| aggregate_level | NOT COMPUTABLE | Current aggregate level (blocked by provenance, lights_out, accepted_work dimensions being unknown/null) |
| dimensions_fresh | 3 of 6 | Dimensions with fresh timestamps (scope_coverage, recovery, sustained_evidence) |
Evidence
Maturity shadow log (7 runs, all NOT COMPUTABLE due to uninstrumented dimensions)
Decision
Running. Shadow mode started 2026-08-07. Promotion session needed ~2026-09-06 after 30 calendar days of stable computation. Three dimensions (provenance, lights_out, accepted_work) remain uninstrumented — the aggregate cannot leave NOT COMPUTABLE until they are populated.