feat: add cli execution contracts
This commit is contained in:
1 parent
1881b63d7b
commit
812b486855
5 files changed
+54
-6
No files matched your search
@@ -188,6 +188,13 @@
|
||||
- Benchmark driver: `./autoresearch.sh` still runs ignored `localllm-cli` tests and reports `cli_surface_score`, `passed_cases`, `failed_cases`, and `total_cases`.
|
||||
- Keep/discard rule remains test-driven: higher `cli_surface_score` wins while the normal non-ignored suites remain green.
|
||||
|
||||
### Run 21: baseline - cli_surface_score=76.667 (KEEP)
|
||||
- Timestamp: 2026-03-26 19:11
|
||||
- What changed: Expanded the benchmark from `CommandPlan` semantics to `ExecutionContract` semantics, requiring the CLI to describe the concrete HTTP or local operation each high-value workflow intends to perform.
|
||||
- Result: cli_surface_score=76.667, passed_cases=23, failed_cases=7, total_cases=30.
|
||||
- Insight: The planning layer is now solid, but `execution_contract()` is still stubbed and returns `SummaryOnly { command: "unplanned" }` for every concrete plan, so the CLI cannot yet describe its backend/control-plane seam accurately.
|
||||
- Next: map the high-value `CommandPlan` variants into concrete HTTP/local execution contracts for status, models scan/load, auth enable, config set, throughput bench, and logs.
|
||||
|
||||
### Run 17: baseline - cli_surface_score=68.75 (KEEP)
|
||||
- Timestamp: 2026-03-26 18:59
|
||||
- What changed: Raised the benchmark with summary-focused ignored tests that require `render_command_summary()` to preserve output format, models directory, command-specific arguments, and key defaults for status, models load, auth enable, bench throughput, and logs.
|
||||
|
||||
Reference in new issue
Block a user