feat: add cli execution contracts

This commit is contained in:
Evandro Fonseca Junior committed 2026-03-26 19:14:00 -03:00
1 parent 1881b63d7b
commit 812b486855
5 files changed
+54 -6

No files matched your search

+7
View File
@@ -188,6 +188,13 @@
- Benchmark driver: `./autoresearch.sh` still runs ignored `localllm-cli` tests and reports `cli_surface_score`, `passed_cases`, `failed_cases`, and `total_cases`.
- Keep/discard rule remains test-driven: higher `cli_surface_score` wins while the normal non-ignored suites remain green.
### Run 21: baseline - cli_surface_score=76.667 (KEEP)
- Timestamp: 2026-03-26 19:11
- What changed: Expanded the benchmark from `CommandPlan` semantics to `ExecutionContract` semantics, requiring the CLI to describe the concrete HTTP or local operation each high-value workflow intends to perform.
- Result: cli_surface_score=76.667, passed_cases=23, failed_cases=7, total_cases=30.
- Insight: The planning layer is now solid, but `execution_contract()` is still stubbed and returns `SummaryOnly { command: "unplanned" }` for every concrete plan, so the CLI cannot yet describe its backend/control-plane seam accurately.
- Next: map the high-value `CommandPlan` variants into concrete HTTP/local execution contracts for status, models scan/load, auth enable, config set, throughput bench, and logs.
### Run 17: baseline - cli_surface_score=68.75 (KEEP)
- Timestamp: 2026-03-26 18:59
- What changed: Raised the benchmark with summary-focused ignored tests that require `render_command_summary()` to preserve output format, models directory, command-specific arguments, and key defaults for status, models load, auth enable, bench throughput, and logs.