feat: add cli execution plans
This commit is contained in:
1 parent
501d39933f
commit
921cead9fd
5 files changed
+94
-10
No files matched your search
@@ -163,3 +163,29 @@
|
||||
- New objective: keep the CLI surface stable, but raise the benchmark so key commands preserve useful semantic detail such as defaults, high-value arguments, and summary/intention contracts.
|
||||
- Benchmark driver: `./autoresearch.sh` still runs ignored `localllm-cli` tests and reports `cli_surface_score`, `passed_cases`, `failed_cases`, and `total_cases`.
|
||||
- Keep/discard rule remains test-driven: higher `cli_surface_score` wins while the normal non-ignored suites remain green.
|
||||
|
||||
## Segment 7: CLI Execution Plans
|
||||
- New objective: raise the CLI benchmark from human-readable semantic summaries to machine-usable execution plans for key workflows.
|
||||
- Benchmark driver: `./autoresearch.sh` still runs ignored `localllm-cli` tests and reports `cli_surface_score`, `passed_cases`, `failed_cases`, and `total_cases`.
|
||||
- Keep/discard rule remains test-driven: higher `cli_surface_score` wins while the normal non-ignored suites remain green.
|
||||
|
||||
### Run 19: baseline - cli_surface_score=69.565 (KEEP)
|
||||
- Timestamp: 2026-03-26 19:04
|
||||
- What changed: Expanded the CLI benchmark with plan-oriented ignored tests that require key commands to map into machine-usable `CommandPlan` variants instead of only semantic summaries.
|
||||
- Result: cli_surface_score=69.565, passed_cases=16, failed_cases=7, total_cases=23.
|
||||
- Insight: The semantic summary layer is already solid, but `plan_command()` is still a stub that collapses everything to `SummaryOnly`, so execution planning has not been wired yet.
|
||||
- Next: implement concrete `CommandPlan` mapping for status, models scan/load, auth enable, config set, bench throughput, and logs.
|
||||
|
||||
### Run 17: baseline - cli_surface_score=68.75 (KEEP)
|
||||
- Timestamp: 2026-03-26 18:59
|
||||
- What changed: Raised the benchmark with summary-focused ignored tests that require `render_command_summary()` to preserve output format, models directory, command-specific arguments, and key defaults for status, models load, auth enable, bench throughput, and logs.
|
||||
- Result: cli_surface_score=68.75, passed_cases=11, failed_cases=5, total_cases=16.
|
||||
- Insight: The parser surface is solid, but the current summary layer is too shallow; it only reports the top-level command and server, losing the semantic detail needed for later execution planning.
|
||||
- Next: upgrade `render_command_summary()` so it emits nested command paths and the highest-value arguments/defaults for each management workflow.
|
||||
|
||||
### Run 18: enrich cli semantic summaries - cli_surface_score=100.0 (KEEP)
|
||||
- Timestamp: 2026-03-26 19:00
|
||||
- What changed: Reworked `render_command_summary()` in `crates/localllm-cli/src/main.rs` to emit nested command paths plus key defaults and high-value arguments for status, models, auth, config, bench, and logs workflows.
|
||||
- Result: cli_surface_score=100.0, passed_cases=16, failed_cases=0, total_cases=16.
|
||||
- Insight: A lightweight semantic/intention layer is enough to make the CLI contract meaningfully richer before wiring real handlers; the CLI now preserves actionable detail instead of collapsing everything to top-level groups.
|
||||
- Next: raise the benchmark again toward execution planning or output contracts, e.g. structured action plans for backend/admin operations rather than just human-readable summaries.
|
||||
Reference in new issue
Block a user