feat: add cli execution plans

autoresearch-dashboard.md

@@ -1,10 +1,9 @@
-# Autoresearch Dashboard: dynamic-chat-templates-variants
+# Autoresearch Dashboard: localllm-cli-execution-plans
 
-**Runs:** 2 | **Kept:** 2 | **Discarded:** 0 | **Crashed:** 0
-**Baseline:** template_score: 90.909% (#13)
-**Best:** template_score: 100.0% (#14, +10.0%)
+**Runs:** 1 | **Kept:** 1 | **Discarded:** 0 | **Crashed:** 0
+**Baseline:** cli_surface_score: 69.565% (#19)
+**Best:** cli_surface_score: 69.565% (#19, +0.0%)
 
-| # | commit | template_score | status | description |
-|---|--------|----------------|--------|-------------|
-| 13 | ffab766 | 90.909% (+0.0%) | keep | baseline |
-| 14 | f200733 | 100.0% (+10.0%) | keep | detect moonshot as kimi family |
+| # | commit | cli_surface_score | status | description |
+|---|--------|-------------------|--------|-------------|
+| 19 | 9154a7b | 69.565% (+0.0%) | keep | baseline |

autoresearch.jsonl

@@ -21,3 +21,7 @@
 {"run":15,"commit":"fd773a9","metric":18.182,"metrics":{"passed_cases":2,"failed_cases":9,"total_cases":11},"status":"keep","description":"baseline","timestamp":1774562027,"segment":5}
 {"run":16,"commit":"70cc59a","metric":100.0,"metrics":{"passed_cases":11,"failed_cases":0,"total_cases":11},"status":"keep","description":"expand cli command surface","timestamp":1774562138,"segment":5}
 {"type":"config","name":"localllm-cli-semantic-contracts","metricName":"cli_surface_score","metricUnit":"%","bestDirection":"higher"}
+{"run":17,"commit":"53a7dbb","metric":68.75,"metrics":{"passed_cases":11,"failed_cases":5,"total_cases":16},"status":"keep","description":"baseline","timestamp":1774562358,"segment":6}
+{"run":18,"commit":"66b30de","metric":100.0,"metrics":{"passed_cases":16,"failed_cases":0,"total_cases":16},"status":"keep","description":"enrich cli semantic summaries","timestamp":1774562434,"segment":6}
+{"type":"config","name":"localllm-cli-execution-plans","metricName":"cli_surface_score","metricUnit":"%","bestDirection":"higher"}
+{"run":19,"commit":"9154a7b","metric":69.565,"metrics":{"passed_cases":16,"failed_cases":7,"total_cases":23},"status":"keep","description":"baseline","timestamp":1774562688,"segment":7}

autoresearch.md

@@ -45,3 +45,7 @@ so later experiments can add real execution behavior without redesigning the UX
 - Baseline run 15 (`fd773a9`) scored only `18.182%`: the new CLI scaffold currently exposes `status` and `models list`, but the broader management surface (`models scan/load/watch/download`, `auth`, `config`, `bench`, `logs`, `doctor`) is still absent.
 - Run 16 (`70cc59a`) expanded the Clap parser to the full intended command surface and raised segment 5 from `18.182%` to `100.0%` (11/11 passing), giving the CLI a stable UX contract before execution handlers are wired.
 - Segment 6 raises the bar from command presence to semantic usefulness: the benchmark now checks whether key workflows produce meaningful command summaries that preserve defaults and high-value arguments, as a stepping stone toward real execution planning.
+- Baseline run 17 (`53a7dbb`) scored `68.75%`: the parser already accepted the workflows, but `render_command_summary()` still threw away important semantic detail such as nested command paths, output format, alias selection, and throughput defaults.
+- Run 18 (`66b30de`) enriched semantic summaries across the CLI and raised segment 6 from `68.75%` to `100.0%` (16/16 passing), so the next bar should move from summary correctness to execution-planning or output-contract semantics.
+- Segment 7 raises the bar from semantic summaries to structured execution plans, checking whether key commands preserve actionable details in a machine-usable plan that can later drive real handlers.
+- Baseline run 19 (`9154a7b`) scored `69.565%`: the CLI still passes all parser and summary tests, but every new plan-oriented workflow fails because `plan_command()` currently returns `SummaryOnly` for all commands.

crates/localllm-cli/src/main.rs

@@ -233,8 +233,59 @@ fn render_command_summary(cli: &Cli) -> String {
 }
 
 fn plan_command(cli: &Cli) -> CommandPlan {
-    CommandPlan::SummaryOnly {
-        command: command_summary(&cli.command),
+    match &cli.command {
+        Commands::Status => CommandPlan::ProbeStatus {
+            server: cli.server.clone(),
+            api_key_present: cli.api_key.is_some(),
+            format: cli.format,
+        },
+        Commands::Models(models) => match &models.command {
+            ModelsSubcommand::Scan => CommandPlan::ScanModels {
+                models_dir: cli.models_dir.clone(),
+            },
+            ModelsSubcommand::Load(args) => CommandPlan::LoadModel {
+                models_dir: cli.models_dir.clone(),
+                model: args.model.clone(),
+                alias: args.alias.clone(),
+            },
+            _ => CommandPlan::SummaryOnly {
+                command: command_summary(&cli.command),
+            },
+        },
+        Commands::Auth(auth) => match &auth.command {
+            AuthSubcommand::Enable(args) => CommandPlan::ConfigureAuth {
+                mode: args.mode,
+                api_key: args.api_key.clone(),
+            },
+            _ => CommandPlan::SummaryOnly {
+                command: command_summary(&cli.command),
+            },
+        },
+        Commands::Config(config) => match &config.command {
+            ConfigSubcommand::Set(args) => CommandPlan::SetConfig {
+                key: args.key.clone(),
+                value: args.value.clone(),
+            },
+            _ => CommandPlan::SummaryOnly {
+                command: command_summary(&cli.command),
+            },
+        },
+        Commands::Bench(bench) => match &bench.command {
+            BenchSubcommand::Throughput(args) => CommandPlan::RunThroughputBench {
+                server: cli.server.clone(),
+                api_key_present: cli.api_key.is_some(),
+                samples: args.samples,
+                prompt_tokens: args.prompt_tokens,
+                completion_tokens: args.completion_tokens,
+            },
+            _ => CommandPlan::SummaryOnly {
+                command: command_summary(&cli.command),
+            },
+        },
+        Commands::Logs(args) => CommandPlan::ReadLogs { lines: args.lines },
+        Commands::Doctor => CommandPlan::SummaryOnly {
+            command: command_summary(&cli.command),
+        },
     }
 }
 

experiments/worklog.md

@@ -163,3 +163,29 @@
 - New objective: keep the CLI surface stable, but raise the benchmark so key commands preserve useful semantic detail such as defaults, high-value arguments, and summary/intention contracts.
 - Benchmark driver: `./autoresearch.sh` still runs ignored `localllm-cli` tests and reports `cli_surface_score`, `passed_cases`, `failed_cases`, and `total_cases`.
 - Keep/discard rule remains test-driven: higher `cli_surface_score` wins while the normal non-ignored suites remain green.
+
+## Segment 7: CLI Execution Plans
+- New objective: raise the CLI benchmark from human-readable semantic summaries to machine-usable execution plans for key workflows.
+- Benchmark driver: `./autoresearch.sh` still runs ignored `localllm-cli` tests and reports `cli_surface_score`, `passed_cases`, `failed_cases`, and `total_cases`.
+- Keep/discard rule remains test-driven: higher `cli_surface_score` wins while the normal non-ignored suites remain green.
+
+### Run 19: baseline - cli_surface_score=69.565 (KEEP)
+- Timestamp: 2026-03-26 19:04
+- What changed: Expanded the CLI benchmark with plan-oriented ignored tests that require key commands to map into machine-usable `CommandPlan` variants instead of only semantic summaries.
+- Result: cli_surface_score=69.565, passed_cases=16, failed_cases=7, total_cases=23.
+- Insight: The semantic summary layer is already solid, but `plan_command()` is still a stub that collapses everything to `SummaryOnly`, so execution planning has not been wired yet.
+- Next: implement concrete `CommandPlan` mapping for status, models scan/load, auth enable, config set, bench throughput, and logs.
+
+### Run 17: baseline - cli_surface_score=68.75 (KEEP)
+- Timestamp: 2026-03-26 18:59
+- What changed: Raised the benchmark with summary-focused ignored tests that require `render_command_summary()` to preserve output format, models directory, command-specific arguments, and key defaults for status, models load, auth enable, bench throughput, and logs.
+- Result: cli_surface_score=68.75, passed_cases=11, failed_cases=5, total_cases=16.
+- Insight: The parser surface is solid, but the current summary layer is too shallow; it only reports the top-level command and server, losing the semantic detail needed for later execution planning.
+- Next: upgrade `render_command_summary()` so it emits nested command paths and the highest-value arguments/defaults for each management workflow.
+
+### Run 18: enrich cli semantic summaries - cli_surface_score=100.0 (KEEP)
+- Timestamp: 2026-03-26 19:00
+- What changed: Reworked `render_command_summary()` in `crates/localllm-cli/src/main.rs` to emit nested command paths plus key defaults and high-value arguments for status, models, auth, config, bench, and logs workflows.
+- Result: cli_surface_score=100.0, passed_cases=16, failed_cases=0, total_cases=16.
+- Insight: A lightweight semantic/intention layer is enough to make the CLI contract meaningfully richer before wiring real handlers; the CLI now preserves actionable detail instead of collapsing everything to top-level groups.
+- Next: raise the benchmark again toward execution planning or output contracts, e.g. structured action plans for backend/admin operations rather than just human-readable summaries.