feat: add cli backend invocation specs

This commit is contained in:
Evandro Fonseca Junior committed 2026-03-26 19:41:38 -03:00
1 parent 3969f12f26
commit f538fc60bb
4 files changed
+108 -8

No files matched your search

+57
View File
@@ -220,3 +220,60 @@
- New objective: raise the CLI benchmark from abstract execution contracts to concrete execution stubs that preserve HTTP method/path/body semantics or named local actions for the highest-value workflows.
- Benchmark driver: `./autoresearch.sh` still runs ignored `localllm-cli` tests and reports `cli_surface_score`, `passed_cases`, `failed_cases`, and `total_cases`.
- Keep/discard rule remains test-driven: higher `cli_surface_score` wins while the normal non-ignored suites remain green.
### Run 23: baseline - cli_surface_score=83.333 (KEEP)
- Timestamp: 2026-03-26 19:20
- What changed: Expanded the benchmark from execution contracts to concrete execution stubs, adding ignored tests for HTTP method/path/body semantics and named local actions for status, model scan/load, auth enable, throughput bench, and logs.
- Result: cli_surface_score=83.333, passed_cases=30, failed_cases=6, total_cases=36.
- Insight: The CLI already knows how to parse commands and describe execution contracts, but `execution_stub()` is still just a placeholder returning `SummaryOnly { command: "unimplemented" }` for every high-value workflow.
- Next: implement concrete stub mapping so status and throughput produce execution-ready HTTP request shapes and local workflows emit named local actions with preserved detail.
### Run 24: add cli execution stubs - cli_surface_score=100.0 (KEEP)
- Timestamp: 2026-03-26 19:22
- What changed: Implemented concrete `execution_stub()` mapping in `crates/localllm-cli/src/main.rs`, turning the highest-value execution contracts into execution-ready HTTP requests and named local actions with preserved detail.
- Result: cli_surface_score=100.0, passed_cases=36, failed_cases=0, total_cases=36.
- Insight: The CLI now has a full design ladder for the key workflows - parser, semantic summary, machine-usable plan, execution contract, and finally execution stub - which is a strong handoff point for wiring real handlers next.
- Next: raise the benchmark from stub shape to actual handler behavior or request payload construction for the same high-value workflows.
## Segment 10: CLI Execution Previews
- New objective: raise the CLI benchmark from execution stubs to dry-run previews that expand the same high-value workflows into full previewable HTTP URLs or local-effect descriptions.
- Benchmark driver: `./autoresearch.sh` still runs ignored `localllm-cli` tests and reports `cli_surface_score`, `passed_cases`, `failed_cases`, and `total_cases`.
- Keep/discard rule remains test-driven: higher `cli_surface_score` wins while the normal non-ignored suites remain green.
### Run 25: baseline - cli_surface_score=85.714 (KEEP)
- Timestamp: 2026-03-26 19:27
- What changed: Expanded the benchmark from execution stubs to dry-run previews, adding ignored tests for full preview URLs and human-readable local-effect descriptions across status, models scan/load, auth enable, throughput bench, and logs.
- Result: cli_surface_score=85.714, passed_cases=36, failed_cases=6, total_cases=42.
- Insight: The CLI already models execution stubs correctly, but `execution_preview()` still collapses every high-value workflow to `SummaryOnly { command: "preview-unimplemented" }`, so there is no user-facing dry-run layer yet.
- Next: map the highest-value execution stubs into concrete `HttpPreview` and `LocalPreview` values using the resolved `--server` base URL and readable local-effect text.
### Run 26: add cli execution previews - cli_surface_score=100.0 (KEEP)
- Timestamp: 2026-03-26 19:28
- What changed: Implemented concrete `execution_preview()` mapping in `crates/localllm-cli/src/main.rs`, expanding HTTP stubs into full URLs with preserved payloads and local actions into readable effect descriptions via `preview_effect()`.
- Result: cli_surface_score=100.0, passed_cases=42, failed_cases=0, total_cases=42.
- Insight: The CLI now has a complete dry-run preview layer for the highest-value workflows, which is a strong bridge from internal planning seams to future real handlers and `--dry-run` UX.
- Next: raise the benchmark from preview shape to actual handler behavior or request-body construction details for status, throughput, and model-management workflows.
## Segment 11: CLI Dry-Run Output
- New objective: raise the CLI benchmark from internal preview objects to final user-facing `--dry-run` output in both `text` and `json`, so the same high-value workflows are ready for a real preview UX at the command line.
- Benchmark driver: `./autoresearch.sh` still runs ignored `localllm-cli` tests and reports `cli_surface_score`, `passed_cases`, `failed_cases`, and `total_cases`.
- Keep/discard rule remains test-driven: higher `cli_surface_score` wins while the normal non-ignored suites remain green.
### Run 27: baseline - cli_surface_score=87.5 (KEEP)
- Timestamp: 2026-03-26 19:32
- What changed: Expanded the benchmark from internal `ExecutionPreview` objects to final `--dry-run` UX in both text and JSON, while keeping `render_execution_preview()` intentionally shallow as `DRY RUN {summary}`.
- Result: cli_surface_score=87.5, passed_cases=42, failed_cases=6, total_cases=48.
- Insight: The CLI already knows how to build the preview object graph, but the final presentation layer still collapses to summaries instead of rendering health/chat HTTP previews and local-effect dry-run text.
- Next: render the existing preview layer into final text/json output using the current `ExecutionPreview` data rather than introducing a new execution seam.
### Run 28: render cli dry-run previews - cli_surface_score=100.0 (KEEP)
- Timestamp: 2026-03-26 19:34
- What changed: Implemented final `--dry-run` rendering in `crates/localllm-cli/src/main.rs`, mapping `ExecutionPreview` values to user-facing text and JSON output and reusing the existing preview pipeline (`plan_command` -> `execution_contract` -> `execution_stub` -> `execution_preview`).
- Result: cli_surface_score=100.0, passed_cases=48, failed_cases=0, total_cases=48.
- Insight: The CLI now has a complete dry-run stack for the highest-value workflows, from parser through final user-visible text/json previews, without yet executing real side effects.
- Next: raise the bar from dry-run rendering to real handler behavior or deterministic request/body construction for backend-facing commands.
## Segment 12: CLI Backend Invocations
- New objective: raise the CLI benchmark from dry-run rendering to the first real backend-call specifications for commands that can already work today, especially `status`, `models list`, and `bench throughput`.
- Benchmark driver: `./autoresearch.sh` still runs ignored `localllm-cli` tests and reports `cli_surface_score`, `passed_cases`, `failed_cases`, and `total_cases`.
- Keep/discard rule remains test-driven: higher `cli_surface_score` wins while the normal non-ignored suites remain green.