chore: expand cli semantic benchmark coverage
This commit is contained in:
1 parent
d1dbc27031
commit
a3f5585700
4 files changed
+121
-5
No files matched your search
@@ -42,3 +42,6 @@ so later experiments can add real execution behavior without redesigning the UX
|
||||
- Segment 5 pivots autoresearch from template hardening to productization, specifically a complete CLI-first control surface on top of the current backend version.
|
||||
- The current repo has no CLI crate, no Tauri app, and no control-plane crate; the backend is still configured almost entirely by `LOCALLLM_*` environment variables and exposes only `/health`, `/v1/models`, and `/v1/chat/completions`.
|
||||
- The first benchmark for this segment is intentionally surface-oriented: ignored tests score whether the CLI presents the expected professional command set before deeper execution behavior is implemented.
|
||||
- Baseline run 15 (`fd773a9`) scored only `18.182%`: the new CLI scaffold currently exposes `status` and `models list`, but the broader management surface (`models scan/load/watch/download`, `auth`, `config`, `bench`, `logs`, `doctor`) is still absent.
|
||||
- Run 16 (`70cc59a`) expanded the Clap parser to the full intended command surface and raised segment 5 from `18.182%` to `100.0%` (11/11 passing), giving the CLI a stable UX contract before execution handlers are wired.
|
||||
- Segment 6 raises the bar from command presence to semantic usefulness: the benchmark now checks whether key workflows produce meaningful command summaries that preserve defaults and high-value arguments, as a stepping stone toward real execution planning.
|
||||
Reference in new issue
Block a user