chore: initialize cli autoresearch segment

autoresearch.md

@@ -1,52 +1,44 @@
-# Autoresearch: dynamic-chat-templates
+# Autoresearch: localllm-cli-control-surface
 
 ## Objective
-Implement a dynamic chat-template system that selects prompt rendering by model family
-for local OpenAI-compatible `POST /v1/chat/completions` requests. The server should ship
-useful built-in templates for common open-source families such as Qwen, Gemma, Llama,
-and Kimi, while preserving a safe generic fallback for unknown models.
+Implement a full management CLI on top of the current local backend version, starting with
+a professional command surface for status, model operations, auth, config, logs, health,
+and throughput benchmarking. This segment focuses first on the CLI surface and contracts,
+so later experiments can add real execution behavior without redesigning the UX shape.
 
 ## Metrics
-- **Primary**: template_score (percent, higher is better)
+- **Primary**: cli_surface_score (percent, higher is better)
 - **Secondary**: passed_cases, failed_cases, total_cases
 
 ## How to Run
-`./autoresearch.sh` - runs targeted ignored tests for dynamic chat-template behavior and outputs `METRIC name=number` lines.
+`./autoresearch.sh` - runs targeted ignored tests for CLI command-surface behavior and outputs `METRIC name=number` lines.
 
 ## Files in Scope
-- `crates/inference-server/src/compat.rs` - prompt preparation, model catalog, template selection, family detection, and template-focused tests
-- `crates/inference-server/src/routes.rs` - route-level integration if template selection needs model-aware wiring
-- `crates/inference-server/src/main.rs` - server boot wiring if template config/env support is needed
-- `crates/api-contracts/src/chat.rs` - request/response roles and content handling that templates consume
-- `experiments/bench_chat_templates.py` - benchmark harness for the autoresearch loop
+- `crates/localllm-cli/` - new CLI crate, parser, command UX, output modes, and CLI-focused tests
+- `crates/inference-server/src/main.rs` - current backend env/config seam the CLI will eventually manage
+- `crates/inference-server/src/routes.rs` - existing inference endpoints the CLI can interrogate today
+- `crates/inference-server/src/compat.rs` - existing auth/config knobs the CLI must understand and expose coherently
+- `crates/api-contracts/` - shared contracts the CLI may eventually consume
+- `experiments/bench_cli_surface.py` - benchmark harness for the autoresearch loop
 - `autoresearch.sh` - loop benchmark entrypoint and fast preflight
 
 ## Off Limits
 - `models/` model binaries
 - `vendor/` third-party sources and prebuilt llama binaries
-- `registry/` Python registry service
+- `registry/` Python registry service unless a later CLI experiment explicitly needs to replace or wire it
 - `ui/` frontend files
-- unrelated latency tuning or runtime/engine experiments unless they are required to support template selection correctly
+- Tauri app implementation in this segment
+- unrelated latency or template experiments
 
 ## Constraints
-- No new external dependencies.
-- Preserve OpenAI-compatible behavior already implemented.
+- Preserve the currently working backend behavior.
 - Regular test suite must stay green for keeps.
-- Prefer explicit, maintainable template definitions over opaque heuristics.
-- Unknown models must continue to work through a generic fallback template.
-- The benchmark is test-driven: keep/discard follows the dynamic-template suite score first.
+- Prefer a stable, professional CLI contract over premature execution internals.
+- Commands should map cleanly to the product plan discussed with the user: status, models, auth, config, bench, logs, doctor.
+- The benchmark is test-driven: keep/discard follows the CLI surface suite score first.
+- A justified CLI-focused dependency is acceptable if it reduces maintenance cost.
 
 ## What's Been Tried
-- Segment 1 switches the autoresearch target from latency to dynamic chat-template selection by model family.
-- The current server still renders every prompt through one generic `System:/User:/Assistant:` formatter in `crates/inference-server/src/compat.rs`.
-- `prepare_prompt()` is the cleanest insertion point for dynamic templates because both blocking and streaming paths already consume its rendered string.
-- `ModelCatalog` currently resolves aliases only; it has no model-family metadata or template registry yet.
-- The benchmark for this segment is a targeted ignored test suite that scores family detection/template rendering coverage for Qwen, Gemma, Llama, Kimi, and the generic fallback.
-- Baseline run 7 scored only `20.0%` because just the generic fallback path passed; Qwen, Gemma, Llama, and Kimi were still rendered through the old generic formatter.
-- Run 8 (`ceb83f2`) added family detection plus built-in template selection inside `compat.rs` and immediately raised coverage to `100.0%` (5/5 passing), making it the current best for segment 1.
-- Segment 2 raises the bar by checking stronger Llama structure, especially whether system+user are merged into the first instruction block and whether the prompt avoids leaving an open `[INST]` prefix at generation time.
-- Run 10 (`fd6d35e`) hardened the Llama renderer with a dedicated formatting path that folds pending system/developer context into the next instruction block and eliminates the dangling open `[INST]` suffix, raising segment 2 from `71.429%` to `100.0%` (7/7 passing).
-- Segment 3 raises the bar again around alias-driven family detection, specifically ensuring a family hint carried only by an alias still selects the right template even when the canonical/root model id is generic.
-- Run 12 (`140c9b2`) fixed alias-driven family detection by checking the requested model string before falling back to resolved alias/root metadata, raising segment 3 from `88.889%` to `100.0%` (9/9 passing).
-- Segment 4 raises the bar around family-name variants, starting with Moonshot-branded Kimi aliases so common `moonshot` names select Kimi markers while unrelated lookalikes still fall back safely to `Generic`.
-- Run 14 (`f200733`) extended family detection so `moonshot` variant names map to the Kimi template, raising segment 4 from `90.909%` to `100.0%` (11/11 passing) without weakening generic fallback behavior.
+- Segment 5 pivots autoresearch from template hardening to productization, specifically a complete CLI-first control surface on top of the current backend version.
+- The current repo has no CLI crate, no Tauri app, and no control-plane crate; the backend is still configured almost entirely by `LOCALLLM_*` environment variables and exposes only `/health`, `/v1/models`, and `/v1/chat/completions`.
+- The first benchmark for this segment is intentionally surface-oriented: ignored tests score whether the CLI presents the expected professional command set before deeper execution behavior is implemented.