feat: add runtime lifecycle health revalidation

autoresearch.md

@@ -1,80 +1,56 @@
-# Autoresearch: localllm-cli-control-surface
+# Autoresearch: tauri-runtime-lifecycle-acceptance
 
 ## Objective
-Implement a full management CLI on top of the current local backend version, starting with
-a professional command surface for status, model operations, auth, config, logs, health,
-and throughput benchmarking. This segment focuses first on the CLI surface and contracts,
-so later experiments can add real execution behavior without redesigning the UX shape.
+Harden the Tauri desktop runtime manager around lifecycle acceptance so the saved
+runtime flow stays trustworthy from configuration through apply/start behavior. This
+segment focuses on the app-surface contract: local/remote mode transitions should show
+the right controls immediately, local startup should stay driven by the saved runtime
+state, and the verification harness should score the desktop runtime instead of the old
+CLI surface.
 
 ## Metrics
-- **Primary**: cli_surface_score (percent, higher is better)
-- **Secondary**: passed_cases, failed_cases, total_cases
+- **Primary**: runtime_lifecycle_score (percent, higher is better)
+- **Secondary**: unit_passed, build_passed, cargo_check_passed, local_mode_cta_passed, saved_startup_flow_passed, local_onboarding_flow_passed, health_revalidation_passed
 
 ## How to Run
-`./autoresearch.sh` - runs targeted ignored tests for CLI command-surface behavior and outputs `METRIC name=number` lines.
+- `./autoresearch.sh`
+- `cd tauri-app && npm test -- src/features/runtime/runtime-utils.test.ts src/features/runtime/components/RuntimeSurface.test.tsx`
+- `cd tauri-app && npm run build`
+- `cargo check --manifest-path tauri-app/src-tauri/Cargo.toml`
 
 ## Files in Scope
-- `crates/localllm-cli/` - new CLI crate, parser, command UX, output modes, and CLI-focused tests
-- `crates/inference-server/src/main.rs` - current backend env/config seam the CLI will eventually manage
-- `crates/inference-server/src/routes.rs` - existing inference endpoints the CLI can interrogate today
-- `crates/inference-server/src/compat.rs` - existing auth/config knobs the CLI must understand and expose coherently
-- `crates/api-contracts/` - shared contracts the CLI may eventually consume
-- `experiments/bench_cli_surface.py` - benchmark harness for the autoresearch loop
-- `autoresearch.sh` - loop benchmark entrypoint and fast preflight
+- `tauri-app/src/App.tsx` - app shell ownership for runtime and model connectivity
+- `tauri-app/src/components/Sidebar.tsx` - sidebar entry point for runtime controls
+- `tauri-app/src/features/runtime/` - unified runtime domain, UI, utils, and Tauri API wrappers
+- `tauri-app/src/index.css` - global visual tokens for the desktop app
+- `tauri-app/src/hooks/useModels.ts` - model loading through resolved runtime connection
+- `tauri-app/src-tauri/src/runtime/` - backend source of truth for runtime config/state
+- `tauri-app/src-tauri/src/lib.rs` - command wiring and legacy compatibility seams
 
 ## Off Limits
 - `models/` model binaries
 - `vendor/` third-party sources and prebuilt llama binaries
-- `registry/` Python registry service unless a later CLI experiment explicitly needs to replace or wire it
-- `ui/` frontend files
-- Tauri app implementation in this segment
-- unrelated latency or template experiments
+- unrelated CLI autoresearch files and crates unless a runtime experiment explicitly needs them
+- unrelated latency/template experiments
 
 ## Constraints
-- Preserve the currently working backend behavior.
-- Regular test suite must stay green for keeps.
-- Prefer a stable, professional CLI contract over premature execution internals.
-- Commands should map cleanly to the product plan discussed with the user: status, models, auth, config, bench, logs, doctor.
-- The benchmark is test-driven: keep/discard follows the CLI surface suite score first.
-- A justified CLI-focused dependency is acceptable if it reduces maintenance cost.
+- Backend/Tauri remains the source of truth for runtime config and process state.
+- The frontend should expose one startup/config flow, not parallel state machines.
+- Remote mode must never silently fall back to localhost.
+- Verification must include `npm test`, `npm run build`, and `cargo check`.
+- Preserve existing chat/model behavior outside the runtime-management refactor.
 
 ## What's Been Tried
-- Segment 5 pivots autoresearch from template hardening to productization, specifically a complete CLI-first control surface on top of the current backend version.
-- The current repo has no CLI crate, no Tauri app, and no control-plane crate; the backend is still configured almost entirely by `LOCALLLM_*` environment variables and exposes only `/health`, `/v1/models`, and `/v1/chat/completions`.
-- The first benchmark for this segment is intentionally surface-oriented: ignored tests score whether the CLI presents the expected professional command set before deeper execution behavior is implemented.
-- Baseline run 15 (`fd773a9`) scored only `18.182%`: the new CLI scaffold currently exposes `status` and `models list`, but the broader management surface (`models scan/load/watch/download`, `auth`, `config`, `bench`, `logs`, `doctor`) is still absent.
-- Run 16 (`70cc59a`) expanded the Clap parser to the full intended command surface and raised segment 5 from `18.182%` to `100.0%` (11/11 passing), giving the CLI a stable UX contract before execution handlers are wired.
-- Segment 6 raises the bar from command presence to semantic usefulness: the benchmark now checks whether key workflows produce meaningful command summaries that preserve defaults and high-value arguments, as a stepping stone toward real execution planning.
-- Baseline run 17 (`53a7dbb`) scored `68.75%`: the parser already accepted the workflows, but `render_command_summary()` still threw away important semantic detail such as nested command paths, output format, alias selection, and throughput defaults.
-- Run 18 (`66b30de`) enriched semantic summaries across the CLI and raised segment 6 from `68.75%` to `100.0%` (16/16 passing), so the next bar should move from summary correctness to execution-planning or output-contract semantics.
-- Segment 7 raises the bar from semantic summaries to structured execution plans, checking whether key commands preserve actionable details in a machine-usable plan that can later drive real handlers.
-- Baseline run 19 (`9154a7b`) scored `69.565%`: the CLI still passes all parser and summary tests, but every new plan-oriented workflow fails because `plan_command()` currently returns `SummaryOnly` for all commands.
-- Run 20 (`bc584be`) introduced concrete `CommandPlan` variants for status, model scan/load, auth enable, config set, throughput bench, and logs, raising segment 7 from `69.565%` to `100.0%` (23/23 passing).
-- Segment 8 raises the bar from machine-usable plans to execution contracts, checking whether key workflows can already describe the concrete HTTP/local action they intend to perform at the backend/control-plane seam.
-- Baseline run 21 (`3d1460b`) scored `76.667%`: the CLI already preserves parser, summary, and plan semantics, but `execution_contract()` still collapses the new workflows to `SummaryOnly { command: "unplanned" }` instead of concrete HTTP/local contracts.
-- Run 22 (`38acb52`) mapped the highest-value `CommandPlan` variants into concrete `ExecutionContract` values for `/health`, `/v1/chat/completions`, local model scan/load, local auth/config mutation, and local log reads, raising segment 8 from `76.667%` to `100.0%` (30/30 passing).
-- Segment 9 raises the bar from contract shape to execution stubs, checking whether the same high-value workflows can already describe concrete HTTP method/path/body details or named local actions that a future real executor can run directly.
-- Baseline run 23 (`7030290`) scored `83.333%`: parser, summaries, plans, and execution contracts still pass, but `execution_stub()` is only a placeholder and returns `SummaryOnly { command: "unimplemented" }` instead of concrete HTTP/local stubs for the highest-value workflows.
-- Run 24 (`50d435a`) mapped the highest-value `ExecutionContract` variants into concrete `ExecutionStub` values for `GET /health`, `POST /v1/chat/completions`, local model scan/load, local auth config, and local log tail actions, raising segment 9 from `83.333%` to `100.0%` (36/36 passing).
-- Segment 10 raises the bar from execution stubs to dry-run previews, checking whether the same high-value workflows can expand into full previewable HTTP URLs or local-effect descriptions that are ready to show in a future `--dry-run` UX.
-- Baseline run 25 (`cd480c4`) scored `85.714%`: parser, summaries, plans, contracts, and stubs still pass, but `execution_preview()` remained a placeholder and returned `SummaryOnly { command: "preview-unimplemented" }` instead of final dry-run previews.
-- Run 26 (`c8ef7cf`) mapped the highest-value execution stubs into concrete `ExecutionPreview` values with full health/chat URLs and readable local-effect descriptions, raising segment 10 from `85.714%` to `100.0%` (42/42 passing).
-- Segment 11 raises the bar from preview objects to final `--dry-run` output, checking whether the CLI can now render those previews as actual text/json UX for the same high-value workflows.
-- Baseline run 27 (`7738acf`) scored `87.5%`: the CLI already had parser/summary/plan/contract/stub/preview layers, but `render_execution_preview()` still collapsed everything to `DRY RUN {summary}` instead of final text/json dry-run output.
-- Run 28 (`99e5041`) rendered the highest-value `ExecutionPreview` values into final user-facing text/json dry-run output, raising segment 11 from `87.5%` to `100.0%` (48/48 passing).
-- Segment 12 raises the bar from dry-run UX to the first real backend-call specifications for commands that can already work today, specifically `status`, `models list`, and `bench throughput`.
-- Baseline run 29 (`1ff5ba3`) scored `94.118%`: every earlier CLI layer still passed, but `backend_invocation()` was still a placeholder and the new backend-facing tests for live status/models/bench requests failed.
-- Run 30 (`bf40b2f`) added concrete backend invocation specs for `GET /health`, `GET /v1/models`, and `POST /v1/chat/completions` with preserved `--model`, token budget, server base URL, and auth presence, raising segment 12 from `94.118%` to `100.0%` (51/51 passing).
-- Segment 13 raises the bar again from backend invocation specs to executable backend request construction, checking whether the same live-compatible commands can now produce final HTTP method/url/header/body request objects with deterministic JSON payloads.
-- Run 31 (`f970c05`) scored `94.444%` because `backend_request()` was still a placeholder and the new backend-request tests failed for `status`, `models list`, and `bench throughput`.
-- Run 32 (`5de35c5`) introduced concrete executable backend request specs with bearer/content-type headers and deterministic JSON benchmark payloads, raising segment 13 from `94.444%` to `100.0%` (54/54 passing).
-- Segment 14 raises the bar from executable backend request specs to backend response interpretation, checking whether the same live-compatible commands can parse real response bodies into CLI-facing previews for health, model lists, and throughput usage metrics.
-- Run 33 (`8cf03e3`) scored `94.737%` because `backend_response_preview()` was still a placeholder and the new response-parsing tests failed for `status`, `models list`, and `bench throughput`.
-- Run 34 (`0506a4c`) introduced CLI-local backend response previews for health payloads, model list ids/count, and throughput usage metrics, raising segment 14 from `94.737%` to `100.0%` (57/57 passing).
-- Segment 15 raises the bar from internal backend response previews to final user-facing live output rendering, checking whether the same live-compatible commands can render text/json output directly from parsed backend responses.
-- Run 35 (`e1ada2c`) scored `90.476%` because `render_backend_response()` still collapsed the live-compatible workflows to `LIVE {summary}` instead of final text/json output.
-- Run 36 (`c54107d`) rendered the first live-compatible backend responses into final user-facing text/json output, raising segment 15 from `90.476%` to `100.0%` (63/63 passing).
-- Segment 16 raises the bar from live output rendering to the first real executable handlers, checking whether the same live-compatible workflows can execute through an injectable transport seam and return the final rendered output end-to-end.
-- Run 37 (`388733a`) scored `95.455%` because `execute_live_command_with()` still ignored the transport response and returned `LIVE EXEC {summary}` instead of final live output.
-- Run 38 (`6920e2a`) wired the injected transport response into `render_backend_response()`, raising segment 16 from `95.455%` to `100.0%` (66/66 passing).
-- Segment 17 adds `ureq` as the real HTTP transport and implements `execute_live_command(cli)` that delegates to `execute_live_command_with()` using `ureq`. Run 39 (`9860d64`) scored `100.0%` (69/69 passing) as a direct keep — the CLI is now functionally complete for the commands the current backend supports.
+- Earlier autoresearch segments 0-17 focused on latency, dynamic chat templates, and a CLI management surface. The latest completed CLI segment (`localllm-cli-ureq-transport`) reached `100.0%` on run 39 (`9860d64`), but that objective is now stale for the desktop app work.
+- The Tauri chat layer was previously fixed so new conversations persist correctly, `<think>...</think>` content is separated into a collapsible thinking panel, and per-message/per-chat token accounting is stored and rendered.
+- The first runtime-manager slice introduced a backend runtime domain in `tauri-app/src-tauri/src/runtime/`, a frontend runtime feature in `tauri-app/src/features/runtime/`, a compact sidebar status card, and a settings drawer.
+- Review of that slice exposed two major correctness issues that were fixed: remote mode no longer falls back to localhost, and runtime config changes that would drift away from a running child process are now blocked with a restart-required error.
+- Even after those fixes, the runtime manager still failed in practice because the app kept two frontend control planes: the new runtime card/drawer started the backend from `RuntimeState.inner.config`, while the embedded legacy `ServerPanel` still edited local model/runtime settings through `save_config`, which only wrote the config file and did not update the in-memory runtime state used by the new start buttons.
+- The latest experiment resolves that split by removing `useServerProcess` and the embedded legacy `ServerPanel` from the active sidebar flow, moving local runtime configuration into the unified runtime drawer, and making the sidebar card either open setup or start the saved runtime config from the same domain.
+- The runtime visuals were also refactored toward a cleaner AI-desktop feel with updated runtime card/drawer styling, a more deliberate font stack, cooler blue-cyan accents, and less of the earlier purple-heavy sidebar look.
+- The current experiment hardens readiness detection in `tauri-app/src-tauri/src/runtime/health.rs` so startup now waits for a successful `GET /health` response with `{"status":"ok"}` instead of treating any open TCP port as ready. Rust tests now cover healthy HTTP, unhealthy payloads, and plain-open-port false positives.
+- The latest experiment adds app-surface smoke coverage for `RuntimeStatusCard` and `RuntimeSettingsDrawer`, then fixes a real regression in `chooseModel()` so a previously saved model path is preserved when a folder rescan returns no models. This keeps the unified drawer start action available for valid saved configs instead of silently disabling it.
+- The current experiment re-initializes the harness around the Tauri desktop surface by replacing the stale CLI benchmark with `experiments/bench_tauri_runtime_surface.py`, updating `autoresearch.sh` to score runtime-focused frontend tests plus build/check verification, and fixing a real drawer regression where switching from remote to local mode hid the primary local CTA until after save.
+- The latest experiment extends the runtime lifecycle acceptance contract with two hook-backed smoke checks: a saved local config must surface `stopped -> starting -> ready` through `useRuntimeManager`, and local onboarding via folder browse must auto-select a model, persist it, and reach a ready local runtime from the same drawer flow.
+- The latest experiment adds periodic runtime truthfulness checks after startup: `refresh_process_state()` now revalidates `/health` while the managed child is still running, downgrades stale `ready` state to `degraded` when the process stops serving correctly, and restores `ready` after recovery. The desktop surface test already rendered degraded snapshots correctly; the missing correctness gap was backend state revalidation.
+- Current keep criteria for this segment are: the desktop runtime benchmark targets the active Tauri surface, switching into local mode keeps the apply/start CTA visible immediately, saved local startup exposes `starting` before `ready`, onboarding can browse/save/start from one local flow, stale managed processes demote from `ready` to `degraded` when `/health` stops reporting ready, and verification stays green across Vitest, TypeScript/Vite build, and Rust compilation.