chore: initialize template autoresearch segment

autoresearch.md

@@ -1,28 +1,24 @@
-# Autoresearch: latency-p95-chat-completions
+# Autoresearch: dynamic-chat-templates
 
 ## Objective
-Optimize end-to-end p95 latency for local OpenAI-compatible `POST /v1/chat/completions`
-requests against the Rust inference server, using the local Qwen GGUF model already
-present in `models/`. The benchmark targets a short coding-oriented request that is
-representative of editor integrations such as VSCode, OpenCode, Cline, and Roo.
+Implement a dynamic chat-template system that selects prompt rendering by model family
+for local OpenAI-compatible `POST /v1/chat/completions` requests. The server should ship
+useful built-in templates for common open-source families such as Qwen, Gemma, Llama,
+and Kimi, while preserving a safe generic fallback for unknown models.
 
 ## Metrics
-- **Primary**: p95_ms (milliseconds, lower is better)
-- **Secondary**: p50_ms, mean_ms, completion_tps_mean, success_count
+- **Primary**: template_score (percent, higher is better)
+- **Secondary**: passed_cases, failed_cases, total_cases
 
 ## How to Run
-`./autoresearch.sh` - builds the release server, starts it locally, runs a fixed HTTP benchmark, and outputs `METRIC name=number` lines.
+`./autoresearch.sh` - runs targeted ignored tests for dynamic chat-template behavior and outputs `METRIC name=number` lines.
 
 ## Files in Scope
-- `crates/inference-server/src/compat.rs` - request shaping, prompt preparation, auth/CORS/config, streaming mode decisions
-- `crates/inference-server/src/routes.rs` - HTTP handlers, blocking/streaming response path, usage/reporting
-- `crates/inference-server/src/main.rs` - runtime/server boot configuration
-- `crates/inference-core/src/engine.rs` - engine load/warmup/session creation path
-- `crates/inference-core/src/session.rs` - prompt prefill and token generation hot path
-- `crates/inference-core/src/profile.rs` - runtime defaults that affect latency
-- `crates/inference-core/src/sampler.rs` - per-request generation config
-- `crates/metrics/src/*` - measurement plumbing if needed
-- `experiments/bench_chat_latency.py` - benchmark harness for the autoresearch loop
+- `crates/inference-server/src/compat.rs` - prompt preparation, model catalog, template selection, family detection, and template-focused tests
+- `crates/inference-server/src/routes.rs` - route-level integration if template selection needs model-aware wiring
+- `crates/inference-server/src/main.rs` - server boot wiring if template config/env support is needed
+- `crates/api-contracts/src/chat.rs` - request/response roles and content handling that templates consume
+- `experiments/bench_chat_templates.py` - benchmark harness for the autoresearch loop
 - `autoresearch.sh` - loop benchmark entrypoint and fast preflight
 
 ## Off Limits
@@ -30,16 +26,19 @@ representative of editor integrations such as VSCode, OpenCode, Cline, and Roo.
 - `vendor/` third-party sources and prebuilt llama binaries
 - `registry/` Python registry service
 - `ui/` frontend files
-- broad API-surface feature additions unrelated to latency unless they directly reduce p95
+- unrelated latency tuning or runtime/engine experiments unless they are required to support template selection correctly
 
 ## Constraints
-- Keep the benchmark reproducible and deterministic enough for comparison.
 - No new external dependencies.
 - Preserve OpenAI-compatible behavior already implemented.
-- Tests must pass for keeps.
-- Prefer changes that improve p95 without making code materially harder to maintain.
-- Aggressive refactors are allowed if they stay within the in-scope server/runtime path.
+- Regular test suite must stay green for keeps.
+- Prefer explicit, maintainable template definitions over opaque heuristics.
+- Unknown models must continue to work through a generic fallback template.
+- The benchmark is test-driven: keep/discard follows the dynamic-template suite score first.
 
 ## What's Been Tried
-- Baseline session initialized; no latency experiments logged yet.
-- Existing work before this autoresearch session already improved token detokenization and broadened OpenAI-compatible request/response handling, but this loop focuses specifically on p95 latency.
+- Segment 1 switches the autoresearch target from latency to dynamic chat-template selection by model family.
+- The current server still renders every prompt through one generic `System:/User:/Assistant:` formatter in `crates/inference-server/src/compat.rs`.
+- `prepare_prompt()` is the cleanest insertion point for dynamic templates because both blocking and streaming paths already consume its rendered string.
+- `ModelCatalog` currently resolves aliases only; it has no model-family metadata or template registry yet.
+- The benchmark for this segment is a targeted ignored test suite that scores family detection/template rendering coverage for Qwen, Gemma, Llama, Kimi, and the generic fallback.