chore: initialize autoresearch baseline

autoresearch.md

@@ -0,0 +1,45 @@
+# Autoresearch: latency-p95-chat-completions
+
+## Objective
+Optimize end-to-end p95 latency for local OpenAI-compatible `POST /v1/chat/completions`
+requests against the Rust inference server, using the local Qwen GGUF model already
+present in `models/`. The benchmark targets a short coding-oriented request that is
+representative of editor integrations such as VSCode, OpenCode, Cline, and Roo.
+
+## Metrics
+- **Primary**: p95_ms (milliseconds, lower is better)
+- **Secondary**: p50_ms, mean_ms, completion_tps_mean, success_count
+
+## How to Run
+`./autoresearch.sh` - builds the release server, starts it locally, runs a fixed HTTP benchmark, and outputs `METRIC name=number` lines.
+
+## Files in Scope
+- `crates/inference-server/src/compat.rs` - request shaping, prompt preparation, auth/CORS/config, streaming mode decisions
+- `crates/inference-server/src/routes.rs` - HTTP handlers, blocking/streaming response path, usage/reporting
+- `crates/inference-server/src/main.rs` - runtime/server boot configuration
+- `crates/inference-core/src/engine.rs` - engine load/warmup/session creation path
+- `crates/inference-core/src/session.rs` - prompt prefill and token generation hot path
+- `crates/inference-core/src/profile.rs` - runtime defaults that affect latency
+- `crates/inference-core/src/sampler.rs` - per-request generation config
+- `crates/metrics/src/*` - measurement plumbing if needed
+- `experiments/bench_chat_latency.py` - benchmark harness for the autoresearch loop
+- `autoresearch.sh` - loop benchmark entrypoint and fast preflight
+
+## Off Limits
+- `models/` model binaries
+- `vendor/` third-party sources and prebuilt llama binaries
+- `registry/` Python registry service
+- `ui/` frontend files
+- broad API-surface feature additions unrelated to latency unless they directly reduce p95
+
+## Constraints
+- Keep the benchmark reproducible and deterministic enough for comparison.
+- No new external dependencies.
+- Preserve OpenAI-compatible behavior already implemented.
+- Tests must pass for keeps.
+- Prefer changes that improve p95 without making code materially harder to maintain.
+- Aggressive refactors are allowed if they stay within the in-scope server/runtime path.
+
+## What's Been Tried
+- Baseline session initialized; no latency experiments logged yet.
+- Existing work before this autoresearch session already improved token detokenization and broadened OpenAI-compatible request/response handling, but this loop focuses specifically on p95 latency.