# Architecture

**Analysis Date:** 2026-03-28

## Pattern Overview

**Overall:** Local-first polyglot workspace with shared contracts and three user surfaces

**Key Characteristics:**
- Keep API contracts in Rust shared types before exposing them to CLI and desktop consumers in `crates/api-contracts/src/lib.rs` and `crates/api-contracts/src/chat.rs`
- Keep inference logic behind a single engine/session boundary in `crates/inference-core/src/engine.rs` and `crates/inference-core/src/session.rs`
- Keep the desktop app as an orchestration shell that manages process lifecycle, model discovery, and chat persistence in `tauri-app/src/App.tsx` and `tauri-app/src-tauri/src/runtime/mod.rs`

## Layers

**Inference Core:**
- Purpose: Load GGUF models, own llama context/model pointers, and generate tokens
- Location: `crates/inference-core/src/`
- Contains: `EngineConfig`, `InferenceEngine`, runtime profiles, sampler config, generation session
- Depends on: `crates/llama-sys`, Rust stdlib, `anyhow`, `tracing`
- Used by: `crates/inference-server/src/main.rs`

**HTTP Compatibility Layer:**
- Purpose: Adapt OpenAI-like chat/model requests to the local engine
- Location: `crates/inference-server/src/`
- Contains: env-driven `ServerConfig`, auth/CORS rules, prompt preparation, routing, SSE streaming
- Depends on: `crates/inference-core`, `crates/api-contracts`, `crates/metrics`
- Used by: local HTTP consumers, CLI, desktop frontend

**Shared API Contract Layer:**
- Purpose: Keep request/response models and error shapes consistent
- Location: `crates/api-contracts/src/`
- Contains: chat completion structs, model list, health payload, error payloads
- Depends on: `serde`, `serde_json`
- Used by: server code directly; mirrored conceptually by frontend/CLI callers

**CLI Surface Layer:**
- Purpose: Expose operator workflows and a future control plane over HTTP/local actions
- Location: `crates/localllm-cli/src/main.rs`
- Contains: command parsing, planning, preview rendering, selective live transport
- Depends on: `clap`, `ureq`, `serde_json`
- Used by: developers/operators

**Desktop Backend Layer:**
- Purpose: Persist runtime config, discover model files, detect GPU, spawn/stop local API
- Location: `tauri-app/src-tauri/src/`
- Contains: Tauri commands, runtime state machine, health probing, file dialog integration
- Depends on: Tauri runtime, native process and filesystem APIs
- Used by: `tauri-app/src/features/runtime/api.ts`

**Desktop Frontend Layer:**
- Purpose: Present runtime controls, model selection, and persistent chat UI
- Location: `tauri-app/src/`
- Contains: shell layout, runtime manager hooks, chat view, sidebar, markdown rendering, local DB wrapper
- Depends on: React, fetch, Tauri invoke bridge, browser storage/CDN-loaded sql.js
- Used by: end users of the desktop app

## Data Flow

**Local chat request flow:**

1. `tauri-app/src/components/ChatView.tsx` builds message history from local sql.js storage and posts to `${server}/v1/chat/completions`
2. `crates/inference-server/src/routes.rs` validates auth/model/mode, prepares the prompt through `crates/inference-server/src/compat.rs`, and locks the shared `InferenceEngine`
3. `crates/inference-core/src/session.rs` tokenizes, pre-fills, samples, and returns streamed or buffered content back through Axum SSE/JSON
4. `tauri-app/src/components/ChatView.tsx` parses `<think>...</think>`, stores assistant content plus usage metrics in `tauri-app/src/lib/db.ts`, and updates the UI

**Runtime management flow:**

1. `tauri-app/src/features/runtime/useRuntimeManager.ts` bootstraps a `RuntimeSnapshot` from `tauri-app/src/features/runtime/api.ts`
2. `tauri-app/src-tauri/src/lib.rs` routes invoke calls into `tauri-app/src-tauri/src/runtime/mod.rs`
3. `tauri-app/src-tauri/src/runtime/process.rs` finds and spawns the `localllm` binary, then `tauri-app/src-tauri/src/runtime/health.rs` probes `/health`
4. `tauri-app/src/features/runtime/components/RuntimeStatusCard.tsx` and `tauri-app/src/features/runtime/components/RuntimeSettingsDrawer.tsx` render and mutate the runtime state

**State Management:**
- Backend API state is a single shared `AppState` with `Arc<Mutex<InferenceEngine>>` in `crates/inference-server/src/routes.rs`
- Desktop runtime state is a single `RuntimeState(pub Mutex<RuntimeInner>)` in `tauri-app/src-tauri/src/runtime/state.rs`
- Frontend state is local React hook state; no global store is used in `tauri-app/src/App.tsx`, `tauri-app/src/hooks/useModels.ts`, and `tauri-app/src/features/runtime/useRuntimeManager.ts`
- Chat persistence is local-first and browser-backed via `localStorage` + sql.js in `tauri-app/src/lib/db.ts`

## Key Abstractions

**InferenceEngine:**
- Purpose: Own one loaded model and one llama context
- Examples: `crates/inference-core/src/engine.rs`, `crates/inference-core/src/session.rs`
- Pattern: Single-owner, RAII-managed native resource wrapper with explicit non-concurrent use

**ServerConfig / ModelCatalog:**
- Purpose: Centralize runtime auth, prompt, CORS, and alias rules
- Examples: `crates/inference-server/src/compat.rs`
- Pattern: Env-derived compatibility adapter layered in front of the engine

**RuntimeSnapshot:**
- Purpose: Serialize the desktop runtime state into a frontend-safe shape
- Examples: `tauri-app/src/features/runtime/types.ts`, `tauri-app/src-tauri/src/runtime/state.rs`
- Pattern: Backend source of truth mirrored into a polling frontend hook

**DB wrapper:**
- Purpose: Hide sql.js statement handling and persistence details behind a tiny query/run API
- Examples: `tauri-app/src/lib/db.ts`
- Pattern: Thin synchronous local repository over browser storage

## Entry Points

**HTTP API binary:**
- Location: `crates/inference-server/src/main.rs`
- Triggers: `cargo run -p inference-server` or spawned local desktop runtime
- Responsibilities: read env, load/warm engine, build router, bind network listener

**CLI binary:**
- Location: `crates/localllm-cli/src/main.rs`
- Triggers: `cargo run -p localllm-cli -- ...`
- Responsibilities: parse commands, build execution plans, optionally perform live HTTP requests

**Desktop frontend:**
- Location: `tauri-app/src/main.tsx` and `tauri-app/src/App.tsx`
- Triggers: Vite dev server or Tauri desktop app startup
- Responsibilities: render app shell, bootstrap DB/runtime hooks, connect sidebar and chat surface

**Desktop backend:**
- Location: `tauri-app/src-tauri/src/lib.rs`
- Triggers: Tauri app startup and frontend `invoke()` calls
- Responsibilities: expose commands, manage runtime state, wire OS dialogs/plugins

## Error Handling

**Strategy:** Convert most backend failures into typed user-facing snapshots/responses rather than panic paths

**Patterns:**
- API uses `ApiError` + `ErrorResponse` with HTTP status mapping in `crates/inference-server/src/routes.rs`
- Desktop runtime stores recoverable errors in `last_error` on `RuntimeSnapshot` in `tauri-app/src-tauri/src/runtime/state.rs`
- Frontend fetch and invoke failures typically degrade to fallback state or inline assistant error messages in `tauri-app/src/components/ChatView.tsx` and `tauri-app/src/features/runtime/api.ts`

## Cross-Cutting Concerns

**Logging:** `tracing` on Rust services in `crates/inference-server/src/main.rs`; frontend logging is minimal and mostly absent
**Validation:** request and mode validation lives in `crates/inference-server/src/compat.rs`; runtime form validation is mostly helper-based in `tauri-app/src/features/runtime/runtime-settings-form.ts`
**Authentication:** optional API-key auth is enforced in `crates/inference-server/src/compat.rs`; desktop frontend forwards the configured API key from runtime config in `tauri-app/src/features/runtime/api.ts`

## Boundaries

**App boundaries:**
- `crates/inference-core/` should stay inference-only and not absorb HTTP/UI concerns
- `crates/inference-server/` should stay the public backend boundary and not own chat persistence or desktop state
- `crates/localllm-cli/` should stay an operator/client surface and not become the runtime itself
- `tauri-app/src-tauri/` should own process/config/filesystem concerns for desktop only
- `tauri-app/src/` should stay a consumer/orchestrator over backend contracts and local chat persistence

---

*Architecture analysis: 2026-03-28*