ForgeLM/CLAUDE.md

<!-- GSD:project-start source:PROJECT.md -->
## Project

**ForgeLM**

ForgeLM (formerly AI Backend Studio) is a local-first desktop workspace for running and managing local LLM runtimes, selecting models, and chatting against the active runtime from a Tauri app. This roadmap focuses on the existing runtime workspace experience: making runtime control and configuration easier to understand, safer to edit, and more reliable for operators using the desktop app.

**Core Value:** A user can configure and operate the runtime from one trustworthy workspace without the UI fighting their in-progress edits.

### Constraints

- **Tech stack**: Keep the existing Rust + Tauri + React/TypeScript architecture - the project is improving the current app, not rewriting it
- **Brownfield compatibility**: Preserve working runtime lifecycle, model selection, and chat flows while reorganizing the UI - regressions in validated capabilities are not acceptable
- **UX reliability**: Unsaved user edits must not be overwritten by polling or snapshot refresh - trust in the settings surface is the main product constraint
- **Local-first**: Continue supporting local desktop runtime management patterns already established in the app - scope should not depend on external hosted infrastructure
<!-- GSD:project-end -->

<!-- GSD:stack-start source:codebase/STACK.md -->
## Technology Stack

## Languages
- Rust 2021 - core backend, inference runtime, CLI, shared contracts, metrics, and Tauri desktop backend in `Cargo.toml`, `crates/inference-server/src/main.rs`, `crates/inference-core/src/engine.rs`, `crates/localllm-cli/src/main.rs`, and `tauri-app/src-tauri/src/lib.rs`
- TypeScript 5.5 - desktop frontend, runtime manager UI, chat UI, and tests in `tauri-app/package.json`, `tauri-app/src/App.tsx`, and `tauri-app/src/features/runtime/components/RuntimeSettingsDrawer.tsx`
- CSS - hand-authored styling with app-wide tokens and feature CSS modules in `tauri-app/src/index.css`, `tauri-app/src/components/Sidebar.css`, and `tauri-app/src/components/ChatView.css`
- Markdown - product and architecture docs in `README.MD`, `docs/api/overview.md`, and `docs/app/overview.md`
- Python - benchmark and experiment scripts in `experiments/bench_chat_latency.py`, `experiments/bench_cli_surface.py`, and related files under `experiments/`
## Runtime
- Rust toolchain targeting native desktop/server binaries; no workspace-level rust-toolchain file detected
- Node.js runtime for Vite/Tauri frontend workflows in `tauri-app/package.json`
- Cargo for Rust workspace dependencies and binaries in `Cargo.toml`
- npm is the documented frontend package manager via `README.MD:47`; lockfiles for npm and pnpm are both present in `tauri-app/package-lock.json` and `tauri-app/pnpm-lock.yaml`
- Lockfile: present for Rust and frontend in `Cargo.lock` and `tauri-app/package-lock.json`
## Frameworks
- Axum 0.8 - local HTTP API in `Cargo.toml` and `crates/inference-server/src/routes.rs`
- React 18 - desktop UI shell and chat surface in `tauri-app/package.json` and `tauri-app/src/App.tsx`
- Tauri 2 - desktop shell, local process control, file dialogs, and frontend-to-Rust command bridge in `tauri-app/src-tauri/Cargo.toml` and `tauri-app/src-tauri/src/lib.rs`
- llama.cpp FFI via custom `llama-sys` - model loading and token generation in `crates/inference-core/src/engine.rs` and `crates/inference-core/src/session.rs`
- Vitest 4 - frontend/unit/component tests in `tauri-app/package.json` and `tauri-app/src/features/runtime/components/RuntimeSurface.test.tsx`
- Rust built-in test harness - backend/unit tests co-located in files such as `crates/api-contracts/src/chat.rs` and `tauri-app/src-tauri/src/runtime/state.rs`
- Vite 5 - frontend dev/build tool in `tauri-app/vite.config.ts`
- TypeScript compiler - typecheck step in `tauri-app/package.json:9`
- Tauri CLI 2 - desktop dev/build entry in `tauri-app/package.json:11`
## Key Dependencies
- `tokio` - async runtime for API server in `Cargo.toml` and `crates/inference-server/src/main.rs`
- `tower-http` - CORS and trace middleware in `Cargo.toml` and `crates/inference-server/src/compat.rs`
- `serde` / `serde_json` - shared request/response contracts across API, CLI, and Tauri in `crates/api-contracts/src/chat.rs` and `tauri-app/src-tauri/src/runtime/config.rs`
- `clap` - CLI command surface in `crates/localllm-cli/src/main.rs`
- `ureq` - synchronous HTTP transport for CLI live execution in `crates/localllm-cli/src/main.rs:404`
- `marked`, `dompurify`, `highlight.js` - markdown rendering/sanitization/code highlighting in `tauri-app/src/lib/markdown.ts`
- `@tauri-apps/api` and `@tauri-apps/plugin-shell` - frontend/Tauri bridge in `tauri-app/package.json` and `tauri-app/src/features/runtime/api.ts`
- `uuid` - response IDs for chat completions in `crates/inference-server/src/routes.rs:126`
- `tracing` and `tracing-subscriber` - backend logging and startup diagnostics in `crates/inference-server/src/main.rs`
- `async-stream` - SSE streaming implementation in `crates/inference-server/Cargo.toml` and `crates/inference-server/src/routes.rs:137`
## Configuration
- Backend runtime is environment-driven via `LOCALLLM_MODEL`, `LOCALLLM_PROFILE`, `LOCALLLM_ADDR`, `LOCALLLM_GPU_LAYERS`, `LOCALLLM_CTX`, `LOCALLLM_API_KEY`, `LOCALLLM_CORS_ORIGINS`, `LOCALLLM_AUTH_HEADER`, `LOCALLLM_AUTH_SCHEME`, `LOCALLLM_SYSTEM_PROMPT`, and `LOCALLLM_MODEL_ALIASES` in `crates/inference-server/src/main.rs` and `crates/inference-server/src/compat.rs`
- Desktop runtime config is persisted as JSON in the app data directory through `tauri-app/src-tauri/src/runtime/config.rs`
- Frontend fallback runtime snapshot is persisted in `localStorage` through `tauri-app/src/features/runtime/api.ts`
- Rust workspace build in `Cargo.toml`
- Tauri desktop build in `tauri-app/src-tauri/Cargo.toml` and `tauri-app/src-tauri/tauri.conf.json`
- Frontend dev server/build in `tauri-app/vite.config.ts` and `tauri-app/package.json`
## Platform Requirements
- Rust + Cargo for backend and desktop backend in `README.MD:56`
- Node.js/npm for the Tauri frontend in `README.MD:57`
- Local `.gguf` model for real inference in `README.MD:58`
- Optional NVIDIA GPU for accelerated local runtime; CPU fallback profile exists in `README.MD:59` and `crates/inference-core/src/profile.rs`
- Local-first desktop deployment via Tauri app in `tauri-app/`
- Local HTTP API deployment as standalone binary `localllm` in `crates/inference-server/Cargo.toml:8`
<!-- GSD:stack-end -->

<!-- GSD:conventions-start source:CONVENTIONS.md -->
## Conventions

Conventions not yet established. Will populate as patterns emerge during development.
<!-- GSD:conventions-end -->

<!-- GSD:architecture-start source:ARCHITECTURE.md -->
## Architecture

## Pattern Overview
- Keep API contracts in Rust shared types before exposing them to CLI and desktop consumers in `crates/api-contracts/src/lib.rs` and `crates/api-contracts/src/chat.rs`
- Keep inference logic behind a single engine/session boundary in `crates/inference-core/src/engine.rs` and `crates/inference-core/src/session.rs`
- Keep the desktop app as an orchestration shell that manages process lifecycle, model discovery, and chat persistence in `tauri-app/src/App.tsx` and `tauri-app/src-tauri/src/runtime/mod.rs`
## Layers
- Purpose: Load GGUF models, own llama context/model pointers, and generate tokens
- Location: `crates/inference-core/src/`
- Contains: `EngineConfig`, `InferenceEngine`, runtime profiles, sampler config, generation session
- Depends on: `crates/llama-sys`, Rust stdlib, `anyhow`, `tracing`
- Used by: `crates/inference-server/src/main.rs`
- Purpose: Adapt OpenAI-like chat/model requests to the local engine
- Location: `crates/inference-server/src/`
- Contains: env-driven `ServerConfig`, auth/CORS rules, prompt preparation, routing, SSE streaming
- Depends on: `crates/inference-core`, `crates/api-contracts`, `crates/metrics`
- Used by: local HTTP consumers, CLI, desktop frontend
- Purpose: Keep request/response models and error shapes consistent
- Location: `crates/api-contracts/src/`
- Contains: chat completion structs, model list, health payload, error payloads
- Depends on: `serde`, `serde_json`
- Used by: server code directly; mirrored conceptually by frontend/CLI callers
- Purpose: Expose operator workflows and a future control plane over HTTP/local actions
- Location: `crates/localllm-cli/src/main.rs`
- Contains: command parsing, planning, preview rendering, selective live transport
- Depends on: `clap`, `ureq`, `serde_json`
- Used by: developers/operators
- Purpose: Persist runtime config, discover model files, detect GPU, spawn/stop local API
- Location: `tauri-app/src-tauri/src/`
- Contains: Tauri commands, runtime state machine, health probing, file dialog integration
- Depends on: Tauri runtime, native process and filesystem APIs
- Used by: `tauri-app/src/features/runtime/api.ts`
- Purpose: Present runtime controls, model selection, and persistent chat UI
- Location: `tauri-app/src/`
- Contains: shell layout, runtime manager hooks, chat view, sidebar, markdown rendering, local DB wrapper
- Depends on: React, fetch, Tauri invoke bridge, browser storage/CDN-loaded sql.js
- Used by: end users of the desktop app
## Data Flow
- Backend API state is a single shared `AppState` with `Arc<Mutex<InferenceEngine>>` in `crates/inference-server/src/routes.rs`
- Desktop runtime state is a single `RuntimeState(pub Mutex<RuntimeInner>)` in `tauri-app/src-tauri/src/runtime/state.rs`
- Frontend state is local React hook state; no global store is used in `tauri-app/src/App.tsx`, `tauri-app/src/hooks/useModels.ts`, and `tauri-app/src/features/runtime/useRuntimeManager.ts`
- Chat persistence is local-first and browser-backed via `localStorage` + sql.js in `tauri-app/src/lib/db.ts`
## Key Abstractions
- Purpose: Own one loaded model and one llama context
- Examples: `crates/inference-core/src/engine.rs`, `crates/inference-core/src/session.rs`
- Pattern: Single-owner, RAII-managed native resource wrapper with explicit non-concurrent use
- Purpose: Centralize runtime auth, prompt, CORS, and alias rules
- Examples: `crates/inference-server/src/compat.rs`
- Pattern: Env-derived compatibility adapter layered in front of the engine
- Purpose: Serialize the desktop runtime state into a frontend-safe shape
- Examples: `tauri-app/src/features/runtime/types.ts`, `tauri-app/src-tauri/src/runtime/state.rs`
- Pattern: Backend source of truth mirrored into a polling frontend hook
- Purpose: Hide sql.js statement handling and persistence details behind a tiny query/run API
- Examples: `tauri-app/src/lib/db.ts`
- Pattern: Thin synchronous local repository over browser storage
## Entry Points
- Location: `crates/inference-server/src/main.rs`
- Triggers: `cargo run -p inference-server` or spawned local desktop runtime
- Responsibilities: read env, load/warm engine, build router, bind network listener
- Location: `crates/localllm-cli/src/main.rs`
- Triggers: `cargo run -p localllm-cli -- ...`
- Responsibilities: parse commands, build execution plans, optionally perform live HTTP requests
- Location: `tauri-app/src/main.tsx` and `tauri-app/src/App.tsx`
- Triggers: Vite dev server or Tauri desktop app startup
- Responsibilities: render app shell, bootstrap DB/runtime hooks, connect sidebar and chat surface
- Location: `tauri-app/src-tauri/src/lib.rs`
- Triggers: Tauri app startup and frontend `invoke()` calls
- Responsibilities: expose commands, manage runtime state, wire OS dialogs/plugins
## Error Handling
- API uses `ApiError` + `ErrorResponse` with HTTP status mapping in `crates/inference-server/src/routes.rs`
- Desktop runtime stores recoverable errors in `last_error` on `RuntimeSnapshot` in `tauri-app/src-tauri/src/runtime/state.rs`
- Frontend fetch and invoke failures typically degrade to fallback state or inline assistant error messages in `tauri-app/src/components/ChatView.tsx` and `tauri-app/src/features/runtime/api.ts`
## Cross-Cutting Concerns
## Boundaries
- `crates/inference-core/` should stay inference-only and not absorb HTTP/UI concerns
- `crates/inference-server/` should stay the public backend boundary and not own chat persistence or desktop state
- `crates/localllm-cli/` should stay an operator/client surface and not become the runtime itself
- `tauri-app/src-tauri/` should own process/config/filesystem concerns for desktop only
- `tauri-app/src/` should stay a consumer/orchestrator over backend contracts and local chat persistence
<!-- GSD:architecture-end -->

<!-- GSD:workflow-start source:GSD defaults -->
## GSD Workflow Enforcement

Before using Edit, Write, or other file-changing tools, start work through a GSD command so planning artifacts and execution context stay in sync.

Use these entry points:
- `/gsd:quick` for small fixes, doc updates, and ad-hoc tasks
- `/gsd:debug` for investigation and bug fixing
- `/gsd:execute-phase` for planned phase work

Do not make direct repo edits outside a GSD workflow unless the user explicitly asks to bypass it.
<!-- GSD:workflow-end -->



<!-- GSD:profile-start -->
## Developer Profile

> Profile not yet configured. Run `/gsd:profile-user` to generate your developer profile.
> This section is managed by `generate-claude-profile` -- do not edit manually.
<!-- GSD:profile-end -->