ForgeLM/CLAUDE.md

Project

ForgeLM

ForgeLM (formerly AI Backend Studio) is a local-first desktop workspace for running and managing local LLM runtimes, selecting models, and chatting against the active runtime from a Tauri app. This roadmap focuses on the existing runtime workspace experience: making runtime control and configuration easier to understand, safer to edit, and more reliable for operators using the desktop app.

Core Value: A user can configure and operate the runtime from one trustworthy workspace without the UI fighting their in-progress edits.

Constraints

• Tech stack: Keep the existing Rust + Tauri + React/TypeScript architecture - the project is improving the current app, not rewriting it
• Brownfield compatibility: Preserve working runtime lifecycle, model selection, and chat flows while reorganizing the UI - regressions in validated capabilities are not acceptable
• UX reliability: Unsaved user edits must not be overwritten by polling or snapshot refresh - trust in the settings surface is the main product constraint
• Local-first: Continue supporting local desktop runtime management patterns already established in the app - scope should not depend on external hosted infrastructure

Technology Stack

Languages

• Rust 2021 - core backend, inference runtime, CLI, shared contracts, metrics, and Tauri desktop backend in Cargo.toml, crates/inference-server/src/main.rs, crates/inference-core/src/engine.rs, crates/localllm-cli/src/main.rs, and tauri-app/src-tauri/src/lib.rs
• TypeScript 5.5 - desktop frontend, runtime manager UI, chat UI, and tests in tauri-app/package.json, tauri-app/src/App.tsx, and tauri-app/src/features/runtime/components/RuntimeSettingsDrawer.tsx
• CSS - hand-authored styling with app-wide tokens and feature CSS modules in tauri-app/src/index.css, tauri-app/src/components/Sidebar.css, and tauri-app/src/components/ChatView.css
• Markdown - product and architecture docs in README.MD, docs/api/overview.md, and docs/app/overview.md
• Python - benchmark and experiment scripts in experiments/bench_chat_latency.py, experiments/bench_cli_surface.py, and related files under experiments/

Runtime

• Rust toolchain targeting native desktop/server binaries; no workspace-level rust-toolchain file detected
• Node.js runtime for Vite/Tauri frontend workflows in tauri-app/package.json
• Cargo for Rust workspace dependencies and binaries in Cargo.toml
• npm is the documented frontend package manager via README.MD:47; lockfiles for npm and pnpm are both present in tauri-app/package-lock.json and tauri-app/pnpm-lock.yaml
• Lockfile: present for Rust and frontend in Cargo.lock and tauri-app/package-lock.json

Frameworks

• Axum 0.8 - local HTTP API in Cargo.toml and crates/inference-server/src/routes.rs
• React 18 - desktop UI shell and chat surface in tauri-app/package.json and tauri-app/src/App.tsx
• Tauri 2 - desktop shell, local process control, file dialogs, and frontend-to-Rust command bridge in tauri-app/src-tauri/Cargo.toml and tauri-app/src-tauri/src/lib.rs
• llama.cpp FFI via custom llama-sys - model loading and token generation in crates/inference-core/src/engine.rs and crates/inference-core/src/session.rs
• Vitest 4 - frontend/unit/component tests in tauri-app/package.json and tauri-app/src/features/runtime/components/RuntimeSurface.test.tsx
• Rust built-in test harness - backend/unit tests co-located in files such as crates/api-contracts/src/chat.rs and tauri-app/src-tauri/src/runtime/state.rs
• Vite 5 - frontend dev/build tool in tauri-app/vite.config.ts
• TypeScript compiler - typecheck step in tauri-app/package.json:9
• Tauri CLI 2 - desktop dev/build entry in tauri-app/package.json:11

Key Dependencies

• tokio - async runtime for API server in Cargo.toml and crates/inference-server/src/main.rs
• tower-http - CORS and trace middleware in Cargo.toml and crates/inference-server/src/compat.rs
• serde / serde_json - shared request/response contracts across API, CLI, and Tauri in crates/api-contracts/src/chat.rs and tauri-app/src-tauri/src/runtime/config.rs
• clap - CLI command surface in crates/localllm-cli/src/main.rs
• ureq - synchronous HTTP transport for CLI live execution in crates/localllm-cli/src/main.rs:404
• marked, dompurify, highlight.js - markdown rendering/sanitization/code highlighting in tauri-app/src/lib/markdown.ts
• @tauri-apps/api and @tauri-apps/plugin-shell - frontend/Tauri bridge in tauri-app/package.json and tauri-app/src/features/runtime/api.ts
• uuid - response IDs for chat completions in crates/inference-server/src/routes.rs:126
• tracing and tracing-subscriber - backend logging and startup diagnostics in crates/inference-server/src/main.rs
• async-stream - SSE streaming implementation in crates/inference-server/Cargo.toml and crates/inference-server/src/routes.rs:137

Configuration

• Backend runtime is environment-driven via LOCALLLM_MODEL, LOCALLLM_PROFILE, LOCALLLM_ADDR, LOCALLLM_GPU_LAYERS, LOCALLLM_CTX, LOCALLLM_API_KEY, LOCALLLM_CORS_ORIGINS, LOCALLLM_AUTH_HEADER, LOCALLLM_AUTH_SCHEME, LOCALLLM_SYSTEM_PROMPT, and LOCALLLM_MODEL_ALIASES in crates/inference-server/src/main.rs and crates/inference-server/src/compat.rs
• Desktop runtime config is persisted as JSON in the app data directory through tauri-app/src-tauri/src/runtime/config.rs
• Frontend fallback runtime snapshot is persisted in localStorage through tauri-app/src/features/runtime/api.ts
• Rust workspace build in Cargo.toml
• Tauri desktop build in tauri-app/src-tauri/Cargo.toml and tauri-app/src-tauri/tauri.conf.json
• Frontend dev server/build in tauri-app/vite.config.ts and tauri-app/package.json

Platform Requirements

• Rust + Cargo for backend and desktop backend in README.MD:56
• Node.js/npm for the Tauri frontend in README.MD:57
• Local .gguf model for real inference in README.MD:58
• Optional NVIDIA GPU for accelerated local runtime; CPU fallback profile exists in README.MD:59 and crates/inference-core/src/profile.rs
• Local-first desktop deployment via Tauri app in tauri-app/
• Local HTTP API deployment as standalone binary localllm in crates/inference-server/Cargo.toml:8

Conventions

Conventions not yet established. Will populate as patterns emerge during development.

Architecture

Pattern Overview

• Keep API contracts in Rust shared types before exposing them to CLI and desktop consumers in crates/api-contracts/src/lib.rs and crates/api-contracts/src/chat.rs
• Keep inference logic behind a single engine/session boundary in crates/inference-core/src/engine.rs and crates/inference-core/src/session.rs
• Keep the desktop app as an orchestration shell that manages process lifecycle, model discovery, and chat persistence in tauri-app/src/App.tsx and tauri-app/src-tauri/src/runtime/mod.rs

Layers

• Purpose: Load GGUF models, own llama context/model pointers, and generate tokens
• Location: crates/inference-core/src/
• Contains: EngineConfig, InferenceEngine, runtime profiles, sampler config, generation session
• Depends on: crates/llama-sys, Rust stdlib, anyhow, tracing
• Used by: crates/inference-server/src/main.rs
• Purpose: Adapt OpenAI-like chat/model requests to the local engine
• Location: crates/inference-server/src/
• Contains: env-driven ServerConfig, auth/CORS rules, prompt preparation, routing, SSE streaming
• Depends on: crates/inference-core, crates/api-contracts, crates/metrics
• Used by: local HTTP consumers, CLI, desktop frontend
• Purpose: Keep request/response models and error shapes consistent
• Location: crates/api-contracts/src/
• Contains: chat completion structs, model list, health payload, error payloads
• Depends on: serde, serde_json
• Used by: server code directly; mirrored conceptually by frontend/CLI callers
• Purpose: Expose operator workflows and a future control plane over HTTP/local actions
• Location: crates/localllm-cli/src/main.rs
• Contains: command parsing, planning, preview rendering, selective live transport
• Depends on: clap, ureq, serde_json
• Used by: developers/operators
• Purpose: Persist runtime config, discover model files, detect GPU, spawn/stop local API
• Location: tauri-app/src-tauri/src/
• Contains: Tauri commands, runtime state machine, health probing, file dialog integration
• Depends on: Tauri runtime, native process and filesystem APIs
• Used by: tauri-app/src/features/runtime/api.ts
• Purpose: Present runtime controls, model selection, and persistent chat UI
• Location: tauri-app/src/
• Contains: shell layout, runtime manager hooks, chat view, sidebar, markdown rendering, local DB wrapper
• Depends on: React, fetch, Tauri invoke bridge, browser storage/CDN-loaded sql.js
• Used by: end users of the desktop app

Data Flow

• Backend API state is a single shared AppState with Arc<Mutex<InferenceEngine>> in crates/inference-server/src/routes.rs
• Desktop runtime state is a single RuntimeState(pub Mutex<RuntimeInner>) in tauri-app/src-tauri/src/runtime/state.rs
• Frontend state is local React hook state; no global store is used in tauri-app/src/App.tsx, tauri-app/src/hooks/useModels.ts, and tauri-app/src/features/runtime/useRuntimeManager.ts
• Chat persistence is local-first and browser-backed via localStorage + sql.js in tauri-app/src/lib/db.ts

Key Abstractions

• Purpose: Own one loaded model and one llama context
• Examples: crates/inference-core/src/engine.rs, crates/inference-core/src/session.rs
• Pattern: Single-owner, RAII-managed native resource wrapper with explicit non-concurrent use
• Purpose: Centralize runtime auth, prompt, CORS, and alias rules
• Examples: crates/inference-server/src/compat.rs
• Pattern: Env-derived compatibility adapter layered in front of the engine
• Purpose: Serialize the desktop runtime state into a frontend-safe shape
• Examples: tauri-app/src/features/runtime/types.ts, tauri-app/src-tauri/src/runtime/state.rs
• Pattern: Backend source of truth mirrored into a polling frontend hook
• Purpose: Hide sql.js statement handling and persistence details behind a tiny query/run API
• Examples: tauri-app/src/lib/db.ts
• Pattern: Thin synchronous local repository over browser storage

Entry Points

• Location: crates/inference-server/src/main.rs
• Triggers: cargo run -p inference-server or spawned local desktop runtime
• Responsibilities: read env, load/warm engine, build router, bind network listener
• Location: crates/localllm-cli/src/main.rs
• Triggers: cargo run -p localllm-cli -- ...
• Responsibilities: parse commands, build execution plans, optionally perform live HTTP requests
• Location: tauri-app/src/main.tsx and tauri-app/src/App.tsx
• Triggers: Vite dev server or Tauri desktop app startup
• Responsibilities: render app shell, bootstrap DB/runtime hooks, connect sidebar and chat surface
• Location: tauri-app/src-tauri/src/lib.rs
• Triggers: Tauri app startup and frontend invoke() calls
• Responsibilities: expose commands, manage runtime state, wire OS dialogs/plugins

Error Handling

• API uses ApiError + ErrorResponse with HTTP status mapping in crates/inference-server/src/routes.rs
• Desktop runtime stores recoverable errors in last_error on RuntimeSnapshot in tauri-app/src-tauri/src/runtime/state.rs
• Frontend fetch and invoke failures typically degrade to fallback state or inline assistant error messages in tauri-app/src/components/ChatView.tsx and tauri-app/src/features/runtime/api.ts

Cross-Cutting Concerns

Boundaries

• crates/inference-core/ should stay inference-only and not absorb HTTP/UI concerns
• crates/inference-server/ should stay the public backend boundary and not own chat persistence or desktop state
• crates/localllm-cli/ should stay an operator/client surface and not become the runtime itself
• tauri-app/src-tauri/ should own process/config/filesystem concerns for desktop only
• tauri-app/src/ should stay a consumer/orchestrator over backend contracts and local chat persistence

GSD Workflow Enforcement

Before using Edit, Write, or other file-changing tools, start work through a GSD command so planning artifacts and execution context stay in sync.

Use these entry points:

• /gsd:quick for small fixes, doc updates, and ad-hoc tasks
• /gsd:debug for investigation and bug fixing
• /gsd:execute-phase for planned phase work

Do not make direct repo edits outside a GSD workflow unless the user explicitly asks to bypass it.

Developer Profile

Profile not yet configured. Run /gsd:profile-user to generate your developer profile. This section is managed by generate-claude-profile -- do not edit manually.