# Codebase Concerns

**Analysis Date:** 2026-03-28

## Tech Debt

**CLI surface vs real implementation:**
- Issue: `crates/localllm-cli/src/main.rs` exposes a broad command tree, but many branches collapse to summary-only behavior instead of real actions
- Files: `crates/localllm-cli/src/main.rs`, `docs/cli/overview.md`
- Impact: product expectations exceed actual operator capability; docs and UX need constant caveats
- Fix approach: split stable commands from roadmap commands, or implement local/config/model actions behind dedicated modules and integration tests

**Monolithic files in core surfaces:**
- Issue: major logic is concentrated in very large files such as `crates/inference-server/src/routes.rs`, `crates/inference-server/src/compat.rs`, `crates/localllm-cli/src/main.rs`, `tauri-app/src/components/ChatView.tsx`, and `tauri-app/src/features/runtime/components/RuntimeSettingsDrawer.tsx`
- Files: `crates/inference-server/src/routes.rs`, `crates/inference-server/src/compat.rs`, `crates/localllm-cli/src/main.rs`, `tauri-app/src/components/ChatView.tsx`, `tauri-app/src/features/runtime/components/RuntimeSettingsDrawer.tsx`
- Impact: high change risk, harder reviewability, and low reuse of logic across features
- Fix approach: extract transport, prompt-building, persistence, and UI section components into smaller feature modules with focused tests

**Frontend fallback state diverges from desktop source of truth:**
- Issue: `tauri-app/src/features/runtime/api.ts` silently falls back to `localStorage` snapshots whenever Tauri invoke fails
- Files: `tauri-app/src/features/runtime/api.ts`, `tauri-app/src/features/runtime/runtime-utils.ts`
- Impact: browser preview works, but runtime state can drift from the real desktop backend and mask integration failures
- Fix approach: make fallback mode explicit in UI and separate browser-demo mode from real desktop mode

## Known Bugs

**Local runtime binary name mismatch risk:**
- Symptoms: desktop runtime start can fail even when the backend was built, because the Tauri helper looks for `localllm` while the error message tells users to build `inference-server`
- Files: `tauri-app/src-tauri/src/runtime/process.rs`, `crates/inference-server/Cargo.toml`
- Trigger: building the wrong cargo target or expecting package name and binary name to match
- Workaround: build the binary defined in `crates/inference-server/Cargo.toml:8` and ensure `target/release/localllm(.exe)` exists

**Streaming parser depends on simple line splitting:**
- Symptoms: malformed SSE framing or larger edge-case chunks can be ignored silently during streaming
- Files: `tauri-app/src/components/ChatView.tsx`, `crates/inference-server/src/routes.rs`
- Trigger: partial `data:` framing, unexpected event fields, or non-content deltas
- Workaround: disable streaming in the UI when debugging transport issues

## Security Considerations

**Remote API keys stored in plain frontend/backend config:**
- Risk: remote API keys are stored in desktop runtime config JSON and also surfaced into frontend state
- Files: `tauri-app/src-tauri/src/runtime/config.rs`, `tauri-app/src/features/runtime/types.ts`, `tauri-app/src/features/runtime/api.ts`
- Current mitigation: password input masks display in `tauri-app/src/features/runtime/components/RuntimeSettingsDrawer.tsx:281`
- Recommendations: move secrets to OS keychain/credential store and store only references in app config

**External CDN dependencies inside desktop app:**
- Risk: chat persistence and syntax highlighting depend on runtime-loaded third-party scripts from jsDelivr
- Files: `tauri-app/src/lib/db.ts`, `tauri-app/src/lib/markdown.ts`
- Current mitigation: markdown output is sanitized with DOMPurify in `tauri-app/src/lib/markdown.ts:68`
- Recommendations: vendor `sql.js` and `highlight.js` assets into the app bundle and pin integrity-checked local assets

## Performance Bottlenecks

**Single-engine serialized generation:**
- Problem: the API holds one `Arc<Mutex<InferenceEngine>>` for all requests
- Files: `crates/inference-server/src/routes.rs:31`, `crates/inference-server/src/routes.rs:159`, `crates/inference-server/src/routes.rs:364`
- Cause: engine/context are single-owner and not safely concurrent in `crates/inference-core/src/engine.rs`
- Improvement path: add queued scheduling or a context/session pool before scaling beyond one active generation

**Polling-heavy desktop runtime updates:**
- Problem: frontend polls runtime snapshot every 3 seconds and models every 30 seconds
- Files: `tauri-app/src/features/runtime/useRuntimeManager.ts:33`, `tauri-app/src/hooks/useModels.ts:38`
- Cause: no push/event channel between Tauri backend and React frontend
- Improvement path: add Tauri event emission or a subscription layer for runtime state and model refresh triggers

## Fragile Areas

**Chat request + persistence loop:**
- Files: `tauri-app/src/components/ChatView.tsx`, `tauri-app/src/lib/db.ts`, `tauri-app/src/lib/chat.ts`
- Why fragile: request orchestration, DB writes, streaming parsing, `<think>` parsing, title updates, and optimistic UI all live in one component
- Safe modification: extract persistence, transport, and stream parsing into dedicated hooks/utilities before adding more chat features
- Test coverage: no direct tests detected for `tauri-app/src/components/ChatView.tsx`

**Compatibility layer prompt/tool-call handling:**
- Files: `crates/inference-server/src/compat.rs`, `crates/api-contracts/src/chat.rs`
- Why fragile: one file owns auth, aliases, prompt shaping, template choice, response finalization, and tool-call compatibility behavior
- Safe modification: add focused unit tests around each compatibility branch before extending OpenAI-style contract coverage
- Test coverage: partial for contracts in `crates/api-contracts/src/chat.rs`; compatibility file-specific tests are not visible in the analyzed slice

## Scaling Limits

**Backend request concurrency:**
- Current capacity: effectively one active generation at a time per server process because one engine/context is mutex-locked
- Limit: parallel requests queue behind one model context and long generations block others
- Scaling path: multi-context pool, worker queue, or multi-process model serving

**Desktop chat storage:**
- Current capacity: local sql.js database exported into one `localStorage` key in `tauri-app/src/lib/db.ts:57`
- Limit: browser storage size, serialization overhead, and startup cost grow with chat history
- Scaling path: move persistence into native Tauri storage or SQLite through the Rust backend

## Dependencies at Risk

**CDN-hosted `sql.js` and `highlight.js`:**
- Risk: runtime availability and supply-chain reliability depend on external network/CDN access
- Impact: chat history DB boot or syntax highlighting can fail in offline/locked-down environments
- Migration plan: bundle assets locally or replace with native persistence plus bundled syntax assets

## Missing Critical Features

**No push observability for runtime lifecycle:**
- Problem: runtime state changes and server logs are not streamed into the UI
- Blocks: rich diagnostics, trustworthy startup progress, and smoother local-runtime UX

**No model library management workflow:**
- Problem: model scanning/import exists, but there is no deeper catalog metadata, download progress, space management, or activation history
- Blocks: desktop app becoming a full local-model manager rather than a thin launcher

## Test Coverage Gaps

**Chat UI and persistence path:**
- What's not tested: chat send/stream/error/edit/delete/memory/persistence flows
- Files: `tauri-app/src/components/ChatView.tsx`, `tauri-app/src/components/MessageBubble.tsx`, `tauri-app/src/lib/db.ts`
- Risk: regressions in the main user workflow can ship unnoticed
- Priority: High

**CLI live behavior:**
- What's not tested: end-to-end live HTTP execution and local action implementations behind `execute_live_command`
- Files: `crates/localllm-cli/src/main.rs`
- Risk: parser tests pass while actual command behavior remains incomplete or broken
- Priority: High

**Server route integration:**
- What's not tested: full Axum route behavior for auth, streaming, aliases, and error payloads against a running router
- Files: `crates/inference-server/src/routes.rs`, `crates/inference-server/src/compat.rs`
- Risk: contract drift between docs and runtime behavior
- Priority: High

---

*Concerns audit: 2026-03-28*