ForgeLM/.planning/codebase/CONCERNS.md

Codebase Concerns

Analysis Date: 2026-03-28

Tech Debt

CLI surface vs real implementation:

• Issue: crates/localllm-cli/src/main.rs exposes a broad command tree, but many branches collapse to summary-only behavior instead of real actions
• Files: crates/localllm-cli/src/main.rs, docs/cli/overview.md
• Impact: product expectations exceed actual operator capability; docs and UX need constant caveats
• Fix approach: split stable commands from roadmap commands, or implement local/config/model actions behind dedicated modules and integration tests

Monolithic files in core surfaces:

• Issue: major logic is concentrated in very large files such as crates/inference-server/src/routes.rs, crates/inference-server/src/compat.rs, crates/localllm-cli/src/main.rs, tauri-app/src/components/ChatView.tsx, and tauri-app/src/features/runtime/components/RuntimeSettingsDrawer.tsx
• Files: crates/inference-server/src/routes.rs, crates/inference-server/src/compat.rs, crates/localllm-cli/src/main.rs, tauri-app/src/components/ChatView.tsx, tauri-app/src/features/runtime/components/RuntimeSettingsDrawer.tsx
• Impact: high change risk, harder reviewability, and low reuse of logic across features
• Fix approach: extract transport, prompt-building, persistence, and UI section components into smaller feature modules with focused tests

Frontend fallback state diverges from desktop source of truth:

• Issue: tauri-app/src/features/runtime/api.ts silently falls back to localStorage snapshots whenever Tauri invoke fails
• Files: tauri-app/src/features/runtime/api.ts, tauri-app/src/features/runtime/runtime-utils.ts
• Impact: browser preview works, but runtime state can drift from the real desktop backend and mask integration failures
• Fix approach: make fallback mode explicit in UI and separate browser-demo mode from real desktop mode

Known Bugs

Local runtime binary name mismatch risk:

• Symptoms: desktop runtime start can fail even when the backend was built, because the Tauri helper looks for localllm while the error message tells users to build inference-server
• Files: tauri-app/src-tauri/src/runtime/process.rs, crates/inference-server/Cargo.toml
• Trigger: building the wrong cargo target or expecting package name and binary name to match
• Workaround: build the binary defined in crates/inference-server/Cargo.toml:8 and ensure target/release/localllm(.exe) exists

Streaming parser depends on simple line splitting:

• Symptoms: malformed SSE framing or larger edge-case chunks can be ignored silently during streaming
• Files: tauri-app/src/components/ChatView.tsx, crates/inference-server/src/routes.rs
• Trigger: partial data: framing, unexpected event fields, or non-content deltas
• Workaround: disable streaming in the UI when debugging transport issues

Security Considerations

Remote API keys stored in plain frontend/backend config:

• Risk: remote API keys are stored in desktop runtime config JSON and also surfaced into frontend state
• Files: tauri-app/src-tauri/src/runtime/config.rs, tauri-app/src/features/runtime/types.ts, tauri-app/src/features/runtime/api.ts
• Current mitigation: password input masks display in tauri-app/src/features/runtime/components/RuntimeSettingsDrawer.tsx:281
• Recommendations: move secrets to OS keychain/credential store and store only references in app config

External CDN dependencies inside desktop app:

• Risk: chat persistence and syntax highlighting depend on runtime-loaded third-party scripts from jsDelivr
• Files: tauri-app/src/lib/db.ts, tauri-app/src/lib/markdown.ts
• Current mitigation: markdown output is sanitized with DOMPurify in tauri-app/src/lib/markdown.ts:68
• Recommendations: vendor sql.js and highlight.js assets into the app bundle and pin integrity-checked local assets

Performance Bottlenecks

Single-engine serialized generation:

• Problem: the API holds one Arc<Mutex<InferenceEngine>> for all requests
• Files: crates/inference-server/src/routes.rs:31, crates/inference-server/src/routes.rs:159, crates/inference-server/src/routes.rs:364
• Cause: engine/context are single-owner and not safely concurrent in crates/inference-core/src/engine.rs
• Improvement path: add queued scheduling or a context/session pool before scaling beyond one active generation

Polling-heavy desktop runtime updates:

• Problem: frontend polls runtime snapshot every 3 seconds and models every 30 seconds
• Files: tauri-app/src/features/runtime/useRuntimeManager.ts:33, tauri-app/src/hooks/useModels.ts:38
• Cause: no push/event channel between Tauri backend and React frontend
• Improvement path: add Tauri event emission or a subscription layer for runtime state and model refresh triggers

Fragile Areas

Chat request + persistence loop:

• Files: tauri-app/src/components/ChatView.tsx, tauri-app/src/lib/db.ts, tauri-app/src/lib/chat.ts
• Why fragile: request orchestration, DB writes, streaming parsing, <think> parsing, title updates, and optimistic UI all live in one component
• Safe modification: extract persistence, transport, and stream parsing into dedicated hooks/utilities before adding more chat features
• Test coverage: no direct tests detected for tauri-app/src/components/ChatView.tsx

Compatibility layer prompt/tool-call handling:

• Files: crates/inference-server/src/compat.rs, crates/api-contracts/src/chat.rs
• Why fragile: one file owns auth, aliases, prompt shaping, template choice, response finalization, and tool-call compatibility behavior
• Safe modification: add focused unit tests around each compatibility branch before extending OpenAI-style contract coverage
• Test coverage: partial for contracts in crates/api-contracts/src/chat.rs; compatibility file-specific tests are not visible in the analyzed slice

Scaling Limits

Backend request concurrency:

• Current capacity: effectively one active generation at a time per server process because one engine/context is mutex-locked
• Limit: parallel requests queue behind one model context and long generations block others
• Scaling path: multi-context pool, worker queue, or multi-process model serving

Desktop chat storage:

• Current capacity: local sql.js database exported into one localStorage key in tauri-app/src/lib/db.ts:57
• Limit: browser storage size, serialization overhead, and startup cost grow with chat history
• Scaling path: move persistence into native Tauri storage or SQLite through the Rust backend

Dependencies at Risk

CDN-hosted sql.js and highlight.js:

• Risk: runtime availability and supply-chain reliability depend on external network/CDN access
• Impact: chat history DB boot or syntax highlighting can fail in offline/locked-down environments
• Migration plan: bundle assets locally or replace with native persistence plus bundled syntax assets

Missing Critical Features

No push observability for runtime lifecycle:

• Problem: runtime state changes and server logs are not streamed into the UI
• Blocks: rich diagnostics, trustworthy startup progress, and smoother local-runtime UX

No model library management workflow:

• Problem: model scanning/import exists, but there is no deeper catalog metadata, download progress, space management, or activation history
• Blocks: desktop app becoming a full local-model manager rather than a thin launcher

Test Coverage Gaps

Chat UI and persistence path:

• What's not tested: chat send/stream/error/edit/delete/memory/persistence flows
• Files: tauri-app/src/components/ChatView.tsx, tauri-app/src/components/MessageBubble.tsx, tauri-app/src/lib/db.ts
• Risk: regressions in the main user workflow can ship unnoticed
• Priority: High

CLI live behavior:

• What's not tested: end-to-end live HTTP execution and local action implementations behind execute_live_command
• Files: crates/localllm-cli/src/main.rs
• Risk: parser tests pass while actual command behavior remains incomplete or broken
• Priority: High

Server route integration:

• What's not tested: full Axum route behavior for auth, streaming, aliases, and error payloads against a running router
• Files: crates/inference-server/src/routes.rs, crates/inference-server/src/compat.rs
• Risk: contract drift between docs and runtime behavior
• Priority: High

---

Concerns audit: 2026-03-28