133 lines
8.2 KiB
Markdown
133 lines
8.2 KiB
Markdown
# Codebase Concerns
|
|
|
|
**Analysis Date:** 2026-03-28
|
|
|
|
## Tech Debt
|
|
|
|
**CLI surface vs real implementation:**
|
|
- Issue: `crates/localllm-cli/src/main.rs` exposes a broad command tree, but many branches collapse to summary-only behavior instead of real actions
|
|
- Files: `crates/localllm-cli/src/main.rs`, `docs/cli/overview.md`
|
|
- Impact: product expectations exceed actual operator capability; docs and UX need constant caveats
|
|
- Fix approach: split stable commands from roadmap commands, or implement local/config/model actions behind dedicated modules and integration tests
|
|
|
|
**Monolithic files in core surfaces:**
|
|
- Issue: major logic is concentrated in very large files such as `crates/inference-server/src/routes.rs`, `crates/inference-server/src/compat.rs`, `crates/localllm-cli/src/main.rs`, `tauri-app/src/components/ChatView.tsx`, and `tauri-app/src/features/runtime/components/RuntimeSettingsDrawer.tsx`
|
|
- Files: `crates/inference-server/src/routes.rs`, `crates/inference-server/src/compat.rs`, `crates/localllm-cli/src/main.rs`, `tauri-app/src/components/ChatView.tsx`, `tauri-app/src/features/runtime/components/RuntimeSettingsDrawer.tsx`
|
|
- Impact: high change risk, harder reviewability, and low reuse of logic across features
|
|
- Fix approach: extract transport, prompt-building, persistence, and UI section components into smaller feature modules with focused tests
|
|
|
|
**Frontend fallback state diverges from desktop source of truth:**
|
|
- Issue: `tauri-app/src/features/runtime/api.ts` silently falls back to `localStorage` snapshots whenever Tauri invoke fails
|
|
- Files: `tauri-app/src/features/runtime/api.ts`, `tauri-app/src/features/runtime/runtime-utils.ts`
|
|
- Impact: browser preview works, but runtime state can drift from the real desktop backend and mask integration failures
|
|
- Fix approach: make fallback mode explicit in UI and separate browser-demo mode from real desktop mode
|
|
|
|
## Known Bugs
|
|
|
|
**Local runtime binary name mismatch risk:**
|
|
- Symptoms: desktop runtime start can fail even when the backend was built, because the Tauri helper looks for `localllm` while the error message tells users to build `inference-server`
|
|
- Files: `tauri-app/src-tauri/src/runtime/process.rs`, `crates/inference-server/Cargo.toml`
|
|
- Trigger: building the wrong cargo target or expecting package name and binary name to match
|
|
- Workaround: build the binary defined in `crates/inference-server/Cargo.toml:8` and ensure `target/release/localllm(.exe)` exists
|
|
|
|
**Streaming parser depends on simple line splitting:**
|
|
- Symptoms: malformed SSE framing or larger edge-case chunks can be ignored silently during streaming
|
|
- Files: `tauri-app/src/components/ChatView.tsx`, `crates/inference-server/src/routes.rs`
|
|
- Trigger: partial `data:` framing, unexpected event fields, or non-content deltas
|
|
- Workaround: disable streaming in the UI when debugging transport issues
|
|
|
|
## Security Considerations
|
|
|
|
**Remote API keys stored in plain frontend/backend config:**
|
|
- Risk: remote API keys are stored in desktop runtime config JSON and also surfaced into frontend state
|
|
- Files: `tauri-app/src-tauri/src/runtime/config.rs`, `tauri-app/src/features/runtime/types.ts`, `tauri-app/src/features/runtime/api.ts`
|
|
- Current mitigation: password input masks display in `tauri-app/src/features/runtime/components/RuntimeSettingsDrawer.tsx:281`
|
|
- Recommendations: move secrets to OS keychain/credential store and store only references in app config
|
|
|
|
**External CDN dependencies inside desktop app:**
|
|
- Risk: chat persistence and syntax highlighting depend on runtime-loaded third-party scripts from jsDelivr
|
|
- Files: `tauri-app/src/lib/db.ts`, `tauri-app/src/lib/markdown.ts`
|
|
- Current mitigation: markdown output is sanitized with DOMPurify in `tauri-app/src/lib/markdown.ts:68`
|
|
- Recommendations: vendor `sql.js` and `highlight.js` assets into the app bundle and pin integrity-checked local assets
|
|
|
|
## Performance Bottlenecks
|
|
|
|
**Single-engine serialized generation:**
|
|
- Problem: the API holds one `Arc<Mutex<InferenceEngine>>` for all requests
|
|
- Files: `crates/inference-server/src/routes.rs:31`, `crates/inference-server/src/routes.rs:159`, `crates/inference-server/src/routes.rs:364`
|
|
- Cause: engine/context are single-owner and not safely concurrent in `crates/inference-core/src/engine.rs`
|
|
- Improvement path: add queued scheduling or a context/session pool before scaling beyond one active generation
|
|
|
|
**Polling-heavy desktop runtime updates:**
|
|
- Problem: frontend polls runtime snapshot every 3 seconds and models every 30 seconds
|
|
- Files: `tauri-app/src/features/runtime/useRuntimeManager.ts:33`, `tauri-app/src/hooks/useModels.ts:38`
|
|
- Cause: no push/event channel between Tauri backend and React frontend
|
|
- Improvement path: add Tauri event emission or a subscription layer for runtime state and model refresh triggers
|
|
|
|
## Fragile Areas
|
|
|
|
**Chat request + persistence loop:**
|
|
- Files: `tauri-app/src/components/ChatView.tsx`, `tauri-app/src/lib/db.ts`, `tauri-app/src/lib/chat.ts`
|
|
- Why fragile: request orchestration, DB writes, streaming parsing, `<think>` parsing, title updates, and optimistic UI all live in one component
|
|
- Safe modification: extract persistence, transport, and stream parsing into dedicated hooks/utilities before adding more chat features
|
|
- Test coverage: no direct tests detected for `tauri-app/src/components/ChatView.tsx`
|
|
|
|
**Compatibility layer prompt/tool-call handling:**
|
|
- Files: `crates/inference-server/src/compat.rs`, `crates/api-contracts/src/chat.rs`
|
|
- Why fragile: one file owns auth, aliases, prompt shaping, template choice, response finalization, and tool-call compatibility behavior
|
|
- Safe modification: add focused unit tests around each compatibility branch before extending OpenAI-style contract coverage
|
|
- Test coverage: partial for contracts in `crates/api-contracts/src/chat.rs`; compatibility file-specific tests are not visible in the analyzed slice
|
|
|
|
## Scaling Limits
|
|
|
|
**Backend request concurrency:**
|
|
- Current capacity: effectively one active generation at a time per server process because one engine/context is mutex-locked
|
|
- Limit: parallel requests queue behind one model context and long generations block others
|
|
- Scaling path: multi-context pool, worker queue, or multi-process model serving
|
|
|
|
**Desktop chat storage:**
|
|
- Current capacity: local sql.js database exported into one `localStorage` key in `tauri-app/src/lib/db.ts:57`
|
|
- Limit: browser storage size, serialization overhead, and startup cost grow with chat history
|
|
- Scaling path: move persistence into native Tauri storage or SQLite through the Rust backend
|
|
|
|
## Dependencies at Risk
|
|
|
|
**CDN-hosted `sql.js` and `highlight.js`:**
|
|
- Risk: runtime availability and supply-chain reliability depend on external network/CDN access
|
|
- Impact: chat history DB boot or syntax highlighting can fail in offline/locked-down environments
|
|
- Migration plan: bundle assets locally or replace with native persistence plus bundled syntax assets
|
|
|
|
## Missing Critical Features
|
|
|
|
**No push observability for runtime lifecycle:**
|
|
- Problem: runtime state changes and server logs are not streamed into the UI
|
|
- Blocks: rich diagnostics, trustworthy startup progress, and smoother local-runtime UX
|
|
|
|
**No model library management workflow:**
|
|
- Problem: model scanning/import exists, but there is no deeper catalog metadata, download progress, space management, or activation history
|
|
- Blocks: desktop app becoming a full local-model manager rather than a thin launcher
|
|
|
|
## Test Coverage Gaps
|
|
|
|
**Chat UI and persistence path:**
|
|
- What's not tested: chat send/stream/error/edit/delete/memory/persistence flows
|
|
- Files: `tauri-app/src/components/ChatView.tsx`, `tauri-app/src/components/MessageBubble.tsx`, `tauri-app/src/lib/db.ts`
|
|
- Risk: regressions in the main user workflow can ship unnoticed
|
|
- Priority: High
|
|
|
|
**CLI live behavior:**
|
|
- What's not tested: end-to-end live HTTP execution and local action implementations behind `execute_live_command`
|
|
- Files: `crates/localllm-cli/src/main.rs`
|
|
- Risk: parser tests pass while actual command behavior remains incomplete or broken
|
|
- Priority: High
|
|
|
|
**Server route integration:**
|
|
- What's not tested: full Axum route behavior for auth, streaming, aliases, and error payloads against a running router
|
|
- Files: `crates/inference-server/src/routes.rs`, `crates/inference-server/src/compat.rs`
|
|
- Risk: contract drift between docs and runtime behavior
|
|
- Priority: High
|
|
|
|
---
|
|
|
|
*Concerns audit: 2026-03-28*
|