ForgeLM/.planning/phases/04-model-source-validation/04-RESEARCH.md

Phase 4: Model Source Validation - Research

Gathered: 2026-03-29 Status: Complete

---

Domain Analysis

Current Model Source System

The app supports two connection modes:

1. Local Mode (connection_mode: "local")

   • Uses local GGUF model files
   • Requires: model_path, models_dir, addr, gpu_layers, ctx_size
   • Supports extra_model_paths for additional search paths

2. Remote Mode (connection_mode: "remote")

   • Uses remote API endpoints (OpenAI, OpenRouter, Anthropic, Custom)
   • Requires: server_url, api_key, remote_presets
   • Does NOT use local model files

Existing Model Selection Components

• RuntimeConfig (types.ts): Contains model_path, models_dir, selected_model_id, remote_presets
• scan_models (Rust): Scans directory for .gguf files
• pick_models_dir (Rust): Opens folder picker, returns dir + scanned models
• createRuntimeConfigPatch (runtime-settings-form.ts): Creates config from form state
• resolveDraftModelSelection: Validates selected model against scanned choices

Draft Field Tracking

Currently tracked draft fields (from Phase 1):

• models_dir
• model_path
• selected_model_id
• server_url
• api_key

---

Source Types

Local Source

• Models directory: User-selected folder containing .gguf files
• Model path: Specific .gguf file selected from directory
• Extra model paths: Additional directories to scan

Remote Presets

• OpenAI, OpenRouter, Anthropic, Custom, Unknown
• Each has: id, label, provider, base_url, default_model

Model Library (Future - implied by MODL-02)

• ModelLibraryEntry: {id, label, source, path}
• ModelLibraryState: {entries, managed_dir}

---

Validation Patterns Needed

1. Source-Mode Compatibility

• When connection_mode is "local": require valid model_path pointing to existing .gguf file
• When connection_mode is "remote": require valid server_url and optionally api_key
• Switching modes should validate the target configuration before allowing save

2. Path Validation

• models_dir: Must be an existing, readable directory
• model_path: Must be an existing .gguf file (if set)
• server_url: Must be valid URL format for remote mode

3. Model-Source Validation

• A model selection is only valid if:
  • It exists in the scanned models from current models_dir, OR
  • It's in extra_model_paths scan results, OR
  • It's a manually entered path that exists on filesystem

4. Invalid Selection Recovery

• When scanned models change (user picks new folder), detect if current model_path is no longer valid
• Show inline validation message: "This draft model is no longer available in the current library or folder scan."
• Provide clear guidance: "Select a model from the list or enter a valid path"

---

Draft Integration

The existing draft ownership model from Phases 1-3 must be extended:

1. Field validation state (RuntimeDraftFieldMeta): Already exists with invalid, pending, message
2. Validation on models_dir change: When user changes models_dir, re-scan and validate model_path
3. Validation on mode switch: Clear/invalidate fields that don't apply to new mode

Validation Flow

User changes models_dir → scan new directory → 
  if model_path invalid → set field.invalid=true, field.message="..." →
  User sees inline error → User selects valid model or clears path →
  field.invalid=false

---

UI/UX Considerations

Invalid Selection Display

• Show validation message below the model dropdown/input
• Use warning/danger tone based on severity
• Disable save/apply if critical fields are invalid

Recovery Guidance

• "Model not found in current folder"
• "Select from available models or enter a valid file path"
• "Click 'Scan' to refresh model list"

Path Selection UX

• Use folder picker dialog for models_dir
• Show scanned model count after scan
• Display "No models found" if directory is empty

---

Dependencies to Reuse

Existing Patterns

1. RuntimeDraftFieldState - Field-level validation tracking
2. createRuntimeDraftFieldState() - Initialize field meta
3. resolveDraftModelSelection() - Already validates model availability
4. scan_models Rust command - Directory scanning
5. pick_models_dir Rust command - Folder picker integration

Types to Extend

• RuntimeConfig: Add validation-specific fields if needed
• RuntimeDraftFieldMeta: Already sufficient for validation state

Files to Modify

• runtime-settings-form.ts: Add validation functions
• useRuntimeManager.ts: Integrate validation into draft flow
• runtime-utils.ts: Add source-specific validation helpers
• UI components for validation message display

---

Technical Approach Summary

1. Source-Mode Validation: Add validation functions that check config based on connection_mode
2. Path Existence: Use backend scan_models to verify paths, or add file existence check
3. Model Availability: Reuse resolveDraftModelSelection pattern for validation
4. Inline Errors: Display validation messages in form using existing field meta
5. Recovery: Clear guidance on how to fix invalid selections

---

Open Questions for Planning

1. Should we validate remote preset selections (e.g., check if API key is present)?
2. Do we need to support model compatibility hints (e.g., "this model requires 8GB VRAM")?
3. Should managed library (MODL-02) include model metadata beyond path/name?
4. How do we handle validation during the "save before apply" flow?

---

Research complete: 2026-03-29