Added orchestrator.rs with parallel dispatching of 4 personas (Security, Architect, Critic, UX) using tokio Semaphores to prevent OOM. Included stream multiplexing and synthesis phase, though still in draft/skeleton form.
When using local aliases like Cooper, Iron, or Steel (which are Qwen-based architectures), the select_chat_template failed to identify them because the alias names didn't contain 'qwen'. This caused the inference server to fall back to the Generic template (User: ... \nAssistant:), which caused Qwen models to hallucinate completion blocks starting with <|im_end|>\n<|im_start|>assistant to 'correct' the missing tags, resulting in the model repeating the user's prompt at the beginning of its response.
1. Reduced repeat_penalty to 1.15 to avoid degrading normal generation.
2. TPS degradation monitor now actively aborts the stream instead of just warning.
3. Removed broad transcript leakage regex from the frontend, relying fully on backend stop sequences to prevent false positives.
4. Fixed race condition in useRuntimeManager by moving model scanning into a reactive useEffect instead of an imperative ref call.
5. Enforced JSON format natively via response_format for ForgeAgent inference requests.
Backend (inference-server/compat.rs):
- PreparedPrompt now carries extra_stop_sequences injected by template
- Generic template injects User:/System:/Assistant:/Human: stop sequences
to prevent the model from continuing the transcript after its reply
- Increased repeat_penalty 1.15→1.3 and repeat_last_n 256→512 to
break repetition loops more aggressively
Backend (inference-server/routes.rs):
- Merge template extra_stop_sequences into sampler before inference
Frontend (chat.ts):
- Added 'System:' and 'Human:' to CHAT_TRANSCRIPT_STOP_SEQUENCES
- Extended TRANSCRIPT_LEAKAGE_PATTERN to include 'Human:'
Frontend (ChatView.tsx):
- buildApiMessages now strips leaked transcript from assistant messages
before sending history back to the model
- Reduced history window to 20 messages to limit context pollution
- Check process_running and ready state before making inference request
- Emit descriptive error messages for each failure scenario
- Handle local vs remote connection modes appropriately
- Validation score: 100 (was 0)
Result: {"status":"keep","validation_score":100,"response_time_ms":500}
- routes.rs: StreamItem::Error now emits content:None so fullContent stays
empty and the frontend does not accumulate 'internal error' as a response
- ChatView.tsx: buildApiMessages filters out empty/error messages from the
history sent to the backend, preventing corrupted context on retries
- ChatView.tsx: assistant INSERT is guarded — skipped when fullContent is
empty, so a failed stream leaves no empty assistant row in the DB
- engine.rs: new_session() now returns Err when llama_get_memory returns NULL
instead of silently continuing, preventing decode corruption on 2nd request
- session.rs: decode_prefill error includes token count for easier diagnosis
Evandro Fonseca Junior