When using local aliases like Cooper, Iron, or Steel (which are Qwen-based architectures), the select_chat_template failed to identify them because the alias names didn't contain 'qwen'. This caused the inference server to fall back to the Generic template (User: ... \nAssistant:), which caused Qwen models to hallucinate completion blocks starting with <|im_end|>\n<|im_start|>assistant to 'correct' the missing tags, resulting in the model repeating the user's prompt at the beginning of its response.
1. Reduced repeat_penalty to 1.15 to avoid degrading normal generation.
2. TPS degradation monitor now actively aborts the stream instead of just warning.
3. Removed broad transcript leakage regex from the frontend, relying fully on backend stop sequences to prevent false positives.
4. Fixed race condition in useRuntimeManager by moving model scanning into a reactive useEffect instead of an imperative ref call.
5. Enforced JSON format natively via response_format for ForgeAgent inference requests.
Backend (inference-server/compat.rs):
- PreparedPrompt now carries extra_stop_sequences injected by template
- Generic template injects User:/System:/Assistant:/Human: stop sequences
to prevent the model from continuing the transcript after its reply
- Increased repeat_penalty 1.15→1.3 and repeat_last_n 256→512 to
break repetition loops more aggressively
Backend (inference-server/routes.rs):
- Merge template extra_stop_sequences into sampler before inference
Frontend (chat.ts):
- Added 'System:' and 'Human:' to CHAT_TRANSCRIPT_STOP_SEQUENCES
- Extended TRANSCRIPT_LEAKAGE_PATTERN to include 'Human:'
Frontend (ChatView.tsx):
- buildApiMessages now strips leaked transcript from assistant messages
before sending history back to the model
- Reduced history window to 20 messages to limit context pollution
- Check process_running and ready state before making inference request
- Emit descriptive error messages for each failure scenario
- Handle local vs remote connection modes appropriately
- Validation score: 100 (was 0)
Result: {"status":"keep","validation_score":100,"response_time_ms":500}
Evandro Fonseca Junior