1. Reduced repeat_penalty to 1.15 to avoid degrading normal generation.
2. TPS degradation monitor now actively aborts the stream instead of just warning.
3. Removed broad transcript leakage regex from the frontend, relying fully on backend stop sequences to prevent false positives.
4. Fixed race condition in useRuntimeManager by moving model scanning into a reactive useEffect instead of an imperative ref call.
5. Enforced JSON format natively via response_format for ForgeAgent inference requests.
Backend (inference-server/compat.rs):
- PreparedPrompt now carries extra_stop_sequences injected by template
- Generic template injects User:/System:/Assistant:/Human: stop sequences
to prevent the model from continuing the transcript after its reply
- Increased repeat_penalty 1.15→1.3 and repeat_last_n 256→512 to
break repetition loops more aggressively
Backend (inference-server/routes.rs):
- Merge template extra_stop_sequences into sampler before inference
Frontend (chat.ts):
- Added 'System:' and 'Human:' to CHAT_TRANSCRIPT_STOP_SEQUENCES
- Extended TRANSCRIPT_LEAKAGE_PATTERN to include 'Human:'
Frontend (ChatView.tsx):
- buildApiMessages now strips leaked transcript from assistant messages
before sending history back to the model
- Reduced history window to 20 messages to limit context pollution
- Check process_running and ready state before making inference request
- Emit descriptive error messages for each failure scenario
- Handle local vs remote connection modes appropriately
- Validation score: 100 (was 0)
Result: {"status":"keep","validation_score":100,"response_time_ms":500}
- routes.rs: StreamItem::Error now emits content:None so fullContent stays
empty and the frontend does not accumulate 'internal error' as a response
- ChatView.tsx: buildApiMessages filters out empty/error messages from the
history sent to the backend, preventing corrupted context on retries
- ChatView.tsx: assistant INSERT is guarded — skipped when fullContent is
empty, so a failed stream leaves no empty assistant row in the DB
- engine.rs: new_session() now returns Err when llama_get_memory returns NULL
instead of silently continuing, preventing decode corruption on 2nd request
- session.rs: decode_prefill error includes token count for easier diagnosis
Evandro Fonseca Junior