Backend (inference-server/compat.rs):
- PreparedPrompt now carries extra_stop_sequences injected by template
- Generic template injects User:/System:/Assistant:/Human: stop sequences
to prevent the model from continuing the transcript after its reply
- Increased repeat_penalty 1.15→1.3 and repeat_last_n 256→512 to
break repetition loops more aggressively
Backend (inference-server/routes.rs):
- Merge template extra_stop_sequences into sampler before inference
Frontend (chat.ts):
- Added 'System:' and 'Human:' to CHAT_TRANSCRIPT_STOP_SEQUENCES
- Extended TRANSCRIPT_LEAKAGE_PATTERN to include 'Human:'
Frontend (ChatView.tsx):
- buildApiMessages now strips leaked transcript from assistant messages
before sending history back to the model
- Reduced history window to 20 messages to limit context pollution
- Check process_running and ready state before making inference request
- Emit descriptive error messages for each failure scenario
- Handle local vs remote connection modes appropriately
- Validation score: 100 (was 0)
Result: {"status":"keep","validation_score":100,"response_time_ms":500}
- routes.rs: StreamItem::Error now emits content:None so fullContent stays
empty and the frontend does not accumulate 'internal error' as a response
- ChatView.tsx: buildApiMessages filters out empty/error messages from the
history sent to the backend, preventing corrupted context on retries
- ChatView.tsx: assistant INSERT is guarded — skipped when fullContent is
empty, so a failed stream leaves no empty assistant row in the DB
- engine.rs: new_session() now returns Err when llama_get_memory returns NULL
instead of silently continuing, preventing decode corruption on 2nd request
- session.rs: decode_prefill error includes token count for easier diagnosis
Evandro Fonseca Junior