fix: map local model aliases to Qwen template to stop prompt echo loops

When using local aliases like Cooper, Iron, or Steel (which are Qwen-based architectures), the select_chat_template failed to identify them because the alias names didn't contain 'qwen'. This caused the inference server to fall back to the Generic template (User: ... \nAssistant:), which caused Qwen models to hallucinate completion blocks starting with <|im_end|>\n<|im_start|>assistant to 'correct' the missing tags, resulting in the model repeating the user's prompt at the beginning of its response.
This commit is contained in:
Evandro Fonseca Junior committed 2026-04-04 00:19:34 -03:00
1 parent 99aff0be6f
commit 8f32b677da
1 file changed
+1 -1
+1 -1
View File
@@ -461,7 +461,7 @@ fn select_chat_template(req: &ChatCompletionRequest, config: &ServerConfig) -> C
for candidate in candidates {
let normalized = candidate.to_ascii_lowercase();
if normalized.contains("qwen") {
if normalized.contains("qwen") || normalized.contains("cooper") || normalized.contains("iron") || normalized.contains("steel") {
return ChatTemplate::Qwen;
}
if normalized.contains("gemma") {