26 lines
2.0 KiB
Plaintext
26 lines
2.0 KiB
Plaintext
[2m2026-03-27T15:10:47.602969Z[0m [32m INFO[0m Loading model via llama-sys [3mmodel[0m[2m=[0mD:\Documents\Github\ai-backend-studio\models\Qwen3.5-4B-Abliterated-Claude-4.6-Opus-Reasoning-Distilled.i1-Q6_K.gguf [3mprofile[0m[2m=[0mTurbo4B [3mn_gpu_layers[0m[2m=[0m999 [3mn_ctx[0m[2m=[0m16384 [3mn_batch[0m[2m=[0m128 [3mn_ubatch[0m[2m=[0m128 [3mflash_attn[0m[2m=[0mtrue [3muse_mmap[0m[2m=[0mfalse
|
|
[2m2026-03-27T15:10:47.603056Z[0m [32m INFO[0m Loading llama library [3mpath[0m[2m=[0mllama.dll
|
|
[2m2026-03-27T15:10:47.691116Z[0m [32m INFO[0m llama backend initialised
|
|
[2m2026-03-27T15:10:49.551993Z[0m [32m INFO[0m Model loaded [3mdesc[0m[2m=[0mqwen35 4B Q6_K [3mparams_b[0m[2m=[0m4.205751296 [3mis_hybrid[0m[2m=[0mtrue [3mis_recurrent[0m[2m=[0mfalse
|
|
[2m2026-03-27T15:10:49.664247Z[0m [32m INFO[0m Context initialised [3mn_ctx[0m[2m=[0m16384
|
|
[2m2026-03-27T15:10:49.664284Z[0m [32m INFO[0m Running engine warmup (dummy decode)…
|
|
[2m2026-03-27T15:10:49.722289Z[0m [32m INFO[0m Warmup complete
|
|
|
|
╔══════════════════════════════════════════════════╗
|
|
║ ForgeLM • Local LLM Runtime ║
|
|
╚══════════════════════════════════════════════════╝
|
|
|
|
Model : Qwen3.5-4B-Abliterated-Claude-4.6-Opus-Reasoning-Distilled.i1-Q6_K
|
|
|
|
API Endpoints:
|
|
GET http://127.0.0.1:18083/health
|
|
GET http://127.0.0.1:18083/v1/models
|
|
POST http://127.0.0.1:18083/v1/chat/completions
|
|
|
|
Chat UI:
|
|
file://$(pwd)/ui/chat.html
|
|
(open ui/chat.html in your browser and set server to http://127.0.0.1:18083)
|
|
|
|
[2m2026-03-27T15:10:50.807302Z[0m [32m INFO[0m Prefill finished in 134.37 ms (2143.28 t/s)
|
|
[2m2026-03-27T15:10:53.092803Z[0m [32m INFO[0m Generation complete [3mtokens_generated[0m[2m=[0m128 [3mtps[0m[2m=[0m"56.0" [3mprefill_ms[0m[2m=[0m"134.4"
|