[autoresearch] baseline: system prompt quality tests (5/5 passing)

program.md

@@ -0,0 +1,53 @@
+# Research Program: Response Quality Optimization
+
+## Goal
+Optimize the SYSTEM_PROMPT to maximize response quality and accuracy from local LLMs (Qwen3 0.8B-4B range).
+
+## Current State
+- System prompt is minimal (1 sentence)
+- Works for basic queries but may lack guidance for:
+  - Consistent response format
+  - Appropriate detail level
+  - Language consistency
+  - Avoiding hallucinations
+
+## Hypotheses to Test
+
+### 1. Role Clarity
+Adding a clearer role definition might improve helpfulness.
+- "You are a helpful assistant" → more specific role?
+- Test: "You are ForgeLM, a helpful local AI assistant"
+
+### 2. Response Guidelines
+Adding lightweight guidance on response style.
+- Keep it short to avoid token overhead
+- Test adding: "Be concise but thorough"
+
+### 3. Language Consistency
+Current: "Respond in the same language as the user"
+- Might be too vague for small models
+- Test: "Detect the user's language from their message and respond in that language"
+
+### 4. Hallucination Reduction
+Adding a gentle reminder to stick to facts.
+- Risk: might cause refusals on small models
+- Test: "Only state facts you are confident about"
+
+### 5. Instruction Following
+Testing if specific formatting helps.
+- "Use markdown for code blocks"
+- "Bullet points for lists"
+
+## Constraints
+- MUST stay under ~50 tokens to preserve context window
+- MUST work with 0.8B parameter models
+- MUST NOT cause refusal loops
+- Positive framing only (no "don't" statements)
+
+## Success Criteria
+- Test pass rate > 95%
+- No regression in existing functionality
+- Maintain compatibility across model sizes
+
+## Past Learnings
+(Space to record what worked/didn't work)