[autoresearch] baseline: system prompt quality tests (5/5 passing)
This commit is contained in:
evandrodevbr committed
1 parent
05cc036f6d
commit
60b0101717
142 files changed
+18589
-9947
No files matched your search
@@ -0,0 +1,129 @@
|
||||
---
|
||||
name: sequential-thinking
|
||||
description: Performs dynamic, reflective problem-solving through iterative thought chains. Use for complex planning requiring revision, branching, backtracking, or hypothesis verification. Ideal for multi-step analysis where context maintenance is required or the full scope isn't initially clear.
|
||||
---
|
||||
|
||||
# Sequential Thinking
|
||||
|
||||
A structured approach to complex problem-solving that breaks down challenges into iterative thought steps with built-in flexibility for revision and course correction.
|
||||
|
||||
## When to Use This Skill
|
||||
|
||||
- Breaking down complex problems into manageable steps
|
||||
- Planning and design requiring iterative refinement
|
||||
- Analysis that might need course correction mid-stream
|
||||
- Problems where the full scope emerges during analysis
|
||||
- Multi-step solutions requiring context across steps
|
||||
- Filtering out irrelevant information
|
||||
- Hypothesis generation and verification workflows
|
||||
|
||||
## Core Methodology
|
||||
|
||||
Sequential thinking follows a dynamic process:
|
||||
|
||||
1. **Initial estimation**: Start with an estimate of thoughts needed, but remain flexible
|
||||
2. **Iterative analysis**: Work through thoughts sequentially while building context
|
||||
3. **Revision capability**: Question or revise previous thoughts as understanding deepens
|
||||
4. **Branch exploration**: Explore alternative approaches when needed
|
||||
5. **Hypothesis cycle**: Generate hypotheses, verify against thought chain, repeat
|
||||
6. **Convergence**: Continue until reaching a satisfactory solution
|
||||
|
||||
## Instructions
|
||||
|
||||
### Thought Structure
|
||||
|
||||
Each thought in the sequence should include:
|
||||
|
||||
- **thought**: Current thinking step content
|
||||
- **thoughtNumber**: Position in sequence (1, 2, 3, ...)
|
||||
- **totalThoughts**: Current estimate of total thoughts needed (adjustable)
|
||||
- **nextThoughtNeeded**: Whether another thought step is required
|
||||
|
||||
Optional revision/branching metadata:
|
||||
- **isRevision**: Boolean indicating if reconsidering previous thinking
|
||||
- **revisesThought**: Which thought number is being revised
|
||||
- **branchFromThought**: Branching point thought number
|
||||
- **branchId**: Identifier for current branch
|
||||
- **needsMoreThoughts**: Flag when reaching end but requiring more analysis
|
||||
|
||||
### Process Guidelines
|
||||
|
||||
**Starting out:**
|
||||
- Estimate initial thoughts needed based on problem complexity
|
||||
- Begin with thought 1, establishing context and approach
|
||||
- Set totalThoughts conservatively; you can adjust later
|
||||
|
||||
**During analysis:**
|
||||
- Build on previous thoughts while maintaining context
|
||||
- Filter out irrelevant information at each step
|
||||
- Express uncertainty when present
|
||||
- Don't hesitate to revise if you spot errors or better approaches
|
||||
- Adjust totalThoughts up/down as the problem's scope becomes clearer
|
||||
|
||||
**Revision pattern:**
|
||||
When reconsidering previous thinking:
|
||||
```json
|
||||
{
|
||||
"thought": "On reflection, thought 3's assumption about X was incorrect because Y...",
|
||||
"thoughtNumber": 6,
|
||||
"totalThoughts": 10,
|
||||
"isRevision": True,
|
||||
"revisesThought": 3,
|
||||
"nextThoughtNeeded": True
|
||||
}
|
||||
```
|
||||
|
||||
**Hypothesis cycle:**
|
||||
1. Generate hypothesis based on current understanding
|
||||
2. Verify against previous thought chain
|
||||
3. If verification fails, revise or branch
|
||||
4. Repeat until hypothesis is validated
|
||||
|
||||
**Completion:**
|
||||
- Only set `nextThoughtNeeded: False` when truly satisfied with the solution
|
||||
- Provide a single, clear final answer
|
||||
- Ensure the answer directly addresses the original problem
|
||||
|
||||
### Working with Context
|
||||
|
||||
**Maintain continuity:**
|
||||
- Reference specific previous thoughts by number
|
||||
- Build logical connections between thoughts
|
||||
- Track which thoughts are still valid vs. revised
|
||||
|
||||
**Filter information:**
|
||||
- Ignore details irrelevant to current thought step
|
||||
- Focus on information that advances understanding
|
||||
- Re-evaluate relevance as context evolves
|
||||
|
||||
**Manage complexity:**
|
||||
- If a thought becomes too complex, break it into multiple thoughts
|
||||
- Increase totalThoughts estimate accordingly
|
||||
- Keep each individual thought focused
|
||||
|
||||
### Output Format
|
||||
|
||||
Present your sequential thinking in a structured format:
|
||||
```
|
||||
Thought [N/Total]: [Current thought content]
|
||||
[If revision: "This revises thought X because..."]
|
||||
[If branching: "Branching from thought X to explore..."]
|
||||
|
||||
[Continue with next thought when nextThoughtNeeded is True]
|
||||
|
||||
Final output after all thoughts complete:
|
||||
Solution: [Clear, direct answer to the original problem]
|
||||
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
For concrete examples of sequential thinking in action, see `resources/examples.md`.
|
||||
|
||||
## Key Principles
|
||||
|
||||
- **Flexibility over rigidity**: Adjust your approach as understanding deepens
|
||||
- **Revision is strength**: Correcting course shows good reasoning
|
||||
- **Hypothesis-driven**: Generate and test hypotheses iteratively
|
||||
- **Context-aware**: Maintain awareness of previous thoughts while progressing
|
||||
- **Clarity at completion**: Deliver a single, clear final answer
|
||||
@@ -0,0 +1,65 @@
|
||||
# Sequential Thinking Examples
|
||||
|
||||
## Example 1: Simple Math Problem
|
||||
|
||||
**Problem:** If a train travels 120 km in 2 hours, then 180 km in the next 3 hours, what's the average speed?
|
||||
|
||||
**Thought 1/5:** Breaking down the problem - we have two segments of travel with different distances and times. Need to calculate total distance and total time first.
|
||||
|
||||
**Thought 2/5:** First segment: 120 km in 2 hours. Second segment: 180 km in 3 hours. Total distance = 120 + 180 = 300 km. Total time = 2 + 3 = 5 hours.
|
||||
|
||||
**Thought 3/5:** Average speed formula = Total distance / Total time = 300 km / 5 hours = 60 km/h.
|
||||
|
||||
**Thought 4/5:** Verifying: Does this make sense? First segment speed = 60 km/h, second segment = 60 km/h. Wait, that can't be right.
|
||||
|
||||
**Thought 5/6 (Revision of Thought 4):** Let me recalculate segment speeds. First: 120/2 = 60 km/h. Second: 180/3 = 60 km/h. Actually, both segments are at 60 km/h, so average is correctly 60 km/h. My suspicion was unfounded.
|
||||
|
||||
**Solution:** The average speed is 60 km/h.
|
||||
|
||||
## Example 2: Complex Design Decision with Branching
|
||||
|
||||
**Problem:** Design a caching strategy for a high-traffic API with both user-specific and global data.
|
||||
|
||||
**Thought 1/8:** Initial analysis - need to consider cache hit rate, memory usage, invalidation complexity, and latency requirements. Two data types suggest a multi-layered approach.
|
||||
|
||||
**Thought 2/8:** Hypothesis: Use Redis with separate key namespaces for user-specific (user:*) and global (global:*) data. User data could have shorter TTL due to personalization.
|
||||
|
||||
**Thought 3/8:** Considering TTL strategy - global data changes infrequently, could use 1-hour TTL. User-specific data needs 5-15 minute TTL to stay fresh.
|
||||
|
||||
**Thought 4/8:** Wait, need to think about invalidation. What happens when global data updates? Broadcasting invalidation to all instances could be expensive.
|
||||
|
||||
**Thought 5/10 (Branching from Thought 4):** Alternative approach - use cache versioning. Include version number in global cache keys. When data updates, increment version. Old cached entries naturally expire.
|
||||
|
||||
**Thought 6/10:** This versioning approach adds complexity but eliminates broadcast invalidation. Trade-off: some stale data until TTL expires vs. system complexity.
|
||||
|
||||
**Thought 7/10:** For user-specific data, simple TTL should work since each user's data is independent. No cross-user invalidation needed.
|
||||
|
||||
**Thought 8/10:** Hypothesis verification - versioned global cache + TTL-based user cache provides good balance. Testing memory usage: assume 100K users, 10KB user cache, 50MB global = ~1GB total. Acceptable.
|
||||
|
||||
**Thought 9/10:** Edge case consideration - what if version wraps around? Use timestamp-based versioning (YYYYMMDDHHMMSS) to prevent collisions.
|
||||
|
||||
**Thought 10/10:** Final design validated. Global cache with timestamp versioning, user cache with 10-minute TTL. Monitoring on cache hit rates and memory usage post-deployment.
|
||||
|
||||
**Solution:** Implement a two-tier Redis caching strategy with timestamp-versioned keys for global data (1-hour TTL) and simple key-value with 10-minute TTL for user-specific data. Monitor hit rates and memory usage after deployment.
|
||||
|
||||
## Example 3: Debugging with Multiple Revisions
|
||||
|
||||
**Problem:** API returning 500 errors intermittently, logs show database timeout.
|
||||
|
||||
**Thought 1/6:** Initial hypothesis - database connection pool exhaustion. Check current pool size and active connections.
|
||||
|
||||
**Thought 2/6:** Pool size = 20, monitoring shows peak usage of 18 connections. Not exhausted. Need to look elsewhere.
|
||||
|
||||
**Thought 3/7 (Revised estimate):** Query performance issue? Check slow query log. Found several queries taking 5-10 seconds during error periods.
|
||||
|
||||
**Thought 4/7:** Slow queries all involve joins on large tables without proper indexes. Hypothesis: missing indexes causing table scans.
|
||||
|
||||
**Thought 5/7:** Wait, these queries worked fine last week. What changed? Checking deployment history... no schema changes, no new queries.
|
||||
|
||||
**Thought 6/8 (Revision of Thought 4):** The queries themselves haven't changed, but data volume has. Tables grew 300% last week due to marketing campaign. Indexes exist but statistics are stale.
|
||||
|
||||
**Thought 7/8:** Hypothesis: Stale query planner statistics causing poor query plans. Solution: Run ANALYZE on affected tables to update statistics.
|
||||
|
||||
**Thought 8/8:** Verified - after ANALYZE, query plans improved and execution times dropped to <500ms. Root cause was stale statistics, not missing indexes.
|
||||
|
||||
**Solution:** Run ANALYZE on the affected database tables to update query planner statistics. Implement automated ANALYZE job to run after bulk data imports to prevent future occurrences.
|
||||
Reference in new issue
Block a user