chore: initialize autoresearch baseline

README.MD

@@ -0,0 +1,243 @@
+# ForgeLM
+
+High-performance local inference runtime for small and mid-sized language models, designed for efficient execution on commodity hardware.
+
+## Overview
+
+ForgeLM is a local-first inference runtime built for developers who need a clean, controllable, and efficient backend for running quantized language models on consumer GPUs and constrained systems.
+
+The project is centered on a practical engineering goal: extracting high throughput and low latency from compact and mid-sized models without turning the runtime into an opaque monolith or an overextended product shell.
+
+The initial focus is on text-only inference for models in the range of Qwen 4B / 9B class deployments, with special attention to NVIDIA GPUs such as the RTX 3070 and to fallback paths for weaker hardware.
+
+ForgeLM is not intended to be a desktop UI product. It is intended to be a modular inference backend that can be embedded, extended, benchmarked, and reasoned about as infrastructure.
+
+---
+
+## Design Goals
+
+ForgeLM is built around a small number of non-negotiable goals:
+
+- **High throughput**
+  - Prioritize sustained token generation performance for small and mid-sized models.
+
+- **Low latency**
+  - Minimize time-to-first-token and reduce generation overhead in the hot path.
+
+- **Efficient memory usage**
+  - Make disciplined use of VRAM and RAM through careful model loading, batching strategy, and KV-cache policies.
+
+- **Predictable architecture**
+  - Keep the inference path understandable, inspectable, and modular.
+
+- **Hardware realism**
+  - Optimize for hardware that developers actually own, including 8 GB consumer GPUs and CPU fallback scenarios.
+
+- **Separation of concerns**
+  - Keep the inference core, API layer, model registry, and tooling clearly separated.
+
+- **Open integration surface**
+  - Provide a clean local API compatible with common inference client patterns.
+
+---
+
+## Project Scope
+
+ForgeLM focuses on the runtime layer of local language model serving.
+
+### In scope
+
+- Local text inference
+- GGUF-based model execution
+- High-performance inference backends
+- OpenAI-compatible HTTP API
+- Streaming responses via SSE
+- Runtime tuning for throughput and memory efficiency
+- Model registry and download workflow
+- Benchmarking and observability
+
+### Out of scope for the initial versions
+
+- Full desktop product UX
+- Multimodal inference
+- Enterprise orchestration features
+- Training and fine-tuning workflows
+- Plugin-heavy product abstractions
+- Broad feature surface at the expense of runtime efficiency
+
+---
+
+## Core Principles
+
+### 1. The hot path must remain small
+
+The token generation path is where performance is won or lost. ForgeLM keeps this path narrow:
+
+- model execution in a native backend
+- Rust in the serving and orchestration layer
+- no Python in the inference hot path
+- controlled allocation and bounded runtime overhead
+
+### 2. Performance comes before feature breadth
+
+A runtime that supports everything but performs poorly is not useful for the target use case. ForgeLM deliberately starts with a constrained feature set in order to produce measurable runtime efficiency first.
+
+### 3. Public architecture should remain understandable
+
+The repository is intended to be inspectable by engineers. The design should be explainable without product-language shortcuts, hidden runtime behavior, or unnecessary framework complexity.
+
+### 4. Consumer hardware is a first-class target
+
+Many local inference projects are evaluated on unrealistic hardware assumptions. ForgeLM explicitly targets practical systems, including 8 GB GPUs and weaker fallback environments.
+
+---
+
+## Technical Direction
+
+ForgeLM is structured as a layered system.
+
+### Inference Core
+
+The inference core is built around a native execution backend for quantized GGUF models. The initial backend strategy is centered on `llama.cpp`-class execution, selected for its strong hardware coverage, mature quantized model support, and practical performance characteristics.
+
+Responsibilities:
+
+- model loading
+- tokenizer access
+- context management
+- decode loop execution
+- GPU offloading control
+- sampling configuration
+- KV-cache handling
+
+### Serving Layer
+
+The HTTP serving layer is implemented in Rust and is responsible for:
+
+- request validation
+- session orchestration
+- streaming token delivery
+- concurrency control
+- backpressure
+- metrics and health endpoints
+
+This layer exists to expose the runtime cleanly without polluting the core execution path.
+
+### Model Registry
+
+Model search, metadata resolution, and download management are intentionally isolated from the hot path. Registry operations may use a separate process and different tooling, but they do not participate in token generation.
+
+Responsibilities:
+
+- model discovery
+- metadata normalization
+- artifact download
+- integrity verification
+- local registry indexing
+
+### Tooling and Benchmarks
+
+Performance claims without reproducible measurement are not useful. Benchmarking and observability are part of the project rather than afterthoughts.
+
+Responsibilities:
+
+- throughput measurement
+- latency measurement
+- memory usage tracking
+- benchmark replay
+- runtime profiling
+
+---
+
+## Initial Optimization Target
+
+The first optimization target is the practical deployment envelope of small and mid-sized text models on commodity NVIDIA hardware.
+
+That means prioritizing:
+
+- compact and mid-sized Qwen-class models
+- strong throughput on RTX 3070-class systems
+- viable degraded execution on weaker hardware
+- text-only serving before broader capability expansion
+
+In practical terms, the project is designed to explore the trade space between:
+
+- model size
+- quantization level
+- context length
+- GPU offloading depth
+- batching strategy
+- latency
+- aggregate throughput
+- output quality
+
+---
+
+## Architectural Priorities
+
+The implementation roadmap is shaped by the following priorities:
+
+### Text-only first
+
+Text inference is the fastest path to a stable and high-performance serving core. Additional modalities, if ever added, must not distort the original runtime goals.
+
+### Single-model efficiency before multi-model orchestration
+
+Before adding model multiplexing or dynamic model switching, the runtime must demonstrate excellent single-model performance and stable serving behavior.
+
+### Controlled concurrency before maximum concurrency
+
+Concurrency is useful only when it improves real system throughput without collapsing latency or wasting memory. The project favors disciplined scheduling over superficial parallelism.
+
+### Measured optimization over speculative abstraction
+
+Every substantial optimization should be benchmarked. Every abstraction in the serving path should justify its cost.
+
+---
+
+## Proposed Stack
+
+The current technical direction is:
+
+### Native backend
+- C/C++
+- `llama.cpp`-class runtime backend
+- CUDA path for NVIDIA acceleration
+
+### Serving layer
+- Rust
+- Tokio
+- Axum
+- Hyper
+- FFI bindings generated with `bindgen`
+
+### Registry and model management
+- Python
+- isolated background process or service
+- Hugging Face integration for model discovery and retrieval
+
+### UI
+- minimal local web interface for runtime validation
+- not a product-grade desktop shell
+
+---
+
+## Repository Structure
+
+```text
+forgelm/
+├── Cargo.toml
+├── crates/
+│   ├── backend-sys/
+│   ├── inference-core/
+│   ├── inference-server/
+│   ├── api-contracts/
+│   └── metrics/
+├── registry/
+│   ├── service.py
+│   └── requirements.txt
+├── ui/
+│   └── index.html
+├── vendor/
+│   └── backend/
+└── models/