Compare commits

5 Commits
Author SHA1 Message Date
evandro a8321ea498 docs(readme): redesenha o README com hero em SVG [skip ci] 2026-10-24 04:57:51 -03:00
evandro c246e68edd docs(readme): padroniza README (hero, badges, quickstart, arquitetura) 2026-10-24 04:13:01 +00:00
Evandro Fonseca Junior 1e0490b1f7 docs: rewrite README with verified setup, web deploy and limitations
Documents only what was executed on Linux (Node 24, npm): install, tsc, expo lint,
web export and a headless-browser E2E against a mock of the LM Studio /v1 API.
Also fixes the clone URL, which pointed at the old LocalAiApp repository name.
2026-09-14 07:55:37 -03:00
Evandro Fonseca Junior 7c624cdce8 chore: drop committed junk files and ignore python caches
- remove the stray '1.2.0' file at the root, an npm install log committed by accident.
- remove the committed __pycache__/*.pyc binaries and ignore __pycache__/ and *.pyc.
- add .gitattributes marking .agent/** as vendored/documentation so GitHub stops
  reporting the repository as Python when the app is TypeScript.
2026-09-14 07:55:33 -03:00
Evandro Fonseca Junior 4cc1addb3c fix: make the web bundle run and repair broken route declarations
- add babel.config.js with unstable_transformImportMeta: zustand/middleware ships
  import.meta.env, which the Metro web bundle (a classic script) cannot parse, so
  every page threw 'SyntaxError: Cannot use import.meta outside a module' and the
  app never hydrated after the static export.
- declare the settings screen as 'settings/index' in the root stack: the old name
  'settings' matched no route, so the modal presentation and the 'Settings' title
  were silently dropped (expo-router logged 'No route named settings exists').
- add the missing app/+not-found.tsx that the root layout already referenced.
- update the About label from LocalAiApp to LocalMind.
2026-09-14 07:55:29 -03:00
25 changed files with 13370 additions and 2948 deletions

No files matched your search

+4
View File
@@ -0,0 +1,4 @@
# O diretorio .agent/ contem ferramentas de apoio ao desenvolvimento (agentes,
# skills e rules de IA), nao faz parte do app React Native.
.agent/** linguist-vendored
.agent/** linguist-documentation
+4
View File
@@ -36,6 +36,10 @@ yarn-error.*
# typescript
*.tsbuildinfo
# python (scripts de apoio em .agent/)
__pycache__/
*.pyc
app-example
# generated native folders
-21
View File
@@ -1,21 +0,0 @@
node_modules/
.next/
dist/
build/
out/
coverage/
.git/
.pnpm-store/
.cache/
*.log
.env
.env.*
__pycache__/
*.pyc
.venv/
venv/
target/
*.exe
*.dll
*.so
*.dylib
-12
View File
@@ -1,12 +0,0 @@
added 885 packages, changed 29 packages, and audited 915 packages in 22s
166 packages are looking for funding
run `npm fund` for details
4 vulnerabilities (1 moderate, 3 high)
To address all issues, run:
npm audit fix
Run `npm audit` for details.
+50 -46
View File
@@ -1,79 +1,83 @@
# LocalMind
<div align="center">
![React Native](https://img.shields.io/badge/React_Native-20232A?style=for-the-badge&logo=react&logoColor=61DAFB) ![Expo](https://img.shields.io/badge/expo-1C1E24?style=for-the-badge&logo=expo&logoColor=#D04A37) ![Zustand](https://img.shields.io/badge/Zustand-443E38?style=for-the-badge&logo=react&logoColor=white) ![LMStudio](https://img.shields.io/badge/Powered_by-LMStudio-5E5DF0?style=for-the-badge)
<img src="assets/readme/hero.svg" width="100%" alt="LocalMind: chat with your LM Studio models from a phone or browser on your own network">
> A fully local, privacy-first AI companion app built with React Native and Expo. Your data never leaves your device.
![React Native 0.81](https://img.shields.io/badge/React_Native-0.81-61DAFB?style=flat-square&logo=react&logoColor=black)
![Expo 54](https://img.shields.io/badge/Expo-54-1C1E24?style=flat-square&logo=expo&logoColor=white)
![TypeScript 5.9](https://img.shields.io/badge/TypeScript-5.9-3178C6?style=flat-square&logo=typescript&logoColor=white)
![Platforms](https://img.shields.io/badge/platforms-Android_%7C_iOS_%7C_Web-555?style=flat-square)
![Status](https://img.shields.io/badge/status-alpha-orange?style=flat-square)
This application connects directly to your own local AI models, completely offline. Currently in alpha, it pairs flawlessly with the [LMStudio](https://lmstudio.ai) local inference API. We developed and tested it against **Qwen3.5 (without thinking)** to guarantee a smooth, zero-latency experience.
[Quick start](#quick-start) · [Issues](https://git.evandro.dev.br/evandro/LocalMind/issues)
*Roadmap: Future updates will introduce direct in-app model downloads and support for external APIs.*
</div>
---
## What it is
## Features
LocalMind is a chat client for the models you run in [LM Studio](https://lmstudio.ai). It is an Expo app for Android, iOS and the web that talks to LM Studio's OpenAI-compatible `/v1` API over your LAN or `localhost`. It is only the client: there is no backend, no account and no server to deploy.
- **100% Offline Inference:** Complete privacy. Zero cloud dependency.
- **Plug & Play LMStudio:** Instantly detects and fetches active models from your local server.
- **Persistent Sessions:** Chat histories and settings are saved locally using Zustand and `AsyncStorage`.
- **Developer-Friendly UI:** Real-time markdown rendering with syntax highlighting and 1-click code copying.
- **Minimalist Dark Mode:** High-contrast, distraction-free interface built on NativeWind principles.
## Why use it
---
- **Your messages stay on your network.** The app only talks to the LM Studio address you set in Settings. There is no cloud API and no fallback to one.
- **Real streaming on every target.** Requests go out with `stream: true` and each SSE `choices[0].delta.content` is appended as it arrives: through `XMLHttpRequest` progress events on native, through `fetch` with a `ReadableStream` on the web. You can stop generation mid-stream.
- **The chat basics are there.** Markdown with copyable code blocks, edit, delete and regenerate, a model selector fed by LM Studio, history saved on the device, and light, dark or system theme.
## Quick Start
**When not to use it:** if you want a cloud model, a server other than LM Studio (the app was built against LM Studio only), or a verified phone build. The Android and iOS builds and the native streaming path have not been verified yet.
To run this application, you will need both the frontend development environment and a local AI engine running.
## How it works
### 1. Model Server (LMStudio)
- Screens are file-based routes (`expo-router`): `app/chat`, `app/chat/history` and `app/settings`. `hooks/useLMStudio` exposes send, regenerate, stop and model fetching.
- `services/lmStudioApi` builds `http(s)://<host>:<port>/v1`, calls `GET /v1/models` for the selector and the connection test, and streams `POST /v1/chat/completions`.
- `store/useAppStore` (Zustand) persists theme, server settings, chat sessions and the current chat in `AsyncStorage`. A finished exchange becomes a session titled from the first user message.
1. Extract and open [LMStudio](https://lmstudio.ai/).
2. Download a compatible model (e.g., Llama 3, Qwen2.5, Mistral).
3. Open the **Local Server** tab (`<->`).
4. Enable **CORS** and set the port (default: `1234`).
5. Click **Start Server**.
## Quick start
> **Note:** If testing on a physical mobile device, locate your machine's local IPv4 address (e.g., `192.168.1.X`).
You need Node.js 20.19.4 or newer (required by React Native 0.81; not declared in `package.json`), npm, and LM Studio with at least one model downloaded. To run the app, use [Expo Go](https://expo.dev/go) on a device, an Android or iOS emulator, or a browser.
### 2. Mobile Client (App)
**1. Start LM Studio's server.** Open the **Local Server** tab (the `<->` icon), load a model, enable **CORS**, keep the port at `1234` (or note yours) and click **Start Server**.
Ensure you have [Node.js](https://nodejs.org/) (v18+) and the [Expo Go](https://expo.dev/go) app installed.
**2. Run the app.**
```bash
git clone https://github.com/evandrodevbr/LocalAiApp.git
cd LocalAiApp
git clone https://git.evandro.dev.br/evandro/LocalMind.git
cd LocalMind
npm install
npx expo start
npm run start # Expo CLI: press w (web), a (Android), i (iOS) or scan the QR code with Expo Go
```
Scan the QR code with Expo Go, or type `i`/`a` to launch a simulator.
`npm run android`, `npm run ios` and `npm run web` open a target directly. Do not run `npm run reset-project`: it is the Expo template script and moves or deletes `app/`, `components/`, `hooks/`, `constants/` and `scripts/`.
### 3. Connection
**3. Point the app at LM Studio.** Open **Settings** (gear icon in the chat header), fill **Host IP / Address** and **Port Number**, then tap **Test Connection**. On a physical device the host must be your computer's LAN address (`192.168.x.x`), not `localhost`. The badge turns to *Connected* and the loaded models appear in the header. Until then the chat shows *Server offline*: the app does not connect on launch.
1. Open the app and tap the **Settings** icon.
2. Enter your LMStudio server IP and Port (e.g., `192.168.1.X:1234`).
3. Tap **Test Connection**.
## Configuration
Once connected, your available models will load automatically.
The app reads no environment variables. The server address is a runtime setting stored on the device:
---
| Setting | Default | Notes |
|---|---|---|
| Host IP / Address | `localhost` | `http://` is prepended when the value does not start with `http` |
| Port Number | `1234` | LM Studio's default port |
| API Key (optional) | empty | Sent as `Authorization: Bearer <key>` only when filled in |
## Bug Reports & Contributing
Everything else lives in code: `temperature: 0.7` and `max_tokens: 2048` in `services/lmStudioApi.ts` (the connection test gives up after 5 s), the fixed system prompt in `constants/SystemPrompt.ts`, and the persisted state under the `localaiapp-storage` key in `store/useAppStore.ts`.
Found an issue or have a suggestion? We welcome community involvement!
## Project status
👉 **[Report an Issue or Request a Feature](https://github.com/evandrodevbr/LocalAiApp/issues)**
Alpha, with no tests and no CI. Known gaps:
When creating an issue, please include:
- The selected model and the model list are not persisted, so you pick a model again after a restart.
- `constants/Models.ts` (a Qwen model catalog) and the `react-native-sse` dependency are not used by any code.
- `app.json` and the storage key still carry the previous project name (`LocalAiApp`, `localaiapp-storage`).
- Both `package-lock.json` and `pnpm-lock.yaml` are committed.
- A clear description of the bug.
- Steps to reproduce.
- Relevant screenshots or environment details.
Before opening a PR, run `npm run lint` (Expo's ESLint config) and `npx tsc --noEmit`. Notes on the LM Studio API, in Portuguese, are in [`docs/`](docs/).
---
## License
## Acknowledgments
No published license: all rights reserved.
Thank you for exploring this project! Our mission is to share knowledge and foster an open-source collaborative ecosystem around offline AI solutions. All feedback, testing, and code contributions are highly appreciated.
<div align="center">
<br>
<br>
<sub>Made by <a href="https://evandro.dev.br">Evandro Fonseca Junior</a> · <a href="https://git.evandro.dev.br/evandro">git.evandro.dev.br</a></sub>
</div>
+14 -8
View File
@@ -1,19 +1,17 @@
import { Link, Stack } from 'expo-router';
import { StyleSheet } from 'react-native';
import { ThemedText } from '@/components/themed-text';
import { ThemedView } from '@/components/themed-view';
import { StyleSheet, Text, View } from 'react-native';
export default function NotFoundScreen() {
return (
<>
<Stack.Screen options={{ title: 'Oops!' }} />
<ThemedView style={styles.container}>
<ThemedText type="title">This screen doesn't exist.</ThemedText>
<View style={styles.container}>
<Text style={styles.title}>This screen does not exist.</Text>
<Link href="/" style={styles.link}>
<ThemedText type="link">Go to home screen!</ThemedText>
<Text style={styles.linkText}>Go to home screen</Text>
</Link>
</ThemedView>
</View>
</>
);
}
@@ -25,8 +23,16 @@ const styles = StyleSheet.create({
justifyContent: 'center',
padding: 20,
},
title: {
fontSize: 20,
fontWeight: 'bold',
},
link: {
marginTop: 15,
paddingVertical: 15,
},
linkText: {
fontSize: 14,
color: '#2e78b7',
},
});
+7 -24
View File
@@ -26,17 +26,6 @@ export default function ChatScreen() {
const editMessage = useAppStore(state => state.editMessage);
const { sendMessage, stopStreaming, regenerateResponse, isStreaming, isConnected } = useLMStudio();
const flatListRef = useRef<FlatList>(null);
const isUserScrollingRef = useRef(false);
const isAutoScrollEnabled = useRef(true);
const prevMessagesLength = useRef(messages.length);
const handleScroll = useCallback((event: any) => {
const { layoutMeasurement, contentOffset, contentSize } = event.nativeEvent;
// if user is within 100 pixels of the bottom, it's considered at the bottom
const paddingToBottom = 100;
const isCloseToBottom = layoutMeasurement.height + contentOffset.y >= contentSize.height - paddingToBottom;
isAutoScrollEnabled.current = isCloseToBottom;
}, []);
// Find the editing message content
const editingMessage = editingMessageId
@@ -80,7 +69,6 @@ export default function ChatScreen() {
}, 100);
} else {
sendMessage(text);
isAutoScrollEnabled.current = true; // explicitly re-enable on manual send
}
}, [editingMessageId, editingMessage, editMessage, deleteMessagesAfter, setEditingMessageId, sendMessage]);
@@ -93,19 +81,17 @@ export default function ChatScreen() {
: null;
useEffect(() => {
const isNewMessage = messages.length > prevMessagesLength.current;
prevMessagesLength.current = messages.length;
if (messages.length > 0 && flatListRef.current) {
setTimeout(() => {
if ((isAutoScrollEnabled.current && !isUserScrollingRef.current) || isNewMessage) {
flatListRef.current?.scrollToEnd({ animated: true });
}
flatListRef.current?.scrollToEnd({ animated: !isStreaming });
}, 100);
}
}, [messages.length, messages[messages.length - 1]?.content?.length]);
}, [messages.length, messages[messages.length - 1]?.content?.length, isStreaming]);
const showTypingIndicator = isStreaming &&
messages.length > 0 &&
messages[messages.length - 1]?.role === 'assistant' &&
messages[messages.length - 1]?.content === '';
return (
<KeyboardAvoidingView
@@ -151,10 +137,6 @@ export default function ChatScreen() {
ref={flatListRef}
data={messages}
keyExtractor={(item) => item.id}
onScroll={handleScroll}
onScrollBeginDrag={() => isUserScrollingRef.current = true}
onScrollEndDrag={() => isUserScrollingRef.current = false}
scrollEventThrottle={16}
renderItem={({ item, index }) => (
<Animated.View entering={FadeInDown.duration(300).delay(index > messages.length - 4 ? 100 : 0)}>
<MessageBubble
@@ -169,6 +151,7 @@ export default function ChatScreen() {
)}
contentContainerStyle={styles.listContent}
/>
{showTypingIndicator && <TypingIndicator />}
</>
)}
+1 -1
View File
@@ -215,7 +215,7 @@ export default function SettingsScreen() {
<View style={[styles.card, { backgroundColor: colors.cardBackground, borderColor: colors.border }]}>
<View style={styles.rowGroup}>
<Info size={18} color={colors.textMuted} style={styles.infoIcon} />
<Text style={[styles.infoText, { color: colors.text }]}>LocalAiApp - Cross Platform Version</Text>
<Text style={[styles.infoText, { color: colors.text }]}>LocalMind - Cross Platform Version</Text>
</View>
<View style={[styles.rowGroup, { marginTop: 8 }]}>
<MonitorCog size={18} color={colors.textMuted} style={styles.infoIcon} />
+46
View File
@@ -0,0 +1,46 @@
<svg xmlns="http://www.w3.org/2000/svg" width="1200" height="420" viewBox="0 0 1200 420" role="img" aria-labelledby="title desc">
<title id="title">LocalMind</title>
<desc id="desc">LocalMind: a phone chat screen marked Connected talks over the local network to LM Studio running on your computer at port 1234, path /v1.</desc>
<rect width="1200" height="420" rx="24" fill="#09090B"/>
<g id="title-block" font-family="system-ui, -apple-system, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif">
<text x="64" y="170" font-size="64" font-weight="700" fill="#FAFAFA">LocalMind</text>
<text x="64" y="226" font-size="28" fill="#FAFAFA">Chat with your LM Studio models</text>
<text x="64" y="264" font-size="28" fill="#FAFAFA">from a phone or browser on your LAN.</text>
<text x="64" y="322" font-size="20" fill="#A1A1AA" font-family="ui-monospace, 'SF Mono', Consolas, monospace">React Native 0.81 · Expo 54 · TypeScript</text>
</g>
<g id="phone" font-family="system-ui, -apple-system, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif">
<rect x="644" y="44" width="216" height="332" rx="34" fill="#18181B" stroke="#3F3F46" stroke-width="3"/>
<rect x="722" y="58" width="60" height="8" rx="4" fill="#3F3F46"/>
<rect x="662" y="82" width="134" height="32" rx="16" fill="#052E16"/>
<circle cx="680" cy="98" r="5" fill="#BBF7D0"/>
<text x="692" y="104" font-size="18" fill="#BBF7D0">Connected</text>
<rect x="696" y="132" width="146" height="40" rx="16" fill="#FAFAFA"/>
<text x="769" y="158" font-size="20" fill="#09090B" text-anchor="middle">Explain SSE</text>
<rect x="662" y="188" width="164" height="104" rx="16" fill="#27272A"/>
<rect x="680" y="208" width="128" height="10" rx="5" fill="#A1A1AA"/>
<rect x="680" y="232" width="112" height="10" rx="5" fill="#A1A1AA"/>
<rect x="680" y="256" width="72" height="10" rx="5" fill="#A1A1AA"/>
<rect x="760" y="252" width="10" height="18" rx="2" fill="#FAFAFA"/>
<rect x="662" y="314" width="180" height="40" rx="20" fill="#09090B" stroke="#3F3F46" stroke-width="2"/>
<text x="682" y="340" font-size="18" fill="#A1A1AA">Message</text>
<circle cx="822" cy="334" r="13" fill="#FAFAFA"/>
<path d="M822 340 v-12 M816 333 l6 -6 l6 6" fill="none" stroke="#09090B" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round"/>
</g>
<g id="lan-link" font-family="system-ui, -apple-system, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif">
<path d="M872 206 h80" stroke="#A1A1AA" stroke-width="2" stroke-dasharray="8 6"/>
<text x="912" y="190" font-size="18" fill="#A1A1AA" text-anchor="middle">LAN</text>
</g>
<g id="computer" font-family="system-ui, -apple-system, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif">
<rect x="960" y="140" width="176" height="124" rx="12" fill="#18181B" stroke="#3F3F46" stroke-width="3"/>
<text x="1048" y="194" font-size="20" font-weight="700" fill="#FAFAFA" text-anchor="middle">LM Studio</text>
<text x="1048" y="228" font-size="20" fill="#A1A1AA" text-anchor="middle" font-family="ui-monospace, 'SF Mono', Consolas, monospace">:1234/v1</text>
<rect x="1032" y="264" width="32" height="24" fill="#3F3F46"/>
<rect x="1004" y="288" width="88" height="10" rx="4" fill="#3F3F46"/>
<text x="1048" y="334" font-size="18" fill="#A1A1AA" text-anchor="middle">your machine</text>
</g>
</svg>

After

Width:  |  Height:  |  Size: 3.3 KiB

+16
View File
@@ -0,0 +1,16 @@
module.exports = function (api) {
api.cache(true);
return {
presets: [
[
'babel-preset-expo',
{
// Dependencias (zustand/middleware) usam `import.meta.env`, que nao existe
// no bundle web do Metro (script classico) e quebrava a hidratacao do app.
// O polyfill do Expo troca por `globalThis.__ExpoImportMetaRegistry`.
unstable_transformImportMeta: true,
},
],
],
};
};
+1 -27
View File
@@ -1,7 +1,6 @@
import { useThemeColor } from '@/hooks/useThemeColor';
import { useAppStore } from '@/store/useAppStore';
import * as Haptics from 'expo-haptics';
import { Globe, Send, Square, X } from 'lucide-react-native';
import { Send, Square, X } from 'lucide-react-native';
import React, { useEffect, useState } from 'react';
import {
Platform,
@@ -23,8 +22,6 @@ interface ChatInputProps {
export function ChatInput({ onSend, onStop, isLoading, editingText, onCancelEdit }: ChatInputProps) {
const [text, setText] = useState('');
const colors = useThemeColor();
const isWebSearchEnabled = useAppStore(state => state.isWebSearchEnabled);
const setIsWebSearchEnabled = useAppStore(state => state.setIsWebSearchEnabled);
// When editingText changes, fill the input
useEffect(() => {
@@ -80,20 +77,6 @@ export function ChatInput({ onSend, onStop, isLoading, editingText, onCancelEdit
{ backgroundColor: colors.inputBg, borderColor: colors.border, borderWidth: 1 },
isEditing && { borderColor: colors.tint, borderWidth: 1.5 }
]}>
<TouchableOpacity
style={styles.webSearchButton}
onPress={() => {
Haptics.impactAsync(Haptics.ImpactFeedbackStyle.Light);
setIsWebSearchEnabled(!isWebSearchEnabled);
}}
activeOpacity={0.7}
>
<Globe
size={20}
color={isWebSearchEnabled ? colors.tint : colors.textMuted}
/>
</TouchableOpacity>
<TextInput
style={[styles.input, { color: colors.text }]}
placeholder="Send a message..."
@@ -181,15 +164,6 @@ const styles = StyleSheet.create({
marginLeft: 8,
marginBottom: 2,
},
webSearchButton: {
width: 34,
height: 34,
borderRadius: 17,
alignItems: 'center',
justifyContent: 'center',
marginRight: 4,
marginBottom: 2,
},
stopButton: {
width: 34,
height: 34,
+9 -90
View File
@@ -3,10 +3,9 @@ import { ChatMessage } from '@/store/useAppStore';
import Markdown from '@ronradtke/react-native-markdown-display';
import * as Clipboard from 'expo-clipboard';
import { Bot, Check, Copy, User } from 'lucide-react-native';
import React, { useMemo, useState, useEffect } from 'react';
import React, { useMemo, useState } from 'react';
import { ScrollView, StyleSheet, Text, TouchableOpacity, useWindowDimensions, View } from 'react-native';
import { MessageActions } from './MessageActions';
import { ThinkingBlock } from './ThinkingBlock';
interface MessageBubbleProps {
message: ChatMessage;
@@ -41,37 +40,6 @@ export function MessageBubble({
const showingStreaming = isStreaming && isLastAssistant;
const shouldShowActions = !isUser || showActions;
const [displayedContent, setDisplayedContent] = useState(message.content);
useEffect(() => {
if (!showingStreaming) {
setDisplayedContent(message.content);
return;
}
let animationFrameId: number;
let currentText = displayedContent;
const targetText = message.content;
if (targetText.length < currentText.length) {
setDisplayedContent(targetText);
return;
}
const animate = () => {
if (currentText.length < targetText.length) {
const diff = targetText.length - currentText.length;
const charsToAdd = Math.max(1, Math.ceil(diff / 4));
currentText = targetText.slice(0, currentText.length + charsToAdd);
setDisplayedContent(currentText);
animationFrameId = requestAnimationFrame(animate);
}
};
animationFrameId = requestAnimationFrame(animate);
return () => cancelAnimationFrame(animationFrameId);
}, [message.content, showingStreaming]);
// UseMemo for dynamic colors
const markdownStyles = useMemo(() => ({
body: {
@@ -204,56 +172,22 @@ export function MessageBubble({
}), [copiedText]);
const renderContent = () => {
let activeContent = showingStreaming ? displayedContent : message.content;
if (showingStreaming && displayedContent === '' && message.content !== '') {
// Prevent empty bubble flicker on the first chunk
activeContent = message.content.slice(0, 1);
}
if (isUser) {
return (
<Text style={[styles.text, { color: colors.messageUserText }]}>
{activeContent}
{message.content}
</Text>
);
}
if (!activeContent && showingStreaming) {
return (
<View style={styles.dotsRow}>
<View style={[styles.dot, styles.dot1, { backgroundColor: colors.textMuted }]} />
<View style={[styles.dot, styles.dot2, { backgroundColor: colors.textMuted }]} />
<View style={[styles.dot, styles.dot3, { backgroundColor: colors.textMuted }]} />
</View>
);
}
const thinkMatch = activeContent.match(/<think>([\s\S]*?)(?:<\/think>|$)/);
let cleanContent = activeContent;
let thinkingContent = null;
let isThinkingComplete = false;
if (thinkMatch) {
thinkingContent = thinkMatch[1].trim();
isThinkingComplete = activeContent.includes('</think>');
cleanContent = activeContent.replace(thinkMatch[0], '').trim();
if (!message.content && showingStreaming) {
return null; // TypingIndicator handled externally
}
return (
<>
{thinkingContent !== null && (
<ThinkingBlock
content={thinkingContent}
isComplete={isThinkingComplete}
isStreaming={showingStreaming}
/>
)}
{cleanContent.length > 0 && (
<Markdown style={markdownStyles} rules={markdownRules}>
{cleanContent}
</Markdown>
)}
</>
<Markdown style={markdownStyles} rules={markdownRules}>
{message.content}
</Markdown>
);
};
@@ -300,10 +234,10 @@ export function MessageBubble({
</View>
{/* Action buttons */}
{(shouldShowActions || !isUser) && (showingStreaming ? displayedContent : message.content).length > 0 && (
{(shouldShowActions || !isUser) && message.content.length > 0 && (
<MessageActions
role={message.role}
content={showingStreaming ? displayedContent : message.content}
content={message.content}
onEdit={isUser ? onEdit : undefined}
onRegenerate={!isUser && isLastAssistant ? onRegenerate : undefined}
onDelete={onDelete}
@@ -423,19 +357,4 @@ const styles = StyleSheet.create({
fontSize: 13,
fontFamily: 'monospace',
},
dotsRow: {
flexDirection: 'row',
gap: 6,
alignItems: 'center',
minHeight: 22,
paddingHorizontal: 4,
},
dot: {
width: 8,
height: 8,
borderRadius: 4,
},
dot1: { opacity: 1.0 },
dot2: { opacity: 0.6 },
dot3: { opacity: 0.3 },
});
-127
View File
@@ -1,127 +0,0 @@
import { useThemeColor } from '@/hooks/useThemeColor';
import { BrainCircuit, ChevronDown, ChevronRight } from 'lucide-react-native';
import React, { useEffect, useState } from 'react';
import { StyleSheet, Text, TouchableOpacity, View } from 'react-native';
import Animated, {
useAnimatedStyle,
useSharedValue,
withRepeat,
withSequence,
withTiming
} from 'react-native-reanimated';
interface ThinkingBlockProps {
content: string;
isComplete: boolean;
isStreaming?: boolean;
}
export function ThinkingBlock({ content, isComplete, isStreaming }: ThinkingBlockProps) {
const colors = useThemeColor();
// Auto-expand while streaming/thinking, auto-collapse when complete
const [isExpanded, setIsExpanded] = useState(!isComplete);
// Auto collapse when it completes streaming
useEffect(() => {
if (isComplete && !isStreaming) {
setIsExpanded(false);
} else if (!isComplete && isStreaming) {
setIsExpanded(true);
}
}, [isComplete, isStreaming]);
// Pulse animation value
const pulseOpacity = useSharedValue(1);
useEffect(() => {
if (!isComplete && isStreaming) {
pulseOpacity.value = withRepeat(
withSequence(
withTiming(0.4, { duration: 800 }),
withTiming(1, { duration: 800 })
),
-1, // infinite
true // reverse
);
} else {
pulseOpacity.value = withTiming(1, { duration: 300 });
}
}, [isComplete, isStreaming, pulseOpacity]);
const animatedHeaderStyle = useAnimatedStyle(() => {
return {
opacity: pulseOpacity.value,
};
});
if (!content && isComplete) {
return null;
}
return (
<View style={[styles.container, { borderColor: colors.border }]}>
<TouchableOpacity
style={styles.header}
activeOpacity={0.7}
onPress={() => setIsExpanded(!isExpanded)}
>
<Animated.View style={[styles.titleRow, animatedHeaderStyle]}>
<BrainCircuit size={16} color={colors.textMuted} />
<Text style={[styles.titleText, { color: colors.textMuted }]}>
{!isComplete && isStreaming ? 'Pensando...' : 'Processo de Raciocínio'}
</Text>
</Animated.View>
{isExpanded ? (
<ChevronDown size={18} color={colors.textMuted} />
) : (
<ChevronRight size={18} color={colors.textMuted} />
)}
</TouchableOpacity>
{isExpanded && content.length > 0 && (
<View style={[styles.contentBlock, { borderTopColor: colors.border }]}>
<Text style={[styles.contentText, { color: colors.textMuted }]}>
{content}
</Text>
</View>
)}
</View>
);
}
const styles = StyleSheet.create({
container: {
borderWidth: 1,
borderRadius: 8,
marginVertical: 8,
overflow: 'hidden',
width: '100%',
},
header: {
flexDirection: 'row',
alignItems: 'center',
justifyContent: 'space-between',
paddingHorizontal: 12,
paddingVertical: 10,
},
titleRow: {
flexDirection: 'row',
alignItems: 'center',
gap: 8,
},
titleText: {
fontSize: 13,
fontWeight: '600',
},
contentBlock: {
paddingHorizontal: 12,
paddingVertical: 10,
borderTopWidth: 1,
},
contentText: {
fontSize: 14,
lineHeight: 20,
fontFamily: 'monospace',
},
});
+14 -116
View File
@@ -3,7 +3,6 @@ import { SYSTEM_PROMPT } from '../constants/SystemPrompt';
import * as api from '../services/lmStudioApi';
import { ChatMessage as ApiMessage } from '../services/types';
import { useAppStore } from '../store/useAppStore';
import { WebSearchLogger } from '../services/webSearchLogger';
export function useLMStudio() {
const serverConfig = useAppStore((s) => s.serverConfig);
@@ -48,13 +47,18 @@ export function useLMStudio() {
const sendMessage = useCallback(async (text: string) => {
if (isStreaming) return;
const pipelineStart = Date.now();
const store = useAppStore.getState();
const isWebSearchEnabled = store.isWebSearchEnabled;
addMessage({ role: 'user', content: text });
const modelId = store.activeModelId;
const currentMessages = useAppStore.getState().messages;
const apiMessages: ApiMessage[] = [
{ role: 'system', content: SYSTEM_PROMPT },
...currentMessages.map((m) => ({
role: m.role,
content: m.content,
}))
];
const modelId = useAppStore.getState().activeModelId;
if (!modelId) {
addMessage({
role: 'assistant',
@@ -67,128 +71,19 @@ export function useLMStudio() {
setIsStreaming(true);
try {
let searchContext = '';
let searchSources: Array<{ title: string; url: string }> = [];
if (isWebSearchEnabled) {
// ── Step 1: Build search service URL ─────────────────────────────
// ServerConfig has { host, port } — construct the search service URL
// from the same host, but on port 3030.
const rawHost = store.serverConfig.host.replace(/\/+$/, '');
const hostWithProtocol = rawHost.startsWith('http') ? rawHost : `http://${rawHost}`;
const searchEndpoint = `${hostWithProtocol}:3030/search-and-extract`;
WebSearchLogger.pipelineStart(text, searchEndpoint);
WebSearchLogger.endpointResolved(
store.serverConfig.host,
store.serverConfig.port,
searchEndpoint,
);
appendToMessage(assistantId, '<think>Searching the web...</think>\n');
try {
const maxContextLength = 10000;
// 60s timeout: SearXNG + 3 páginas em paralelo (max ~8s cada) + parsing
// Sequential era ~24s worst case; paralelo cabe em ~10-15s. 60s é margem segura.
const SEARCH_TIMEOUT_MS = 60000;
const searchController = new AbortController();
const searchTimeout = setTimeout(() => {
searchController.abort();
}, SEARCH_TIMEOUT_MS);
WebSearchLogger.requestSent(text, maxContextLength, SEARCH_TIMEOUT_MS);
const searchRes = await fetch(searchEndpoint, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
query: text,
maxContextLength,
}),
signal: searchController.signal,
});
clearTimeout(searchTimeout);
if (searchRes.ok) {
const searchData = await searchRes.json();
searchContext = searchData.context ?? '';
searchSources = searchData.sources ?? [];
WebSearchLogger.contextReceived(
searchContext,
searchSources,
searchData.meta,
);
} else {
WebSearchLogger.requestFailed(searchRes.status, searchRes.statusText);
}
} catch (error) {
const isTimeout = error instanceof Error && error.name === 'AbortError';
WebSearchLogger.serviceUnreachable(error, isTimeout);
}
// Clear the temporary "searching..." thought bubble
store.updateMessageContent(assistantId, '');
}
// ── Step 2: Build API messages ──────────────────────────────────────
const currentMessages = useAppStore.getState().messages;
// Exclude the empty assistant placeholder from the history
const historyToSend = currentMessages.slice(0, currentMessages.length - 1);
// Warn explicitly when web search was enabled but returned no context
if (isWebSearchEnabled && !searchContext) {
console.warn(
'[WebSearch/Fallback] ⚠️ Web search was enabled but returned NO context. ' +
'Proceeding with LLM response without web data. ' +
'Check if local-search-service is running at the correct address.',
);
}
const systemContent = SYSTEM_PROMPT + (searchContext ? `\n\n${searchContext}` : '');
const apiMessages: ApiMessage[] = [
{ role: 'system', content: systemContent },
...historyToSend.map((m) => ({
role: m.role,
content: m.content,
})),
];
// ── Step 3: Context Window Analysis ────────────────────────────────
if (isWebSearchEnabled) {
const historyChars = historyToSend.reduce((sum, m) => sum + m.content.length, 0);
WebSearchLogger.contextAnalysis(
systemContent.length,
historyChars,
searchContext.length,
);
}
// ── Step 4: Stream LLM response ────────────────────────────────────
WebSearchLogger.llmRequestStart(modelId, apiMessages.length);
let chunkCount = 0;
abortRef.current = api.chatCompletionStream(
serverConfig,
modelId,
apiMessages,
(chunk) => {
chunkCount++;
appendToMessage(assistantId, chunk);
},
() => {
WebSearchLogger.llmStreamComplete(assistantId, chunkCount);
WebSearchLogger.pipelineComplete(text, Date.now() - pipelineStart, isWebSearchEnabled);
setIsStreaming(false);
abortRef.current = null;
useAppStore.getState().saveCurrentChat();
},
(error) => {
WebSearchLogger.llmError(error);
const errorMsg = `\n\n⚠️ Error: ${error.message}`;
appendToMessage(assistantId, errorMsg);
setIsStreaming(false);
@@ -208,6 +103,7 @@ export function useLMStudio() {
if (isStreaming) return;
const msgs = useAppStore.getState().messages;
// Find the last user message
let lastUserIdx = -1;
for (let i = msgs.length - 1; i >= 0; i--) {
if (msgs[i].role === 'user') {
@@ -218,16 +114,18 @@ export function useLMStudio() {
if (lastUserIdx === -1) return;
const lastUserMsg = msgs[lastUserIdx];
// Remove messages after the last user message
const store = useAppStore.getState();
store.deleteMessagesAfter(lastUserMsg.id);
// Build API messages from remaining
const remaining = useAppStore.getState().messages;
const apiMessages: ApiMessage[] = [
{ role: 'system', content: SYSTEM_PROMPT },
...remaining.map((m) => ({
role: m.role,
content: m.content,
})),
}))
];
const modelId = useAppStore.getState().activeModelId;
-83
View File
@@ -1,83 +0,0 @@
# Local AI Database Strategy (LanceDB + SQLite)
## Overview
A 100% offline-first local database architecture for storing AI memories in a React Native (Expo) application. The solution splits the data into two highly optimized engines:
1. **`expo-sqlite`**: Manages structured text, sessions, dates, and relationships.
2. **`LanceDB`**: Manages the high-dimensional vector embeddings, pointed by SQLite relationship IDs.
> [!WARNING] **Architectural Risk (LanceDB on Mobile)**
> LanceDB relies on robust Apache Arrow and native dependencies. Running it completely on-device in a React Native/Expo environment needs careful validation (e.g., verifying polyfills or WebAssembly compatibility), since most vector DBs are backend/Node.js native.
## Project Type
**MOBILE** (React Native / Expo)
## Success Criteria
- [ ] Both databases can correctly initialize on iOS and Android simulators/devices.
- [ ] A memory string can be successfully indexed (stored in SQLite + embedding stored in LanceDB).
- [ ] A semantic search query returns the correct related memories locally.
- [ ] Application does not crash due to native library incompatibilities.
## Tech Stack
- **`expo-sqlite`**: The official, lightweight SQLite solution for the Expo SDK, providing fully synchronous mapping properties.
- **`@lancedb/lancedb`** (or equivalent mobile-compatible API): Vector store for fast embedding similarity searches locally.
- **`zustand`** (Already installed): State management for caching the queries/memories if needed in the UI.
## File Structure
```
app/
...
services/
db/
sqliteClient.ts # SQLite connection and raw queries
lanceDbClient.ts # LanceDB vector search and connection
memoryService.ts # Unifies both DBs to create the "AI Memories" logic
hooks/
useAiMemories.ts # React Hook to expose database queries
types/
database.d.ts # TS interfaces for Tables and Documents
```
## Task Breakdown
### Task 1: Environment Research & Setup
- **Agent**: `mobile-developer`
- **Skill**: `mobile-design`
- **INPUT**: Current `package.json`
- **OUTPUT**: Installation of `expo-sqlite` and LanceDB dependencies. Verified bare React Native compatibility for LanceDB.
- **VERIFY**: Running a successful `npx expo start` and `npx expo run:android` without build failures.
### Task 2: Database Schema & Layer
- **Agent**: `database-architect` (simulated by `mobile-developer`)
- **Skill**: `database-design`
- **INPUT**: Dependencies installed.
- **OUTPUT**:
- `sqliteClient.ts`: `Sessions`, `Messages`, `Metadata` tables.
- `lanceDbClient.ts`: LanceDB table initialization with vector dimensions (e.g. 384 or 1536).
- **VERIFY**: Launching the app correctly fires the initialization queries (logs "DB Connected").
### Task 3: Unifying Memory Service
- **Agent**: `mobile-developer`
- **Skill**: `clean-code`
- **INPUT**: Initialized Database clients.
- **OUTPUT**: `memoryService.ts` containing functions like `saveMemory(text, embedding)` and `searchSimilar(embedding, limit)`.
- **VERIFY**: Hardcoding a mock vector search and getting the SQLite structured data back.
### Task 4: Hook UI Integration
- **Agent**: `mobile-developer`
- **Skill**: `mobile-design`
- **INPUT**: `memoryService.ts` logic.
- **OUTPUT**: `useAiMemories.ts` hook ready to be plugged into chat components.
- **VERIFY**: Test components renders data from the DB layer.
---
## ✅ PHASE X: Verification Checklist
*To be completed after Phase 4.*
- **Linting:** ⬜ Pass (`npm run lint` & `tsc`)
- **Build (Android):** ⬜ Pass (`npx expo run:android`)
- **Build (iOS):** ⬜ Pass (`npx expo run:ios`)
- **Feature Check:** ⬜ Vector search and relation query works offline.
- **Color/UI Rules:** ⬜ UI follows standard themes (Not applicable for DB layer but required by protocol).
- **Date:** TBD
-32
View File
@@ -1,32 +0,0 @@
# Local Search Service
This is the background Web Search extraction service for the **LocalAiApp**. It acts as a lightweight proxy and DOM parser that connects your mobile app (React Native) to SearXNG, ensuring the Local AI context is pristine Markdown.
## How it Works
1. Receives a search query from the LocalAiApp via `POST /search-and-extract`.
2. Asks your local `SearXNG` instance for the top results.
3. Fetches the HTML of the resulting pages.
4. Uses `linkedom` and `Readability` to strip out all menus, footers, scripts, and clutter.
5. Converts the clean article payload into `Markdown` using `node-html-markdown`.
6. Returns the pure Markdown text to the LocalAiApp to inject into the AI's prompt.
## Setup
### Requirements
- Node.js v18+
- A running instance of [SearXNG](https://github.com/searxng/searxng) (running locally at `http://127.0.0.1:8888` by default).
### Running
1. Install dependencies:
```bash
npm install
```
2. Start the service (runs on port `3030`):
```bash
npm start
```
## Integrations
Inside the main `LocalAiApp`, the chat input now has a Web Search toggle (the Globe icon). When activated, every prompt sent will first trigger the search service. The service fetches context online and feeds the result as an invisible pre-system-prompt to LM Studio!
File diff suppressed because it is too large. Load diff
-24
View File
@@ -1,24 +0,0 @@
{
"name": "local-search-service",
"version": "1.0.0",
"description": "Helper service for LocalAiApp to fetch and extract web context",
"scripts": {
"start": "ts-node server.ts",
"dev": "ts-node server.ts"
},
"dependencies": {
"@mozilla/readability": "^0.5.0",
"cors": "^2.8.5",
"express": "^4.21.0",
"linkedom": "^0.18.5",
"node-html-markdown": "^1.3.0"
},
"devDependencies": {
"@types/cors": "^2.8.17",
"@types/express": "^5.0.0",
"@types/mozilla-readability": "^0.2.1",
"@types/node": "^22.0.0",
"ts-node": "^10.9.2",
"typescript": "^5.6.2"
}
}
-435
View File
@@ -1,435 +0,0 @@
import express, { Request, Response } from 'express';
import cors from 'cors';
import { parseHTML } from 'linkedom';
import { Readability } from '@mozilla/readability';
import { NodeHtmlMarkdown } from 'node-html-markdown';
const app = express();
const PORT = process.env.PORT || 3030;
app.use(cors());
app.use(express.json());
// ─── Logger ────────────────────────────────────────────────────────────────────
type LogLevel = 'INFO' | 'WARN' | 'ERROR' | 'DEBUG';
function log(level: LogLevel, phase: string, message: string, meta?: Record<string, any>) {
const ts = new Date().toISOString();
const metaStr = meta ? ` | ${JSON.stringify(meta)}` : '';
const icons: Record<LogLevel, string> = { INFO: 'ℹ️', WARN: '⚠️', ERROR: '❌', DEBUG: '🔍' };
console.log(`[${ts}] ${icons[level]} [${phase}] ${message}${metaStr}`);
}
function logSection(title: string) {
const border = '═'.repeat(60);
console.log(`\n${border}`);
console.log(` ${title}`);
console.log(`${border}`);
}
// ─── Helpers ───────────────────────────────────────────────────────────────────
/** Rough token count estimate (~4 chars per token) */
function estimateTokens(text: string): number {
return Math.ceil(text.length / 4);
}
/**
* Context Analysis — scores each extracted source by:
* - content length (longer = more info)
* - keyword overlap with the query
* - penalizes sources with very short content (likely failed extractions)
*/
function scoreSource(
source: { title: string; url: string; content: string },
query: string,
): number {
const queryWords = query
.toLowerCase()
.split(/\s+/)
.filter((w) => w.length > 3);
const contentLower = source.content.toLowerCase();
const titleLower = source.title.toLowerCase();
// Keyword relevance: count query words present in content/title
const keywordMatches = queryWords.filter(
(w) => contentLower.includes(w) || titleLower.includes(w),
).length;
const keywordScore = queryWords.length > 0 ? keywordMatches / queryWords.length : 0;
// Length score (0–1): prefer 500–3000 char range
const len = source.content.length;
const lengthScore = len < 100 ? 0.1 : len > 500 ? Math.min(1, len / 3000) : 0.5;
// Penalize if content is just a fallback snippet (very short)
const snippetPenalty = len < 200 ? 0.4 : 1.0;
const total = (keywordScore * 0.6 + lengthScore * 0.4) * snippetPenalty;
log('DEBUG', 'ContextAnalysis', `Scored source: "${source.title}"`, {
url: source.url,
contentLength: len,
keywordMatches,
keywordScore: keywordScore.toFixed(2),
lengthScore: lengthScore.toFixed(2),
snippetPenalty,
totalScore: total.toFixed(3),
});
return total;
}
/**
* Intelligent Context Pruning:
* 1. Sort sources by relevance score (descending).
* 2. Include full sources one by one until the budget is reached.
* 3. If a single source already exceeds budget, truncate it at a sentence boundary.
* This avoids cutting in the middle of a word / sentence.
*/
function buildAndPruneContext(
sources: Array<{ title: string; url: string; content: string }>,
query: string,
maxChars: number,
): { markdown: string; pruned: boolean; keptSources: number; droppedSources: number } {
logSection('🧠 Context Analysis & Pruning');
// Score and rank sources
const scored = sources
.map((src) => ({ ...src, score: scoreSource(src, query) }))
.sort((a, b) => b.score - a.score);
log('INFO', 'ContextPruning', `Ranked ${scored.length} sources by relevance`, {
ranking: scored.map((s, i) => ({
rank: i + 1,
title: s.title.substring(0, 50),
score: s.score.toFixed(3),
chars: s.content.length,
estimatedTokens: estimateTokens(s.content),
})),
});
const header = `# Web Search Context for: "${query}"\n\n`;
let markdown = header;
let budgetUsed = header.length;
let keptSources = 0;
let droppedSources = 0;
let pruned = false;
for (const src of scored) {
const entry =
`## Source ${keptSources + 1}: [${src.title}](${src.url})\n\n` +
`${src.content}\n\n` +
`---\n\n`;
if (budgetUsed + entry.length <= maxChars) {
markdown += entry;
budgetUsed += entry.length;
keptSources++;
log('INFO', 'ContextPruning', `✅ Included source (budget OK)`, {
rank: keptSources,
title: src.title.substring(0, 50),
entryChars: entry.length,
budgetUsed,
budgetRemaining: maxChars - budgetUsed,
});
} else {
const remaining = maxChars - budgetUsed;
if (keptSources === 0 && remaining > 200) {
// First source is already too large — truncate at last sentence boundary
const entryHeader =
`## Source 1: [${src.title}](${src.url})\n\n`;
const contentBudget = remaining - entryHeader.length - 60; // room for suffix
let truncatedContent = src.content.substring(0, contentBudget);
// Find last sentence boundary (. ! ?)
const lastSentence = Math.max(
truncatedContent.lastIndexOf('. '),
truncatedContent.lastIndexOf('! '),
truncatedContent.lastIndexOf('? '),
truncatedContent.lastIndexOf('\n'),
);
if (lastSentence > contentBudget * 0.5) {
truncatedContent = truncatedContent.substring(0, lastSentence + 1);
}
markdown +=
entryHeader +
truncatedContent +
'\n\n...[CONTENT TRUNCATED — token budget exceeded]...\n\n---\n\n';
keptSources++;
pruned = true;
droppedSources = scored.length - keptSources;
log('WARN', 'ContextPruning', `✂️ Truncated oversized first source at sentence boundary`, {
original: src.content.length,
truncated: truncatedContent.length,
budgetRemaining: remaining,
});
break;
}
// Drop this source entirely — not enough budget
droppedSources++;
pruned = true;
log('WARN', 'ContextPruning', `🗑️ Dropped source (insufficient budget)`, {
title: src.title.substring(0, 50),
entryChars: entry.length,
budgetRemaining: remaining,
score: src.score.toFixed(3),
});
}
}
return { markdown, pruned, keptSources, droppedSources };
}
// ─── Main Search Endpoint ──────────────────────────────────────────────────────
app.post('/search-and-extract', async (req: Request, res: Response): Promise<any> => {
const requestId = `req_${Date.now().toString(36)}`;
const startTime = Date.now();
try {
const {
query,
searxngUrl = 'http://127.0.0.1:8888',
topK = 3,
maxContextLength = 8000,
} = req.body;
logSection(`🌍 Web Search Pipeline — [${requestId}]`);
log('INFO', 'Request', `Received search request`, {
requestId,
query,
searxngUrl,
topK,
maxContextLength,
estimatedTokenBudget: estimateTokens(String(maxContextLength)),
});
if (!query) {
log('WARN', 'Validation', `Rejected: query is missing`, { requestId });
return res.status(400).json({ error: 'Query is missing' });
}
// ── Step 1: SearXNG Search ─────────────────────────────────────────────
log('INFO', 'SearXNG', `Querying SearXNG...`, { url: searxngUrl, query });
const searxStart = Date.now();
const searchRes = await fetch(
`${searxngUrl}/search?q=${encodeURIComponent(query)}&format=json`,
);
const searxLatency = Date.now() - searxStart;
if (!searchRes.ok) {
log('ERROR', 'SearXNG', `Search failed`, {
status: searchRes.status,
latencyMs: searxLatency,
});
throw new Error(
`SearXNG search failed. Status: ${searchRes.status}. ` +
`Make sure SearXNG is running and 'json' format is enabled in settings.yml.`,
);
}
const searchData = await searchRes.json();
const results = searchData.results || [];
const topResults = results.slice(0, topK).filter((r: any) => r.url);
log('INFO', 'SearXNG', `Search complete`, {
requestId,
latencyMs: searxLatency,
totalResultsFound: results.length,
processingTopK: topResults.length,
urls: topResults.map((r: any) => r.url),
});
// ── Step 2: Page Extraction (parallel) ────────────────────────────────
// All pages are fetched concurrently with Promise.allSettled so a single
// slow or failing page never blocks the others.
logSection('📄 Page Extraction (parallel)');
log('INFO', 'Extractor', `Starting parallel fetch for ${topResults.length} pages`, {
urls: topResults.map((r: any) => r.url),
});
async function extractPage(result: any): Promise<{
ctx: { title: string; url: string; content: string } | null;
stat: Record<string, any>;
}> {
const pageStart = Date.now();
log('INFO', 'Extractor', `Fetching page...`, { url: result.url });
try {
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), 8000);
const pageRes = await fetch(result.url, {
signal: controller.signal,
headers: { 'User-Agent': 'Mozilla/5.0 (compatible; LocalSearchBot/1.0)' },
});
clearTimeout(timeoutId);
const pageLatency = Date.now() - pageStart;
if (!pageRes.ok) {
log('WARN', 'Extractor', `Skipping page — bad HTTP status`, {
url: result.url,
status: pageRes.status,
latencyMs: pageLatency,
});
return { ctx: null, stat: { url: result.url, status: 'skipped', reason: `HTTP ${pageRes.status}`, latencyMs: pageLatency } };
}
const html = await pageRes.text();
const htmlSize = html.length;
// ── Readability Extraction ──────────────────────────────────────
const { document } = parseHTML(html);
const reader = new Readability(document as any);
const article = reader.parse();
if (!article || !article.content) {
log('WARN', 'Extractor', `Readability failed — using snippet fallback`, {
url: result.url,
htmlSizeChars: htmlSize,
latencyMs: pageLatency,
});
const fallbackContent = result.content || 'Content could not be parsed.';
return {
ctx: { title: result.title || 'Unknown', url: result.url, content: fallbackContent },
stat: { url: result.url, status: 'fallback_snippet', contentChars: fallbackContent.length, latencyMs: pageLatency },
};
}
// ── HTML → Markdown ─────────────────────────────────────────────
const markdown = NodeHtmlMarkdown.translate(article.content);
const pageLatencyFinal = Date.now() - pageStart;
log('INFO', 'Extractor', `✅ Extracted successfully`, {
url: result.url,
title: (article.title || result.title).substring(0, 60),
htmlSizeChars: htmlSize,
markdownChars: markdown.length,
estimatedTokens: estimateTokens(markdown),
latencyMs: pageLatencyFinal,
});
return {
ctx: { title: article.title || result.title, url: result.url, content: markdown },
stat: { url: result.url, status: 'success', htmlChars: htmlSize, markdownChars: markdown.length, estimatedTokens: estimateTokens(markdown), latencyMs: pageLatencyFinal },
};
} catch (err: any) {
const pageLatency = Date.now() - pageStart;
const isTimeout = err.name === 'AbortError';
log('ERROR', 'Extractor', `Failed to process page`, {
url: result.url,
error: err.message,
isTimeout,
latencyMs: pageLatency,
});
return { ctx: null, stat: { url: result.url, status: isTimeout ? 'timeout' : 'error', error: err.message, latencyMs: pageLatency } };
}
}
// Run all extractions in parallel
const parallelStart = Date.now();
const settled = await Promise.allSettled(topResults.map((r: any) => extractPage(r)));
const parallelLatency = Date.now() - parallelStart;
const extractedContexts: Array<{ title: string; url: string; content: string }> = [];
const extractionStats: Array<Record<string, any>> = [];
for (const result of settled) {
if (result.status === 'fulfilled') {
if (result.value.ctx) extractedContexts.push(result.value.ctx);
extractionStats.push(result.value.stat);
} else {
// Promise itself rejected (shouldn't happen since extractPage catches internally)
extractionStats.push({ status: 'unexpected_rejection', error: result.reason?.message });
}
}
log('INFO', 'Extractor', `Extraction phase complete (parallel)`, {
requestId,
parallelLatencyMs: parallelLatency,
attempted: topResults.length,
succeeded: extractedContexts.length,
failed: topResults.length - extractedContexts.length,
extractionStats,
});
// ── Step 3: Context Analysis + Intelligent Pruning ────────────────────
const totalRawChars = extractedContexts.reduce((sum, c) => sum + c.content.length, 0);
const totalRawTokens = estimateTokens(totalRawChars.toString()) * 4; // rough
log('INFO', 'ContextAnalysis', `Pre-pruning context stats`, {
requestId,
sources: extractedContexts.length,
totalRawChars,
estimatedRawTokens: estimateTokens(extractedContexts.reduce((s, c) => s + c.content, '')),
maxContextLength,
maxContextTokensEstimate: estimateTokens('x'.repeat(maxContextLength)),
pruningNeeded: totalRawChars > maxContextLength,
});
const { markdown: combinedMarkdown, pruned, keptSources, droppedSources } =
buildAndPruneContext(extractedContexts, query, maxContextLength);
// ── Step 4: Final Stats & Response ────────────────────────────────────
const totalLatency = Date.now() - startTime;
const finalChars = combinedMarkdown.length;
const finalTokens = estimateTokens(combinedMarkdown);
logSection('📤 Pipeline Complete');
log('INFO', 'Response', `Sending context to LocalAiApp`, {
requestId,
query,
totalPipelineLatencyMs: totalLatency,
searxLatencyMs: searxLatency,
finalContextChars: finalChars,
finalContextEstimatedTokens: finalTokens,
sourcesKept: keptSources,
sourcesDropped: droppedSources,
pruned,
sourceTitles: extractedContexts.map((c) => c.title.substring(0, 50)),
});
console.log(`${'═'.repeat(60)}\n`);
return res.json({
success: true,
context: combinedMarkdown,
sources: extractedContexts.map((c) => ({ title: c.title, url: c.url })),
meta: {
requestId,
totalLatencyMs: totalLatency,
finalContextChars: finalChars,
finalContextEstimatedTokens: finalTokens,
sourcesKept: keptSources,
sourcesDropped: droppedSources,
pruned,
},
});
} catch (error: any) {
const totalLatency = Date.now() - startTime;
log('ERROR', 'Fatal', `Pipeline failed`, {
requestId,
error: error.message,
stack: error.stack?.split('\n').slice(0, 5),
totalLatencyMs: totalLatency,
});
return res.status(500).json({ error: error.message, requestId });
}
});
app.listen(Number(PORT), '0.0.0.0', () => {
log('INFO', 'Server', `Local Search Service started`, {
port: PORT,
address: `http://0.0.0.0:${PORT}`,
endpoints: ['/search-and-extract'],
});
});
+13203
View File
File diff suppressed because it is too large. Load diff
+1 -290
View File
@@ -8,18 +8,9 @@ importers:
.:
dependencies:
'@babel/plugin-transform-runtime':
specifier: ^7.29.0
version: 7.29.0(@babel/[email protected])
'@babel/runtime':
specifier: ^7.28.6
version: 7.28.6
'@expo/vector-icons':
specifier: ^15.0.3
version: 15.1.1([email protected]([email protected])([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected]))([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected])
'@react-native-async-storage/async-storage':
specifier: 2.2.0
version: 2.2.0([email protected](@babel/[email protected])(@types/[email protected])([email protected]))
'@react-navigation/bottom-tabs':
specifier: ^7.4.0
version: 7.15.5(@react-navigation/[email protected]([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected]))([email protected]([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected]))([email protected]([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected]))([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected])
@@ -29,15 +20,9 @@ importers:
'@react-navigation/native':
specifier: ^7.1.8
version: 7.1.33([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected])
'@ronradtke/react-native-markdown-display':
specifier: ^8.1.0
version: 8.1.0([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected])
expo:
specifier: ~54.0.31
version: 54.0.33(@babel/[email protected])(@expo/[email protected])([email protected])([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected])
expo-clipboard:
specifier: ^55.0.8
version: 55.0.9([email protected])([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected])
expo-constants:
specifier: ~18.0.13
version: 18.0.13([email protected])([email protected](@babel/[email protected])(@types/[email protected])([email protected]))
@@ -71,14 +56,8 @@ importers:
expo-web-browser:
specifier: ~15.0.10
version: 15.0.10([email protected])([email protected](@babel/[email protected])(@types/[email protected])([email protected]))
immer:
specifier: ^11.1.4
version: 11.1.4
lucide-react-native:
specifier: ^0.576.0
version: 0.576.0([email protected]([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected]))([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected])
react:
specifier: ^19.1.0
specifier: 19.1.0
version: 19.1.0
react-dom:
specifier: 19.1.0
@@ -98,24 +77,12 @@ importers:
react-native-screens:
specifier: ~4.16.0
version: 4.16.0([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected])
react-native-sse:
specifier: ^1.2.1
version: 1.2.1
react-native-svg:
specifier: 15.12.1
version: 15.12.1([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected])
react-native-web:
specifier: ~0.21.0
version: 0.21.2([email protected]([email protected]))([email protected])
react-native-worklets:
specifier: 0.5.1
version: 0.5.1(@babel/[email protected])([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected])
use-sync-external-store:
specifier: ^1.6.0
version: 1.6.0([email protected])
zustand:
specifier: ^5.0.11
version: 5.0.12(@types/[email protected])([email protected])([email protected])([email protected]([email protected]))
devDependencies:
'@types/react':
specifier: ~19.1.0
@@ -1120,11 +1087,6 @@ packages:
'@types/react':
optional: true
'@react-native-async-storage/[email protected]':
resolution: {integrity: sha512-gvRvjR5JAaUZF8tv2Kcq/Gbt3JHwbKFYfmb445rhOj6NUMx3qPLixmDx5pZAyb9at1bYvJ4/eTUipU5aki45xw==}
peerDependencies:
react-native: ^0.0.0-0 || >=0.65 <1.0
'@react-native/[email protected]':
resolution: {integrity: sha512-705B6x/5Kxm1RKRvSv0ADYWm5JOnoiQ1ufW7h8uu2E6G9Of/eE6hP/Ivw3U5jI16ERqZxiKQwk34VJbB0niX9w==}
engines: {node: '>= 20.19.4'}
@@ -1234,12 +1196,6 @@ packages:
'@react-navigation/[email protected]':
resolution: {integrity: sha512-1tJHg4KKRJuQ1/EvJxatrMef3NZXEPzwUIUZ3n1yJ2t7Q97siwRtbynRpQG9/69ebbtiZ8W3ScOZF/OmhvM4Rg==}
'@ronradtke/[email protected]':
resolution: {integrity: sha512-pAtefWI76vpkxsEgIFivyq1q6ej8rDyR7oVM/cWAxUydyBej9LOvULjLAeFuFLbYAelHTNoYXmGxQOlFLBa0+w==}
peerDependencies:
react: '>=16.2.0'
react-native: '>=0.50.4'
'@rtsao/[email protected]':
resolution: {integrity: sha512-zt6OdqaDoOnJ1ZYsCYGt9YmWzDXl4vQdKTyJev62gFhRGKdx7mcT54V9KIjg+d2wi9EXsPvAPKe7i7WjfVWB8g==}
@@ -1692,9 +1648,6 @@ packages:
resolution: {integrity: sha512-QxD8cf2eVqJOOz63z6JIN9BzvVs/dlySa5HGSBH5xtR8dPteIRQnBxxKqkNTiT6jbDTF6jAfrd4oMcND9RGbQg==}
engines: {node: '>=0.6'}
[email protected]:
resolution: {integrity: sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww==}
[email protected]:
resolution: {integrity: sha512-sXaHZicyEEmY86WyueLTQesbeoH/mquvarJaQNbjuOQO+7gbFcDEWqKmcWA4cOTLzFlfgvkiVxolk1k5bBIpmg==}
@@ -1762,9 +1715,6 @@ packages:
resolution: {integrity: sha512-Gmy6FhYlCY7uOElZUSbxo2UCDH8owEk996gkbrpsgGtrJLM3J7jGxl9Ic7Qwwj4ivOE5AWZWRMecDdF7hqGjFA==}
engines: {node: '>=10'}
[email protected]:
resolution: {integrity: sha512-dU+Tx2fsypxTgtLoE36npi3UqcjSSMNYfkqgmoEhtZrraP5VWq0K7FkWVTYa8eMPtnU/G2txVsfdCJTn9uzpuQ==}
[email protected]:
resolution: {integrity: sha512-sg01JDPzZ9jGshqKSckOQthXnYwOEP50jeVFhaSFbZcOy05TiuuaffDOfcwtCisJ9kNQuLBFibYywv2Bgm9osw==}
@@ -1877,27 +1827,9 @@ packages:
resolution: {integrity: sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA==}
engines: {node: '>= 8'}
[email protected]:
resolution: {integrity: sha512-FyyrDHZKEjXDpNJYvVsV960FiqQyXc/LlYmsxl2BcdMb2WPx0OGRVgTg55rPSyLSNMqP52R9r8geSp7apN3Ofg==}
engines: {node: '>=4'}
[email protected]:
resolution: {integrity: sha512-fJAcud6B3rRu+KHYk+Bwf+WFL2MDCJJ1XG9x137tJQ0xYxor7XziQtuGFbWNdqrvF4Tk26O3H73nfVqXt/fW1A==}
[email protected]:
resolution: {integrity: sha512-TizTzUddG/xYLA3NXodFM0fSbNizXjOKhqiQQwvhlspadZokn1KDy0NZFS0wuEubIYAV5/c1/lAr0TaaFXEXzw==}
[email protected]:
resolution: {integrity: sha512-e8RKaLXMOFii+02mOlqwjbD00KSEKqblnpO9e++1aXS1fPQOpS1YoqdVHBqPjHNoxeF2mimzVqawm2KCbEdtHQ==}
[email protected]:
resolution: {integrity: sha512-tRpdppF7TRazZrjJ6v3stzv93qxRcSsFmW6cX0Zm2NVKpxE1WV1HblnghVv9TreireHkqI/VDEsfolRF1p6y7Q==}
engines: {node: '>=8.0.0'}
[email protected]:
resolution: {integrity: sha512-u/O3vwbptzhMs3L1fQE82ZSLHQQfto5gyZzwteVIEyeaY5Fc7R4dapF/BvRoSYFeqfBk4m0V1Vafq5Pjv25wvA==}
engines: {node: '>= 6'}
[email protected]:
resolution: {integrity: sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==}
@@ -1987,19 +1919,6 @@ packages:
resolution: {integrity: sha512-35mSku4ZXK0vfCuHEDAwt55dg2jNajHZ1odvF+8SSr82EsZY4QmXfuWso8oEd8zRhVObSN18aM0CjSdoBX7zIw==}
engines: {node: '>=0.10.0'}
[email protected]:
resolution: {integrity: sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg==}
[email protected]:
resolution: {integrity: sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==}
[email protected]:
resolution: {integrity: sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w==}
engines: {node: '>= 4'}
[email protected]:
resolution: {integrity: sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw==}
[email protected]:
resolution: {integrity: sha512-zIHwmZPRshsCdpMDyVsqGmgyP0yT8GAgXUnkdAoJisxvf33k7yO6OuoKmcTGuXPWSsm8Oh88nZicRLA9Y0rUeA==}
engines: {node: '>=12'}
@@ -2029,14 +1948,6 @@ packages:
resolution: {integrity: sha512-Q0n9HRi4m6JuGIV1eFlmvJB7ZEVxu93IrMyiMsGC0lrMJMWzRgx6WGquyfQgZVb31vhGgXnfmPNNXmxnOkRBrg==}
engines: {node: '>= 0.8'}
[email protected]:
resolution: {integrity: sha512-WiyBqoomrwMdFG1e0kqvASYfnlb0lp8M5o5Fw2OFq1hNZxxcNk8Ik0Xm7LxzBhuidnZB/UtBqVCgUz3kBOP51Q==}
engines: {node: '>=0.12'}
[email protected]:
resolution: {integrity: sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==}
engines: {node: '>=0.12'}
[email protected]:
resolution: {integrity: sha512-ObFo8v4rQJAE59M69QzwloxPZtd33TpYEIjtKD1rrFDcM1Gd7IkDxEBU+HriziN6HSHQnBJi8Dmy+JWkav5HKA==}
engines: {node: '>=8'}
@@ -2231,13 +2142,6 @@ packages:
react: '*'
react-native: '*'
[email protected]:
resolution: {integrity: sha512-WJ9ougE8fEDu3/RV5Vz3gE5aNgnj1C0WByXCz3WUj8y/06bJyaIUDMq8FDLv3n0AO3guiErmkUunsD0EzSDUUw==}
peerDependencies:
expo: '*'
react: '*'
react-native: '*'
[email protected]:
resolution: {integrity: sha512-FnZn12E1dRYKDHlAdIyNFhBurKTS3F9CrfrBDJI5m3D7U17KBHMQ6JEfYlSj7LG7t+Ulr+IKaj58L1k5gBwTcQ==}
peerDependencies:
@@ -2633,9 +2537,6 @@ packages:
engines: {node: '>=16.x'}
hasBin: true
[email protected]:
resolution: {integrity: sha512-XREFCPo6ksxVzP4E0ekD5aMdf8WMwmdNaz6vuvxgI40UaEiu6q3p8X52aU6GdyvLY3XXX/8R7JOTXStz/nBbRw==}
[email protected]:
resolution: {integrity: sha512-TR3KfrTZTYLPB6jUjfx6MF9WcWrHL9su5TObK4ZkYgBdWKPOFoSoQIdEuTuR82pmtxH2spWG9h6etwfr1pLBqQ==}
engines: {node: '>=6'}
@@ -2743,10 +2644,6 @@ packages:
resolution: {integrity: sha512-41Cifkg6e8TylSpdtTpeLVMqvSBEVzTttHvERD741+pnZ8ANv0004MRL43QKPDlK9cGvNp6NZWZUBlbGXYxxng==}
engines: {node: '>=0.12.0'}
[email protected]:
resolution: {integrity: sha512-YWnfyRwxL/+SsrWYfOpUtz5b3YD+nyfkHvjbcanzk8zgyO4ASD67uVMRt8k5bM4lLMDnXfriRhOpemw+NfT1eA==}
engines: {node: '>=8'}
[email protected]:
resolution: {integrity: sha512-MjYsKHO5O7mCsmRGxWcLWheFqN9DJ/2TmngvjKXihe6efViPqc274+Fx/4fYj/r03+ESvBdTXK0V6tA3rgez1g==}
engines: {node: '>= 0.4'}
@@ -2984,9 +2881,6 @@ packages:
[email protected]:
resolution: {integrity: sha512-7ylylesZQ/PV29jhEDl3Ufjo6ZX7gCqJr5F7PKrqc93v7fzSymt1BpwEU8nAUXs8qzzvqhbjhK5QZg6Mt/HkBg==}
[email protected]:
resolution: {integrity: sha512-C7bfi1UZmoj8+PQx22XyeXCuBlokoyWQL5pWSP+EI6nzRylyThouddufc2c1NDIcP9k5agmN9fLpA7VNJfIiqw==}
[email protected]:
resolution: {integrity: sha512-t7hw9pI+WvuwNJXwk5zVHpyhIqzg2qTlklJOf0mVxGSbe3Fp2VieZcduNYjaLDoy6p9uGpQEGWG87WpMKlNq8g==}
engines: {node: '>=8'}
@@ -3022,20 +2916,9 @@ packages:
[email protected]:
resolution: {integrity: sha512-KpNARQA3Iwv+jTA0utUVVbrh+Jlrr1Fv0e56GGzAFOXN7dk/FviaDW8LHmK52DlcH4WP2n6gI8vN1aesBFgo9w==}
[email protected]:
resolution: {integrity: sha512-z38PwB7Rknhs1FrT1oMTp1MThnXpv//UH0usmhHAL8DrMoaBtKOKtRvdJOUleHmQZXMX8NcjgzOkelhCilUIlQ==}
peerDependencies:
react: ^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0
react-native: '*'
react-native-svg: ^12.0.0 || ^13.0.0 || ^14.0.0 || ^15.0.0
[email protected]:
resolution: {integrity: sha512-JmqCvUhmt43madlpFzG4BQzG2Z3m6tvQDNKdClZnO3VbIudJYmxsT0FNJMeiB2+JTSlTQTSbU8QdesVmwJcmLg==}
[email protected]:
resolution: {integrity: sha512-FtwnEuuK+2yVU7goGn/MJ0WBZMM9ZPgU9spqlFs7/A/pDIUNSOQZhUgOqYCficIuR2QaFnrt8LHqBWsbTAoI5w==}
hasBin: true
[email protected]:
resolution: {integrity: sha512-ocnPZQLNpvbedwTy9kNrQEsknEfgvcLMvOtz3sFeWApDq1MXH1TqkCIx58xlpESsfwQOnuBO9beyQuNGzVvuhQ==}
@@ -3043,22 +2926,12 @@ packages:
resolution: {integrity: sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==}
engines: {node: '>= 0.4'}
[email protected]:
resolution: {integrity: sha512-dn6wd0uw5GsdswPFfsgMp5NSB0/aDe6fK94YJV/AJDYXL6HVLWBsxeq7js7Ad+mU2K9LAlwpk6kN2D5mwCPVow==}
[email protected]:
resolution: {integrity: sha512-/sKlQJCBYVY9Ers9hqzKou4H6V5UWc/M59TH2dvkt+84itfnq7uFOMLpOiOS4ujvHP4etln18fmIxA5R5fll0g==}
[email protected]:
resolution: {integrity: sha512-zYiwtZUcYyXKo/np96AGZAckk+FWWsUdJ3cHGGmld7+AhvcWmQyGCYUh1hc4Q/pkOhb65dQR/pqCyK0cOaHz4Q==}
[email protected]:
resolution: {integrity: sha512-rkpe71W0N0c0Xz6QD0eJETuWAJGnJ9afsl1srmwPrI+yBCkge5EycXXbYRyvL29zZVUWQCY7InPRCv3GDXuZNw==}
[email protected]:
resolution: {integrity: sha512-2Sug1+knBjkaMsMgf1ctR1Ujx+Ayku4EdJN4Z+C2+JzoeF7A3OZ9KM2GY0CpQS51NR61LTurMJrRKPhSs3ZRTQ==}
engines: {node: '>=10'}
[email protected]:
resolution: {integrity: sha512-abv/qOcuPfk3URPfDzmZU1LKmuw8kT+0nIHvKrKgFrwifol/doWcdA4ZqsWQ8ENrFKkd67Mfpo/LovbIUsbt3w==}
@@ -3302,9 +3175,6 @@ packages:
resolution: {integrity: sha512-sHGJy8sOC1YraBywpzQlIKBE4pBbGbiF95U6Auspzyem956E0+FtDtsx1ZxlOJkQCZ1AFXAY/yuvtFYrOxF+Bw==}
engines: {node: ^16.14.0 || >=18.0.0}
[email protected]:
resolution: {integrity: sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w==}
[email protected]:
resolution: {integrity: sha512-2vPPEi+Z7WqML2jZYddDIfy5Dqb0r2fze2zTxNNknZaFpVHU3mFB3R+DWeJWGVx0ecvttSGlJTI+WG+8Z4cDWw==}
@@ -3558,9 +3428,6 @@ packages:
[email protected]:
resolution: {integrity: sha512-W+EWGn2v0ApPKgKKCy/7s7WHXkboGcsrXE+2joLyVxkbyVQfO3MUEaUQDHoSmb8TFFrSKYa9mw64WZHNHSDzYA==}
[email protected]:
resolution: {integrity: sha512-Wl3Vq2DQzxgsWKuW4USfck9zS7YzhvLNPpkwUUCF90bL32e1a0zOVQ3WsJILJOwzmPdHfzZmWasiiAUNBkhNkg==}
[email protected]:
resolution: {integrity: sha512-0msfJ1vRxXKVgTgvL+1ZOoYw3/0z1R+Ked0+udoJhyplC2jbVKIJ8Z1bzWdpQRCV3QcQ87Op0zJVE5DhKK2A0A==}
peerDependencies:
@@ -3593,15 +3460,6 @@ packages:
react: '*'
react-native: '*'
[email protected]:
resolution: {integrity: sha512-zejanlScF+IB9tYnbdry0MT34qjBXbiV/E72qGz33W/tX1bx8MXsbB4lxiuPETc9v/008vYZ60yjIstW22VlVg==}
[email protected]:
resolution: {integrity: sha512-vCuZJDf8a5aNC2dlMovEv4Z0jjEUET53lm/iILFnFewa15b4atjVxU6Wirm6O9y6dEsdjDZVD7Q3QM4T1wlI8g==}
peerDependencies:
react: '*'
react-native: '*'
[email protected]:
resolution: {integrity: sha512-SO2t9/17zM4iEnFvlu2DA9jqNbzNhoUP+AItkoCOyFmDMOhUnBBznBDCYN92fGdfAkfQlWzPoez6+zLxFNsZEg==}
peerDependencies:
@@ -4105,9 +3963,6 @@ packages:
resolution: {integrity: sha512-LbBDqdIC5s8iROCUjMbW1f5dJQTEFB1+KO9ogbvlb3nm9n4YHa5p4KTvFPWvh2Hs8gZMBuiB1/8+pdfe/tDPug==}
hasBin: true
[email protected]:
resolution: {integrity: sha512-8Y75pvTYkLJW2hWQHXxoqRgV7qb9B+9vFEtidML+7koHUFapnVJAZ6cKs+Qjz5Aw3aZWHMC6u0wJE3At+nSGwA==}
[email protected]:
resolution: {integrity: sha512-nWJ91DjeOkej/TA8pXQ3myruKpKEYgqvpw9lz4OPHj/NWFNluYrjbz9j01CJ8yKQd2g4jFoOkINCTW2I5LEEyw==}
engines: {node: '>= 0.4'}
@@ -4350,24 +4205,6 @@ packages:
resolution: {integrity: sha512-rVksvsnNCdJ/ohGc6xgPwyN8eheCxsiLM8mxuE/t/mOVqJewPuO1miLpTHQiRgTKCLexL4MeAFVagts7HmNZ2Q==}
engines: {node: '>=10'}
[email protected]:
resolution: {integrity: sha512-i77ae3aZq4dhMlRhJVCYgMLKuSiZAaUPAct2AksxQ+gOtimhGMdXljRT21P5BNpeT4kXlLIckvkPM029OljD7g==}
engines: {node: '>=12.20.0'}
peerDependencies:
'@types/react': '>=18.0.0'
immer: '>=9.0.6'
react: '>=18.0.0'
use-sync-external-store: '>=1.2.0'
peerDependenciesMeta:
'@types/react':
optional: true
immer:
optional: true
react:
optional: true
use-sync-external-store:
optional: true
snapshots:
'@0no-co/[email protected]': {}
@@ -5648,11 +5485,6 @@ snapshots:
optionalDependencies:
'@types/react': 19.1.17
'@react-native-async-storage/[email protected]([email protected](@babel/[email protected])(@types/[email protected])([email protected]))':
dependencies:
merge-options: 3.0.4
react-native: 0.81.5(@babel/[email protected])(@types/[email protected])([email protected])
'@react-native/[email protected]': {}
'@react-native/[email protected](@babel/[email protected])':
@@ -5837,15 +5669,6 @@ snapshots:
dependencies:
nanoid: 3.3.11
'@ronradtke/[email protected]([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected])':
dependencies:
css-to-react-native: 3.2.0
markdown-it: 13.0.2
prop-types: 15.8.1
react: 19.1.0
react-native: 0.81.5(@babel/[email protected])(@types/[email protected])([email protected])
react-native-fit-image: 1.5.5
'@rtsao/[email protected]': {}
'@sinclair/[email protected]': {}
@@ -6374,8 +6197,6 @@ snapshots:
[email protected]: {}
[email protected]: {}
[email protected]:
dependencies:
stream-buffers: 2.2.0
@@ -6449,8 +6270,6 @@ snapshots:
[email protected]: {}
[email protected]: {}
[email protected]: {}
[email protected]:
@@ -6581,33 +6400,10 @@ snapshots:
shebang-command: 2.0.0
which: 2.0.2
[email protected]: {}
[email protected]:
dependencies:
hyphenate-style-name: 1.1.0
[email protected]:
dependencies:
boolbase: 1.0.0
css-what: 6.2.2
domhandler: 5.0.3
domutils: 3.2.2
nth-check: 2.1.1
[email protected]:
dependencies:
camelize: 1.0.1
css-color-keywords: 1.0.0
postcss-value-parser: 4.2.0
[email protected]:
dependencies:
mdn-data: 2.0.14
source-map: 0.6.1
[email protected]: {}
[email protected]: {}
[email protected]:
@@ -6678,24 +6474,6 @@ snapshots:
dependencies:
esutils: 2.0.3
[email protected]:
dependencies:
domelementtype: 2.3.0
domhandler: 5.0.3
entities: 4.5.0
[email protected]: {}
[email protected]:
dependencies:
domelementtype: 2.3.0
[email protected]:
dependencies:
dom-serializer: 2.0.0
domelementtype: 2.3.0
domhandler: 5.0.3
[email protected]:
dependencies:
dotenv: 16.4.7
@@ -6718,10 +6496,6 @@ snapshots:
[email protected]: {}
[email protected]: {}
[email protected]: {}
[email protected]: {}
[email protected]:
@@ -7038,12 +6812,6 @@ snapshots:
transitivePeerDependencies:
- supports-color
[email protected]([email protected])([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected]):
dependencies:
expo: 54.0.33(@babel/[email protected])(@expo/[email protected])([email protected])([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected])
react: 19.1.0
react-native: 0.81.5(@babel/[email protected])(@types/[email protected])([email protected])
[email protected]([email protected])([email protected](@babel/[email protected])(@types/[email protected])([email protected])):
dependencies:
'@expo/config': 12.0.13
@@ -7469,8 +7237,6 @@ snapshots:
dependencies:
queue: 6.0.2
[email protected]: {}
[email protected]:
dependencies:
parent-module: 1.0.1
@@ -7580,8 +7346,6 @@ snapshots:
[email protected]: {}
[email protected]: {}
[email protected]:
dependencies:
call-bound: 1.0.4
@@ -7831,10 +7595,6 @@ snapshots:
[email protected]: {}
[email protected]:
dependencies:
uc.micro: 1.0.6
[email protected]:
dependencies:
p-locate: 4.1.0
@@ -7865,40 +7625,18 @@ snapshots:
dependencies:
yallist: 3.1.1
[email protected]([email protected]([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected]))([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected]):
dependencies:
react: 19.1.0
react-native: 0.81.5(@babel/[email protected])(@types/[email protected])([email protected])
react-native-svg: 15.12.1([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected])
[email protected]:
dependencies:
tmpl: 1.0.5
[email protected]:
dependencies:
argparse: 2.0.1
entities: 3.0.1
linkify-it: 4.0.1
mdurl: 1.0.1
uc.micro: 1.0.6
[email protected]: {}
[email protected]: {}
[email protected]: {}
[email protected]: {}
[email protected]: {}
[email protected]: {}
[email protected]:
dependencies:
is-plain-obj: 2.1.0
[email protected]: {}
[email protected]:
@@ -8343,10 +8081,6 @@ snapshots:
semver: 7.7.4
validate-npm-package-name: 5.0.1
[email protected]:
dependencies:
boolbase: 1.0.0
[email protected]: {}
[email protected]:
@@ -8603,10 +8337,6 @@ snapshots:
[email protected]: {}
[email protected]:
dependencies:
prop-types: 15.8.1
[email protected]([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected]):
dependencies:
'@egjs/hammerjs': 2.0.17
@@ -8642,16 +8372,6 @@ snapshots:
react-native-is-edge-to-edge: 1.3.1([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected])
warn-once: 0.1.1
[email protected]: {}
[email protected]([email protected](@babel/[email protected])(@types/[email protected])([email protected]))([email protected]):
dependencies:
css-select: 5.2.2
css-tree: 1.1.3
react: 19.1.0
react-native: 0.81.5(@babel/[email protected])(@types/[email protected])([email protected])
warn-once: 0.1.1
[email protected]([email protected]([email protected]))([email protected]):
dependencies:
'@babel/runtime': 7.28.6
@@ -9253,8 +8973,6 @@ snapshots:
[email protected]: {}
[email protected]: {}
[email protected]:
dependencies:
call-bound: 1.0.4
@@ -9487,10 +9205,3 @@ snapshots:
yargs-parser: 21.1.1
[email protected]: {}
[email protected](@types/[email protected])([email protected])([email protected])([email protected]([email protected])):
optionalDependencies:
'@types/react': 19.1.17
immer: 11.1.4
react: 19.1.0
use-sync-external-store: 1.6.0([email protected])
-190
View File
@@ -1,190 +0,0 @@
/**
* webSearchLogger.ts
* Centralized structured logger for the web search pipeline on the client side.
* Outputs consistently formatted, emoji-tagged logs to the React Native / Metro console.
*/
type LogLevel = 'INFO' | 'WARN' | 'ERROR' | 'DEBUG';
const ICONS: Record<LogLevel, string> = {
INFO: 'ℹ️ ',
WARN: '⚠️ ',
ERROR: '❌',
DEBUG: '🔍',
};
function formatMeta(meta?: Record<string, any>): string {
if (!meta) return '';
try {
return '\n ' + JSON.stringify(meta, null, 2).replace(/\n/g, '\n ');
} catch {
return '';
}
}
function log(level: LogLevel, phase: string, message: string, meta?: Record<string, any>) {
const ts = new Date().toISOString();
const metaStr = formatMeta(meta);
const prefix = `[${ts}] ${ICONS[level]} [WebSearch/${phase}]`;
const fullMsg = `${prefix} ${message}${metaStr}`;
switch (level) {
case 'ERROR':
console.error(fullMsg);
break;
case 'WARN':
console.warn(fullMsg);
break;
default:
console.log(fullMsg);
}
}
function logSection(title: string) {
const border = '─'.repeat(55);
console.log(`\n${border}`);
console.log(` ${title}`);
console.log(`${border}`);
}
/** Rough tokens estimate (~4 chars per token) */
function estimateTokens(text: string): number {
return Math.ceil(text.length / 4);
}
// ─── Public API ────────────────────────────────────────────────────────────────
export const WebSearchLogger = {
pipelineStart(query: string, searchEndpoint: string) {
logSection(`🚀 Web Search Pipeline Started`);
log('INFO', 'Init', `New search initiated`, {
query,
searchEndpoint,
timestamp: new Date().toISOString(),
});
},
endpointResolved(host: string, port: number, searchEndpoint: string) {
log('INFO', 'Endpoint', `Search service URL resolved`, {
lmStudioHost: host,
lmStudioPort: port,
searchServiceEndpoint: searchEndpoint,
});
},
requestSent(query: string, maxContextLength: number, timeoutMs = 30000) {
log('INFO', 'Request', `Sending search request to local service`, {
query,
maxContextLength,
estimatedTokenBudget: estimateTokens('x'.repeat(maxContextLength)),
timeoutMs,
});
},
requestFailed(status: number, statusText: string) {
log('ERROR', 'Request', `Search service returned error`, {
httpStatus: status,
statusText,
});
},
serviceUnreachable(error: unknown, isTimeout = false) {
const err = error instanceof Error ? error : new Error(String(error));
const isConnectionRefused =
err.message.includes('Network request failed') ||
err.message.includes('ECONNREFUSED') ||
err.message.includes('Connection refused');
const message = isTimeout
? `Search service timeout (30s) — service may be overloaded or unreachable`
: isConnectionRefused
? `Search service is NOT running — connection refused`
: `Cannot reach local search service`;
log('ERROR', 'Request', message, {
error: err.message,
isTimeout,
isConnectionRefused,
diagnosis: isConnectionRefused
? 'The local-search-service Node.js process is not running on this machine'
: isTimeout
? 'Service is running but not responding within 30s (SearXNG may be slow)'
: 'Unknown network error',
hint: 'Run: cd local-search-service && npx ts-node server.ts',
});
},
contextReceived(
context: string,
sources: Array<{ title: string; url: string }>,
meta?: {
requestId?: string;
totalLatencyMs?: number;
finalContextChars?: number;
finalContextEstimatedTokens?: number;
sourcesKept?: number;
sourcesDropped?: number;
pruned?: boolean;
},
) {
logSection(`✅ Context Received from Search Service`);
log('INFO', 'Context', `Web search context ready`, {
contextChars: context.length,
estimatedTokens: estimateTokens(context),
sourcesReceived: sources.length,
sourceTitles: sources.map((s) => s.title.substring(0, 60)),
sourceUrls: sources.map((s) => s.url),
...(meta ?? {}),
});
},
contextAnalysis(systemPromptChars: number, historyChars: number, searchContextChars: number) {
logSection(`🧠 Context Window Analysis`);
const totalChars = systemPromptChars + historyChars + searchContextChars;
const totalTokens = estimateTokens('x'.repeat(totalChars));
log('INFO', 'Analysis', `Total context before sending to LLM`, {
systemPromptChars,
systemPromptTokens: estimateTokens('x'.repeat(systemPromptChars)),
chatHistoryChars: historyChars,
chatHistoryTokens: estimateTokens('x'.repeat(historyChars)),
searchContextChars,
searchContextTokens: estimateTokens('x'.repeat(searchContextChars)),
totalChars,
estimatedTotalTokens: totalTokens,
note: 'Token estimates are approximate (~4 chars/token). Actual tokenization varies by model.',
});
},
llmRequestStart(model: string, messageCount: number) {
logSection(`🤖 LLM Request`);
log('INFO', 'LLM', `Sending messages to LM Studio`, {
model,
totalMessages: messageCount,
streamingEnabled: true,
});
},
llmStreamComplete(assistantMessageId: string, chunkCount?: number) {
log('INFO', 'LLM', `Stream complete — response generated`, {
assistantMessageId,
...(chunkCount !== undefined ? { chunkCount } : {}),
});
},
llmError(error: Error) {
log('ERROR', 'LLM', `LLM stream error`, {
error: error.message,
stack: error.stack?.split('\n').slice(0, 3).join(' | '),
});
},
pipelineComplete(query: string, totalLatencyMs: number, withSearch: boolean) {
logSection(`🏁 Pipeline Complete`);
log('INFO', 'Done', `Full request cycle finished`, {
query,
totalLatencyMs,
webSearchUsed: withSearch,
});
console.log('─'.repeat(55) + '\n');
},
};
-8
View File
@@ -49,10 +49,6 @@ interface AppState {
isStreaming: boolean;
setIsStreaming: (streaming: boolean) => void;
// Web Search
isWebSearchEnabled: boolean;
setIsWebSearchEnabled: (enabled: boolean) => void;
// Edit mode
editingMessageId: string | null;
setEditingMessageId: (id: string | null) => void;
@@ -123,9 +119,6 @@ export const useAppStore = create<AppState>()(
isStreaming: false,
setIsStreaming: (isStreaming) => set({ isStreaming }),
isWebSearchEnabled: false,
setIsWebSearchEnabled: (isWebSearchEnabled) => set({ isWebSearchEnabled }),
editingMessageId: null,
setEditingMessageId: (editingMessageId) => set({ editingMessageId }),
@@ -178,7 +171,6 @@ export const useAppStore = create<AppState>()(
chatSessions: state.chatSessions,
messages: state.messages,
currentChatId: state.currentChatId,
isWebSearchEnabled: state.isWebSearchEnabled,
}),
}
)