feat(ui): enhance chat UI with new components and styling

refactor(chat): improve message display and actions
feat(chat): add high contrast mode toggle
style(chat): update spacing and sizing for better readability
feat(chat): add new Section and Stack layout components
docs: add UI style guide documentation
fix(chat): improve browser connection handling in scraper
feat(settings): expand default sources and improve UI
refactor: consolidate chat page layout and components
This commit is contained in:
Evandro Fonseca Junior committed 2025-11-20 15:13:54 -03:00
1 parent c06e7ae5cf
commit 4006a72eec
20 files changed
+741 -235

No files matched your search

@@ -0,0 +1,114 @@
## 📋 Análise
**Solicitação**: Corrigir falhas de conexão do browser headless e o erro de compilação `Send`, tornar o scraper resiliente com retry e ampliar resultados de busca.
**Objetivo real**: Garantir estabilidade contínua do backend de scraping, mesmo quando o Chrome headless cai, evitando violações de `Send` e melhorando a cobertura de fontes.
## 🔍 Contexto Identificado
### Arquivos Examinados
- `src-tauri/src/lib.rs:1176–1193` get_or_create_browser: cria/reutiliza `Browser` singleton.
- `src-tauri/src/lib.rs:1247–1277` search_web_metadata: comando Tauri que exige `Future + Send`.
- `src-tauri/src/web_scraper.rs:136–240` search_duckduckgo_metadata: parsing HTML, previously segurava `Html` atravessando `await`.
- `src-tauri/src/web_scraper.rs:469–538` search_and_scrape_with_config: coordena scraping paralelo e coleta resultados.
- `src-tauri/src/web_scraper.rs:605–736` fetch_and_convert_sync: cria aba, navega, extrai HTML e usa Readability.
- `src-tauri/src/lib.rs:1306–1399` force_kill_browser: encerra processos Chrome headless.
### Padrões Detectados
- **Singleton Browser** via `State<BrowserState>` com `Arc<Browser>`.
- **Concorrência controlada** com `Semaphore` e `tokio::spawn_blocking` para operações de Chrome.
- **Fallbacks**: limpeza de URL, filtros anti-ads, Readability + fallback por parágrafos.
### Dependências Mapeadas
```
web_scraper (modificado)
├─ headless_chrome::Browser/Tab
├─ reqwest (HTTP)
├─ scraper (Html, Selector) [não Send]
└─ readability, html2text
lib (modificado)
└─ tauri commands (Future + Send obrigatório)
```
## 🎯 Abordagens Viáveis
### Abordagem A: Verificação ativa de Browser + Retry cooperativo
**Conceito**: Validar liveness do `Browser` antes de uso; se cair durante scraping, recriar e repetir uma passada de URLs com limite de tentativas.
```
request → get_or_create_browser (is_alive?) → spawn scraping
↳ erro conn fechada → reset + recreate → retry restante (1x)
```
**Vantagens**: ✓ Simples de integrar, ✓ Minimiza impacto, ✓ Boa resiliência
**Desvantagens**: ✗ Retry global pode repetir algumas URLs
**Complexidade**: O(n) | Tempo: 3–4h | Risco: Médio
### Abordagem B: Pool/Auto-heal por task
**Conceito**: Cada task recupera o browser ao falhar e re-executa só sua URL.
```
spawn_blocking(url) → new_tab → erro conn → sinaliza reset → reobtem browser → reprocessa url
```
**Vantagens**: ✓ Isolamento por URL
**Desvantagens**: ✗ Coordenação de estado entre threads, ✗ Maior complexidade
**Complexidade**: O(n) | Tempo: 5–6h | Risco: Médio-Alto
## 🏆 Recomendação
**Abordagem A** porque:
- Alinha com o padrão singleton já existente.
- Menor acoplamento entre tarefas e estado global.
- Implementação direta com menor risco de condições de corrida.
## ⚠️ Riscos & Mitigações
| Risco | Prob | Impacto | Mitigação |
|-------|------|---------|-----------|
| Loop de retry infinito | Baixa | Médio | Limitar a 1–2 tentativas, backoff curto |
| Recriação concorrente do Browser | Média | Médio | Guardar lock no `BrowserState` e checar antes de recriar |
| Demasiadas abas simultâneas | Média | Médio | Reduzir `max_concurrent_tabs` ao detectar falha |
| DDG bloqueios 429 | Média | Baixo | Rotação User-Agent e backoff |
## 📝 Plano de Execução
1. Ajustes `Send` (consolidação)
- Isolar parsing `scraper::Html` em escopo síncrono que retorna `Vec<SearchResultMetadata>` (já aplicado em parte na função; revisar para todos os caminhos).
- Garantir que nenhum `Html`/`Selector` vive além do `await` nos métodos `search_*`.
- Verificar pontos: `src-tauri/src/web_scraper.rs:136–240` e demais que usem `Html`.
2. Browser Resurrection
- Adicionar `fn is_browser_alive(browser: &Browser) -> bool` em `web_scraper.rs` ou `lib.rs` (ex.: tentar `new_tab()` e descartar; ou `get_tabs()`; se falhar → false).
- Alterar `get_or_create_browser` (`src-tauri/src/lib.rs:1176–1193`) para: se `Some(browser)` e `!is_alive(browser)` → `reset` e `create_browser()`.
- Em `force_kill_browser` (`src-tauri/src/lib.rs:1306–1399`): após kill, setar `BrowserState` para `None` via lock (ou orientar UI a chamar `reset_browser` antes do kill), garantindo recriação no próximo uso.
3. Retry resiliente no Scraper
- Envolver `fetch_and_convert_sync` com camada de retry (máx. 2 tentativas) na orquestração async (`search_and_scrape_with_config` em `src-tauri/src/web_scraper.rs:469–538`).
- Se erro contiver "underlying connection is closed"/timeout global do browser:
- Executar `reset_browser` e `get_or_create_browser` para novo `Arc<Browser)`.
- Reprocessar as URLs que falharam apenas uma vez com o novo browser.
- Reduzir temporariamente `max_concurrent_tabs` para 3 ao detectar queda, evitando sobrecarga imediata.
4. Paginação no DuckDuckGo
- Em `search_duckduckgo` (`src-tauri/src/web_scraper.rs:78–133`): suportar paginação via parâmetro `s=<offset>` até atingir `limit` (30–50).
- Extrair links por página, deduplicar, respeitar `excluded_domains`/ads.
5. Telemetria e Logs
- Logar eventos de recreação de browser, tentativas de retry e motivos.
- Contadores: quedas de browser, tentativas, tempo médio por URL.
6. Testes/Validação
- Caso 1: Forçar `force_kill_browser` durante scraping → validar que a próxima chamada recria o browser e conclui.
- Caso 2: Simular DDG com >= 40 resultados → paginar e coletar sem violar `Send`.
- Caso 3: Stress com `max_concurrent_tabs=5` → se cair, reduzir e recuperar.
## Key Changes (Resumo de Implementação)
- `lib.rs:1176–1193`: `get_or_create_browser` verifica liveness; recria se morto.
- `lib.rs:1306–1399`: sincronia com `reset_browser` ao matar processos.
- `web_scraper.rs:469–538`: retry global (1x) e recriação de browser ao detectar conexão fechada; ajuste dinamicamente `Semaphore` na segunda passada.
- `web_scraper.rs:78–133`: paginação DDG com `s=` offset.
- `web_scraper.rs:136–240`: parsing isolado; garantir que `Html` é dropado antes do `await`.
## Entregáveis
- Código robusto em `web_scraper.rs` e `lib.rs` com:
- Correção total do `Send`.
- Resurreição automática do browser.
- Retry controlado por erro de conexão.
- Paginação para 30–50 resultados.
- Logs melhorados e validação manual automática.
## ❓ Aguardando Aprovação
- [ ] Posso prosseguir com as mudanças propostas?
- [ ] Preferências de `max_concurrent_tabs` e `limit` padrão (sugestão: 5 e 40)?
- [ ] Deseja manter `force_kill_browser` como utilitário avançado, com alerta na UI para reiniciar o scraper?
+64 -87
View File
@@ -1,17 +1,14 @@
'use client';
import { ResizableHandle, ResizablePanel, ResizablePanelGroup } from "@/components/ui/resizable";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
import { Button } from "@/components/ui/button";
import { Textarea } from "@/components/ui/textarea";
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select";
import { Switch } from "@/components/ui/switch";
import { Label } from "@/components/ui/label";
import { MessageSquare, Settings, Server, Moon, Sun, PanelLeftClose, PanelLeftOpen, Loader2, ChevronDown } from "lucide-react";
import { ChatInput } from "@/components/chat/chat-input";
import { ChatMessage } from "@/components/chat/chat-message";
import { SystemPanel } from "@/components/chat/system-panel";
import { PromptGeneratorDialog } from "@/components/chat/prompt-generator";
import { SidebarList } from "@/components/chat/sidebar-list";
import { useChat } from "@/hooks/use-chat";
import { useLocalModels } from "@/hooks/use-local-models";
@@ -36,7 +33,7 @@ import defaultFormatPrompt from "@/data/prompts/default-format.md";
export default function ChatPage() {
const router = useRouter();
const { messages, setMessages, sendMessage, isLoading, stop, clearChat } = useChat();
const { models } = useLocalModels();
const { models, refresh } = useLocalModels();
const { theme, setTheme } = useTheme();
const {
@@ -56,6 +53,8 @@ export default function ChatPage() {
const settings = useSettingsStore();
const [selectedModel, setSelectedModel] = useState("");
const [mounted, setMounted] = useState(false);
const initializedRef = useRef(false);
// Initialize with default format prompt
const [systemPrompt, setSystemPrompt] = useState(defaultFormatPrompt || "Você é um assistente útil e prestativo.");
const [isChatsSidebarCollapsed, setIsChatsSidebarCollapsed] = useState(false);
@@ -71,10 +70,22 @@ export default function ChatPage() {
// Auto-select first model
useEffect(() => {
if (models.length > 0 && !selectedModel) {
setSelectedModel(models[0].name);
setMounted(true);
}, []);
useEffect(() => {
if (initializedRef.current) return;
const stored = settings.selectedModel;
const first = models.length > 0 ? models[0].name : "";
const next = stored || first || "";
if (next) {
setSelectedModel(next);
if (!stored) {
settings.setSelectedModel(next);
}
}
}, [models, selectedModel]);
initializedRef.current = true;
}, [settings.selectedModel, models]);
// Auto-scroll para o final durante streaming
useEffect(() => {
@@ -372,7 +383,7 @@ Ao responder sobre fatos atuais ou notícias, inicie mencionando explicitamente
onClick={() => setTheme(theme === "dark" ? "light" : "dark")}
className="rounded-lg text-muted-foreground hover:text-foreground"
>
{theme === "dark" ? <Sun className="w-5 h-5" /> : <Moon className="w-5 h-5" />}
{mounted ? (theme === "dark" ? <Sun className="w-5 h-5" /> : <Moon className="w-5 h-5" />) : null}
</Button>
<Button
variant="ghost"
@@ -436,17 +447,27 @@ Ao responder sobre fatos atuais ou notícias, inicie mencionando explicitamente
</div>
</div>
<div className="flex-1 max-w-[300px]">
<Select value={selectedModel} onValueChange={setSelectedModel}>
<SelectTrigger className="h-9">
<SelectValue placeholder="Selecione um modelo..." />
</SelectTrigger>
<SelectContent>
{models.map(m => (
<SelectItem key={m.name} value={m.name}>{m.name}</SelectItem>
))}
</SelectContent>
</Select>
<div className="flex-1 max-w-[420px] flex items-center gap-2">
<div className="flex-1">
<Select value={selectedModel} onValueChange={(v) => { setSelectedModel(v); settings.setSelectedModel(v); }}>
<SelectTrigger className="h-9">
<SelectValue placeholder="Selecione um modelo..." />
</SelectTrigger>
<SelectContent>
{models.map(m => (
<SelectItem key={m.name} value={m.name}>
<span className="flex items-center justify-between w-full">
<span className="truncate">{m.name}</span>
<span className="ml-3 text-xs text-muted-foreground">{m.size}</span>
</span>
</SelectItem>
))}
</SelectContent>
</Select>
</div>
<Button variant="outline" size="icon" className="h-9 w-9" onClick={refresh} title="Atualizar modelos">
<Loader2 className="w-4 h-4" />
</Button>
</div>
</div>
@@ -460,7 +481,7 @@ Ao responder sobre fatos atuais ou notícias, inicie mencionando explicitamente
<p>Inicie uma conversa com {selectedModel}</p>
</div>
) : (
<div className="flex flex-col pb-4 max-w-4xl mx-auto w-full">
<div className="flex flex-col pb-4 max-w-3xl mx-auto w-full space-y-6">
{messages.map((msg, i) => {
// Verificar se a mensagem anterior teve busca web
const hasWebSources = i > 0 &&
@@ -471,6 +492,19 @@ Ao responder sobre fatos atuais ou notícias, inicie mencionando explicitamente
return (
<div key={i}>
<ChatMessage message={msg} />
{(msg.role === 'user') && (
(webSearch.status === 'searching' ||
webSearch.status === 'scraping' ||
webSearch.status === 'completed' ||
webSearch.status === 'error') && (
<SearchProgress
status={webSearch.status}
query={webSearch.currentQuery}
sources={webSearch.scrapedSources}
error={webSearch.error}
/>
)
)}
{/* Mostrar fontes após resposta do assistente que usou web search */}
{msg.role === 'assistant' && hasWebSources && (
<div className="px-6 pb-4">
@@ -507,18 +541,7 @@ Ao responder sobre fatos atuais ou notícias, inicie mencionando explicitamente
</div>
);
})}
{/* Web Search Progress - mostrar antes da resposta da IA */}
{(webSearch.status === 'searching' ||
webSearch.status === 'scraping' ||
webSearch.status === 'completed' ||
webSearch.status === 'error') && (
<SearchProgress
status={webSearch.status}
query={webSearch.currentQuery}
sources={webSearch.scrapedSources}
error={webSearch.error}
/>
)}
{isLoading && thinkingStep === 'complete' && (
<div className="px-6 py-4 text-xs text-muted-foreground animate-pulse">
Gerando resposta...
@@ -530,8 +553,8 @@ Ao responder sobre fatos atuais ou notícias, inicie mencionando explicitamente
<div ref={messagesEndRef} className="h-1" />
</div>
{/* Thinking Indicator */}
{thinkingStep && thinkingStep !== 'complete' && (
{/* Thinking Indicator - aparece apenas quando não há web search ativo */}
{thinkingStep && thinkingStep !== 'complete' && (webSearch.status === 'idle') && (
<ThinkingIndicator
currentStep={thinkingStep}
searchQuery={webSearch.currentQuery}
@@ -593,62 +616,16 @@ Ao responder sobre fatos atuais ou notícias, inicie mencionando explicitamente
isLoading={isLoading}
webSearchEnabled={webSearch.isEnabled}
onWebSearchToggle={webSearch.setEnabled}
categories={settings.webSearch.categories}
onToggleCategory={(id, enabled) => {
const cat = settings.webSearch.categories.find(c => c.id === id);
if (!cat) return;
settings.updateCategory({ ...cat, enabled });
}}
/>
</div>
</ResizablePanel>
<ResizableHandle withHandle />
{/* Right Sidebar (Settings & Monitor) */}
<ResizablePanel defaultSize={20} minSize={15} maxSize={30} className="bg-muted/10">
<Tabs defaultValue="params" className="h-full flex flex-col overflow-hidden">
<div className="border-b px-4">
<TabsList className="w-full justify-start h-12 bg-transparent p-0 gap-4">
<TabsTrigger
value="params"
className="data-[state=active]:bg-transparent data-[state=active]:shadow-none data-[state=active]:border-b-2 data-[state=active]:border-primary rounded-none px-0"
>
Parâmetros
</TabsTrigger>
<TabsTrigger
value="system"
className="data-[state=active]:bg-transparent data-[state=active]:shadow-none data-[state=active]:border-b-2 data-[state=active]:border-primary rounded-none px-0"
>
Sistema
</TabsTrigger>
</TabsList>
</div>
<TabsContent value="params" className="flex-1 p-4 space-y-6 overflow-y-auto overflow-x-hidden m-0">
<div className="space-y-6">
{/* System Prompt */}
<div className="space-y-2">
<div className="flex items-center justify-between">
<label className="text-sm font-medium">System Prompt</label>
<PromptGeneratorDialog
defaultModel={selectedModel}
onPromptGenerated={setSystemPrompt}
/>
</div>
<Textarea
value={systemPrompt}
onChange={(e) => setSystemPrompt(e.target.value)}
className="min-h-[300px] resize-none font-mono text-sm"
placeholder="Defina como a IA deve se comportar..."
/>
<p className="text-xs text-muted-foreground">
Instruções globais para o comportamento do modelo.
</p>
</div>
</div>
</TabsContent>
<TabsContent value="system" className="flex-1 m-0 overflow-y-auto overflow-x-hidden">
<SystemPanel />
</TabsContent>
</Tabs>
</ResizablePanel>
</ResizablePanelGroup>
</div>
);
+32
View File
@@ -2,6 +2,7 @@
@import "tw-animate-css";
@custom-variant dark (&:is(.dark *));
@custom-variant hc (&:is([data-high-contrast=true] *));
@theme inline {
--color-background: var(--background);
@@ -37,10 +38,23 @@
--color-popover: var(--popover);
--color-card-foreground: var(--card-foreground);
--color-card: var(--card);
--color-message-user: var(--message-user);
--color-message-assistant: var(--message-assistant);
--color-message-highlight: var(--message-highlight);
--color-message-selected: var(--message-selected);
--color-message-high-contrast: var(--message-high-contrast);
--radius-sm: calc(var(--radius) - 4px);
--radius-md: calc(var(--radius) - 2px);
--radius-lg: var(--radius);
--radius-xl: calc(var(--radius) + 4px);
--space-1: 8px;
--space-2: 16px;
--space-3: 24px;
--space-4: 32px;
--space-5: 40px;
--space-6: 48px;
--space-7: 56px;
--space-8: 64px;
}
:root {
@@ -76,6 +90,11 @@
--sidebar-accent-foreground: oklch(0.21 0.006 285.885);
--sidebar-border: oklch(0.92 0.004 286.32);
--sidebar-ring: oklch(0.705 0.015 286.067);
--message-user: oklch(0.965 0.02 245);
--message-assistant: oklch(0.972 0.012 300);
--message-highlight: oklch(0.94 0.02 250);
--message-selected: oklch(0.92 0.02 285);
--message-high-contrast: oklch(0.99 0 0);
}
.dark {
@@ -110,6 +129,19 @@
--sidebar-accent-foreground: oklch(0.985 0 0);
--sidebar-border: oklch(1 0 0 / 10%);
--sidebar-ring: oklch(0.552 0.016 285.938);
--message-user: oklch(0.23 0.02 245);
--message-assistant: oklch(0.24 0.015 300);
--message-highlight: oklch(0.34 0.02 250);
--message-selected: oklch(0.25 0.02 285);
--message-high-contrast: oklch(0.18 0.005 286);
}
:root[data-high-contrast="true"] {
--message-user: oklch(0.9 0 0);
--message-assistant: oklch(0.9 0 0);
--message-highlight: oklch(0.8 0 0);
--message-selected: oklch(0.7 0 0);
--message-high-contrast: oklch(0.99 0 0);
}
@layer base {
+70 -41
View File
@@ -425,19 +425,36 @@ export default function SettingsPage() {
<div className="space-y-2">
<Label htmlFor="max-results">Máximo de Resultados</Label>
<div className="space-y-2">
<div className="flex justify-between text-sm">
<div className="flex items-center gap-3">
<Input
id="max-results"
type="number"
value={settings.webSearch.maxResults}
onChange={(e) => {
const v = parseInt(e.target.value || '0', 10);
const clamped = Math.min(100, Math.max(1, isNaN(v) ? 1 : v));
settings.setWebSearchMaxResults(clamped);
}}
className="w-24"
min={1}
max={100}
disabled={!settings.webSearch.enabled}
/>
<div className="flex-1">
<Slider
value={[settings.webSearch.maxResults]}
onValueChange={([value]) => settings.setWebSearchMaxResults(value)}
min={1}
max={100}
step={1}
disabled={!settings.webSearch.enabled}
/>
</div>
</div>
<div className="flex justify-between text-xs text-muted-foreground">
<span>1</span>
<span className="font-medium">{settings.webSearch.maxResults}</span>
<span>10</span>
<span>100</span>
</div>
<Slider
value={[settings.webSearch.maxResults]}
onValueChange={([value]) => settings.setWebSearchMaxResults(value)}
min={1}
max={10}
step={1}
disabled={!settings.webSearch.enabled}
/>
</div>
</div>
@@ -516,9 +533,9 @@ export default function SettingsPage() {
variant="outline"
size="sm"
onClick={async () => {
if (confirm('Restaurar categorias padrão? Isso substituirá a configuração atual.')) {
if (confirm('Mesclar categorias padrão curadas com sua configuração (sem duplicatas)?')) {
try {
const defaultConfig: SourcesConfig = {
const curated: SourcesConfig = {
version: 1,
last_updated: new Date().toISOString(),
categories: [
@@ -526,11 +543,10 @@ export default function SettingsPage() {
id: 'academico',
name: 'Acadêmico',
base_sites: [
'scholar.google.com',
'arxiv.org',
'pubmed.ncbi.nlm.nih.gov',
'ieee.org',
'acm.org',
'scholar.google.com', 'arxiv.org', 'pubmed.ncbi.nlm.nih.gov', 'ieee.org', 'acm.org',
'sciencedirect.com', 'springer.com', 'jstor.org', 'researchgate.net', 'plos.org',
'nature.com', 'sciencemag.org', 'mit.edu', 'phys.org', 'semanticscholar.org',
'core.ac.uk', 'paperswithcode.com', 'base-search.net', 'doaj.org', 'worldscientific.com'
],
enabled: true,
},
@@ -538,11 +554,10 @@ export default function SettingsPage() {
id: 'tech',
name: 'Tech',
base_sites: [
'github.com',
'stackoverflow.com',
'dev.to',
'medium.com',
'reddit.com/r/programming',
'stackoverflow.com', 'github.com', 'reddit.com/r/programming', 'dev.to', 'hackernoon.com',
'tabnews.com.br', 'lobste.rs', 'news.ycombinator.com', 'hashnode.com', 'codesandbox.io',
'developer.mozilla.org', 'web.dev', 'smashingmagazine.com', 'martinfowler.com', 'infoq.com',
'dzone.com', 'freecodecamp.org', 'digitalocean.com', 'realpython.com', 'css-tricks.com'
],
enabled: true,
},
@@ -550,10 +565,10 @@ export default function SettingsPage() {
id: 'news',
name: 'News',
base_sites: [
'news.ycombinator.com',
'techcrunch.com',
'theverge.com',
'arstechnica.com',
'theverge.com', 'techcrunch.com', 'arstechnica.com', 'engadget.com', 'wired.com',
'venturebeat.com', 'cnet.com', 'mashable.com', 'gizmodo.com', 'zdnet.com',
'tomshardware.com', 'anandtech.com', 'bleepingcomputer.com', 'theregister.com', '9to5mac.com',
'androidauthority.com', 'techradar.com', 'fastcompany.com', 'g1.globo.com', 'techtudo.com.br'
],
enabled: true,
},
@@ -561,33 +576,47 @@ export default function SettingsPage() {
id: 'financeiro',
name: 'Financeiro',
base_sites: [
'bloomberg.com',
'reuters.com',
'financialtimes.com',
'wsj.com',
'bloomberg.com', 'reuters.com', 'wsj.com', 'ft.com', 'cnbc.com',
'marketwatch.com', 'economist.com', 'forbes.com', 'investopedia.com', 'seekingalpha.com',
'infomoney.com.br', 'valor.globo.com', 'exame.com', 'braziljournal.com', 'coindesk.com',
'cointelegraph.com', 'moneytimes.com.br', 'suno.com.br', 'b3.com.br', 'imf.org'
],
enabled: true,
},
],
};
await settings.saveSources(defaultConfig);
if (!settings.sourcesConfig) {
await settings.saveSources(curated);
} else {
const merged: SourcesConfig = {
...settings.sourcesConfig,
categories: settings.sourcesConfig.categories.map((existing) => {
const cur = curated.categories.find(c => c.id === existing.id);
if (!cur) return existing;
const set = new Set<string>([...existing.base_sites, ...cur.base_sites].map(s => s.toLowerCase()));
return { ...existing, base_sites: Array.from(set) } as any;
}),
last_updated: new Date().toISOString(),
};
await settings.saveSources(merged);
}
toast({
title: 'Padrões restaurados',
description: 'Categorias padrão foram restauradas',
title: 'Fontes mescladas',
description: 'Categorias curadas foram mescladas sem duplicatas',
});
} catch (error) {
toast({
title: 'Erro ao restaurar',
description: error instanceof Error ? error.message : 'Falha ao restaurar',
title: 'Erro ao mesclar',
description: error instanceof Error ? error.message : 'Falha ao mesclar categorias',
variant: 'destructive',
});
}
}
}}
>
<RotateCcw className="w-4 h-4 mr-2" />
Restaurar Padrões
</Button>
}}
>
<RotateCcw className="w-4 h-4 mr-2" />
Mesclar Padrões Curados
</Button>
</div>
<div className="grid gap-4">
{settings.sourcesConfig.categories.map((category) => {
+71 -12
View File
@@ -1,6 +1,6 @@
import { Button } from "@/components/ui/button";
import { Textarea } from "@/components/ui/textarea";
import { Send, Square, Globe, WifiOff } from "lucide-react";
import { Send, Square, Globe, WifiOff, ChevronUp } from "lucide-react";
import { useState, useRef, useEffect } from "react";
import {
Tooltip,
@@ -15,6 +15,8 @@ interface ChatInputProps {
isLoading: boolean;
webSearchEnabled?: boolean;
onWebSearchToggle?: (enabled: boolean) => void;
categories?: SearchCategory[];
onToggleCategory?: (id: string, enabled: boolean) => void;
}
export function ChatInput({
@@ -23,9 +25,12 @@ export function ChatInput({
isLoading,
webSearchEnabled = true,
onWebSearchToggle,
categories = [],
onToggleCategory,
}: ChatInputProps) {
const [input, setInput] = useState("");
const textareaRef = useRef<HTMLTextAreaElement>(null);
const [webDialogOpen, setWebDialogOpen] = useState(false);
const handleKeyDown = (e: React.KeyboardEvent) => {
if (e.key === "Enter" && !e.shiftKey) {
@@ -49,8 +54,8 @@ export function ChatInput({
}, [input]);
return (
<div className="p-4 border-t bg-background">
<div className="relative flex items-end gap-2 max-w-4xl mx-auto w-full">
<div className="p-6 bg-transparent">
<div className="relative max-w-3xl mx-auto w-full">
<div className="flex-1 relative">
<Textarea
ref={textareaRef}
@@ -58,7 +63,7 @@ export function ChatInput({
onChange={(e) => setInput(e.target.value)}
onKeyDown={handleKeyDown}
placeholder="Digite sua mensagem..."
className="min-h-[50px] max-h-[200px] resize-none py-3 pr-12"
className="min-h-[48px] resize-none overflow-hidden py-4 pr-12 rounded-xl border shadow-lg"
rows={1}
/>
<div className="absolute right-2 bottom-2 flex items-center gap-2">
@@ -71,16 +76,9 @@ export function ChatInput({
variant={webSearchEnabled ? "default" : "ghost"}
onClick={() => {
if (!webSearchEnabled) {
// Ao ativar, não precisa de confirmação
onWebSearchToggle(true);
} else {
// Ao desativar, mostrar aviso
const confirmed = window.confirm(
"Atenção: Desativar a pesquisa web pode reduzir a qualidade das respostas para fatos recentes, especialmente em modelos menores. Deseja continuar?"
);
if (confirmed) {
onWebSearchToggle(false);
}
setWebDialogOpen(true);
}
}}
className="h-8 w-8"
@@ -103,6 +101,28 @@ export function ChatInput({
</Tooltip>
</TooltipProvider>
)}
{onToggleCategory && categories && categories.length > 0 && (
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button size="icon" variant="outline" className="h-8 w-8" title="Gerenciar fontes">
<ChevronUp className="h-4 w-4" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end" sideOffset={6}>
<DropdownMenuLabel>Fontes de conhecimento</DropdownMenuLabel>
<DropdownMenuSeparator />
{categories.map((cat) => (
<DropdownMenuCheckboxItem
key={cat.id}
checked={cat.enabled}
onCheckedChange={(checked) => onToggleCategory(cat.id, !!checked)}
>
{cat.name}
</DropdownMenuCheckboxItem>
))}
</DropdownMenuContent>
</DropdownMenu>
)}
{isLoading ? (
<Button size="icon" variant="destructive" onClick={onStop} className="h-8 w-8">
<Square className="h-4 w-4 fill-current" />
@@ -120,6 +140,26 @@ export function ChatInput({
</div>
</div>
</div>
<Dialog open={webDialogOpen} onOpenChange={setWebDialogOpen}>
<DialogContent>
<DialogHeader>
<DialogTitle>Desativar pesquisa web?</DialogTitle>
<DialogDescription>
Desativar a busca pode reduzir a qualidade em perguntas sobre fatos recentes, preços, notícias e conteúdo em rápida mudança.
Use com cautela — modelos locais sem contexto web tendem a alucinar mais nesses cenários.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button variant="outline" onClick={() => setWebDialogOpen(false)}>Manter ativado</Button>
<Button
variant="destructive"
onClick={() => { onWebSearchToggle?.(false); setWebDialogOpen(false); }}
>
Desativar pesquisa web
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
<div className="text-xs text-center text-muted-foreground mt-2">
OllaHub pode cometer erros. Verifique informações importantes.
</div>
@@ -128,4 +168,23 @@ export function ChatInput({
}
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from "@/components/ui/dialog";
import {
DropdownMenu,
DropdownMenuCheckboxItem,
DropdownMenuContent,
DropdownMenuLabel,
DropdownMenuSeparator,
DropdownMenuTrigger,
} from "@/components/ui/dropdown-menu";
import type { SearchCategory } from "@/store/settings-store";
+13 -9
View File
@@ -1,29 +1,35 @@
import ReactMarkdown from 'react-markdown';
import remarkGfm from 'remark-gfm';
import { useState } from 'react';
import { cn } from "@/lib/utils";
import { Message } from "@/hooks/use-chat";
import { Bot, User } from "lucide-react";
import { markdownRenderers } from "./markdown-renderers";
import { MessageActions } from "./message-actions";
import { MessageBackground } from "./message-background";
export function ChatMessage({ message }: { message: Message }) {
const isUser = message.role === 'user';
const [selected, setSelected] = useState(false);
return (
<div className={cn(
"group flex w-full gap-4 p-6 relative",
isUser ? "bg-background" : "bg-muted/30"
)}>
<MessageBackground
role={isUser ? 'user' : 'assistant'}
state={selected ? 'selected' : 'normal'}
className={cn("cursor-pointer")}
onClick={() => setSelected((v) => !v)}
tabIndex={0}
>
<div className="flex-shrink-0 mt-1">
<div className={cn(
"w-8 h-8 rounded-sm flex items-center justify-center",
"w-8 h-8 rounded-md flex items-center justify-center",
isUser ? "bg-primary text-primary-foreground" : "bg-secondary text-secondary-foreground"
)}>
{isUser ? <User className="w-5 h-5" /> : <Bot className="w-5 h-5" />}
</div>
</div>
<div className="flex-1 overflow-hidden min-w-0">
<div className="prose dark:prose-invert max-w-none break-words">
<div className="prose dark:prose-invert max-w-none break-words leading-relaxed">
<ReactMarkdown
remarkPlugins={[remarkGfm]}
components={markdownRenderers}
@@ -55,13 +61,11 @@ export function ChatMessage({ message }: { message: Message }) {
</ReactMarkdown>
</div>
</div>
{/* Actions Toolbar - Absolute positioned or flex end */}
{!isUser && (
<div className="absolute top-4 right-4">
<MessageActions content={message.content} role={message.role} />
</div>
)}
</div>
</MessageBackground>
);
}
+28 -2
View File
@@ -5,8 +5,8 @@ import {
DropdownMenuItem,
DropdownMenuTrigger,
} from "@/components/ui/dropdown-menu";
import { Copy, FileJson, FileText, MoreHorizontal } from "lucide-react";
import { useState } from "react";
import { Copy, FileJson, FileText, MoreHorizontal, Contrast } from "lucide-react";
import { useEffect, useState } from "react";
interface MessageActionsProps {
content: string;
@@ -15,6 +15,16 @@ interface MessageActionsProps {
export function MessageActions({ content, role }: MessageActionsProps) {
const [copied, setCopied] = useState(false);
const [highContrast, setHighContrast] = useState(false);
useEffect(() => {
const saved = typeof window !== 'undefined' ? localStorage.getItem('highContrast') === 'true' : false;
setHighContrast(saved);
if (typeof document !== 'undefined') {
if (saved) document.documentElement.setAttribute('data-high-contrast', 'true');
else document.documentElement.removeAttribute('data-high-contrast');
}
}, []);
const handleCopy = (text: string) => {
navigator.clipboard.writeText(text);
@@ -73,6 +83,22 @@ export function MessageActions({ content, role }: MessageActionsProps) {
<FileJson className="mr-2 h-4 w-4" />
<span>Copiar JSON</span>
</DropdownMenuItem>
<DropdownMenuItem
onClick={() => {
const next = !highContrast;
setHighContrast(next);
if (typeof document !== 'undefined') {
if (next) document.documentElement.setAttribute('data-high-contrast', 'true');
else document.documentElement.removeAttribute('data-high-contrast');
}
if (typeof window !== 'undefined') {
localStorage.setItem('highContrast', String(next));
}
}}
>
<Contrast className="mr-2 h-4 w-4" />
<span>{highContrast ? 'Desativar Alto Contraste' : 'Ativar Alto Contraste'}</span>
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
+37
View File
@@ -0,0 +1,37 @@
import * as React from "react";
import { cn } from "@/lib/utils";
type MessageRole = "user" | "assistant";
type MessageVisualState = "normal" | "highlighted" | "selected";
interface MessageBackgroundProps extends React.ComponentProps<"div"> {
role: MessageRole;
state?: MessageVisualState;
highContrast?: boolean;
}
export function MessageBackground({
role,
state = "normal",
highContrast = false,
className,
children,
...props
}: MessageBackgroundProps) {
const base = "group flex w-full gap-3 md:gap-4 relative transition-all focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring/50";
const roleCl = role === "user"
? "rounded-2xl bg-zinc-100 dark:bg-zinc-800 text-zinc-900 dark:text-zinc-100 px-4 py-3"
: "bg-transparent px-0 py-0";
const stateCl = state === "selected" ? "ring-2 ring-primary/40" : state === "highlighted" ? "ring-1 ring-accent/30" : "";
const hc = highContrast ? "hc:bg-message-high-contrast hc:text-foreground" : "";
return (
<div
data-selected={state === "selected"}
className={cn(base, roleCl, stateCl, hc, className)}
{...props}
>
{children}
</div>
);
}
+1 -1
View File
@@ -130,7 +130,7 @@ export function PromptGeneratorDialog({ defaultModel, onPromptGenerated }: Promp
value={goal}
onChange={(e) => setGoal(e.target.value)}
placeholder="Ex: Quero um assistente especialista em Python que explique conceitos complexos de forma simples..."
className="min-h-[150px]"
className="min-h-[160px]"
disabled={isGenerating}
/>
</div>
+21 -24
View File
@@ -1,6 +1,5 @@
import { Search, BookOpen, CheckCircle2, AlertCircle, Loader2, ChevronDown, ChevronUp } from 'lucide-react';
import { Button } from '@/components/ui/button';
import { Card, CardContent } from '@/components/ui/card';
import { cn } from '@/lib/utils';
import { ScrapedContent } from '@/services/webSearch';
import { SearchStatus } from '@/hooks/use-web-search';
@@ -73,24 +72,23 @@ export function SearchProgress({
const showExpandButton = status === 'completed' && sources.length > 0;
return (
<Card className={cn(
"mx-4 mb-2 border border-border/50 bg-muted/30 transition-all",
status === 'completed' && "opacity-60 hover:opacity-100",
status === 'error' && "border-destructive/30 bg-destructive/5"
<div className={cn(
"mx-6 mb-2 transition-all",
status === 'completed' && "opacity-70 hover:opacity-100",
status === 'error' && "text-destructive"
)}>
<CardContent className="p-2">
<div className="flex items-center justify-between gap-2">
<div className="flex items-center gap-2 flex-1 min-w-0">
<div className="flex-shrink-0">
{isActive ? (
<Loader2 className="w-3 h-3 animate-spin text-muted-foreground" />
) : (
<div className="w-3 h-3">
{getStatusIcon()}
</div>
)}
</div>
<span className="text-xs text-muted-foreground truncate">
<div className="p-2 pl-3 border-l-2 border-muted flex items-center justify-between gap-2 text-xs">
<div className="flex items-center gap-2 flex-1 min-w-0">
<div className="flex-shrink-0">
{isActive ? (
<Loader2 className="w-3 h-3 animate-spin text-muted-foreground" />
) : (
<div className="w-3 h-3">
{getStatusIcon()}
</div>
)}
</div>
<span className="text-muted-foreground truncate">
{getStatusText()}
</span>
</div>
@@ -100,7 +98,7 @@ export function SearchProgress({
variant="ghost"
size="sm"
onClick={toggleExpanded}
className="h-7 w-7 p-0 flex-shrink-0"
className="h-8 w-8 p-0 flex-shrink-0"
>
{isExpanded ? (
<ChevronUp className="w-4 h-4" />
@@ -109,15 +107,15 @@ export function SearchProgress({
)}
</Button>
)}
</div>
</div>
{/* Lista de fontes expandida */}
{isExpanded && sources.length > 0 && (
<div className="mt-3 pt-3 border-t space-y-2">
<div className="mt-2 ml-6 pl-3 border-l space-y-2">
{sources.map((source, idx) => (
<div
key={idx}
className="flex items-start gap-2 p-2 rounded-md bg-background/50 hover:bg-background/80 transition-colors"
className="flex items-start gap-2 p-2 rounded-md bg-muted/30 hover:bg-muted/50 transition-colors"
>
<img
src={getFaviconUrl(source.url)}
@@ -145,8 +143,7 @@ export function SearchProgress({
))}
</div>
)}
</CardContent>
</Card>
</div>
);
}
+2 -2
View File
@@ -59,8 +59,8 @@ export function ThinkingIndicator({
}
return (
<div className="px-6 py-3 border-b border-muted/50 bg-muted/20">
<div className="flex items-center gap-3 text-xs text-muted-foreground">
<div className="px-6 py-4 border-b border-muted/50 bg-muted/20">
<div className="flex items-center gap-4 text-xs text-muted-foreground">
{steps.slice(0, currentStepIndex + 1).map((step, idx) => {
const isActive = idx === currentStepIndex;
const isPast = idx < currentStepIndex;
+1 -1
View File
@@ -74,7 +74,7 @@ export function WebSearchStatus({
status === 'error' && "border-l-red-500",
(status === 'searching' || status === 'scraping') && "border-l-blue-500"
)}>
<CardContent className="p-3">
<CardContent className="p-4">
<div className="flex items-center justify-between gap-2">
<div className="flex items-center gap-2 flex-1 min-w-0">
<div className="flex-shrink-0">
+4 -4
View File
@@ -21,12 +21,12 @@ const buttonVariants = cva(
link: "text-primary underline-offset-4 hover:underline",
},
size: {
default: "h-9 px-4 py-2 has-[>svg]:px-3",
default: "h-10 px-4 py-2 has-[>svg]:px-3",
sm: "h-8 rounded-md gap-1.5 px-3 has-[>svg]:px-2.5",
lg: "h-10 rounded-md px-6 has-[>svg]:px-4",
icon: "size-9",
lg: "h-12 rounded-md px-6 has-[>svg]:px-4",
icon: "size-10",
"icon-sm": "size-8",
"icon-lg": "size-10",
"icon-lg": "size-12",
},
},
defaultVariants: {
+29
View File
@@ -0,0 +1,29 @@
import { cn } from "@/lib/utils";
type Padding = "sm" | "md" | "lg";
interface SectionProps extends React.ComponentProps<"section"> {
padding?: Padding;
fullWidth?: boolean;
}
function paddingToClass(padding?: Padding) {
switch (padding) {
case "sm": return "px-4 md:px-6 lg:px-8 py-4 md:py-6";
case "lg": return "px-4 md:px-6 lg:px-8 py-8 md:py-12";
case "md":
default:
return "px-4 md:px-6 lg:px-8 py-6 md:py-8";
}
}
export function Section({ padding = "md", fullWidth = false, className, children, ...props }: SectionProps) {
return (
<section
className={cn(paddingToClass(padding), fullWidth ? "w-full" : "w-full max-w-6xl mx-auto", className)}
{...props}
>
{children}
</section>
);
}
+65
View File
@@ -0,0 +1,65 @@
import { cn } from "@/lib/utils";
type Gap = 8 | 16 | 24 | 32 | 40 | 48;
type Direction = "vertical" | "horizontal";
type Align = "start" | "center" | "end" | "stretch";
type Justify = "start" | "center" | "end" | "between";
interface StackProps extends React.ComponentProps<"div"> {
direction?: Direction;
gap?: Gap;
align?: Align;
justify?: Justify;
}
function gapToClass(gap?: Gap) {
switch (gap) {
case 8: return "gap-2";
case 16: return "gap-4";
case 24: return "gap-6";
case 32: return "gap-8";
case 40: return "gap-10";
case 48: return "gap-12";
default: return "gap-4";
}
}
function alignToClass(align?: Align) {
switch (align) {
case "start": return "items-start";
case "center": return "items-center";
case "end": return "items-end";
case "stretch": return "items-stretch";
default: return "items-start";
}
}
function justifyToClass(justify?: Justify) {
switch (justify) {
case "start": return "justify-start";
case "center": return "justify-center";
case "end": return "justify-end";
case "between": return "justify-between";
default: return "justify-start";
}
}
export function Stack({
direction = "vertical",
gap = 16,
align = "start",
justify = "start",
className,
children,
...props
}: StackProps) {
const dirClass = direction === "vertical" ? "flex flex-col" : "flex";
return (
<div
className={cn(dirClass, gapToClass(gap), alignToClass(align), justifyToClass(justify), className)}
{...props}
>
{children}
</div>
);
}
+34
View File
@@ -0,0 +1,34 @@
# Guia de Estilo de UI – OllaHub
## Espaçamento (escala 8px)
- Tokens: 8, 16, 24, 32, 40, 48, 56, 64
- Classes Tailwind equivalentes: `2, 4, 6, 8, 10, 12, 14, 16`
- Recomendações:
- Conteúdo: `p-4` (mobile), `p-6` (tablet), `p-8` (desktop)
- Listas/Stacks: `gap-4` (16px) padrão
- Separações de seção: `py-6` (24px) ou `py-8` (32px)
## Raios de Borda
- Token base: `--radius` (10px)
- Derivados: `rounded-sm`, `rounded-md`, `rounded-lg`, `rounded-xl` mapeiam para `--radius-*`
- Interativos: usar `rounded-md` como padrão; cartões e contêineres: `rounded-lg/xl`
## Responsividade
- Breakpoints: `sm` (mobile), `md` (tablet), `lg` (desktop)
- Padrões:
- Padding: `px-4 md:px-6 lg:px-8`
- Tipografia e gaps escalam suavemente (ex: `gap-4 md:gap-6`)
## Acessibilidade
- Contraste: seguir WCAG AA; tokens base usam OKLCH calibrado
- Alto contraste: atributo global `data-high-contrast="true"` habilita variações de mensagem
- Foco: `focus-visible:ring-2` com `ring-ring/50`
## Componentes Base
- `Section`: container com padding responsivo padrão
- `Stack`: layout de coluna/linha com `gap` na escala de 8px
## Aplicação Consistente
- Mensagens: gradientes sutis, `rounded-xl`, estados (hover/selecionado)
- Botões: alturas `h-8`, `h-10`, `h-12` (32/40/48px)
- Card/Content: `p-4` (mínimo), evitar `p-3`
+6 -1
View File
@@ -1178,7 +1178,12 @@ pub fn get_or_create_browser(state: State<BrowserState>) -> Result<Arc<Browser>,
let mut browser_opt = state.lock().map_err(|e| format!("Erro ao acessar estado do browser: {}", e))?;
if let Some(ref browser) = *browser_opt {
return Ok(browser.clone());
let alive = browser.new_tab().is_ok();
if alive {
return Ok(browser.clone());
} else {
*browser_opt = None;
}
}
// Criar nova instância
+68 -6
View File
@@ -46,6 +46,21 @@ impl Default for SourcesConfig {
"pubmed.ncbi.nlm.nih.gov".to_string(),
"ieee.org".to_string(),
"acm.org".to_string(),
"sciencedirect.com".to_string(),
"springer.com".to_string(),
"jstor.org".to_string(),
"researchgate.net".to_string(),
"plos.org".to_string(),
"nature.com".to_string(),
"sciencemag.org".to_string(),
"mit.edu".to_string(),
"phys.org".to_string(),
"semanticscholar.org".to_string(),
"core.ac.uk".to_string(),
"paperswithcode.com".to_string(),
"base-search.net".to_string(),
"doaj.org".to_string(),
"worldscientific.com".to_string(),
],
enabled: true,
},
@@ -53,11 +68,26 @@ impl Default for SourcesConfig {
id: "tech".to_string(),
name: "Tech".to_string(),
base_sites: vec![
"github.com".to_string(),
"stackoverflow.com".to_string(),
"dev.to".to_string(),
"medium.com".to_string(),
"github.com".to_string(),
"reddit.com/r/programming".to_string(),
"dev.to".to_string(),
"hackernoon.com".to_string(),
"tabnews.com.br".to_string(),
"lobste.rs".to_string(),
"news.ycombinator.com".to_string(),
"hashnode.com".to_string(),
"codesandbox.io".to_string(),
"developer.mozilla.org".to_string(),
"web.dev".to_string(),
"smashingmagazine.com".to_string(),
"martinfowler.com".to_string(),
"infoq.com".to_string(),
"dzone.com".to_string(),
"freecodecamp.org".to_string(),
"digitalocean.com".to_string(),
"realpython.com".to_string(),
"css-tricks.com".to_string(),
],
enabled: true,
},
@@ -65,10 +95,26 @@ impl Default for SourcesConfig {
id: "news".to_string(),
name: "News".to_string(),
base_sites: vec![
"news.ycombinator.com".to_string(),
"techcrunch.com".to_string(),
"theverge.com".to_string(),
"techcrunch.com".to_string(),
"arstechnica.com".to_string(),
"engadget.com".to_string(),
"wired.com".to_string(),
"venturebeat.com".to_string(),
"cnet.com".to_string(),
"mashable.com".to_string(),
"gizmodo.com".to_string(),
"zdnet.com".to_string(),
"tomshardware.com".to_string(),
"anandtech.com".to_string(),
"bleepingcomputer.com".to_string(),
"theregister.com".to_string(),
"9to5mac.com".to_string(),
"androidauthority.com".to_string(),
"techradar.com".to_string(),
"fastcompany.com".to_string(),
"g1.globo.com".to_string(),
"techtudo.com.br".to_string(),
],
enabled: true,
},
@@ -78,8 +124,24 @@ impl Default for SourcesConfig {
base_sites: vec![
"bloomberg.com".to_string(),
"reuters.com".to_string(),
"financialtimes.com".to_string(),
"wsj.com".to_string(),
"ft.com".to_string(),
"cnbc.com".to_string(),
"marketwatch.com".to_string(),
"economist.com".to_string(),
"forbes.com".to_string(),
"investopedia.com".to_string(),
"seekingalpha.com".to_string(),
"infomoney.com.br".to_string(),
"valor.globo.com".to_string(),
"exame.com".to_string(),
"braziljournal.com".to_string(),
"coindesk.com".to_string(),
"cointelegraph.com".to_string(),
"moneytimes.com.br".to_string(),
"suno.com.br".to_string(),
"b3.com.br".to_string(),
"imf.org".to_string(),
],
enabled: true,
},
+80 -44
View File
@@ -54,7 +54,7 @@ fn default_max_concurrent() -> usize {
}
fn default_total_sources() -> usize {
40
100
}
/// Pool de User-Agents para rotação (evita bloqueios 429)
@@ -80,55 +80,51 @@ pub async fn search_duckduckgo(query: &str, limit: usize) -> Result<Vec<String>>
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(10))
.build()?;
let url = format!("https://html.duckduckgo.com/html/?q={}",
urlencoding::encode(query));
// User-Agent rotativo para evitar bloqueios
let user_agent = get_random_user_agent();
log::debug!("Usando User-Agent: {}", user_agent);
let res = client
.get(&url)
.header(USER_AGENT, user_agent)
.send()
.await?
.text()
.await?;
let document = Html::parse_document(&res);
// Seletores para links orgânicos do DuckDuckGo
// Tenta múltiplos seletores para maior compatibilidade
let mut links = Vec::new();
let mut offset = 0usize;
let max_pages = 5usize;
let selectors = vec![
".result__a",
".web-result__link",
"a.result__a",
];
let mut links = Vec::new();
for selector_str in selectors {
if let Ok(selector) = Selector::parse(selector_str) {
for element in document.select(&selector).take(limit) {
if let Some(href) = element.value().attr("href") {
// DuckDuckGo usa redirecionamento, extrair URL real
if let Some(real_url) = extract_real_url(href) {
if !links.contains(&real_url) {
links.push(real_url);
if links.len() >= limit {
break;
for _ in 0..max_pages {
if links.len() >= limit { break; }
let url = format!(
"https://html.duckduckgo.com/html/?q={}&s={}",
urlencoding::encode(query),
offset
);
let res = client
.get(&url)
.header(USER_AGENT, user_agent)
.send()
.await?
.text()
.await?;
{
let document = Html::parse_document(&res);
for selector_str in &selectors {
if let Ok(selector) = Selector::parse(selector_str) {
for element in document.select(&selector) {
if let Some(href) = element.value().attr("href") {
if let Some(real_url) = extract_real_url(href) {
if !links.contains(&real_url) {
links.push(real_url);
if links.len() >= limit { break; }
}
}
}
}
if links.len() >= limit { break; }
}
}
if !links.is_empty() {
break;
}
}
offset += 50;
if res.is_empty() { break; }
}
links.truncate(limit);
Ok(links)
}
@@ -485,23 +481,25 @@ pub async fn search_and_scrape_with_config(
let semaphore = Arc::new(Semaphore::new(config.max_concurrent_tabs));
let mut handles = Vec::new();
for url in urls {
for url in urls.clone() {
let browser_clone = browser.clone();
let permit = semaphore.clone().acquire_owned().await.unwrap();
let url_clone = url.clone();
let handle = tokio::task::spawn_blocking(move || {
let result = fetch_and_convert_sync(&browser_clone, &url);
drop(permit); // Libera o semáforo após processar
result
let res = fetch_and_convert_sync(&browser_clone, &url_clone);
drop(permit);
(url_clone, res)
});
handles.push(handle);
}
// 3. Coletar resultados (ignorar erros individuais, continuar com sucessos)
let mut results = Vec::new();
let mut failed_urls = Vec::new();
let mut connection_closed = false;
for handle in handles {
match handle.await {
Ok(Ok(content)) => {
Ok((_, Ok(content))) => {
// Filtrar conteúdo muito curto (< 200 caracteres)
let content_length = content.content.chars().count();
let markdown_length = content.markdown.chars().count();
@@ -516,12 +514,17 @@ pub async fn search_and_scrape_with_config(
results.push(content);
}
}
Ok(Err(e)) => {
Ok((url, Err(e))) => {
let err_msg = format!("{}", e);
if err_msg.contains("Timeout") || err_msg.contains("ERR_HTTP") {
log::debug!("URL ignorada (timeout/erro HTTP): {}", err_msg);
failed_urls.push(url);
} else {
log::warn!("Erro ao processar URL: {}", e);
if err_msg.contains("underlying connection is closed") {
connection_closed = true;
failed_urls.push(url);
}
}
}
Err(e) => {
@@ -530,6 +533,39 @@ pub async fn search_and_scrape_with_config(
}
}
if connection_closed && !failed_urls.is_empty() {
let retry_concurrency = std::cmp::min(3, config.max_concurrent_tabs.max(1));
let semaphore = Arc::new(Semaphore::new(retry_concurrency));
let browser_new = Arc::new(create_browser()?);
let mut retry_handles = Vec::new();
for url in failed_urls.clone() {
let browser_clone = browser_new.clone();
let permit = semaphore.clone().acquire_owned().await.unwrap();
let url_clone = url.clone();
let handle = tokio::task::spawn_blocking(move || {
let res = fetch_and_convert_sync(&browser_clone, &url_clone);
drop(permit);
(url_clone, res)
});
retry_handles.push(handle);
}
for h in retry_handles {
match h.await {
Ok((_, Ok(content))) => {
let content_length = content.content.chars().count();
let markdown_length = content.markdown.chars().count();
if content_length >= 200 || markdown_length >= 200 {
results.push(content);
}
}
Ok((url, Err(e))) => {
log::warn!("Falha após retry para URL {}: {}", url, e);
}
Err(e) => log::warn!("Erro na task de retry: {}", e),
}
}
}
if results.is_empty() {
log::warn!("Nenhuma fonte foi extraída com sucesso para a query: {}", query);
} else {
+1 -1
View File
@@ -159,7 +159,7 @@ const initialState = {
timeout: 10,
excludedDomains: ['youtube.com', 'linkedin.com'],
maxConcurrentTabs: 5,
totalSourcesLimit: 40,
totalSourcesLimit: 100,
categories: defaultCategories,
userCustomSites: [],
},