diff --git a/docs/OpencodeAPI.md b/docs/OpencodeAPI.md new file mode 100644 index 0000000..31f79d0 --- /dev/null +++ b/docs/OpencodeAPI.md @@ -0,0 +1,266 @@ +**✅ Documento Completo: Integração com Custom Endpoint do OpenCode (Provedores Custom / OpenAI-Compatible)** + +OpenCode (https://opencode.ai) é um agente de IA para terminal que usa o **Vercel AI SDK** para se comunicar com provedores de LLM. Ele suporta nativamente provedores customizados via o pacote `@ai-sdk/openai-compatible`, que segue o padrão **OpenAI Chat Completions API** (`/v1/chat/completions`). + +Isso significa que **sua API precisa implementar exatamente o formato OpenAI-compatible** para que o OpenCode consiga conversar com ela diretamente. + +--- + +### 1. Como Configurar um Provedor Custom no OpenCode + +Crie ou edite o arquivo de configuração (recomendado: `~/.config/opencode/opencode.json` ou `opencode.json` na raiz do projeto): + +```json +{ + "$schema": "https://opencode.ai/config.json", + "provider": { + "minha-ia-custom": { + "npm": "@ai-sdk/openai-compatible", + "name": "Minha IA Custom (OpenCode)", + "options": { + "baseURL": "https://sua-api.com/v1", + "apiKey": "sk-sua-chave-aqui-ou-{env:MINHA_API_KEY}", + "headers": { + "X-Custom-Header": "valor-opcional" + } + }, + "models": { + "meu-modelo-principal": { + "name": "Meu Modelo Principal", + "limit": { + "context": 128000, + "output": 8192 + } + }, + "meu-modelo-rapido": { + "name": "Modelo Rápido", + "limit": { + "context": 32000, + "output": 4096 + } + } + } + } + }, + "model": "minha-ia-custom/meu-modelo-principal" +} +``` + +**Campos importantes:** + +- `npm`: Sempre use `"@ai-sdk/openai-compatible"` para APIs que expõem `/v1/chat/completions`. +- `baseURL`: URL base da sua API (deve terminar com `/v1`). +- `apiKey`: Pode ser valor direto ou `{env:NOME_VARIAVEL}`. +- `models`: Defina os modelos que sua API expõe. O `limit` ajuda o OpenCode a gerenciar contexto. +- `model`: Define o modelo padrão no formato `provider_id/model_id`. + +Depois de salvar, reinicie o OpenCode ou use `/models` para selecionar. + +**Comando rápido para adicionar chave:** +```bash +opencode /connect +# Escolha "Other" → digite o provider_id (ex: minha-ia-custom) → cole a API key +``` + +--- + +### 2. Como o OpenCode Envia Requisições + +O OpenCode (via AI SDK) faz chamadas **POST** para: + +``` +POST {baseURL}/chat/completions +``` + +**Headers enviados:** +- `Authorization: Bearer ` +- `Content-Type: application/json` +- Headers customizados que você definir em `options.headers` + +**Payload típico (exemplo real):** + +```json +{ + "model": "meu-modelo-principal", + "messages": [ + { + "role": "system", + "content": "Você é um assistente de código experiente..." + }, + { + "role": "user", + "content": "Crie uma função em Python que..." + } + ], + "stream": true, + "temperature": 0.7, + "max_tokens": 8192, + "tools": [ /* array de tools se o agente estiver usando ferramentas */ ], + "tool_choice": "auto" +} +``` + +**Parâmetros comuns que o OpenCode envia:** +- `messages` (array completo do histórico) +- `stream: true` (quase sempre, para UX em tempo real) +- `temperature`, `max_tokens`, `top_p` +- `tools` + `tool_choice` (quando usa agentes com ferramentas/MCP) +- `response_format` (às vezes) +- `providerOptions` (opções específicas do provedor – raras no openai-compatible) + +--- + +### 3. Como o OpenCode Recebe e Processa Respostas + +#### A. Streaming (recomendado – usado na maioria das interações) + +Sua API deve responder com `Content-Type: text/event-stream` e eventos SSE: + +``` +data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1712345678,"model":"meu-modelo-principal","choices":[{"index":0,"delta":{"content":"Olá"},"finish_reason":null}]} + +data: {"id":"chatcmpl-123","choices":[{"index":0,"delta":{"content":"! Como"},"finish_reason":null}]} + +data: [DONE] +``` + +O OpenCode processa cada chunk em tempo real. + +#### B. Resposta não-streaming (completa) + +```json +{ + "id": "chatcmpl-123", + "object": "chat.completion", + "created": 1712345678, + "model": "meu-modelo-principal", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "Resposta completa aqui..." + }, + "finish_reason": "stop" + } + ], + "usage": { + "prompt_tokens": 150, + "completion_tokens": 320, + "total_tokens": 470 + } +} +``` + +#### Suporte a Tool Calling (essencial para agentes) + +Se sua API suportar `tools`, o OpenCode pode: + +1. Enviar `tools` na requisição. +2. Receber `tool_calls` na resposta. +3. Executar a ferramenta (via MCP ou built-in). +4. Enviar o resultado de volta como mensagem `tool`. + +**Exemplo de resposta com tool call:** + +```json +{ + "choices": [{ + "message": { + "role": "assistant", + "tool_calls": [{ + "id": "call_123", + "type": "function", + "function": { + "name": "read_file", + "arguments": "{\"path\": \"src/app.ts\"}" + } + }] + } + }] +} +``` + +--- + +### 4. Como é Exibido no Chat do OpenCode (TUI) + +- **Streaming**: Texto aparece caractere por caractere com efeito de digitação. +- **Markdown**: Suporte completo a código (syntax highlight), listas, tabelas, etc. +- **Tool Calls**: Mostra “Executando ferramenta: read_file...” + resultado. +- **Multi-turn**: Mantém histórico completo na conversa. +- **Erros**: Mostra mensagens de erro claras no terminal. +- **Contexto**: Respeita os limites de `context` e `output` que você definiu no config. + +O OpenCode usa o TUI (Terminal User Interface) baseado em React/Ink para renderizar tudo de forma bonita. + +--- + +### 5. Requisitos Mínimos da Sua API para Funcionar Perfeitamente + +| Recurso | Obrigatório? | Recomendado? | Observação | +|----------------------|--------------|--------------|----------| +| `/v1/chat/completions` | Sim | - | Endpoint principal | +| Streaming (SSE) | Sim | - | Essencial para UX | +| Tool Calling | Não | Sim | Para agentes completos | +| System messages | Sim | - | - | +| Multi-turn history | Sim | - | - | +| Usage stats | Não | Sim | Mostra tokens gastos | +| Error handling | Sim | - | Retorne JSON de erro padrão OpenAI | + +**Endpoint exato esperado:** +`POST https://sua-api.com/v1/chat/completions` + +--- + +### 6. Exemplo Completo de Configuração + Teste + +1. Configure o `opencode.json` como acima. +2. Rode: + ```bash + opencode + ``` +3. Digite algo ou use `/models` para selecionar seu provedor. +4. Teste com: + ```bash + curl -X POST https://sua-api.com/v1/chat/completions \ + -H "Authorization: Bearer sk-..." \ + -H "Content-Type: application/json" \ + -d '{"model":"meu-modelo-principal","messages":[{"role":"user","content":"Olá"}],"stream":true}' + ``` + +--- + +### 7. Troubleshooting Comum + +- **“Route not found” ou requests indo para lugar errado** → Verifique se `baseURL` termina com `/v1` e não com `/v1/chat/completions`. +- **Custom provider não aparece** → Certifique-se que o ID do provider no `/connect` bate com a chave no `opencode.json`. +- **Sem streaming / resposta lenta** → Implemente SSE corretamente. +- **Tool calls não funcionam** → Sua API precisa retornar `tool_calls` no formato OpenAI exato. +- **Erro de autenticação** → Use `apiKey` ou `headers` corretamente. + +--- + +### Referências Oficiais + +- **Docs Principais**: https://opencode.ai/docs/providers/ +- **Models**: https://opencode.ai/docs/models/ +- **Config**: https://opencode.ai/docs/config/ +- **Schema de Config**: https://opencode.ai/config.json +- **GitHub**: https://github.com/anomalyco/opencode +- **AI SDK OpenAI-Compatible**: https://ai-sdk.dev/providers/openai-compatible-providers + +--- + +**Resumo Final para Você (que está construindo a API):** + +Sua API **precisa ser 100% compatível** com o endpoint OpenAI `/v1/chat/completions` (streaming + tool calling recomendado). +O OpenCode vai: +1. Configurar via `opencode.json` (baseURL + apiKey) +2. Enviar requisições padrão OpenAI +3. Receber streaming ou resposta completa +4. Renderizar tudo lindamente no TUI com suporte a ferramentas + +Se você implementar exatamente o padrão OpenAI Chat Completions, vai funcionar perfeitamente com o OpenCode. + +Quer que eu monte um **exemplo de servidor mínimo** (em Node.js/Fastify ou Python/FastAPI) que já funcione com o OpenCode? Posso criar um template completo para você. \ No newline at end of file diff --git a/src/server/modules/proxy/proxy.service.ts b/src/server/modules/proxy/proxy.service.ts index fce7eca..1457270 100644 --- a/src/server/modules/proxy/proxy.service.ts +++ b/src/server/modules/proxy/proxy.service.ts @@ -33,6 +33,7 @@ import { import { getMaxOutputTokens, getThinkingBudget } from '../../../lib/geminiNexus/ModelSpecs'; import { resolveRequestUserAgent } from './request-user-agent'; import { UpstreamRequestError } from './clients/upstream-error'; +import { trafficLogger } from '../../../utils/traffic-logger'; @Injectable() export class ProxyService { @@ -691,7 +692,7 @@ export class ProxyService { async handleChatCompletions( request: OpenAIChatRequest, - _requestId?: string, + requestId?: string, headerSessionKey?: string, ): Promise> { // Prefer the x-session-affinity header value (sent by OpenCode) over body-level session ids. @@ -745,13 +746,14 @@ export class ProxyService { // Use v1internal API (same as Anthropic handler) if (request.stream) { try { + trafficLogger.logUpstream(requestId || 'unknown', '/v1/chat/completions', geminiBody, extraHeaders, { model: effectiveTargetModel }); const stream = await this.geminiClient.streamGenerateInternal( geminiBody, token.token.access_token, token.token.upstream_proxy_url, extraHeaders, ); - return this.processStreamResponse(stream, request.model); + return this.processStreamResponse(stream, request.model, requestId); } catch (streamError) { this.logger.warn( `Stream path failed for model=${request.model}; falling back to non-stream generation: ${ @@ -759,12 +761,14 @@ export class ProxyService { }`, ); + trafficLogger.logUpstream(requestId || 'unknown', '/v1/chat/completions', geminiBody, extraHeaders, { model: effectiveTargetModel }); const response = await this.generateInternalWithStreamFallback( geminiBody, token.token.access_token, token.token.upstream_proxy_url, extraHeaders, ); + trafficLogger.logUpstreamResponse(requestId || 'unknown', '/v1/chat/completions', 200, response, 0); this.logger.log( `Upstream response snippet after stream fallback: ${JSON.stringify(response).substring(0, 500)}`, ); @@ -804,21 +808,24 @@ export class ProxyService { fallbackBody.model = effectiveTargetModel; this.applyInternalGenerationConstraints(fallbackBody, effectiveTargetModel, token.id); if (request.stream) { + trafficLogger.logUpstream(requestId || 'unknown', '/v1/chat/completions', fallbackBody, extraHeaders, { model: effectiveTargetModel }); const stream = await this.geminiClient.streamGenerateInternal( fallbackBody, token.token.access_token, token.token.upstream_proxy_url, extraHeaders, ); - return this.processStreamResponse(stream, request.model); + return this.processStreamResponse(stream, request.model, requestId); } + trafficLogger.logUpstream(requestId || 'unknown', '/v1/chat/completions', fallbackBody, extraHeaders, { model: effectiveTargetModel }); const response = await this.generateInternalWithStreamFallback( fallbackBody, token.token.access_token, token.token.upstream_proxy_url, extraHeaders, ); + trafficLogger.logUpstreamResponse(requestId || 'unknown', '/v1/chat/completions', 200, response, 0); const claudeResponse = transformResponse(response); return this.convertClaudeToOpenAIResponse(claudeResponse, request.model); } catch (fallbackErr) { @@ -957,17 +964,38 @@ export class ProxyService { private processStreamResponse( upstreamStream: NodeJS.ReadableStream, model: string, + requestId?: string, ): Observable { return new Observable((subscriber) => { const decoder = new TextDecoder(); let buffer = ''; let hasEmittedChunk = false; + let hasSentRoleChunk = false; let hasSentDone = false; let toolCallIndex = 0; const streamId = `chatcmpl-${uuidv4()}`; const created = Math.floor(Date.now() / 1000); + const pushRoleChunk = (): void => { + if (hasSentRoleChunk) return; + hasSentRoleChunk = true; + const roleChunk = { + id: streamId, + object: 'chat.completion.chunk', + created, + model, + choices: [ + { + index: 0, + delta: { role: 'assistant', content: '' }, + finish_reason: null, + }, + ], + }; + subscriber.next(`data: ${JSON.stringify(roleChunk)}\n\n`); + }; + const pushChunk = (payload: Record): void => { hasEmittedChunk = true; subscriber.next(`data: ${JSON.stringify(payload)}\n\n`); @@ -987,7 +1015,10 @@ export class ProxyService { try { const json = JSON.parse(dataStr); - const candidate = json.candidates?.[0]; + trafficLogger.logUpstreamResponse(requestId || 'unknown', '/v1/chat/completions', 200, json, 0, { rawLine: trimmed }); + // v1internal API wraps response in {"response": {"candidates": [...]}} + const responsePayload = json.response ?? json; + const candidate = responsePayload.candidates?.[0]; const parts = candidate?.content?.parts || []; for (const part of parts) { @@ -1007,11 +1038,13 @@ export class ProxyService { }, ], }; + pushRoleChunk(); pushChunk(reasoningChunk); continue; } if (part.functionCall) { + pushRoleChunk(); const currentToolIndex = toolCallIndex++; const toolCallChunk = { id: streamId, @@ -1059,11 +1092,13 @@ export class ProxyService { }, ], }; + pushRoleChunk(); pushChunk(imageChunk); continue; } if (part.text) { + pushRoleChunk(); const contentChunk = { id: streamId, object: 'chat.completion.chunk', diff --git a/src/tests/unit/proxy-controller.integration.test.ts b/src/tests/unit/proxy-controller.integration.test.ts index 653bcc2..215fb47 100644 --- a/src/tests/unit/proxy-controller.integration.test.ts +++ b/src/tests/unit/proxy-controller.integration.test.ts @@ -11,6 +11,10 @@ function createReplyMock() { return reply; } +function createReqMock() { + return { headers: {} } as any; +} + describe('ProxyController Integration', () => { it('routes Claude OpenAI requests to protocol parity path', async () => { const proxyService = { @@ -19,6 +23,7 @@ describe('ProxyController Integration', () => { }; const controller = new ProxyController(proxyService as any); const reply = createReplyMock(); + const req = createReqMock(); await controller.chatCompletions( { @@ -26,6 +31,7 @@ describe('ProxyController Integration', () => { stream: false, messages: [{ role: 'user', content: 'hello' }], } as any, + req, reply as any, ); @@ -42,6 +48,7 @@ describe('ProxyController Integration', () => { }; const controller = new ProxyController(proxyService as any); const reply = createReplyMock(); + const req = createReqMock(); await controller.chatCompletions( { @@ -49,6 +56,7 @@ describe('ProxyController Integration', () => { stream: true, messages: [{ role: 'user', content: 'hello' }], } as any, + req, reply as any, ); diff --git a/src/tests/unit/proxy-retry-mock.test.ts b/src/tests/unit/proxy-retry-mock.test.ts index ee7868a..93c1334 100644 --- a/src/tests/unit/proxy-retry-mock.test.ts +++ b/src/tests/unit/proxy-retry-mock.test.ts @@ -648,6 +648,7 @@ describe('ProxyService Protocol Parity Fixtures', () => { }); const output = chunks.join(''); + expect(output).toContain('"role":"assistant"'); expect(output).toContain('"reasoning_content":"reasoning text"'); expect(output).toContain('"tool_calls"'); expect(output).toContain('"content":"final answer"'); @@ -689,6 +690,7 @@ describe('ProxyService Protocol Parity Fixtures', () => { expect(streamResult.completed).not.toBe(true); expect(streamResult.error?.message).toContain('socket hang up'); + expect(chunks.join('')).toContain('"role":"assistant"'); expect(chunks.join('')).toContain('"content":"partial output"'); expect(chunks.join('')).not.toContain('data: [DONE]'); }); diff --git a/src/tests/unit/proxy-stream-debug.test.ts b/src/tests/unit/proxy-stream-debug.test.ts new file mode 100644 index 0000000..477b0da --- /dev/null +++ b/src/tests/unit/proxy-stream-debug.test.ts @@ -0,0 +1,329 @@ +import { describe, it, expect, vi } from 'vitest'; +import { EventEmitter } from 'events'; +import { ProxyService } from '../../server/modules/proxy/proxy.service'; + +// Minimal mocks so ProxyService can instantiate +const mockTokenManager = { + getNextToken: vi.fn(), + markAsRateLimited: vi.fn(), + markAsForbidden: vi.fn(), + forceRefreshToken: vi.fn(), + markFromUpstreamError: vi.fn(), + recordParityError: vi.fn(), + getModelOutputLimitForAccount: vi.fn(), + getModelThinkingBudgetForAccount: vi.fn(), + resolveDynamicModelForAccount: vi.fn((_token: unknown, model: string) => model), +}; +const mockGeminiClient = { streamGenerateInternal: vi.fn(), generateInternal: vi.fn() }; + +class TestableProxyService extends ProxyService { + constructor() { + super(mockTokenManager as any, mockGeminiClient as any); + } + + public testProcessStreamResponse( + stream: any, + model: string = 'gemini-3.1-pro-low', + requestId?: string, + ) { + return (this as any).processStreamResponse(stream, model, requestId); + } +} + +function collectChunks(observable: any): Promise { + return new Promise((resolve, reject) => { + const chunks: string[] = []; + observable.subscribe({ + next: (chunk: string) => chunks.push(chunk), + error: reject, + complete: () => resolve(chunks), + }); + }); +} + +function emitSseLine(stream: EventEmitter, payload: object): void { + stream.emit('data', Buffer.from(`data: ${JSON.stringify(payload)}\n`)); +} + +describe('ProxyService Stream Debug', () => { + it('emits role chunk followed by content for a simple text response', async () => { + const service = new TestableProxyService(); + const stream = new EventEmitter(); + const observable = service.testProcessStreamResponse(stream, 'gemini-3.1-pro-low'); + + const promise = collectChunks(observable); + + emitSseLine(stream, { + candidates: [ + { + content: { + parts: [{ text: 'Hello world' }], + }, + }, + ], + }); + + emitSseLine(stream, { + candidates: [ + { + content: { + parts: [{ text: '!' }], + }, + finishReason: 'STOP', + }, + ], + }); + + stream.emit('end'); + + const chunks = await promise; + console.log('=== Simple text response chunks ==='); + chunks.forEach((c, i) => console.log(`Chunk ${i}:`, c.trim())); + + // Should have at least role + content + finish + DONE + expect(chunks.length).toBeGreaterThanOrEqual(4); + expect(chunks[0]).toContain('"role":"assistant"'); + expect(chunks.some((c) => c.includes('"content":"Hello world"'))).toBe(true); + expect(chunks.some((c) => c.includes('"finish_reason":"stop"'))).toBe(true); + expect(chunks[chunks.length - 1]).toContain('[DONE]'); + }); + + it('handles empty parts array gracefully', async () => { + const service = new TestableProxyService(); + const stream = new EventEmitter(); + const observable = service.testProcessStreamResponse(stream, 'gemini-3.1-pro-low'); + + const promise = collectChunks(observable); + + emitSseLine(stream, { + candidates: [ + { + content: { + parts: [], + }, + finishReason: 'STOP', + }, + ], + }); + + stream.emit('end'); + + const chunks = await promise; + console.log('=== Empty parts response chunks ==='); + chunks.forEach((c, i) => console.log(`Chunk ${i}:`, c.trim())); + + // With empty parts, only finish + DONE should be emitted + expect(chunks.length).toBeGreaterThanOrEqual(2); + expect(chunks.some((c) => c.includes('"finish_reason":"stop"'))).toBe(true); + expect(chunks[chunks.length - 1]).toContain('[DONE]'); + }); + + it('handles reasoning content (thought parts)', async () => { + const service = new TestableProxyService(); + const stream = new EventEmitter(); + const observable = service.testProcessStreamResponse(stream, 'gemini-3.1-pro-low'); + + const promise = collectChunks(observable); + + emitSseLine(stream, { + candidates: [ + { + content: { + parts: [{ thought: true, text: 'Let me think...' }], + }, + }, + ], + }); + + emitSseLine(stream, { + candidates: [ + { + content: { + parts: [{ text: 'Final answer' }], + }, + finishReason: 'STOP', + }, + ], + }); + + stream.emit('end'); + + const chunks = await promise; + console.log('=== Reasoning response chunks ==='); + chunks.forEach((c, i) => console.log(`Chunk ${i}:`, c.trim())); + + expect(chunks.some((c) => c.includes('"reasoning_content":"Let me think..."'))).toBe(true); + expect(chunks.some((c) => c.includes('"content":"Final answer"'))).toBe(true); + }); + + it('handles function call parts', async () => { + const service = new TestableProxyService(); + const stream = new EventEmitter(); + const observable = service.testProcessStreamResponse(stream, 'gemini-3.1-pro-low'); + + const promise = collectChunks(observable); + + emitSseLine(stream, { + candidates: [ + { + content: { + parts: [ + { + functionCall: { + id: 'fc-1', + name: 'get_weather', + args: { city: 'London' }, + }, + }, + ], + }, + finishReason: 'STOP', + }, + ], + }); + + stream.emit('end'); + + const chunks = await promise; + console.log('=== Function call response chunks ==='); + chunks.forEach((c, i) => console.log(`Chunk ${i}:`, c.trim())); + + expect(chunks.some((c) => c.includes('"tool_calls"'))).toBe(true); + expect(chunks.some((c) => c.includes('"finish_reason":"tool_calls"'))).toBe(true); + }); + + it('handles multiple SSE lines in a single buffer', async () => { + const service = new TestableProxyService(); + const stream = new EventEmitter(); + const observable = service.testProcessStreamResponse(stream, 'gemini-3.1-pro-low'); + + const promise = collectChunks(observable); + + // Simulate two complete SSE lines in one data chunk + const payload1 = JSON.stringify({ + candidates: [{ content: { parts: [{ text: 'First' }] } }], + }); + const payload2 = JSON.stringify({ + candidates: [{ content: { parts: [{ text: 'Second' }] }, finishReason: 'STOP' }], + }); + stream.emit('data', Buffer.from(`data: ${payload1}\ndata: ${payload2}\n`)); + stream.emit('end'); + + const chunks = await promise; + console.log('=== Multi-line buffer chunks ==='); + chunks.forEach((c, i) => console.log(`Chunk ${i}:`, c.trim())); + + expect(chunks.some((c) => c.includes('"content":"First"'))).toBe(true); + expect(chunks.some((c) => c.includes('"content":"Second"'))).toBe(true); + expect(chunks[chunks.length - 1]).toContain('[DONE]'); + }); + + it('handles partial SSE lines across multiple data events', async () => { + const service = new TestableProxyService(); + const stream = new EventEmitter(); + const observable = service.testProcessStreamResponse(stream, 'gemini-3.1-pro-low'); + + const promise = collectChunks(observable); + + const payload = JSON.stringify({ + candidates: [{ content: { parts: [{ text: 'Split' }] }, finishReason: 'STOP' }], + }); + const line = `data: ${payload}\n`; + + // Split the line across two data events + const mid = Math.floor(line.length / 2); + stream.emit('data', Buffer.from(line.slice(0, mid))); + stream.emit('data', Buffer.from(line.slice(mid))); + stream.emit('end'); + + const chunks = await promise; + console.log('=== Split-line buffer chunks ==='); + chunks.forEach((c, i) => console.log(`Chunk ${i}:`, c.trim())); + + expect(chunks.some((c) => c.includes('"content":"Split"'))).toBe(true); + expect(chunks[chunks.length - 1]).toContain('[DONE]'); + }); + + it('handles Gemini v1internal wrapped response format (response.candidates)', async () => { + const service = new TestableProxyService(); + const stream = new EventEmitter(); + const observable = service.testProcessStreamResponse(stream, 'gemini-3.1-pro-low'); + + const promise = collectChunks(observable); + + // v1internal API wraps the payload in {"response": {"candidates": [...]}} + stream.emit( + 'data', + Buffer.from( + `data: ${JSON.stringify({ + response: { + candidates: [ + { + content: { + parts: [{ text: 'Hello from wrapped' }], + }, + }, + ], + }, + traceId: 'abc123', + metadata: {}, + })}\n`, + ), + ); + + stream.emit( + 'data', + Buffer.from( + `data: ${JSON.stringify({ + response: { + candidates: [ + { + content: { + parts: [{ text: ' response' }], + }, + finishReason: 'STOP', + }, + ], + }, + traceId: 'abc123', + metadata: {}, + })}\n`, + ), + ); + + stream.emit('end'); + + const chunks = await promise; + console.log('=== Wrapped response format chunks ==='); + chunks.forEach((c, i) => console.log(`Chunk ${i}:`, c.trim())); + + expect(chunks.some((c) => c.includes('"content":"Hello from wrapped"'))).toBe(true); + expect(chunks.some((c) => c.includes('"content":" response"'))).toBe(true); + expect(chunks[chunks.length - 1]).toContain('[DONE]'); + }); + + it('handles Gemini streaming chunks with no content field', async () => { + const service = new TestableProxyService(); + const stream = new EventEmitter(); + const observable = service.testProcessStreamResponse(stream, 'gemini-3.1-pro-low'); + + const promise = collectChunks(observable); + + // Sometimes Gemini sends candidates without content at all + emitSseLine(stream, { + candidates: [{ index: 0 }], + }); + + emitSseLine(stream, { + candidates: [{ content: { parts: [{ text: 'Late content' }] }, finishReason: 'STOP' }], + }); + + stream.emit('end'); + + const chunks = await promise; + console.log('=== No-content-then-late-content chunks ==='); + chunks.forEach((c, i) => console.log(`Chunk ${i}:`, c.trim())); + + expect(chunks.some((c) => c.includes('"content":"Late content"'))).toBe(true); + }); +}); diff --git a/src/utils/traffic-logger.ts b/src/utils/traffic-logger.ts index 0e8874d..c45f1db 100644 --- a/src/utils/traffic-logger.ts +++ b/src/utils/traffic-logger.ts @@ -87,9 +87,22 @@ class TrafficLogger { console.error('Traffic DailyRotateFile transport error', error); }); + const consoleFormat = winston.format.combine( + winston.format.colorize(), + winston.format.printf(({ level, message }) => { + return `[TRAFFIC] ${level}: ${message}`; + }), + ); + this.winstonLogger = winston.createLogger({ level: 'debug', - transports: [rotateTransport], + transports: [ + rotateTransport, + new winston.transports.Console({ + level: 'debug', + format: consoleFormat, + }), + ], exitOnError: false, }); }