wdevs/vecna
5
0
Fork 0
mirror of https://github.com/Warky-Devs/vecna.git synced 2026-08-07 10:47:38 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(server): add passthrough proxy for OpenAI-compatible endpoints

Browse Source 
* implement proxy handler for various OpenAI API routes
* add error handling for request body and response streaming
* introduce new error response format for API compatibility
* add tests for recover middleware to handle panics gracefully
This commit is contained in:
warkanum
2026-08-01 20:15:59 +02:00
parent 52902a8383
commit 1953a4f4f9
11 changed files with 1087 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+31 -1
View File
@@ -271,12 +271,17 @@ When unsure, run `vecna test` before and after and compare the reported L2 norm.
```
POST /v1/embeddings
GET /v1/embeddings?input=text&input=text2&model=nomic-embed-text
Authorization: Bearer <api_key>
Content-Type: application/json
{"input": "text or array of texts", "model": "nomic-embed-text"}
```
Embeddings are the only endpoint that gets dimension adaptation. Everything below is
forwarded verbatim (no adaptation) — request/response shape is whatever the backing model's
API defines.
### Google Gemini-compatible
```
@@ -284,14 +289,39 @@ POST /v1/models/{model}:embedContent
POST /v1/models/{model}:batchEmbedContents
```
### Generic passthrough
Target is chosen from the request body's `"model"` field (must name a key under
`forward.targets`), falling back to `forward.default`. `"stream": true` requests are
relayed as SSE without buffering.
| Method | Path | Notes |
|--------|-------------------------------|-------------------------------|
| GET | `/v1/models` | uses `forward.default` |
| GET | `/v1/models/{model}` | |
| POST | `/v1/chat/completions` | streaming supported |
| POST | `/v1/completions` | streaming supported |
| POST | `/v1/moderations` | |
| POST | `/v1/images/generations` | |
| POST | `/v1/images/edits` | multipart/form-data |
| POST | `/v1/images/variations` | multipart/form-data |
| POST | `/v1/audio/speech` | |
| POST | `/v1/audio/transcriptions` | multipart/form-data |
| POST | `/v1/audio/translations` | multipart/form-data |
| POST | `/v1/rerank` | Cohere/Infinity/vLLM-style |
### Extra-map routes
Serve the same backing model with a different adapter per endpoint. The `{mapping}` segment matches a key in `extra_maps`.
Serve the same backing model with a different adapter (or forced target) per endpoint.
The `{mapping}` segment matches a key in `extra_maps`. Every route above — embeddings,
Google, and generic passthrough — also exists under `/map/{mapping}/...`, e.g.:
```
POST /map/{mapping}/v1/embeddings
POST /map/{mapping}/v1/models/{model}:embedContent
POST /map/{mapping}/v1/models/{model}:batchEmbedContents
POST /map/{mapping}/v1/chat/completions
GET /map/{mapping}/v1/models
```
All extra-map routes require the same authentication as the standard API routes.