Add multi-provider routing policy inside the app #13

New Issue

sam · 2026-04-03T11:12:38Z

sam commented

2026-04-03 11:12:38 +00:00

Summary

Implement provider-aware routing so AMCS can choose between LiteLLM, OpenRouter, Ollama, or future backends based on task type, policy, availability, cost, or fallback rules.

Why

The implementation plan defers multi-provider routing inside the app. Right now the provider model is primarily configuration-driven; richer routing would improve resilience and operational control.

Scope ideas

Define routing policy inputs: operation type, preferred provider, fallback order, health, latency, or cost
Separate embedding-routing and metadata-routing decisions where useful
Add provider health/probe state if required
Make policy observable so operators can see why a provider was chosen
Ensure failures degrade cleanly to fallback providers
Document config and runtime behavior

Acceptance criteria

AMCS can route requests between multiple providers according to explicit policy
Fallback behavior is deterministic and observable
Embeddings and metadata extraction can use different routing rules if needed
Docs explain configuration and operational tradeoffs

Source

Deferred feature from llm/plan.md: multi-provider routing policy inside the app.

## Summary Implement provider-aware routing so AMCS can choose between LiteLLM, OpenRouter, Ollama, or future backends based on task type, policy, availability, cost, or fallback rules. ## Why The implementation plan defers multi-provider routing inside the app. Right now the provider model is primarily configuration-driven; richer routing would improve resilience and operational control. ## Scope ideas - Define routing policy inputs: operation type, preferred provider, fallback order, health, latency, or cost - Separate embedding-routing and metadata-routing decisions where useful - Add provider health/probe state if required - Make policy observable so operators can see why a provider was chosen - Ensure failures degrade cleanly to fallback providers - Document config and runtime behavior ## Acceptance criteria - AMCS can route requests between multiple providers according to explicit policy - Fallback behavior is deterministic and observable - Embeddings and metadata extraction can use different routing rules if needed - Docs explain configuration and operational tradeoffs ## Source Deferred feature from `llm/plan.md`: multi-provider routing policy inside the app.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: wdevs/amcs#13