wdevs/amcs
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Add multi-provider routing policy inside the app #13

New Issue
Closed
opened 2026-04-03 11:12:38 +00:00 by sam · 1 comment
sam commented 2026-04-03 11:12:38 +00:00
Member
Copy Link
Copy Source

Summary

Implement provider-aware routing so AMCS can choose between LiteLLM, OpenRouter, Ollama, or future backends based on task type, policy, availability, cost, or fallback rules.

Why

The implementation plan defers multi-provider routing inside the app. Right now the provider model is primarily configuration-driven; richer routing would improve resilience and operational control.

Scope ideas

  • Define routing policy inputs: operation type, preferred provider, fallback order, health, latency, or cost
  • Separate embedding-routing and metadata-routing decisions where useful
  • Add provider health/probe state if required
  • Make policy observable so operators can see why a provider was chosen
  • Ensure failures degrade cleanly to fallback providers
  • Document config and runtime behavior

Acceptance criteria

  • AMCS can route requests between multiple providers according to explicit policy
  • Fallback behavior is deterministic and observable
  • Embeddings and metadata extraction can use different routing rules if needed
  • Docs explain configuration and operational tradeoffs

Source

Deferred feature from llm/plan.md: multi-provider routing policy inside the app.

## Summary Implement provider-aware routing so AMCS can choose between LiteLLM, OpenRouter, Ollama, or future backends based on task type, policy, availability, cost, or fallback rules. ## Why The implementation plan defers multi-provider routing inside the app. Right now the provider model is primarily configuration-driven; richer routing would improve resilience and operational control. ## Scope ideas - Define routing policy inputs: operation type, preferred provider, fallback order, health, latency, or cost - Separate embedding-routing and metadata-routing decisions where useful - Add provider health/probe state if required - Make policy observable so operators can see why a provider was chosen - Ensure failures degrade cleanly to fallback providers - Document config and runtime behavior ## Acceptance criteria - AMCS can route requests between multiple providers according to explicit policy - Fallback behavior is deterministic and observable - Embeddings and metadata extraction can use different routing rules if needed - Docs explain configuration and operational tradeoffs ## Source Deferred feature from `llm/plan.md`: multi-provider routing policy inside the app.
sgcommand commented 2026-04-21 22:10:56 +00:00
Member
Copy Link
Copy Source

Latest commit from today:

  • 894fa3fc1d95661c645cefac0f30b446cad25122
  • fix: include project names in thought text search
Latest commit from today: - `894fa3fc1d95661c645cefac0f30b446cad25122` - `fix: include project names in thought text search`
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
architecture
debugging
integration
planned
ux
Compat/Breaking
Breaking change that won't be backward compatible
 Kind/Bug
Something is not working
 Kind/Documentation
Documentation changes
 Kind/Enhancement
Improve existing functionality
 Kind/Feature
New functionality
 Kind/Security
This is security issue
 Kind/Testing
Issue or pull request related to testing
Priority
Critical
1
The priority is critical
Priority
High
2
The priority is high
Priority
Low
4
The priority is low
Priority
Medium
3
The priority is medium
Reviewed
Confirmed
1
Issue has been confirmed
Reviewed
Duplicate
2
This issue or pull request already exists
Reviewed
Invalid
3
Invalid issue
Reviewed
Won't Fix
3
This issue won't be fixed
Status
Abandoned
3
Somebody has started to work on this but abandoned work
Status
Blocked
1
Something is blocking this issue or pull request
Status
Need More Info
2
Feedback is required to reproduce issue or to continue work
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
hein (Hein) root sam sgcommand (SG Command) warkanum (Warky)
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: wdevs/amcs#13