wdevs/amcs
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Add metadata extraction retry queue and scheduled sweeper #27

New Issue
Closed
opened 2026-04-13 20:13:37 +00:00 by sgcommand · 0 comments
sgcommand commented 2026-04-13 20:13:37 +00:00
Member
Copy Link
Copy Source

Problem

AMCS appears to have embedding backfill behavior, but metadata extraction retry/sweeper coverage is missing or incomplete. When metadata extraction fails or times out, there should be a reliable background path to retry later.

Impact

  • Thoughts can remain permanently missing metadata
  • Failures may require manual intervention
  • Background enrichment state is inconsistent across records

Expected behavior

Metadata extraction failures should be tracked per thought and retried by a recurring background sweeper with backoff.

Acceptance criteria

  • Metadata extraction attempts persist per-thought status fields such as pending/running/succeeded/failed
  • Failed or timed-out metadata work records retry count, last error, and next retry time
  • A recurring sweeper finds eligible thoughts and retries metadata extraction without duplicate processing
  • Retry behavior uses bounded backoff and stops hammering persistent failures
  • Operators can inspect counts of pending/failed/retried metadata jobs via logs or metrics
## Problem AMCS appears to have embedding backfill behavior, but metadata extraction retry/sweeper coverage is missing or incomplete. When metadata extraction fails or times out, there should be a reliable background path to retry later. ## Impact - Thoughts can remain permanently missing metadata - Failures may require manual intervention - Background enrichment state is inconsistent across records ## Expected behavior Metadata extraction failures should be tracked per thought and retried by a recurring background sweeper with backoff. ## Acceptance criteria - Metadata extraction attempts persist per-thought status fields such as pending/running/succeeded/failed - Failed or timed-out metadata work records retry count, last error, and next retry time - A recurring sweeper finds eligible thoughts and retries metadata extraction without duplicate processing - Retry behavior uses bounded backoff and stops hammering persistent failures - Operators can inspect counts of pending/failed/retried metadata jobs via logs or metrics
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
architecture
debugging
integration
planned
ux
Compat/Breaking
Breaking change that won't be backward compatible
 Kind/Bug
Something is not working
 Kind/Documentation
Documentation changes
 Kind/Enhancement
Improve existing functionality
 Kind/Feature
New functionality
 Kind/Security
This is security issue
 Kind/Testing
Issue or pull request related to testing
Priority
Critical
1
The priority is critical
Priority
High
2
The priority is high
Priority
Low
4
The priority is low
Priority
Medium
3
The priority is medium
Reviewed
Confirmed
1
Issue has been confirmed
Reviewed
Duplicate
2
This issue or pull request already exists
Reviewed
Invalid
3
Invalid issue
Reviewed
Won't Fix
3
This issue won't be fixed
Status
Abandoned
3
Somebody has started to work on this but abandoned work
Status
Blocked
1
Something is blocking this issue or pull request
Status
Need More Info
2
Feedback is required to reproduce issue or to continue work
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
hein (Hein) root sam sgcommand (SG Command) warkanum (Warky)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: wdevs/amcs#27