* Introduce project_personas table with foreign keys to projects and agent_personas * Add project_skills table with foreign key to projects and agent_skills * Include override boolean field in agent_persona_skills and project_skills * Update schema and migration files to reflect new tables and fields * Enhance CORS handling to reflect request origin
This commit is contained in:
+99
@@ -1,5 +1,104 @@
|
||||
# AMCS TODO
|
||||
|
||||
## Memory Consolidation and Retrieval Quality
|
||||
|
||||
No prior consolidation implementation notes were found in the project. This roadmap starts clean and follows AMCS's current Go, PostgreSQL/pgvector, thoughts, learnings, projects, and model-specific embeddings architecture.
|
||||
|
||||
### Current-shape decisions
|
||||
|
||||
- Use bigint thought IDs for foreign keys and queue references, matching the current schema. Keep thought GUIDs as external identifiers rather than introducing UUID foreign keys without a specific interoperability need.
|
||||
- Embeddings are stored in the separate, model-specific `embeddings` table. Similarity queries and indexes must include the embedding model and project scope where applicable.
|
||||
- Verify the existing pgvector index strategy before adding another HNSW index. Add or change an index only when the current generated schema and query plan show it is needed.
|
||||
- All consolidation writes are soft, auditable, idempotent, and reversible.
|
||||
|
||||
### Phase 0 - Schema additions
|
||||
|
||||
Add to `thoughts`:
|
||||
|
||||
- `confidence smallint`
|
||||
- `source text`
|
||||
- `valid_until timestamptz`
|
||||
- `superseded_by bigint` referencing `thoughts.id`
|
||||
- `salience real`
|
||||
- `outcome text` constrained to `confirmed`, `refuted`, or null
|
||||
|
||||
Add tables:
|
||||
|
||||
- `consolidation_queue(id, kind, thought_a, thought_b, status, created_at)`
|
||||
- `consolidation_audit(id, action, before jsonb, after jsonb, actor, ts)`
|
||||
|
||||
The audit record must contain enough information to reverse every consolidation mutation. Define queue uniqueness and status transitions so retries cannot enqueue or process the same pair twice.
|
||||
|
||||
Confidence must be available through the existing admin thought resource. Return both the normalized numeric value and enough provenance to explain whether it was user-supplied, heuristic, outcome-derived, or consolidation-derived. Existing thoughts need a documented neutral default or backfill policy rather than silently appearing maximally confident.
|
||||
|
||||
### Phase 1 - Algorithmic candidate generation
|
||||
|
||||
Run nightly SQL candidate generation without an LLM:
|
||||
|
||||
- Near duplicates: self-join model-compatible embeddings and enqueue canonical thought pairs where `1 - (a.embedding <=> b.embedding) > 0.92`.
|
||||
- Contradiction suspects: combine high cosine similarity with low `pg_trgm` `similarity()` and enqueue as `kind = 'contradiction'`.
|
||||
- Decay: demote thoughts past `valid_until`; apply recency weighting during retrieval rather than rewriting content.
|
||||
|
||||
Candidate queries must de-duplicate ordered pairs, exclude already superseded thoughts, respect project boundaries, and avoid comparing embeddings from different models.
|
||||
|
||||
### Phase 2 - Retrieval ranking (ship first)
|
||||
|
||||
Change `recall_context` from cosine-only top-k ranking to a bounded composite score based on:
|
||||
|
||||
```text
|
||||
cosine_similarity * exp(-lambda * age_days) * confidence_weight * outcome_weight
|
||||
```
|
||||
|
||||
Exclude superseded and low-salience thoughts from default recall, with explicit options for diagnostic retrieval. Tune defaults conservatively and preserve the current full-text fallback when embeddings are unavailable.
|
||||
|
||||
This phase should ship first because it provides the largest recall-quality improvement without requiring consolidation writes or an adjudication model.
|
||||
|
||||
### Phase 3 - LLM adjudication worker
|
||||
|
||||
Add a resumable Go worker that claims queued pairs and calls a low-cost configured provider only for flagged candidates. Support two structured prompts:
|
||||
|
||||
- Merge: produce one normalized schema entry from genuine duplicates.
|
||||
- Resolve conflict: verify a real contradiction and select or synthesize the supported result.
|
||||
|
||||
Writes remain soft: set `superseded_by`, never delete source thoughts, and append every mutation to `consolidation_audit`. Queue claiming, provider calls, and writes must be idempotent and safe across restarts.
|
||||
|
||||
### Phase 4 - Salience gate
|
||||
|
||||
Compute initial salience during `capture_thought` using explainable heuristics such as length, code presence, decision verbs, and thought type. Store low-salience thoughts but exclude them from default recall. Consider a logistic model only after measured retrieval outcomes show the heuristics have plateaued.
|
||||
|
||||
### Phase 5 - Outcome loop
|
||||
|
||||
Add a tool or update flag that marks a thought as `confirmed` or `refuted` when its decision plays out. Preserve the change in the consolidation audit and feed the outcome into retrieval ranking as the real-world error signal.
|
||||
|
||||
### Phase 6 - Admin confidence and SQL job observability
|
||||
|
||||
Extend the Svelte admin UI and its ResolveSpec-compatible backend resources with two read-oriented views.
|
||||
|
||||
#### Thought confidence
|
||||
|
||||
- Show confidence in thought list and detail views without requiring the operator to open raw metadata.
|
||||
- Display the numeric value, a human-readable band, provenance, outcome, salience, validity window, and superseded state.
|
||||
- Allow sorting and filtering by confidence, outcome, validity, and superseded state.
|
||||
- Visually distinguish unknown/unscored confidence from low confidence; never coerce null into zero or one.
|
||||
- Explain ranking inputs in the detail view so operators can understand why a thought was included or excluded from default recall.
|
||||
- Keep confidence mutation behind the normal thought update/audit path rather than allowing unaudited inline table edits.
|
||||
|
||||
#### Consolidation SQL jobs
|
||||
|
||||
- Show configured `pg_cron` consolidation jobs and their schedule, enabled state, last start/end time, duration, status, and sanitized result/error message.
|
||||
- Show currently running consolidation jobs separately from recent completed/failed runs using `cron.job` and `cron.job_run_details` through a permission-limited backend projection.
|
||||
- Include queue counts by status and consolidation kind so a successful cron invocation cannot hide a stalled or growing queue.
|
||||
- Add filters for job, status, and time range, plus clear empty/unavailable states when `pg_cron` is not installed or its views are inaccessible.
|
||||
- Do not expose raw SQL command text, connection strings, credentials, or unrestricted `pg_stat_activity` data.
|
||||
- Keep the first version read-only. Manual run, retry, pause, or cancel controls require separate authorization, audit logging, and explicit confirmation.
|
||||
- Poll conservatively only while the jobs screen is visible; stop polling when hidden and avoid loading run history during normal admin startup.
|
||||
|
||||
Backend tests must cover `pg_cron` unavailable/permission-denied behavior, sanitized errors, running-versus-finished classification, pagination, and project-independent admin authorization. Frontend tests must cover confidence null handling, failed/running job states, queue backlog warnings, and polling cleanup.
|
||||
|
||||
### Delivery principle
|
||||
|
||||
SQL handles high-volume candidate recall and ranking; the LLM handles low-volume precision decisions for a bounded number of queued pairs. Implement Phase 2 first, then Phase 0/1 foundations needed for consolidation, followed by the worker, salience gate, outcome feedback loop, and admin observability. Confidence fields should appear in the admin UI as soon as Phase 0 lands; SQL job observability should land with the first scheduled Phase 1 jobs.
|
||||
|
||||
## Future Plugin: Lifestyle Tools (calendar, meals, household, CRM)
|
||||
|
||||
The following tool groups have been removed from the core server and are candidates for a separate optional plugin or extension server. The store/tool implementations remain in the codebase but are no longer registered.
|
||||
|
||||
Reference in New Issue
Block a user