feat(db): add project personas and skills tables

* Introduce project_personas table with foreign keys to projects and agent_personas
* Add project_skills table with foreign key to projects and agent_skills
* Include override boolean field in agent_persona_skills and project_skills
* Update schema and migration files to reflect new tables and fields
* Enhance CORS handling to reflect request origin

llm/instructions.go

@@ -5,4 +5,7 @@ import _ "embed"
 var (
 	//go:embed memory.md
 	MemoryInstructions []byte
+
+	//go:embed world_model_intro.md
+	WorldModelIntro []byte
 )

llm/memory.md

@@ -28,7 +28,20 @@ Use AMCS as memory with two scopes:
 
 At the very start of any session with AMCS:
 
-1. Call `describe_tools` to get the full list of available tools with their categories and any notes you have previously annotated. Read the notes before using a tool — they contain accumulated gotchas, workflow patterns, and field-ordering requirements you have recorded from prior sessions.
+1. Read the `amcs://world-model/intro` resource.
+2. Identify the current project and call `bootstrap_world_model` with its explicit name or ID and the smallest useful `context_limit`.
+3. Apply the returned skills and guardrails, but keep persona manifests and memory summaries as indexes. Load detailed content only when the current task requires it.
+4. Call `describe_tools` only when tool discovery or saved usage notes are needed. Prefer a category filter instead of loading the full catalog.
+
+## Token Budget and Progressive Loading
+
+- Minimize startup and working context. Do not load data merely because it is available.
+- Start with names, summaries, manifests, and metadata. Fetch full skill, guardrail, persona, trait, part, thought, plan, learning, chat, or file content only when it is relevant to the current task.
+- Use the smallest practical `limit` or `context_limit`, narrow queries, and project/category filters. Increase them only when the initial result is insufficient.
+- Do not preload every persona, skill, guardrail, plan, file, or historical memory. Follow references on demand.
+- Avoid repeating unchanged world-model content in the conversation. Keep a concise working summary and refresh only when the project or task changes, or when stale context is suspected.
+- Prefer `get_persona_manifest` and compiled summaries before `get_agent_persona(detail=true)`. Prefer `list_skills` metadata before `get_skill`, and file metadata before `load_file`.
+- Guardrails and directly applicable skills are mandatory even under a tight token budget; token minimization must never omit an applicable constraint.
 
 ## Project Session Startup
 
@@ -51,7 +64,7 @@ Do not abandon the project scope or retry without a project. The project simply
 ## Project Memory Rules
 
 - Use project memory for code decisions, architecture, TODOs, debugging findings, and context specific to the current repo or workstream.
-- Before substantial work, always retrieve context with `get_project_context` or `recall_context` so prior decisions inform your approach.
+- Before substantial work, retrieve focused context with `get_project_context` or `recall_context` only when prior decisions may affect the task. Use a narrow query and small limit first.
 - Save durable project facts with `capture_thought` after completing meaningful work.
 - Use structured learnings for curated, reusable lessons that should remain distinct from raw thought capture.
 - Use `save_file` or `upload_file` for project assets the memory should retain, such as screenshots, PDFs, audio notes, and other documents.
@@ -135,4 +148,4 @@ Notes are returned by `describe_tools` in future sessions. Annotate whenever you
 
 ## Short Operational Form
 
-At the start of every session, call `describe_tools` to read the full tool list and any accumulated usage notes. Use AMCS memory in project scope when the current work matches a known project; if no clear project matches, global notebook memory is allowed for non-project-specific information. At the start of every project session call `list_project_skills` and `list_project_guardrails` and apply what is returned; only create new skills or guardrails if none exist. If your MCP client does not preserve sessions across calls, pass `project` explicitly instead of relying on `set_active_project`. Store raw/durable notes with `capture_thought`, store curated durable lessons with `add_learning`, and track structured multi-step goals with `create_plan`. Use `get_plan` to load a plan's full context including dependencies, related plans, and linked skills/guardrails. Stamp `last_reviewed_at` on plans you review with `update_plan mark_reviewed: true`. For binary files or files larger than 10 MB, call `upload_file` with `content_path` to stage the file and get an `amcs://files/{id}` URI, then pass that URI to `save_file` as `content_uri` to link it to a thought. For small files, use `save_file` or `upload_file` with `content_base64` directly. Browse stored files with `list_files`, and load them with `load_file` only when their contents are needed. Stored files can also be read as raw binary via MCP resources at `amcs://files/{id}`. Never store project-specific memory globally when a matching project exists, and never store memory in the wrong project. If project matching is ambiguous, ask the user. If a tool returns `project_not_found`, call `create_project` with that name and retry — never drop the project scope. Whenever you discover a non-obvious tool behaviour, gotcha, or workflow pattern, record it with `annotate_tool` so future sessions benefit.
+At session start, identify the project and call `bootstrap_world_model` with a small `context_limit`. Apply mandatory skills and guardrails, but otherwise use summaries and manifests as indexes and load details only when needed. Call `describe_tools` only for discovery or saved notes, preferably with a category filter. Use project scope when the work matches a known project; pass `project` explicitly for stateless clients. Store durable notes with `capture_thought`, curated lessons with `add_learning`, and multi-step work with `create_plan`. Retrieve plans, memories, personas, and files on demand using narrow queries and the smallest useful limits. Never store memory in the wrong scope, silently choose an ambiguous project, or omit an applicable guardrail to save tokens. Record non-obvious tool behavior with `annotate_tool`.

llm/todo.md

@@ -1,5 +1,104 @@
 # AMCS TODO
 
+## Memory Consolidation and Retrieval Quality
+
+No prior consolidation implementation notes were found in the project. This roadmap starts clean and follows AMCS's current Go, PostgreSQL/pgvector, thoughts, learnings, projects, and model-specific embeddings architecture.
+
+### Current-shape decisions
+
+- Use bigint thought IDs for foreign keys and queue references, matching the current schema. Keep thought GUIDs as external identifiers rather than introducing UUID foreign keys without a specific interoperability need.
+- Embeddings are stored in the separate, model-specific `embeddings` table. Similarity queries and indexes must include the embedding model and project scope where applicable.
+- Verify the existing pgvector index strategy before adding another HNSW index. Add or change an index only when the current generated schema and query plan show it is needed.
+- All consolidation writes are soft, auditable, idempotent, and reversible.
+
+### Phase 0 - Schema additions
+
+Add to `thoughts`:
+
+- `confidence smallint`
+- `source text`
+- `valid_until timestamptz`
+- `superseded_by bigint` referencing `thoughts.id`
+- `salience real`
+- `outcome text` constrained to `confirmed`, `refuted`, or null
+
+Add tables:
+
+- `consolidation_queue(id, kind, thought_a, thought_b, status, created_at)`
+- `consolidation_audit(id, action, before jsonb, after jsonb, actor, ts)`
+
+The audit record must contain enough information to reverse every consolidation mutation. Define queue uniqueness and status transitions so retries cannot enqueue or process the same pair twice.
+
+Confidence must be available through the existing admin thought resource. Return both the normalized numeric value and enough provenance to explain whether it was user-supplied, heuristic, outcome-derived, or consolidation-derived. Existing thoughts need a documented neutral default or backfill policy rather than silently appearing maximally confident.
+
+### Phase 1 - Algorithmic candidate generation
+
+Run nightly SQL candidate generation without an LLM:
+
+- Near duplicates: self-join model-compatible embeddings and enqueue canonical thought pairs where `1 - (a.embedding <=> b.embedding) > 0.92`.
+- Contradiction suspects: combine high cosine similarity with low `pg_trgm` `similarity()` and enqueue as `kind = 'contradiction'`.
+- Decay: demote thoughts past `valid_until`; apply recency weighting during retrieval rather than rewriting content.
+
+Candidate queries must de-duplicate ordered pairs, exclude already superseded thoughts, respect project boundaries, and avoid comparing embeddings from different models.
+
+### Phase 2 - Retrieval ranking (ship first)
+
+Change `recall_context` from cosine-only top-k ranking to a bounded composite score based on:
+
+```text
+cosine_similarity * exp(-lambda * age_days) * confidence_weight * outcome_weight
+```
+
+Exclude superseded and low-salience thoughts from default recall, with explicit options for diagnostic retrieval. Tune defaults conservatively and preserve the current full-text fallback when embeddings are unavailable.
+
+This phase should ship first because it provides the largest recall-quality improvement without requiring consolidation writes or an adjudication model.
+
+### Phase 3 - LLM adjudication worker
+
+Add a resumable Go worker that claims queued pairs and calls a low-cost configured provider only for flagged candidates. Support two structured prompts:
+
+- Merge: produce one normalized schema entry from genuine duplicates.
+- Resolve conflict: verify a real contradiction and select or synthesize the supported result.
+
+Writes remain soft: set `superseded_by`, never delete source thoughts, and append every mutation to `consolidation_audit`. Queue claiming, provider calls, and writes must be idempotent and safe across restarts.
+
+### Phase 4 - Salience gate
+
+Compute initial salience during `capture_thought` using explainable heuristics such as length, code presence, decision verbs, and thought type. Store low-salience thoughts but exclude them from default recall. Consider a logistic model only after measured retrieval outcomes show the heuristics have plateaued.
+
+### Phase 5 - Outcome loop
+
+Add a tool or update flag that marks a thought as `confirmed` or `refuted` when its decision plays out. Preserve the change in the consolidation audit and feed the outcome into retrieval ranking as the real-world error signal.
+
+### Phase 6 - Admin confidence and SQL job observability
+
+Extend the Svelte admin UI and its ResolveSpec-compatible backend resources with two read-oriented views.
+
+#### Thought confidence
+
+- Show confidence in thought list and detail views without requiring the operator to open raw metadata.
+- Display the numeric value, a human-readable band, provenance, outcome, salience, validity window, and superseded state.
+- Allow sorting and filtering by confidence, outcome, validity, and superseded state.
+- Visually distinguish unknown/unscored confidence from low confidence; never coerce null into zero or one.
+- Explain ranking inputs in the detail view so operators can understand why a thought was included or excluded from default recall.
+- Keep confidence mutation behind the normal thought update/audit path rather than allowing unaudited inline table edits.
+
+#### Consolidation SQL jobs
+
+- Show configured `pg_cron` consolidation jobs and their schedule, enabled state, last start/end time, duration, status, and sanitized result/error message.
+- Show currently running consolidation jobs separately from recent completed/failed runs using `cron.job` and `cron.job_run_details` through a permission-limited backend projection.
+- Include queue counts by status and consolidation kind so a successful cron invocation cannot hide a stalled or growing queue.
+- Add filters for job, status, and time range, plus clear empty/unavailable states when `pg_cron` is not installed or its views are inaccessible.
+- Do not expose raw SQL command text, connection strings, credentials, or unrestricted `pg_stat_activity` data.
+- Keep the first version read-only. Manual run, retry, pause, or cancel controls require separate authorization, audit logging, and explicit confirmation.
+- Poll conservatively only while the jobs screen is visible; stop polling when hidden and avoid loading run history during normal admin startup.
+
+Backend tests must cover `pg_cron` unavailable/permission-denied behavior, sanitized errors, running-versus-finished classification, pagination, and project-independent admin authorization. Frontend tests must cover confidence null handling, failed/running job states, queue backlog warnings, and polling cleanup.
+
+### Delivery principle
+
+SQL handles high-volume candidate recall and ranking; the LLM handles low-volume precision decisions for a bounded number of queued pairs. Implement Phase 2 first, then Phase 0/1 foundations needed for consolidation, followed by the worker, salience gate, outcome feedback loop, and admin observability. Confidence fields should appear in the admin UI as soon as Phase 0 lands; SQL job observability should land with the first scheduled Phase 1 jobs.
+
 ## Future Plugin: Lifestyle Tools (calendar, meals, household, CRM)
 
 The following tool groups have been removed from the core server and are candidates for a separate optional plugin or extension server. The store/tool implementations remain in the codebase but are no longer registered.

llm/world_model_intro.md

@@ -0,0 +1,15 @@
+# AMCS World Model
+
+AMCS provides a project-scoped world model containing memory, skills, guardrails, personas, and recent context.
+
+At the start of an agent or chat session:
+
+1. Identify the current project from explicit user input or the workspace.
+2. Call `list_projects` when the project has not already been confirmed.
+3. Call `bootstrap_world_model` with the explicit project name or ID.
+4. Apply every returned guardrail and effective skill before continuing.
+5. Use the default persona when present. Other linked persona manifests describe personas available on demand.
+
+Never silently choose or create a project. If no project matches, ask the user or create one only with explicit approval. Stateless clients must continue passing the project explicitly on project-scoped calls.
+
+Skills are additive by default. A linked skill with `override: true` replaces a lower-precedence skill with the same stable name. Guardrails are additive and later layers cannot weaken them.