feat(personas): add implementation plan for agent personas

llm/core_tools.md

@@ -0,0 +1,263 @@
+# AMCS Core Tools — Implementation Plan
+
+Tools that expose runtime context to agents and provide general-purpose capabilities: time, location, connectivity, health, data quality, credentials, background jobs, change detection, HTTP, computation, secrets, webhooks, and context management.
+
+---
+
+## 1. Date / Time / Timezone
+
+**Tool:** `get_datetime`
+
+- Return current UTC time, local server time, and timezone identifier (IANA)
+- Include Unix timestamp, ISO 8601 string, day-of-week, week number
+- Relative helpers: is_business_hours, start_of_day, start_of_week (relative to server TZ)
+
+---
+
+## 2. Location
+
+**Tool:** `get_location`
+
+Three scopes returned in one call:
+
+| Scope | Data |
+|-------|------|
+| Server | hostname, OS, datacenter/cloud region if detectable |
+| Project | active project name, working directory, git repo root, current branch, last commit hash+message |
+| User | IP-derived city/country (GeoIP), configurable home location override in config |
+
+---
+
+## 3. Connectivity & Health
+
+**Tool:** `get_health`
+
+- AMCS server version and uptime
+- Database connection status and round-trip latency (ms)
+- pgvector extension present and version
+- LiteLLM / OpenRouter reachability (HEAD ping, latency, last-ok timestamp)
+- Configured integration endpoints (n8n, Telegram, etc.) — status and last-ok timestamp
+- Overall status: `ok` / `degraded` / `down`
+
+---
+
+## 4. Data Quality & Maintenance
+
+**Tool:** `get_data_quality`
+
+- Total thoughts, plans, learnings, projects
+- Thoughts missing embeddings (count + oldest)
+- Failed metadata reparse queue depth
+- Orphaned records: thoughts with no project, broken thought links, plans with missing dependencies
+- Last embedding backfill run timestamp and result
+
+---
+
+## 5. Security & Credentials
+
+**Tool:** `get_credential_status`
+
+- Per-configured integration: name, valid/expired boolean, days until expiry (no values exposed)
+- Last successful authentication timestamp per integration
+- AMCS token expiry if applicable
+
+---
+
+## 6. Background Job Status
+
+**Tool:** `get_job_status`
+
+- List of known async job types (embedding backfill, metadata reparse, file indexing)
+- Per job: status (idle/running/failed), last run timestamp, last run duration, error if failed
+- Pending queue depth per job type
+
+---
+
+## 7. Change Detection
+
+**Tool:** `get_changes`
+
+Parameters:
+- `since` — ISO 8601 timestamp or relative shorthand (`1h`, `24h`, `7d`)
+- `scope` — `all` | `thoughts` | `plans` | `learnings` | `projects`
+
+Returns:
+- Counts of created, updated, deleted per entity type since the given time
+- Top 5 most recently modified items per type (id, title, updated_at)
+- Active project changes highlighted separately
+
+---
+
+---
+
+## 8. HTTP Client
+
+**Tool:** `http_request`
+
+Parameters:
+- `method` — GET | POST | PUT | PATCH | DELETE
+- `url` — target URL
+- `headers` — optional map of header key/value pairs
+- `body` — optional string body (JSON, form, etc.)
+- `timeout_seconds` — default 30
+
+Returns: status code, response headers, body (truncated at configurable limit), latency (ms)
+
+Restrictions:
+- Blocklist for private/internal IP ranges (configurable allow-override)
+- Max response body size configurable (`http_client.max_body_bytes`)
+- Requires `tools.http_client: true` in config to enable
+
+---
+
+## 9. Token Counter
+
+**Tool:** `count_tokens`
+
+Parameters:
+- `text` — string to count
+- `model` — optional model name to select tokenizer (default: configured primary model)
+
+Returns: token count, estimated cost (input only), tokenizer used
+
+- Uses tiktoken-compatible tokenizer; falls back to char/4 estimate if model unknown
+
+---
+
+## 10. Text Diff
+
+**Tool:** `diff_text`
+
+Parameters:
+- `a` — original text
+- `b` — revised text
+- `format` — `unified` | `side_by_side` | `summary` (default: `summary`)
+
+Returns: diff output and change stats (lines added, removed, unchanged)
+
+---
+
+## 11. Data Parser
+
+**Tool:** `parse_data`
+
+Parameters:
+- `input` — raw string
+- `format` — `json` | `csv` | `xml` | `toml` | `yaml`
+- `query` — optional JSONPath / XPath / key path to extract a subset
+
+Returns: parsed structure as JSON, or extracted value if query provided
+
+---
+
+## 12. Template Renderer
+
+**Tool:** `render_template`
+
+Parameters:
+- `template` — Go `text/template` or Mustache string
+- `vars` — key/value map of variables to inject
+
+Returns: rendered string
+
+Use case: build dynamic prompts or messages from stored templates and runtime context.
+
+---
+
+## 13. Secret Retrieval
+
+**Tool:** `get_secret`
+
+Parameters:
+- `name` — secret name as configured in AMCS secrets store
+
+Returns: secret value (transmitted but never logged by server)
+
+- Secrets defined in config or environment under `secrets.*`
+- Access requires authenticated token with `secrets:read` scope
+- Audit log entry written on every retrieval (name, caller, timestamp — not value)
+
+---
+
+## 14. Webhook Trigger
+
+**Tool:** `trigger_webhook`
+
+Parameters:
+- `name` — named webhook as configured in `webhooks.*`
+- `payload` — optional JSON body to merge with default payload
+
+Returns: HTTP status from target, latency (ms)
+
+- Named webhooks defined in config (URL + default headers/auth) — agents never see the raw URL or credentials
+- Supports n8n, Zapier, generic HTTP targets
+
+---
+
+## 15. Summarize / Compress
+
+**Tool:** `summarize`
+
+Parameters:
+- `text` — content to summarize
+- `max_tokens` — target output length in tokens (default: 200)
+- `style` — `bullets` | `paragraph` | `headline` (default: `paragraph`)
+- `focus` — optional hint string ("focus on action items", "focus on errors")
+
+Returns: summarized text, input token count, output token count
+
+- Calls the configured LLM; billed against the AMCS service account
+- Useful for compressing large file loads or long thought chains before embedding
+
+---
+
+## 16. Chunk Text
+
+**Tool:** `chunk_text`
+
+Parameters:
+- `text` — content to split
+- `chunk_size` — max tokens per chunk (default: 512)
+- `overlap` — token overlap between chunks (default: 50)
+- `model` — tokenizer model (default: configured primary)
+
+Returns: array of chunk strings with index and token count per chunk
+
+---
+
+## 17. Schedule Action
+
+**Tool:** `schedule_action`
+
+Parameters:
+- `run_at` — ISO 8601 datetime or cron expression
+- `tool` — tool name to invoke
+- `args` — arguments for the tool
+- `label` — optional human-readable label
+
+Returns: scheduled action ID
+
+**Tool:** `list_scheduled_actions`
+- Returns pending actions: id, label, tool, run_at, status
+
+**Tool:** `cancel_scheduled_action`
+- Parameter: `id`
+
+---
+
+## Implementation Notes
+
+**Context & read-only tools (1–7):**
+- All read-only, no write permissions required
+- GeoIP opt-in via config (`core_tools.geoip: true`), offline MaxMind GeoLite2 DB
+- Credential expiry checks never log or return secret values — metadata only
+- `get_health` callable unauthenticated (mirrors existing `/health` endpoint)
+- `get_changes` is query-heavy — 30s cache TTL to avoid repeated aggregation
+
+**Agent utility tools (8–17):**
+- `http_request` disabled by default; requires `tools.http_client: true` in config; private IP ranges blocked
+- `get_secret` requires `secrets:read` token scope; every retrieval audit-logged (name + caller only)
+- `trigger_webhook` never exposes raw URLs or credentials to agents — named config only
+- `summarize` and `chunk_text` call the configured LLM; usage billed to AMCS service account
+- `schedule_action` backed by the existing cron/trigger infrastructure
+- All tools registered under the `core` category in `describe_tools`

llm/personas.md

@@ -0,0 +1,288 @@
+# Agent Personas — Implementation Plan
+
+## Purpose
+
+Composable agent settings system. An agent loads a named persona which assembles behavior from reusable parts. Parts can be overridden at load time without modifying the persona.
+
+---
+
+## Schema: 5 New Tables
+
+### `agent_personas`
+Named, loadable agent configurations.
+
+| Column | Type | Notes |
+|---|---|---|
+| id | bigserial | PK |
+| guid | uuid | DEFAULT gen_random_uuid() |
+| name | text UNIQUE NOT NULL | load-by-name key |
+| description | text | |
+| summary | text NOT NULL | returned by default on load |
+| detail | text | returned only when detail=true |
+| tags | text[] | DEFAULT '{}' |
+| created_at | timestamptz | |
+| updated_at | timestamptz | |
+
+### `agent_parts`
+Reusable behavior building blocks. Name is globally unique for reference by name.
+
+| Column | Type | Notes |
+|---|---|---|
+| id | bigserial | PK |
+| guid | uuid | DEFAULT gen_random_uuid() |
+| name | text UNIQUE NOT NULL | reference key for overrides |
+| part_type | text NOT NULL | system\|agent\|soul\|identity\|skill\|specialization\|tone\|goal\|context\|protocol |
+| description | text | |
+| summary | text NOT NULL | returned in summary view |
+| content | text | returned in detail view |
+| tags | text[] | DEFAULT '{}' |
+| created_at | timestamptz | |
+| updated_at | timestamptz | |
+
+### `agent_persona_parts`
+Junction — links reusable parts to a persona. `part_order` controls assembly sequence.
+
+| Column | Type |
+|---|---|
+| persona_id | bigint FK → agent_personas.id |
+| part_id | bigint FK → agent_parts.id |
+| part_order | int DEFAULT 0 |
+
+Composite PK: (persona_id, part_id)
+
+### `agent_persona_skills`
+Links existing `agent_skills` to a persona.
+
+| Column | Type |
+|---|---|
+| persona_id | bigint FK → agent_personas.id |
+| skill_id | bigint FK → agent_skills.id |
+
+Composite PK: (persona_id, skill_id)
+
+### `agent_persona_guardrails`
+Links existing `agent_guardrails` to a persona.
+
+| Column | Type |
+|---|---|
+| persona_id | bigint FK → agent_personas.id |
+| guardrail_id | bigint FK → agent_guardrails.id |
+
+Composite PK: (persona_id, guardrail_id)
+
+---
+
+## Part Types
+
+| Type | Purpose |
+|---|---|
+| system | Base rules and constraints |
+| agent | Core behavior definition |
+| soul | Fundamental character and values |
+| identity | Name, role, backstory |
+| skill | Specific capabilities (distinct from agent_skills) |
+| specialization | Domain expertise |
+| tone | Communication style, formality, voice |
+| goal | Objectives and success criteria |
+| context | Operational environment and available tools |
+| protocol | Step-by-step workflows or procedures |
+| backstory | Formative history and past experiences (what happened to them, distinct from identity which is who they are) |
+| motivation | Deep inner drives and why the character acts (distinct from goal which is task objective) |
+| voice | Speech patterns, vocabulary, mannerisms, catchphrases (distinct from tone which is formality level) |
+| archetype | Narrative template — hero, mentor, trickster, etc. Composable base layer |
+| flaw | Weaknesses, fears, internal conflicts |
+| relationship | How the character relates to other agents, users, or entities |
+
+---
+
+## Override Mechanism
+
+`get_agent_persona(name, detail=false, overrides={"tone":"part-name","goal":"part-name"})`
+
+- Overrides replace all linked parts of that `part_type` with the named part at query time
+- No DB write — runtime substitution only
+- Works for any part type, not just tone/goal
+- Parts resolved by unique `name`
+
+---
+
+## Response Shape
+
+**Summary (default):** persona `summary` + each linked part's `summary` + skill/guardrail `description`
+
+**Detail (detail=true):** persona `detail` + each part's full `content` + skill/guardrail full `content` + guardrail `severity`
+
+---
+
+## Tools (16 total)
+
+| Tool | Purpose |
+|---|---|
+| `create_agent_persona` | Create persona |
+| `update_agent_persona` | Update persona fields |
+| `delete_agent_persona` | Delete by name |
+| `list_agent_personas` | List, filter by tags |
+| `get_agent_persona` | Load by name; `detail` bool + `overrides` map |
+| `create_agent_part` | Create reusable part |
+| `update_agent_part` | Update part fields |
+| `delete_agent_part` | Delete by name |
+| `list_agent_parts` | Filter by part_type, tags |
+| `get_agent_part` | Get single part by name |
+| `add_persona_part` | Link part to persona |
+| `remove_persona_part` | Unlink part from persona |
+| `add_persona_skill` | Link existing agent_skill |
+| `remove_persona_skill` | Unlink skill |
+| `add_persona_guardrail` | Link existing guardrail |
+| `remove_persona_guardrail` | Unlink guardrail |
+
+---
+
+## Character Evolution
+
+Characters can evolve through defined arcs. Stage parts override the persona's base parts for matching types.
+
+### Additional Tables
+
+**`character_arcs`** — named progression definition
+- `id`, `name` UNIQUE NOT NULL, `description`, `summary`, `created_at`, `updated_at`
+
+**`arc_stages`** — ordered stages within an arc
+- `id`, `arc_id → character_arcs.id`, `name`, `stage_order int`, `description`, `condition text` (trigger description — evaluated externally by the calling agent), `created_at`
+
+**`arc_stage_parts`** — parts active at a given stage, override matching types from persona base
+- `stage_id → arc_stages.id`, `part_id → agent_parts.id` — composite PK
+
+**`persona_arc`** — links a persona to an arc and tracks current stage (one arc per persona)
+- `persona_id → agent_personas.id` UNIQUE, `arc_id → character_arcs.id`, `current_stage_id → arc_stages.id`, `updated_at`
+
+### Part Assembly Priority (highest wins per type)
+
+1. Runtime `overrides` passed in the call
+2. Active arc stage parts
+3. Persona's own linked parts
+
+### Additional Tools
+
+| Tool | Purpose |
+|---|---|
+| `create_character_arc` | Define a new arc |
+| `list_character_arcs` | List arcs |
+| `add_arc_stage` | Add a stage to an arc |
+| `add_stage_part` | Link a part to a stage |
+| `remove_stage_part` | Unlink part from stage |
+| `assign_persona_arc` | Attach an arc to a persona, set starting stage |
+| `advance_persona_stage` | Move persona to next stage in its arc |
+| `reset_persona_stage` | Reset to first stage |
+
+---
+
+## Limited Context Support
+
+For agents with tight context budgets: fewer round-trips, fewer tokens.
+
+### Schema Changes
+
+**`agent_personas`** — add columns:
+- `compiled_summary text` — pre-merged part summaries + persona summary, ready to use directly in a system prompt
+- `compiled_detail text` — pre-merged full part content
+- `compiled_at timestamptz` — when last regenerated
+
+**`agent_persona_parts`** — add column:
+- `priority int DEFAULT 0` — higher priority parts load first when trimming
+
+### `get_agent_persona` Additional Parameters
+
+| Param | Purpose |
+|---|---|
+| `types []string` | Return only parts of specified types e.g. `["soul","goal","tone"]` |
+| `limit int` | Return only top-N parts by priority |
+
+### Additional Tool
+
+| Tool | Purpose |
+|---|---|
+| `compile_persona` | Regenerate `compiled_summary` and `compiled_detail` from current parts + active arc stage |
+
+### Usage Pattern
+
+Agents with limited context call `get_agent_persona(name)` and use `compiled_summary` directly — one fetch, no assembly, minimal tokens. Call `compile_persona` after any part or arc stage change.
+
+---
+
+## On-Demand Loading & Traits
+
+Core philosophy: **load nothing upfront, discover everything, fetch on demand.**
+
+### `agent_traits` — Atomic Personality Units
+
+Parts are too coarse for on-demand loading. Traits are individual, atomic units fetched one at a time.
+
+**`agent_traits`** table:
+- `id` bigserial PK, `guid` uuid, `name` text UNIQUE NOT NULL, `trait_type` text NOT NULL, `description` text, `instruction` text (how to apply this trait in practice), `tags` text[] DEFAULT '{}', `created_at`, `updated_at`
+
+**`agent_persona_traits`** junction:
+- `persona_id → agent_personas.id`, `trait_id → agent_traits.id` — composite PK
+
+| Trait Type | Examples |
+|---|---|
+| `personality` | curious, warm, stubborn, irreverent |
+| `cognitive` | analytical, creative, methodical, lateral |
+| `emotional` | empathetic, stoic, excitable, measured |
+| `social` | assertive, collaborative, reserved, direct |
+| `behavioral` | cautious, thorough, impulsive, adaptive |
+
+### Manifest — Lightweight Discovery
+
+`get_persona_manifest(name)` — agent calls this first. Returns structure only, no content.
+
+Response includes:
+- Persona name + description
+- Available parts: `{name, type, description}` — no content
+- Available traits: `{name, trait_type, description}` — no instruction
+- Linked skill + guardrail names
+- **`on_demand_tools` hint block** — tells the agent which tools to call and when
+
+```
+on_demand_tools:
+  get_agent_part(name)       — load full content of a specific part
+  get_agent_trait(name)      — load instruction for a specific trait
+  get_agent_skill(name)      — load skill content
+  get_agent_guardrail(name)  — load guardrail content
+  compile_persona(name)      — load pre-merged compiled_summary if full context needed
+```
+
+### Additional Tools
+
+| Tool | Purpose |
+|---|---|
+| `get_persona_manifest` | Lightweight discovery — structure only, includes tool hints |
+| `create_agent_trait` | Create an atomic trait |
+| `update_agent_trait` | Update trait fields |
+| `delete_agent_trait` | Delete by name |
+| `list_agent_traits` | Filter by trait_type, tags |
+| `get_agent_trait` | Load single trait instruction by name |
+| `add_persona_trait` | Link trait to persona |
+| `remove_persona_trait` | Unlink trait |
+
+### Loading Strategy
+
+| Situation | Call |
+|---|---|
+| First load, unknown persona | `get_persona_manifest(name)` |
+| Need a specific behavior | `get_agent_part(name)` |
+| Need a personality nuance | `get_agent_trait(name)` |
+| Full context available | `compile_persona(name)` → use `compiled_summary` |
+| Arc stage changed | `advance_persona_stage` → `compile_persona` |
+
+---
+
+## Implementation Order
+
+1. `schema/agent_personas.dbml` — DBML table definitions
+2. `migrations/NNN_agent_personas.sql` — SQL migration
+3. Regenerate models via relspecgo → `internal/generatedmodels/`
+4. `internal/store/agent_personas.go` — DB access methods
+5. `internal/types/agent_persona.go` — Go types and I/O structs
+6. `internal/tools/agent_personas.go` — Tool implementation
+7. `internal/mcpserver/server.go` — Register tools + catalog entries
+8. `internal/app/app.go` — Instantiate and wire tool