Compare commits
13
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
631bb64109 | ||
|
|
4f8f2f4190 | ||
|
|
1689167a7b | ||
|
|
5cd81f325f | ||
|
|
cd010fc7a1 | ||
|
|
e3a4a3c5c7 | ||
|
|
4b3f0b1b55 | ||
|
|
e459775740 | ||
|
|
5718685c40 | ||
|
|
5c899d1635 | ||
|
|
8db2141d45 | ||
|
|
3198600031 | ||
|
|
cfc78e0493 |
@@ -18,6 +18,14 @@ jobs:
|
||||
with:
|
||||
go-version: '1.26'
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '24'
|
||||
|
||||
- name: Enable pnpm
|
||||
run: corepack enable
|
||||
|
||||
- name: Cache Go modules
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
@@ -31,9 +39,37 @@ jobs:
|
||||
- name: Download dependencies
|
||||
run: go mod download
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 'lts/*'
|
||||
|
||||
- name: Install pnpm
|
||||
run: npm install -g pnpm
|
||||
|
||||
- name: Build UI
|
||||
run: |
|
||||
cd ui
|
||||
pnpm install --frozen-lockfile
|
||||
pnpm run build
|
||||
|
||||
- name: Tidy modules
|
||||
run: go mod tidy
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 'lts/*'
|
||||
|
||||
- name: Install pnpm
|
||||
run: npm install -g pnpm
|
||||
|
||||
- name: Build UI
|
||||
run: |
|
||||
cd ui
|
||||
pnpm install --frozen-lockfile
|
||||
pnpm run build
|
||||
|
||||
- name: Run tests
|
||||
run: go test ./...
|
||||
|
||||
|
||||
@@ -74,6 +74,38 @@ The AMCS directory is used to store configuration and code for the Avalon Memory
|
||||
| `describe_tools` | List all available MCP tools with names, descriptions, categories, and model-authored usage notes; call this at the start of a session to orient yourself |
|
||||
| `annotate_tool` | Persist your own usage notes for a specific tool; notes are returned by `describe_tools` in future sessions |
|
||||
|
||||
## Webhook ingestion
|
||||
|
||||
External automation can create thoughts without speaking MCP by posting JSON to `POST /webhooks/thoughts`. The endpoint is protected by the same AMCS authentication middleware as MCP and file uploads, so pass one configured API key via `x-brain-key`, an authorization bearer token header, or another enabled auth method.
|
||||
|
||||
Example:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/webhooks/thoughts \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H 'x-brain-key: <api-key>' \
|
||||
-H 'Idempotency-Key: n8n-run-123' \
|
||||
-d '{
|
||||
"content": "External system observed build failure on main",
|
||||
"project": "amcs",
|
||||
"source": "n8n",
|
||||
"type": "task",
|
||||
"topics": ["ci", "webhook"],
|
||||
"metadata": {"workflow": "ci-monitor", "run_id": "123"}
|
||||
}'
|
||||
```
|
||||
|
||||
Payload fields:
|
||||
|
||||
- `content` is required and becomes the thought content.
|
||||
- `project` is optional; when present it must match an existing AMCS project.
|
||||
- `source`, `type`, `topics`, `people`, `action_items`, and `dates_mentioned` are normalized into the standard thought metadata schema. Unknown `type` values fall back to `observation`.
|
||||
- `metadata` or `source_metadata` may contain safe source-specific JSON values; unsupported values and overly deep objects are dropped rather than persisted.
|
||||
- `idempotency_key` or the `Idempotency-Key` header can be supplied to make repeated webhook deliveries return the existing thought with `duplicate: true`.
|
||||
- `external_id` is stored under `metadata.webhook.external_id` for source-side traceability.
|
||||
|
||||
Successful new ingestion returns `201` with the created thought. Duplicate idempotency-key delivery returns `200` and the previously created thought. Invalid JSON, missing content, missing/unknown projects, or unauthenticated requests are rejected before persistence. Metadata and embedding enrichment are queued after the thought is stored.
|
||||
|
||||
## Learnings
|
||||
|
||||
Learnings are curated, structured memory records for durable insights you want to keep distinct from raw thoughts. Use them for normalized lessons, decisions, and evidence-backed findings that should be easy to retrieve and review over time.
|
||||
@@ -478,7 +510,7 @@ metadata_retry:
|
||||
include_archived: false
|
||||
```
|
||||
|
||||
**Search fallback**: when no embeddings exist for the active model in scope, `search_thoughts`, `recall_context`, `get_project_context`, `summarize_thoughts`, and `related_thoughts` automatically fall back to Postgres full-text search so results are never silently empty.
|
||||
**Search fallback**: when no embeddings exist for the active model in scope, `search_thoughts`, `recall_context`, `get_project_context`, `summarize_thoughts`, and `related_thoughts` automatically fall back to Postgres full-text search so results are never silently empty. All five tools include a `retrieval_mode` field in their response (`"semantic"` or `"text"`) so callers can see which path was taken.
|
||||
|
||||
## Client Setup
|
||||
|
||||
|
||||
@@ -0,0 +1,161 @@
|
||||
# Per-user tenancy implementation plan
|
||||
|
||||
## Scope and current state
|
||||
|
||||
Gitea issue #10 asks AMCS to isolate memories by user, tenant, or workspace. The current branch already carries the first implementation pass, so this plan records the intended model and the remaining hardening work another worker should verify or finish.
|
||||
|
||||
Observed code paths:
|
||||
|
||||
- Authentication enters through `internal/auth/middleware.go`. API-key auth uses the configured header, defaulting to `x-brain-key`; bearer tokens can resolve through OAuth `TokenStore` or API keyring; HTTP Basic resolves OAuth client credentials.
|
||||
- `internal/auth/middleware.go` writes both `auth.key_id` and `tenancy.tenant_key` into the request context. The tenant key is intentionally opaque and currently equals the authenticated key id or OAuth client id.
|
||||
- `internal/tenancy/tenancy.go` exposes `WithTenantKey` and `KeyFromContext` only; callers should not infer user semantics from the tenant string.
|
||||
- Schema already has `tenant_key` on `projects`, `thoughts`, `stored_files`, `learnings`, `plans`, and `chat_histories` in `schema/*.dbml`.
|
||||
- `internal/store/tenancy.go` provides helper functions for appending tenant predicates to SQL.
|
||||
- Tenant scoping is already present in project, thought, and stored-file store paths. Some other project-owned domains still need explicit tenant enforcement.
|
||||
|
||||
## Tenant/user identity source
|
||||
|
||||
Use the authenticated principal id as the tenant boundary:
|
||||
|
||||
1. API key: resolve token through `auth.Keyring.Lookup`; use returned key id as `tenant_key`.
|
||||
2. OAuth bearer token: resolve token through `TokenStore.Lookup`; use returned client id/key id as `tenant_key`.
|
||||
3. OAuth Basic client credentials: resolve through `OAuthRegistry.Lookup`; use returned client id as `tenant_key`.
|
||||
4. Unauthenticated requests must not get tenant context and must not reach protected MCP/API handlers.
|
||||
|
||||
Do not store raw API keys or bearer tokens in tenant columns. Store only stable configured key ids/client ids. Yes, it is less flashy than inventing an account service before breakfast, but it keeps the trust boundary small and auditable.
|
||||
|
||||
## Required schema/model changes
|
||||
|
||||
Source-of-truth DBML changes:
|
||||
|
||||
- Add nullable `tenant_key text` to all user-owned tables:
|
||||
- `projects`
|
||||
- `thoughts`
|
||||
- `stored_files`
|
||||
- `learnings`
|
||||
- `plans`
|
||||
- `chat_histories`
|
||||
- Add tenant indexes:
|
||||
- single-column `tenant_key` for direct filtering
|
||||
- `(tenant_key, name)` unique on `projects`, replacing global `projects.name` uniqueness
|
||||
- `(tenant_key, project_id)` on `thoughts` for common project-scoped memory lookups
|
||||
- Regenerate SQL migrations and generated models from DBML; do not hand-edit generated Go models except as a temporary debugging step.
|
||||
|
||||
Important follow-up: `agent_skills`, `agent_guardrails`, `agent_personas`, `agent_parts`, traits, and arcs are currently global catalogs. Keep them global unless product requirements say skills/personas are private per tenant. Project join tables inherit protection through tenant-scoped project ids, but direct join queries must verify the project belongs to the tenant.
|
||||
|
||||
## Query scoping strategy
|
||||
|
||||
All request-facing store methods that touch tenant-owned rows must include tenant predicates whenever `tenancy.KeyFromContext(ctx)` returns a key.
|
||||
|
||||
Rules:
|
||||
|
||||
- Inserts must populate `tenant_key` from context.
|
||||
- Gets/updates/deletes by id or guid must add `and tenant_key = $n`.
|
||||
- Lists/searches must add `tenant_key = $n` before other filters.
|
||||
- Joins must scope the tenant-owned root table. Example: project summaries should count only thoughts that belong to the same tenant as the project, not merely thoughts with the same `project_id`.
|
||||
- Background maintenance jobs must either run per tenant, carry tenant context explicitly, or intentionally operate cross-tenant with an internal-only code path documented in the job.
|
||||
|
||||
Already-scoped paths to keep:
|
||||
|
||||
- `internal/store/projects.go`: create/get/list/touch projects.
|
||||
- `internal/store/thoughts.go`: create/list/get/update/delete/archive/search/stats/embedding repair paths.
|
||||
- `internal/store/files.go`: insert/get/list stored files.
|
||||
|
||||
Paths needing audit/hardening:
|
||||
|
||||
- `internal/store/learnings.go`: `CreateLearning`, `GetLearning`, and `ListLearnings` should populate/filter by tenant.
|
||||
- `internal/store/plans.go`: `CreatePlan`, `GetPlan`, `GetPlanDetail`, `UpdatePlan`, `DeletePlan`, `ListPlans`, dependency/related-plan operations, and plan skill/guardrail joins should scope to tenant-owned plans.
|
||||
- `internal/store/chat_histories.go`: chat history create/get/list/update paths should populate/filter by tenant.
|
||||
- `internal/store/thought_learning_links.go`: links traverse tenant-owned thoughts/learnings; queries should join and scope both sides or at least the tenant-owned root.
|
||||
- `internal/store/skills.go` and `internal/store/project_personas.go`: project-scoped joins should verify the project is visible to the current tenant before returning linked global catalog records.
|
||||
- ResolveSpec/admin CRUD endpoints exposing generated models must not bypass tenant-aware store methods. If they use direct generic model access, add middleware-level filter injection or disable tenant-owned tables from generic admin writes.
|
||||
|
||||
## Migration and backfill
|
||||
|
||||
Migration approach for existing single-tenant installs:
|
||||
|
||||
1. Add nullable `tenant_key` columns first; do not make them `not null` initially because existing deployments have historical rows without a principal.
|
||||
2. Backfill existing rows to a configured default tenant only if the instance enables multi-tenant mode or defines `auth.default_tenant_key`. Otherwise leave `NULL` rows visible only to unauthenticated/internal single-tenant contexts.
|
||||
3. Replace global project-name uniqueness with `(tenant_key, name)` uniqueness. For PostgreSQL, preserve legacy `NULL` semantics carefully: multiple null-tenant projects with the same name may be possible unless a partial unique index is added for null tenant rows.
|
||||
4. Add indexes concurrently where practical for production-sized tables.
|
||||
5. Document that after enabling auth-backed tenancy, legacy null-tenant data is not visible to authenticated tenants unless backfilled.
|
||||
|
||||
Recommended config addition:
|
||||
|
||||
- `auth.default_tenant_key` or `tenancy.default_key` for one-time/self-hosted backfill and development.
|
||||
- Optional `tenancy.mode: single|authenticated` so operators can keep current single-user behavior deliberately instead of discovering isolation by accident. An accident in auth is just a breach with better branding.
|
||||
|
||||
## Authorization checks
|
||||
|
||||
Authorization is row ownership by tenant key:
|
||||
|
||||
- A request may only see or mutate rows whose `tenant_key` equals the authenticated tenant key.
|
||||
- Cross-tenant ids/gids should behave as not found, not forbidden, to avoid existence leaks.
|
||||
- Tenant key is server-derived only. Ignore any client-provided `tenant_key` fields on tool/API inputs.
|
||||
- Project id references in create/update operations must be validated against the same tenant before use. This prevents attaching a new thought/file/plan to another tenant's project id.
|
||||
- Relationship operations must verify both endpoints are in the same tenant before creating links.
|
||||
- Global catalogs can be read across tenants only if intentionally shared; project-specific associations must be tenant-checked through the project.
|
||||
|
||||
## Affected endpoints/services/UI surfaces
|
||||
|
||||
Backend/MCP tools:
|
||||
|
||||
- Project tools: create/get/list/set active project.
|
||||
- Thought tools: capture, retrieve, update, delete/archive, semantic search, text search, stats, metadata and embedding repair queues.
|
||||
- File tools and binary file upload/download APIs.
|
||||
- Learning tools and thought-learning link tools.
|
||||
- Plan/task tools including dependencies, related plans, skills, and guardrails.
|
||||
- Chat history/session persistence tools.
|
||||
- Persona/skill/guardrail project-association tools.
|
||||
|
||||
HTTP/API boundaries:
|
||||
|
||||
- MCP SSE and streamable HTTP handlers must run behind auth middleware when auth is configured.
|
||||
- Any non-MCP REST endpoints under the admin/API server must either use tenant-aware stores or explicitly be internal/admin-only.
|
||||
- ResolveSpec admin CRUD views need tenant filter injection or table-level access restrictions for tenant-owned models.
|
||||
|
||||
UI:
|
||||
|
||||
- Project selector/list should naturally show tenant-scoped projects.
|
||||
- Admin tables for projects/thoughts/files/learnings/plans/chat histories must not show `tenant_key` as an editable field.
|
||||
- If a tenant switcher is ever added, it must map to a server-side authenticated principal or admin impersonation path, not a client-side query parameter. Obviously.
|
||||
|
||||
## Test cases
|
||||
|
||||
Minimum test matrix:
|
||||
|
||||
1. Auth middleware propagates tenant key for API-key header auth.
|
||||
2. Auth middleware propagates tenant key for bearer token via OAuth token store.
|
||||
3. Auth middleware propagates tenant key for HTTP Basic OAuth client credentials.
|
||||
4. Tenant A and tenant B can create projects with the same name; each only lists its own project.
|
||||
5. Tenant A cannot get/update/delete/archive Tenant B's thought by guid or numeric id.
|
||||
6. Semantic and text search return only same-tenant thoughts, including project-filtered searches.
|
||||
7. Stored file get/list/download is scoped; a file id from another tenant returns not found.
|
||||
8. Learning create/list/get is scoped, including links to thoughts.
|
||||
9. Plan create/list/get/update/delete and dependency/related-plan operations are scoped.
|
||||
10. Project skill/persona/guardrail association reads verify the project belongs to the tenant.
|
||||
11. Background metadata/embedding retry queues do not cross tenants unless intentionally internal.
|
||||
12. Migration test covers legacy null-tenant rows and backfill to default tenant.
|
||||
13. ResolveSpec/admin API tests prove tenant-owned resources are filtered or unavailable without admin override.
|
||||
|
||||
## Assumptions and blockers
|
||||
|
||||
Assumptions:
|
||||
|
||||
- The first tenancy boundary is authenticated principal id, not human account, org, or workspace. This is consistent with the current auth system and avoids adding an account model prematurely.
|
||||
- Null `tenant_key` remains the compatibility path for existing single-tenant/internal flows.
|
||||
- Global skills/personas/guardrails remain shared catalogs until a separate product decision makes them tenant-private.
|
||||
|
||||
Blockers/decisions needed:
|
||||
|
||||
- Decide whether legacy rows should be backfilled automatically to a configured default tenant during migration or left null until an operator runs an explicit backfill.
|
||||
- Decide whether ResolveSpec admin endpoints are trusted admin-only or must enforce tenant predicates like MCP tools.
|
||||
- Decide whether future OAuth subjects should distinguish user id from client id; the current implementation only has client/key id available.
|
||||
|
||||
## Implementation order
|
||||
|
||||
1. Finish tenant scoping audits for learnings, plans, chat histories, thought-learning links, and project catalog joins.
|
||||
2. Add tests proving cross-tenant invisibility for each store/tool domain before widening coverage. Yes, tests first; future us has enough enemies.
|
||||
3. Add config/backfill migration behavior and document operator steps.
|
||||
4. Lock down or filter ResolveSpec/admin tenant-owned model access.
|
||||
5. Run `make test` and `make build`; include exact results in the issue/PR comment.
|
||||
@@ -202,6 +202,7 @@ func routes(logger *slog.Logger, cfg *config.Config, info buildinfo.Info, db *st
|
||||
Update: tools.NewUpdateTool(db, embeddings, metadata, cfg.Capture, logger),
|
||||
Delete: tools.NewDeleteTool(db),
|
||||
Archive: tools.NewArchiveTool(db),
|
||||
DuplicateAudit: tools.NewDuplicateAuditTool(db, cfg.Search, activeProjects),
|
||||
Projects: tools.NewProjectsTool(db, activeProjects),
|
||||
Version: tools.NewVersionTool(cfg.MCP.ServerName, info),
|
||||
Learnings: tools.NewLearningsTool(db, activeProjects, cfg.Search),
|
||||
@@ -238,6 +239,7 @@ func routes(logger *slog.Logger, cfg *config.Config, info buildinfo.Info, db *st
|
||||
}
|
||||
mux.Handle("/files", authMiddleware(fileHandler(filesTool)))
|
||||
mux.Handle("/files/{id}", authMiddleware(fileHandler(filesTool)))
|
||||
mux.Handle("/webhooks/thoughts", authMiddleware(newWebhookThoughtHandler(db, embeddings, cfg.Capture, enrichmentRetryer, backfillTool)))
|
||||
mux.HandleFunc("/.well-known/oauth-authorization-server", oauthMetadataHandler())
|
||||
mux.HandleFunc("/api/oauth/register", oauthRegisterHandler(dynClients, logger))
|
||||
mux.HandleFunc("/api/oauth/authorize", oauthAuthorizeHandler(dynClients, authCodes, logger))
|
||||
|
||||
@@ -0,0 +1,214 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.warky.dev/wdevs/amcs/internal/ai"
|
||||
"git.warky.dev/wdevs/amcs/internal/config"
|
||||
"git.warky.dev/wdevs/amcs/internal/metadata"
|
||||
"git.warky.dev/wdevs/amcs/internal/store"
|
||||
"git.warky.dev/wdevs/amcs/internal/tools"
|
||||
thoughttypes "git.warky.dev/wdevs/amcs/internal/types"
|
||||
)
|
||||
|
||||
const maxWebhookBodyBytes = 1 << 20
|
||||
|
||||
type webhookThoughtRequest struct {
|
||||
Content string `json:"content"`
|
||||
Project string `json:"project,omitempty"`
|
||||
Source string `json:"source,omitempty"`
|
||||
Type string `json:"type,omitempty"`
|
||||
Topics []string `json:"topics,omitempty"`
|
||||
People []string `json:"people,omitempty"`
|
||||
ActionItems []string `json:"action_items,omitempty"`
|
||||
DatesMentioned []string `json:"dates_mentioned,omitempty"`
|
||||
Metadata map[string]any `json:"metadata,omitempty"`
|
||||
SourceMetadata map[string]any `json:"source_metadata,omitempty"`
|
||||
IDempotencyKey string `json:"idempotency_key,omitempty"`
|
||||
ExternalID string `json:"external_id,omitempty"`
|
||||
}
|
||||
|
||||
type webhookThoughtResponse struct {
|
||||
Thought thoughttypes.Thought `json:"thought"`
|
||||
Duplicate bool `json:"duplicate"`
|
||||
WebhookMeta thoughttypes.WebhookMetadata `json:"webhook"`
|
||||
}
|
||||
|
||||
type webhookThoughtHandler struct {
|
||||
store *store.DB
|
||||
embeddings *ai.EmbeddingRunner
|
||||
capture config.CaptureConfig
|
||||
retryer tools.MetadataQueuer
|
||||
embedRetryer tools.EmbeddingQueuer
|
||||
}
|
||||
|
||||
func newWebhookThoughtHandler(db *store.DB, embeddings *ai.EmbeddingRunner, capture config.CaptureConfig, retryer tools.MetadataQueuer, embedRetryer tools.EmbeddingQueuer) http.Handler {
|
||||
return &webhookThoughtHandler{store: db, embeddings: embeddings, capture: capture, retryer: retryer, embedRetryer: embedRetryer}
|
||||
}
|
||||
|
||||
func (h *webhookThoughtHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path != "/webhooks/thoughts" {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
if r.Method != http.MethodPost {
|
||||
w.Header().Set("Allow", http.MethodPost)
|
||||
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
|
||||
return
|
||||
}
|
||||
|
||||
r.Body = http.MaxBytesReader(w, r.Body, maxWebhookBodyBytes)
|
||||
in, err := parseWebhookThoughtRequest(r)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
webhookMeta := buildWebhookMetadata(in, r.Header.Get("Idempotency-Key"), time.Now().UTC())
|
||||
if webhookMeta.IDempotencyKey != "" {
|
||||
if existing, err := h.store.GetThoughtByWebhookIDempotencyKey(r.Context(), webhookMeta.IDempotencyKey); err == nil {
|
||||
writeWebhookThoughtResponse(w, http.StatusOK, webhookThoughtResponse{Thought: existing, Duplicate: true, WebhookMeta: webhookMeta})
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
projectID, err := h.resolveWebhookProject(r, in.Project)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
thought := thoughttypes.Thought{
|
||||
Content: strings.TrimSpace(in.Content),
|
||||
Metadata: normalizeWebhookThoughtMetadata(in, webhookMeta, h.capture),
|
||||
ProjectID: projectID,
|
||||
}
|
||||
created, err := h.store.InsertThought(r.Context(), thought, h.embeddings.PrimaryModel())
|
||||
if err != nil {
|
||||
http.Error(w, "insert thought: "+err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
if projectID != nil {
|
||||
_ = h.store.TouchProject(r.Context(), *projectID)
|
||||
}
|
||||
if h.retryer != nil {
|
||||
h.retryer.QueueThought(created.ID)
|
||||
}
|
||||
if h.embedRetryer != nil {
|
||||
h.embedRetryer.QueueThought(r.Context(), created.ID, created.Content)
|
||||
}
|
||||
|
||||
writeWebhookThoughtResponse(w, http.StatusCreated, webhookThoughtResponse{Thought: created, WebhookMeta: webhookMeta})
|
||||
}
|
||||
|
||||
func parseWebhookThoughtRequest(r *http.Request) (webhookThoughtRequest, error) {
|
||||
if !strings.Contains(r.Header.Get("Content-Type"), "application/json") {
|
||||
return webhookThoughtRequest{}, errors.New("webhook requires application/json")
|
||||
}
|
||||
defer r.Body.Close()
|
||||
decoder := json.NewDecoder(r.Body)
|
||||
decoder.DisallowUnknownFields()
|
||||
var in webhookThoughtRequest
|
||||
if err := decoder.Decode(&in); err != nil {
|
||||
return webhookThoughtRequest{}, err
|
||||
}
|
||||
if strings.TrimSpace(in.Content) == "" {
|
||||
return webhookThoughtRequest{}, errors.New("content is required")
|
||||
}
|
||||
return in, nil
|
||||
}
|
||||
|
||||
func (h *webhookThoughtHandler) resolveWebhookProject(r *http.Request, projectName string) (*int64, error) {
|
||||
projectName = strings.TrimSpace(projectName)
|
||||
if projectName == "" {
|
||||
return nil, nil
|
||||
}
|
||||
project, err := h.store.GetProject(r.Context(), projectName)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &project.NumericID, nil
|
||||
}
|
||||
|
||||
func buildWebhookMetadata(in webhookThoughtRequest, headerKey string, now time.Time) thoughttypes.WebhookMetadata {
|
||||
sourceMetadata := in.SourceMetadata
|
||||
if len(sourceMetadata) == 0 {
|
||||
sourceMetadata = in.Metadata
|
||||
}
|
||||
return thoughttypes.WebhookMetadata{
|
||||
ReceivedAt: now.Format(time.RFC3339),
|
||||
IDempotencyKey: firstNonEmpty(in.IDempotencyKey, headerKey),
|
||||
ExternalID: strings.TrimSpace(in.ExternalID),
|
||||
SourceMetadata: sanitizeWebhookMetadata(sourceMetadata),
|
||||
}
|
||||
}
|
||||
|
||||
func normalizeWebhookThoughtMetadata(in webhookThoughtRequest, webhookMeta thoughttypes.WebhookMetadata, capture config.CaptureConfig) thoughttypes.ThoughtMetadata {
|
||||
return metadata.Normalize(thoughttypes.ThoughtMetadata{
|
||||
People: in.People,
|
||||
ActionItems: in.ActionItems,
|
||||
DatesMentioned: in.DatesMentioned,
|
||||
Topics: in.Topics,
|
||||
Type: in.Type,
|
||||
Source: firstNonEmpty(in.Source, "webhook"),
|
||||
Webhook: &webhookMeta,
|
||||
}, capture)
|
||||
}
|
||||
|
||||
func sanitizeWebhookMetadata(in map[string]any) map[string]any {
|
||||
if len(in) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make(map[string]any, len(in))
|
||||
for key, value := range in {
|
||||
key = strings.TrimSpace(key)
|
||||
if key == "" {
|
||||
continue
|
||||
}
|
||||
if sanitized, ok := sanitizeWebhookMetadataValue(value, 0); ok {
|
||||
out[key] = sanitized
|
||||
}
|
||||
}
|
||||
if len(out) == 0 {
|
||||
return nil
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func sanitizeWebhookMetadataValue(value any, depth int) (any, bool) {
|
||||
if depth > 3 {
|
||||
return nil, false
|
||||
}
|
||||
switch v := value.(type) {
|
||||
case nil, bool, float64, string:
|
||||
return v, true
|
||||
case []any:
|
||||
if len(v) > 50 {
|
||||
v = v[:50]
|
||||
}
|
||||
out := make([]any, 0, len(v))
|
||||
for _, item := range v {
|
||||
if sanitized, ok := sanitizeWebhookMetadataValue(item, depth+1); ok {
|
||||
out = append(out, sanitized)
|
||||
}
|
||||
}
|
||||
return out, true
|
||||
case map[string]any:
|
||||
if len(v) > 50 {
|
||||
return nil, false
|
||||
}
|
||||
return sanitizeWebhookMetadata(v), true
|
||||
default:
|
||||
return nil, false
|
||||
}
|
||||
}
|
||||
|
||||
func writeWebhookThoughtResponse(w http.ResponseWriter, status int, out webhookThoughtResponse) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
_ = json.NewEncoder(w).Encode(out)
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.warky.dev/wdevs/amcs/internal/config"
|
||||
)
|
||||
|
||||
func TestParseWebhookThoughtRequestRequiresJSON(t *testing.T) {
|
||||
req := httptestRequest("text/plain", `{"content":"hello"}`)
|
||||
|
||||
_, err := parseWebhookThoughtRequest(req)
|
||||
if err == nil {
|
||||
t.Fatal("expected error for non-json content type")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseWebhookThoughtRequestRequiresContent(t *testing.T) {
|
||||
req := httptestRequest("application/json", `{"source":"n8n"}`)
|
||||
|
||||
_, err := parseWebhookThoughtRequest(req)
|
||||
if err == nil || !strings.Contains(err.Error(), "content is required") {
|
||||
t.Fatalf("error = %v, want content required", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildWebhookMetadataUsesHeaderIdempotencyAndSanitizesMetadata(t *testing.T) {
|
||||
now := time.Date(2026, 7, 15, 4, 0, 0, 0, time.UTC)
|
||||
got := buildWebhookMetadata(webhookThoughtRequest{
|
||||
ExternalID: " ext-1 ",
|
||||
Metadata: map[string]any{
|
||||
"service": "n8n",
|
||||
"unsafe": struct{}{},
|
||||
"nested": map[string]any{"ok": true},
|
||||
},
|
||||
}, " key-1 ", now)
|
||||
|
||||
if got.IDempotencyKey != "key-1" {
|
||||
t.Fatalf("IDempotencyKey = %q, want key-1", got.IDempotencyKey)
|
||||
}
|
||||
if got.ExternalID != "ext-1" {
|
||||
t.Fatalf("ExternalID = %q, want ext-1", got.ExternalID)
|
||||
}
|
||||
if got.ReceivedAt != "2026-07-15T04:00:00Z" {
|
||||
t.Fatalf("ReceivedAt = %q", got.ReceivedAt)
|
||||
}
|
||||
if _, ok := got.SourceMetadata["unsafe"]; ok {
|
||||
t.Fatal("unsafe metadata value was not removed")
|
||||
}
|
||||
if got.SourceMetadata["service"] != "n8n" {
|
||||
t.Fatalf("service metadata = %#v", got.SourceMetadata["service"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizeWebhookThoughtMetadata(t *testing.T) {
|
||||
webhookMeta := buildWebhookMetadata(webhookThoughtRequest{IDempotencyKey: "abc"}, "", time.Date(2026, 7, 15, 4, 0, 0, 0, time.UTC))
|
||||
got := normalizeWebhookThoughtMetadata(webhookThoughtRequest{
|
||||
Source: "github",
|
||||
Type: "task",
|
||||
Topics: []string{"ci", "ci", ""},
|
||||
People: []string{" Sam "},
|
||||
}, webhookMeta, config.CaptureConfig{})
|
||||
|
||||
if got.Source != "github" {
|
||||
t.Fatalf("Source = %q, want github", got.Source)
|
||||
}
|
||||
if got.Type != "task" {
|
||||
t.Fatalf("Type = %q, want task", got.Type)
|
||||
}
|
||||
if len(got.Topics) != 1 || got.Topics[0] != "ci" {
|
||||
t.Fatalf("Topics = %#v, want [ci]", got.Topics)
|
||||
}
|
||||
if got.Webhook == nil || got.Webhook.IDempotencyKey != "abc" {
|
||||
t.Fatalf("Webhook = %#v, want idempotency key abc", got.Webhook)
|
||||
}
|
||||
}
|
||||
|
||||
func httptestRequest(contentType, body string) *http.Request {
|
||||
req := httptest.NewRequest(http.MethodPost, "/webhooks/thoughts", strings.NewReader(body))
|
||||
req.Header.Set("Content-Type", contentType)
|
||||
return req
|
||||
}
|
||||
@@ -11,6 +11,7 @@ import (
|
||||
"git.warky.dev/wdevs/amcs/internal/config"
|
||||
"git.warky.dev/wdevs/amcs/internal/observability"
|
||||
"git.warky.dev/wdevs/amcs/internal/requestip"
|
||||
"git.warky.dev/wdevs/amcs/internal/tenancy"
|
||||
)
|
||||
|
||||
type contextKey string
|
||||
@@ -50,6 +51,10 @@ func Middleware(cfg config.AuthConfig, keyring *Keyring, oauthRegistry *OAuthReg
|
||||
)
|
||||
}
|
||||
}
|
||||
withTenant := func(ctx context.Context, keyID string) context.Context {
|
||||
ctx = context.WithValue(ctx, keyIDContextKey, keyID)
|
||||
return tenancy.WithTenantKey(ctx, keyID)
|
||||
}
|
||||
return func(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
remoteAddr := requestip.FromRequest(r)
|
||||
@@ -63,7 +68,7 @@ func Middleware(cfg config.AuthConfig, keyring *Keyring, oauthRegistry *OAuthReg
|
||||
return
|
||||
}
|
||||
recordAccess(r, keyID)
|
||||
next.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), keyIDContextKey, keyID)))
|
||||
next.ServeHTTP(w, r.WithContext(withTenant(r.Context(), keyID)))
|
||||
return
|
||||
}
|
||||
}
|
||||
@@ -73,14 +78,14 @@ func Middleware(cfg config.AuthConfig, keyring *Keyring, oauthRegistry *OAuthReg
|
||||
if tokenStore != nil {
|
||||
if keyID, ok := tokenStore.Lookup(bearer); ok {
|
||||
recordAccess(r, keyID)
|
||||
next.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), keyIDContextKey, keyID)))
|
||||
next.ServeHTTP(w, r.WithContext(withTenant(r.Context(), keyID)))
|
||||
return
|
||||
}
|
||||
}
|
||||
if keyring != nil {
|
||||
if keyID, ok := keyring.Lookup(bearer); ok {
|
||||
recordAccess(r, keyID)
|
||||
next.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), keyIDContextKey, keyID)))
|
||||
next.ServeHTTP(w, r.WithContext(withTenant(r.Context(), keyID)))
|
||||
return
|
||||
}
|
||||
}
|
||||
@@ -103,7 +108,7 @@ func Middleware(cfg config.AuthConfig, keyring *Keyring, oauthRegistry *OAuthReg
|
||||
return
|
||||
}
|
||||
recordAccess(r, keyID)
|
||||
next.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), keyIDContextKey, keyID)))
|
||||
next.ServeHTTP(w, r.WithContext(withTenant(r.Context(), keyID)))
|
||||
return
|
||||
}
|
||||
|
||||
@@ -117,7 +122,7 @@ func Middleware(cfg config.AuthConfig, keyring *Keyring, oauthRegistry *OAuthReg
|
||||
return
|
||||
}
|
||||
recordAccess(r, keyID)
|
||||
next.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), keyIDContextKey, keyID)))
|
||||
next.ServeHTTP(w, r.WithContext(withTenant(r.Context(), keyID)))
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
"git.warky.dev/wdevs/amcs/internal/config"
|
||||
"git.warky.dev/wdevs/amcs/internal/tenancy"
|
||||
)
|
||||
|
||||
func TestMiddlewareAddsTenantKeyToContext(t *testing.T) {
|
||||
keyring, err := NewKeyring([]config.APIKey{{ID: "user-a", Value: "secret-a"}})
|
||||
if err != nil {
|
||||
t.Fatalf("NewKeyring error = %v", err)
|
||||
}
|
||||
|
||||
var gotKeyID, gotTenant string
|
||||
handler := Middleware(config.AuthConfig{}, keyring, nil, nil, nil, slog.Default())(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
var ok bool
|
||||
gotKeyID, ok = KeyIDFromContext(r.Context())
|
||||
if !ok {
|
||||
t.Fatal("KeyIDFromContext ok = false")
|
||||
}
|
||||
gotTenant, ok = tenancy.KeyFromContext(r.Context())
|
||||
if !ok {
|
||||
t.Fatal("tenancy.KeyFromContext ok = false")
|
||||
}
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}))
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/mcp", nil)
|
||||
req.Header.Set("x-brain-key", "secret-a")
|
||||
rr := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rr, req)
|
||||
|
||||
if rr.Code != http.StatusNoContent {
|
||||
t.Fatalf("status = %d, want %d", rr.Code, http.StatusNoContent)
|
||||
}
|
||||
if gotKeyID != "user-a" || gotTenant != "user-a" {
|
||||
t.Fatalf("keyID=%q tenant=%q, want user-a/user-a", gotKeyID, gotTenant)
|
||||
}
|
||||
}
|
||||
@@ -28,6 +28,7 @@ type ToolSet struct {
|
||||
Update *tools.UpdateTool
|
||||
Delete *tools.DeleteTool
|
||||
Archive *tools.ArchiveTool
|
||||
DuplicateAudit *tools.DuplicateAuditTool
|
||||
Projects *tools.ProjectsTool
|
||||
Context *tools.ContextTool
|
||||
Recall *tools.RecallTool
|
||||
@@ -227,6 +228,12 @@ func registerThoughtTools(server *mcp.Server, logger *slog.Logger, toolSet ToolS
|
||||
}, toolSet.Archive.Handle); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := addTool(server, logger, &mcp.Tool{
|
||||
Name: "audit_duplicates",
|
||||
Description: "Dry-run duplicate audit for projects, thoughts, and metadata normalization candidates; performs no writes.",
|
||||
}, toolSet.DuplicateAudit.Handle); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := addTool(server, logger, &mcp.Tool{
|
||||
Name: "summarize_thoughts",
|
||||
Description: "LLM summary of a filtered set of thoughts.",
|
||||
@@ -764,6 +771,7 @@ func BuildToolCatalog() []tools.ToolEntry {
|
||||
{Name: "update_thought", Description: "Update thought content or merge metadata.", Category: "thoughts"},
|
||||
{Name: "delete_thought", Description: "Hard-delete a thought by id.", Category: "thoughts"},
|
||||
{Name: "archive_thought", Description: "Archive a thought so it is hidden from default search and listing.", Category: "thoughts"},
|
||||
{Name: "audit_duplicates", Description: "Dry-run duplicate audit for exact/normalized project names, thought content, and metadata value variants. Reports candidates and recommendations; performs no writes.", Category: "admin"},
|
||||
{Name: "summarize_thoughts", Description: "Produce an LLM prose summary of a filtered or searched set of thoughts.", Category: "thoughts"},
|
||||
{Name: "recall_context", Description: "Recall semantically relevant and recent context for prompt injection. Combines vector similarity with recency. Falls back to full-text search when no embeddings exist.", Category: "thoughts"},
|
||||
{Name: "link_thoughts", Description: "Create a typed relationship between two thoughts.", Category: "thoughts"},
|
||||
|
||||
@@ -53,6 +53,7 @@ func Normalize(in thoughttypes.ThoughtMetadata, capture config.CaptureConfig) th
|
||||
Type: normalizeType(in.Type),
|
||||
Source: normalizeSource(in.Source),
|
||||
Attachments: normalizeAttachments(in.Attachments),
|
||||
Webhook: normalizeWebhook(in.Webhook),
|
||||
MetadataStatus: normalizeMetadataStatus(in.MetadataStatus),
|
||||
MetadataUpdatedAt: strings.TrimSpace(in.MetadataUpdatedAt),
|
||||
MetadataLastAttemptedAt: strings.TrimSpace(in.MetadataLastAttemptedAt),
|
||||
@@ -201,10 +202,31 @@ func Merge(base, patch thoughttypes.ThoughtMetadata, capture config.CaptureConfi
|
||||
if len(patch.Attachments) > 0 {
|
||||
merged.Attachments = append(append([]thoughttypes.ThoughtAttachment{}, merged.Attachments...), patch.Attachments...)
|
||||
}
|
||||
if patch.Webhook != nil {
|
||||
merged.Webhook = patch.Webhook
|
||||
}
|
||||
|
||||
return Normalize(merged, capture)
|
||||
}
|
||||
|
||||
func normalizeWebhook(value *thoughttypes.WebhookMetadata) *thoughttypes.WebhookMetadata {
|
||||
if value == nil {
|
||||
return nil
|
||||
}
|
||||
out := &thoughttypes.WebhookMetadata{
|
||||
ReceivedAt: strings.TrimSpace(value.ReceivedAt),
|
||||
IDempotencyKey: strings.TrimSpace(value.IDempotencyKey),
|
||||
ExternalID: strings.TrimSpace(value.ExternalID),
|
||||
}
|
||||
if len(value.SourceMetadata) > 0 {
|
||||
out.SourceMetadata = value.SourceMetadata
|
||||
}
|
||||
if out.ReceivedAt == "" && out.IDempotencyKey == "" && out.ExternalID == "" && len(out.SourceMetadata) == 0 {
|
||||
return nil
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func normalizeAttachments(values []thoughttypes.ThoughtAttachment) []thoughttypes.ThoughtAttachment {
|
||||
seen := make(map[string]struct{}, len(values))
|
||||
result := make([]thoughttypes.ThoughtAttachment, 0, len(values))
|
||||
|
||||
@@ -15,10 +15,10 @@ import (
|
||||
|
||||
func (db *DB) InsertStoredFile(ctx context.Context, file thoughttypes.StoredFile) (thoughttypes.StoredFile, error) {
|
||||
row := db.pool.QueryRow(ctx, `
|
||||
insert into stored_files (thought_id, project_id, name, media_type, kind, encoding, size_bytes, sha256, content)
|
||||
values ($1, $2, $3, $4, $5, $6, $7, $8, $9)
|
||||
insert into stored_files (thought_id, project_id, tenant_key, name, media_type, kind, encoding, size_bytes, sha256, content)
|
||||
values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
|
||||
returning id, guid, thought_id, project_id, name, media_type, kind, encoding, size_bytes, sha256, created_at, updated_at
|
||||
`, file.ThoughtID, file.ProjectID, file.Name, file.MediaType, file.Kind, file.Encoding, file.SizeBytes, file.SHA256, file.Content)
|
||||
`, file.ThoughtID, file.ProjectID, tenantKeyPtr(ctx), file.Name, file.MediaType, file.Kind, file.Encoding, file.SizeBytes, file.SHA256, file.Content)
|
||||
|
||||
var model generatedmodels.ModelPublicStoredFiles
|
||||
if err := row.Scan(
|
||||
@@ -42,11 +42,11 @@ func (db *DB) InsertStoredFile(ctx context.Context, file thoughttypes.StoredFile
|
||||
}
|
||||
|
||||
func (db *DB) GetStoredFile(ctx context.Context, id uuid.UUID) (thoughttypes.StoredFile, error) {
|
||||
args := []any{id}
|
||||
row := db.pool.QueryRow(ctx, `
|
||||
select id, guid, thought_id, project_id, name, media_type, kind, encoding, size_bytes, sha256, content, created_at, updated_at
|
||||
from stored_files
|
||||
where guid = $1
|
||||
`, id)
|
||||
where guid = $1`+tenantSQL(ctx, &args, "tenant_key"), args...)
|
||||
|
||||
var model generatedmodels.ModelPublicStoredFiles
|
||||
if err := row.Scan(
|
||||
@@ -77,6 +77,7 @@ func (db *DB) ListStoredFiles(ctx context.Context, filter thoughttypes.StoredFil
|
||||
args := make([]any, 0, 4)
|
||||
conditions := make([]string, 0, 3)
|
||||
|
||||
addTenantCondition(ctx, &args, &conditions, "tenant_key")
|
||||
if filter.ThoughtID != nil {
|
||||
args = append(args, *filter.ThoughtID)
|
||||
conditions = append(conditions, fmt.Sprintf("thought_id = $%d", len(args)))
|
||||
|
||||
@@ -14,10 +14,10 @@ import (
|
||||
|
||||
func (db *DB) CreateProject(ctx context.Context, name, description string) (thoughttypes.Project, error) {
|
||||
row := db.pool.QueryRow(ctx, `
|
||||
insert into projects (name, description)
|
||||
values ($1, $2)
|
||||
insert into projects (name, description, tenant_key)
|
||||
values ($1, $2, $3)
|
||||
returning id, guid, name, description, created_at, last_active_at
|
||||
`, name, description)
|
||||
`, name, description, tenantKeyPtr(ctx))
|
||||
|
||||
var model generatedmodels.ModelPublicProjects
|
||||
if err := row.Scan(&model.ID, &model.GUID, &model.Name, &model.Description, &model.CreatedAt, &model.LastActiveAt); err != nil {
|
||||
@@ -45,20 +45,20 @@ func (db *DB) GetProject(ctx context.Context, nameOrID string) (thoughttypes.Pro
|
||||
}
|
||||
|
||||
func (db *DB) getProjectByGUID(ctx context.Context, id uuid.UUID) (thoughttypes.Project, error) {
|
||||
args := []any{id}
|
||||
row := db.pool.QueryRow(ctx, `
|
||||
select id, guid, name, description, created_at, last_active_at
|
||||
from projects
|
||||
where guid = $1
|
||||
`, id)
|
||||
where guid = $1`+tenantSQL(ctx, &args, "tenant_key"), args...)
|
||||
return scanProject(row)
|
||||
}
|
||||
|
||||
func (db *DB) getProjectByName(ctx context.Context, name string) (thoughttypes.Project, error) {
|
||||
args := []any{name}
|
||||
row := db.pool.QueryRow(ctx, `
|
||||
select id, guid, name, description, created_at, last_active_at
|
||||
from projects
|
||||
where name = $1
|
||||
`, name)
|
||||
where name = $1`+tenantSQL(ctx, &args, "tenant_key"), args...)
|
||||
return scanProject(row)
|
||||
}
|
||||
|
||||
@@ -74,13 +74,20 @@ func scanProject(row pgx.Row) (thoughttypes.Project, error) {
|
||||
}
|
||||
|
||||
func (db *DB) ListProjects(ctx context.Context) ([]thoughttypes.ProjectSummary, error) {
|
||||
args := []any{}
|
||||
where := ""
|
||||
if key, ok := tenantKey(ctx); ok {
|
||||
args = append(args, key)
|
||||
where = "where p.tenant_key = $1"
|
||||
}
|
||||
rows, err := db.pool.Query(ctx, `
|
||||
select p.id, p.guid, p.name, p.description, p.created_at, p.last_active_at, count(t.id) as thought_count
|
||||
from projects p
|
||||
left join thoughts t on t.project_id = p.id and t.archived_at is null
|
||||
`+where+`
|
||||
group by p.id, p.guid, p.name, p.description, p.created_at, p.last_active_at
|
||||
order by p.last_active_at desc, p.created_at desc
|
||||
`)
|
||||
`, args...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list projects: %w", err)
|
||||
}
|
||||
@@ -105,7 +112,8 @@ func (db *DB) ListProjects(ctx context.Context) ([]thoughttypes.ProjectSummary,
|
||||
}
|
||||
|
||||
func (db *DB) TouchProject(ctx context.Context, id int64) error {
|
||||
tag, err := db.pool.Exec(ctx, `update projects set last_active_at = now() where id = $1`, id)
|
||||
args := []any{id}
|
||||
tag, err := db.pool.Exec(ctx, `update projects set last_active_at = now() where id = $1`+tenantSQL(ctx, &args, "tenant_key"), args...)
|
||||
if err != nil {
|
||||
return fmt.Errorf("touch project: %w", err)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"git.warky.dev/wdevs/amcs/internal/tenancy"
|
||||
)
|
||||
|
||||
func tenantKeyPtr(ctx context.Context) *string {
|
||||
if key, ok := tenancy.KeyFromContext(ctx); ok {
|
||||
return &key
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func tenantKey(ctx context.Context) (string, bool) {
|
||||
return tenancy.KeyFromContext(ctx)
|
||||
}
|
||||
|
||||
func addTenantCondition(ctx context.Context, args *[]any, conditions *[]string, column string) {
|
||||
if key, ok := tenancy.KeyFromContext(ctx); ok {
|
||||
*args = append(*args, key)
|
||||
*conditions = append(*conditions, fmt.Sprintf("%s = $%d", column, len(*args)))
|
||||
}
|
||||
}
|
||||
|
||||
func tenantSQL(ctx context.Context, args *[]any, column string) string {
|
||||
if key, ok := tenancy.KeyFromContext(ctx); ok {
|
||||
*args = append(*args, key)
|
||||
return fmt.Sprintf(" and %s = $%d", column, len(*args))
|
||||
}
|
||||
return ""
|
||||
}
|
||||
+42
-15
@@ -31,10 +31,10 @@ func (db *DB) InsertThought(ctx context.Context, thought thoughttypes.Thought, e
|
||||
}()
|
||||
|
||||
row := tx.QueryRow(ctx, `
|
||||
insert into thoughts (content, metadata, project_id)
|
||||
values ($1, $2::jsonb, $3)
|
||||
insert into thoughts (content, metadata, project_id, tenant_key)
|
||||
values ($1, $2::jsonb, $3, $4)
|
||||
returning id, guid, created_at, updated_at
|
||||
`, thought.Content, metadata, thought.ProjectID)
|
||||
`, thought.Content, metadata, thought.ProjectID, tenantKeyPtr(ctx))
|
||||
|
||||
created := thought
|
||||
created.Embedding = nil
|
||||
@@ -68,6 +68,22 @@ func (db *DB) InsertThought(ctx context.Context, thought thoughttypes.Thought, e
|
||||
return created, nil
|
||||
}
|
||||
|
||||
func (db *DB) GetThoughtByWebhookIDempotencyKey(ctx context.Context, key string) (thoughttypes.Thought, error) {
|
||||
row := db.pool.QueryRow(ctx, `
|
||||
select id, guid, content, metadata, project_id, archived_at, created_at, updated_at
|
||||
from thoughts
|
||||
where metadata->'webhook'->>'idempotency_key' = $1
|
||||
order by created_at desc
|
||||
limit 1
|
||||
`, strings.TrimSpace(key))
|
||||
|
||||
var model generatedmodels.ModelPublicThoughts
|
||||
if err := row.Scan(&model.ID, &model.GUID, &model.Content, &model.Metadata, &model.ProjectID, &model.ArchivedAt, &model.CreatedAt, &model.UpdatedAt); err != nil {
|
||||
return thoughttypes.Thought{}, err
|
||||
}
|
||||
return thoughtFromModel(model)
|
||||
}
|
||||
|
||||
func (db *DB) SearchThoughts(ctx context.Context, embedding []float32, embeddingModel string, threshold float64, limit int, filter map[string]any) ([]thoughttypes.SearchResult, error) {
|
||||
filterJSON, err := json.Marshal(filter)
|
||||
if err != nil {
|
||||
@@ -107,6 +123,7 @@ func (db *DB) ListThoughts(ctx context.Context, filter thoughttypes.ListFilter)
|
||||
args := make([]any, 0, 6)
|
||||
conditions := []string{}
|
||||
|
||||
addTenantCondition(ctx, &args, &conditions, "tenant_key")
|
||||
if !filter.IncludeArchived {
|
||||
conditions = append(conditions, "archived_at is null")
|
||||
}
|
||||
@@ -170,11 +187,14 @@ func (db *DB) ListThoughts(ctx context.Context, filter thoughttypes.ListFilter)
|
||||
|
||||
func (db *DB) Stats(ctx context.Context) (thoughttypes.ThoughtStats, error) {
|
||||
var total int
|
||||
if err := db.pool.QueryRow(ctx, `select count(*) from thoughts where archived_at is null`).Scan(&total); err != nil {
|
||||
statsArgs := []any{}
|
||||
statsConditions := []string{"archived_at is null"}
|
||||
addTenantCondition(ctx, &statsArgs, &statsConditions, "tenant_key")
|
||||
if err := db.pool.QueryRow(ctx, `select count(*) from thoughts where `+strings.Join(statsConditions, " and "), statsArgs...).Scan(&total); err != nil {
|
||||
return thoughttypes.ThoughtStats{}, fmt.Errorf("count thoughts: %w", err)
|
||||
}
|
||||
|
||||
rows, err := db.pool.Query(ctx, `select metadata from thoughts where archived_at is null`)
|
||||
rows, err := db.pool.Query(ctx, `select metadata from thoughts where `+strings.Join(statsConditions, " and "), statsArgs...)
|
||||
if err != nil {
|
||||
return thoughttypes.ThoughtStats{}, fmt.Errorf("query stats metadata: %w", err)
|
||||
}
|
||||
@@ -217,11 +237,11 @@ func (db *DB) Stats(ctx context.Context) (thoughttypes.ThoughtStats, error) {
|
||||
}
|
||||
|
||||
func (db *DB) GetThought(ctx context.Context, id uuid.UUID) (thoughttypes.Thought, error) {
|
||||
args := []any{id}
|
||||
row := db.pool.QueryRow(ctx, `
|
||||
select id, guid, content, metadata, project_id, archived_at, created_at, updated_at
|
||||
from thoughts
|
||||
where guid = $1
|
||||
`, id)
|
||||
where guid = $1`+tenantSQL(ctx, &args, "tenant_key"), args...)
|
||||
|
||||
var model generatedmodels.ModelPublicThoughts
|
||||
if err := row.Scan(&model.ID, &model.GUID, &model.Content, &model.Metadata, &model.ProjectID, &model.ArchivedAt, &model.CreatedAt, &model.UpdatedAt); err != nil {
|
||||
@@ -240,11 +260,11 @@ func (db *DB) GetThought(ctx context.Context, id uuid.UUID) (thoughttypes.Though
|
||||
}
|
||||
|
||||
func (db *DB) GetThoughtByID(ctx context.Context, id int64) (thoughttypes.Thought, error) {
|
||||
args := []any{id}
|
||||
row := db.pool.QueryRow(ctx, `
|
||||
select id, guid, content, metadata, project_id, archived_at, created_at, updated_at
|
||||
from thoughts
|
||||
where id = $1
|
||||
`, id)
|
||||
where id = $1`+tenantSQL(ctx, &args, "tenant_key"), args...)
|
||||
|
||||
var model generatedmodels.ModelPublicThoughts
|
||||
if err := row.Scan(&model.ID, &model.GUID, &model.Content, &model.Metadata, &model.ProjectID, &model.ArchivedAt, &model.CreatedAt, &model.UpdatedAt); err != nil {
|
||||
@@ -276,14 +296,14 @@ func (db *DB) UpdateThought(ctx context.Context, id uuid.UUID, content string, e
|
||||
_ = tx.Rollback(ctx)
|
||||
}()
|
||||
|
||||
args := []any{id, content, metadataBytes, projectID}
|
||||
tag, err := tx.Exec(ctx, `
|
||||
update thoughts
|
||||
set content = $2,
|
||||
metadata = $3::jsonb,
|
||||
project_id = $4,
|
||||
updated_at = now()
|
||||
where guid = $1
|
||||
`, id, content, metadataBytes, projectID)
|
||||
where guid = $1`+tenantSQL(ctx, &args, "tenant_key"), args...)
|
||||
if err != nil {
|
||||
return thoughttypes.Thought{}, fmt.Errorf("update thought: %w", err)
|
||||
}
|
||||
@@ -317,12 +337,12 @@ func (db *DB) UpdateThoughtMetadata(ctx context.Context, id int64, metadata thou
|
||||
return thoughttypes.Thought{}, fmt.Errorf("marshal updated metadata: %w", err)
|
||||
}
|
||||
|
||||
args := []any{id, metadataBytes}
|
||||
tag, err := db.pool.Exec(ctx, `
|
||||
update thoughts
|
||||
set metadata = $2::jsonb,
|
||||
updated_at = now()
|
||||
where id = $1
|
||||
`, id, metadataBytes)
|
||||
where id = $1`+tenantSQL(ctx, &args, "tenant_key"), args...)
|
||||
if err != nil {
|
||||
return thoughttypes.Thought{}, fmt.Errorf("update thought metadata: %w", err)
|
||||
}
|
||||
@@ -334,7 +354,8 @@ func (db *DB) UpdateThoughtMetadata(ctx context.Context, id int64, metadata thou
|
||||
}
|
||||
|
||||
func (db *DB) DeleteThought(ctx context.Context, id uuid.UUID) error {
|
||||
tag, err := db.pool.Exec(ctx, `delete from thoughts where guid = $1`, id)
|
||||
args := []any{id}
|
||||
tag, err := db.pool.Exec(ctx, `delete from thoughts where guid = $1`+tenantSQL(ctx, &args, "tenant_key"), args...)
|
||||
if err != nil {
|
||||
return fmt.Errorf("delete thought: %w", err)
|
||||
}
|
||||
@@ -345,7 +366,8 @@ func (db *DB) DeleteThought(ctx context.Context, id uuid.UUID) error {
|
||||
}
|
||||
|
||||
func (db *DB) ArchiveThought(ctx context.Context, id uuid.UUID) error {
|
||||
tag, err := db.pool.Exec(ctx, `update thoughts set archived_at = now(), updated_at = now() where guid = $1`, id)
|
||||
args := []any{id}
|
||||
tag, err := db.pool.Exec(ctx, `update thoughts set archived_at = now(), updated_at = now() where guid = $1`+tenantSQL(ctx, &args, "tenant_key"), args...)
|
||||
if err != nil {
|
||||
return fmt.Errorf("archive thought: %w", err)
|
||||
}
|
||||
@@ -424,6 +446,7 @@ func (db *DB) SearchSimilarThoughts(ctx context.Context, embedding []float32, em
|
||||
"1 - (e.embedding <=> $1) > $2",
|
||||
"e.model = $3",
|
||||
}
|
||||
addTenantCondition(ctx, &args, &conditions, "t.tenant_key")
|
||||
if projectID != nil {
|
||||
args = append(args, *projectID)
|
||||
conditions = append(conditions, fmt.Sprintf("t.project_id = $%d", len(args)))
|
||||
@@ -472,6 +495,7 @@ func (db *DB) HasEmbeddingsForModel(ctx context.Context, model string, projectID
|
||||
"e.model = $1",
|
||||
"t.archived_at is null",
|
||||
}
|
||||
addTenantCondition(ctx, &args, &conditions, "t.tenant_key")
|
||||
if projectID != nil {
|
||||
args = append(args, *projectID)
|
||||
conditions = append(conditions, fmt.Sprintf("t.project_id = $%d", len(args)))
|
||||
@@ -490,6 +514,7 @@ func (db *DB) HasEmbeddingsForModel(ctx context.Context, model string, projectID
|
||||
func (db *DB) ListThoughtsMissingEmbedding(ctx context.Context, model string, limit int, projectID *int64, includeArchived bool, olderThanDays int) ([]thoughttypes.Thought, error) {
|
||||
args := []any{model}
|
||||
conditions := []string{"e.id is null"}
|
||||
addTenantCondition(ctx, &args, &conditions, "t.tenant_key")
|
||||
|
||||
if !includeArchived {
|
||||
conditions = append(conditions, "t.archived_at is null")
|
||||
@@ -539,6 +564,7 @@ func (db *DB) ListThoughtsMissingEmbedding(ctx context.Context, model string, li
|
||||
func (db *DB) ListThoughtsForMetadataReparse(ctx context.Context, limit int, projectID *int64, includeArchived bool, olderThanDays int) ([]thoughttypes.Thought, error) {
|
||||
args := make([]any, 0, 3)
|
||||
conditions := make([]string, 0, 4)
|
||||
addTenantCondition(ctx, &args, &conditions, "tenant_key")
|
||||
|
||||
if !includeArchived {
|
||||
conditions = append(conditions, "archived_at is null")
|
||||
@@ -608,6 +634,7 @@ func (db *DB) SearchThoughtsText(ctx context.Context, query string, limit int, p
|
||||
"t.archived_at is null",
|
||||
"(to_tsvector('simple', t.content) || to_tsvector('simple', coalesce(p.name, ''))) @@ websearch_to_tsquery('simple', $1)",
|
||||
}
|
||||
addTenantCondition(ctx, &args, &conditions, "t.tenant_key")
|
||||
if projectID != nil {
|
||||
args = append(args, *projectID)
|
||||
conditions = append(conditions, fmt.Sprintf("t.project_id = $%d", len(args)))
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
package tenancy
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type contextKey string
|
||||
|
||||
const tenantKeyContextKey contextKey = "tenancy.tenant_key"
|
||||
|
||||
// WithTenantKey returns a context scoped to the authenticated tenant boundary.
|
||||
// The tenant key is intentionally opaque; today it is the authenticated API key
|
||||
// or OAuth client id, and callers should not interpret it as a human username.
|
||||
func WithTenantKey(ctx context.Context, tenantKey string) context.Context {
|
||||
tenantKey = strings.TrimSpace(tenantKey)
|
||||
if tenantKey == "" {
|
||||
return ctx
|
||||
}
|
||||
return context.WithValue(ctx, tenantKeyContextKey, tenantKey)
|
||||
}
|
||||
|
||||
func KeyFromContext(ctx context.Context) (string, bool) {
|
||||
if ctx == nil {
|
||||
return "", false
|
||||
}
|
||||
value, ok := ctx.Value(tenantKeyContextKey).(string)
|
||||
value = strings.TrimSpace(value)
|
||||
return value, ok && value != ""
|
||||
}
|
||||
@@ -39,6 +39,7 @@ type ProjectContextOutput struct {
|
||||
Project thoughttypes.Project `json:"project"`
|
||||
Context string `json:"context"`
|
||||
Items []ContextItem `json:"items"`
|
||||
RetrievalMode string `json:"retrieval_mode,omitempty"`
|
||||
}
|
||||
|
||||
func NewContextTool(db *store.DB, embeddings *ai.EmbeddingRunner, search config.SearchConfig, sessions *session.ActiveProjects) *ContextTool {
|
||||
@@ -70,12 +71,14 @@ func (t *ContextTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in P
|
||||
})
|
||||
}
|
||||
|
||||
var retrievalMode string
|
||||
query := strings.TrimSpace(in.Query)
|
||||
if query != "" {
|
||||
semantic, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, t.search.DefaultThreshold, &project.NumericID, nil)
|
||||
semantic, mode, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, t.search.DefaultThreshold, &project.NumericID, nil)
|
||||
if err != nil {
|
||||
return nil, ProjectContextOutput{}, err
|
||||
}
|
||||
retrievalMode = mode
|
||||
for _, result := range semantic {
|
||||
key := fmt.Sprint(result.ID)
|
||||
if _, ok := seen[key]; ok {
|
||||
@@ -103,5 +106,6 @@ func (t *ContextTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in P
|
||||
Project: *project,
|
||||
Context: contextBlock,
|
||||
Items: items,
|
||||
RetrievalMode: retrievalMode,
|
||||
}, nil
|
||||
}
|
||||
|
||||
@@ -0,0 +1,305 @@
|
||||
package tools
|
||||
|
||||
import (
|
||||
"context"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/modelcontextprotocol/go-sdk/mcp"
|
||||
|
||||
"git.warky.dev/wdevs/amcs/internal/config"
|
||||
"git.warky.dev/wdevs/amcs/internal/session"
|
||||
"git.warky.dev/wdevs/amcs/internal/store"
|
||||
thoughttypes "git.warky.dev/wdevs/amcs/internal/types"
|
||||
)
|
||||
|
||||
const defaultDuplicateAuditLimit = 50
|
||||
|
||||
var duplicateWhitespace = regexp.MustCompile(`\s+`)
|
||||
|
||||
type DuplicateAuditTool struct {
|
||||
store *store.DB
|
||||
search config.SearchConfig
|
||||
sessions *session.ActiveProjects
|
||||
}
|
||||
|
||||
type DuplicateAuditInput struct {
|
||||
Project string `json:"project,omitempty" jsonschema:"optional project name or id to scope thought duplicate scanning"`
|
||||
Limit int `json:"limit,omitempty" jsonschema:"maximum candidate groups to return"`
|
||||
ThoughtLimit int `json:"thought_limit,omitempty" jsonschema:"maximum thoughts to scan for duplicate content; defaults to search limit"`
|
||||
IncludeArchived bool `json:"include_archived,omitempty" jsonschema:"include archived thoughts in the scan"`
|
||||
}
|
||||
|
||||
type DuplicateAuditOutput struct {
|
||||
Report DuplicateAuditReport `json:"report"`
|
||||
}
|
||||
|
||||
type DuplicateAuditReport struct {
|
||||
Summary DuplicateAuditSummary `json:"summary"`
|
||||
ProjectCandidates []DuplicateProjectCandidate `json:"project_candidates"`
|
||||
ThoughtCandidates []DuplicateThoughtCandidate `json:"thought_candidates"`
|
||||
MetadataCandidates []MetadataCleanupCandidate `json:"metadata_candidates"`
|
||||
RecommendedNextStep string `json:"recommended_next_step"`
|
||||
}
|
||||
|
||||
type DuplicateAuditSummary struct {
|
||||
DryRun bool `json:"dry_run"`
|
||||
ScannedProjects int `json:"scanned_projects"`
|
||||
ScannedThoughts int `json:"scanned_thoughts"`
|
||||
ProjectCandidateGroups int `json:"project_candidate_groups"`
|
||||
ThoughtCandidateGroups int `json:"thought_candidate_groups"`
|
||||
MetadataCandidateGroups int `json:"metadata_candidate_groups"`
|
||||
Truncated bool `json:"truncated"`
|
||||
}
|
||||
|
||||
type DuplicateProjectCandidate struct {
|
||||
MatchType string `json:"match_type"`
|
||||
NormalizedValue string `json:"normalized_value"`
|
||||
Confidence float64 `json:"confidence"`
|
||||
RecommendedCanonicalID uuid.UUID `json:"recommended_canonical_id"`
|
||||
Projects []thoughttypes.ProjectSummary `json:"projects"`
|
||||
}
|
||||
|
||||
type DuplicateThoughtCandidate struct {
|
||||
MatchType string `json:"match_type"`
|
||||
NormalizedValue string `json:"normalized_value"`
|
||||
Confidence float64 `json:"confidence"`
|
||||
RecommendedCanonicalID uuid.UUID `json:"recommended_canonical_id"`
|
||||
Thoughts []thoughttypes.Thought `json:"thoughts"`
|
||||
}
|
||||
|
||||
type MetadataCleanupCandidate struct {
|
||||
Field string `json:"field"`
|
||||
NormalizedValue string `json:"normalized_value"`
|
||||
Values []string `json:"values"`
|
||||
Occurrences int `json:"occurrences"`
|
||||
RecommendedValue string `json:"recommended_value"`
|
||||
RecommendedAction string `json:"recommended_action"`
|
||||
}
|
||||
|
||||
func NewDuplicateAuditTool(db *store.DB, search config.SearchConfig, sessions *session.ActiveProjects) *DuplicateAuditTool {
|
||||
return &DuplicateAuditTool{store: db, search: search, sessions: sessions}
|
||||
}
|
||||
|
||||
func (t *DuplicateAuditTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in DuplicateAuditInput) (*mcp.CallToolResult, DuplicateAuditOutput, error) {
|
||||
project, err := resolveProject(ctx, t.store, t.sessions, req, in.Project, false)
|
||||
if err != nil {
|
||||
return nil, DuplicateAuditOutput{}, err
|
||||
}
|
||||
var projectID *int64
|
||||
if project != nil {
|
||||
projectID = &project.NumericID
|
||||
}
|
||||
|
||||
projects, err := t.store.ListProjects(ctx)
|
||||
if err != nil {
|
||||
return nil, DuplicateAuditOutput{}, err
|
||||
}
|
||||
|
||||
thoughtLimit := normalizeLimit(in.ThoughtLimit, t.search)
|
||||
if in.ThoughtLimit <= 0 {
|
||||
thoughtLimit = normalizeLimit(0, t.search)
|
||||
}
|
||||
thoughts, err := t.store.ListThoughts(ctx, thoughttypes.ListFilter{Limit: thoughtLimit, ProjectID: projectID, IncludeArchived: in.IncludeArchived})
|
||||
if err != nil {
|
||||
return nil, DuplicateAuditOutput{}, err
|
||||
}
|
||||
|
||||
if project != nil {
|
||||
_ = t.store.TouchProject(ctx, project.NumericID)
|
||||
}
|
||||
return nil, DuplicateAuditOutput{Report: buildDuplicateAuditReport(projects, thoughts, in)}, nil
|
||||
}
|
||||
|
||||
func buildDuplicateAuditReport(projects []thoughttypes.ProjectSummary, thoughts []thoughttypes.Thought, in DuplicateAuditInput) DuplicateAuditReport {
|
||||
limit := in.Limit
|
||||
if limit <= 0 {
|
||||
limit = defaultDuplicateAuditLimit
|
||||
}
|
||||
|
||||
projectCandidates, projectTruncated := duplicateProjectCandidates(projects, limit)
|
||||
thoughtCandidates, thoughtTruncated := duplicateThoughtCandidates(thoughts, limit)
|
||||
metadataCandidates, metadataTruncated := metadataCleanupCandidates(thoughts, limit)
|
||||
|
||||
return DuplicateAuditReport{
|
||||
Summary: DuplicateAuditSummary{
|
||||
DryRun: true,
|
||||
ScannedProjects: len(projects),
|
||||
ScannedThoughts: len(thoughts),
|
||||
ProjectCandidateGroups: len(projectCandidates),
|
||||
ThoughtCandidateGroups: len(thoughtCandidates),
|
||||
MetadataCandidateGroups: len(metadataCandidates),
|
||||
Truncated: projectTruncated || thoughtTruncated || metadataTruncated,
|
||||
},
|
||||
ProjectCandidates: projectCandidates,
|
||||
ThoughtCandidates: thoughtCandidates,
|
||||
MetadataCandidates: metadataCandidates,
|
||||
RecommendedNextStep: "Review candidates manually, then use explicit archive/update/merge tools; this audit performs no writes.",
|
||||
}
|
||||
}
|
||||
|
||||
func duplicateProjectCandidates(projects []thoughttypes.ProjectSummary, limit int) ([]DuplicateProjectCandidate, bool) {
|
||||
groups := map[string][]thoughttypes.ProjectSummary{}
|
||||
for _, project := range projects {
|
||||
key := normalizeDuplicateText(project.Name)
|
||||
if key != "" {
|
||||
groups[key] = append(groups[key], project)
|
||||
}
|
||||
}
|
||||
keys := sortedDuplicateKeys(groups)
|
||||
out := make([]DuplicateProjectCandidate, 0, min(len(keys), limit))
|
||||
for _, key := range keys {
|
||||
items := groups[key]
|
||||
sort.SliceStable(items, func(i, j int) bool {
|
||||
if items[i].ThoughtCount != items[j].ThoughtCount {
|
||||
return items[i].ThoughtCount > items[j].ThoughtCount
|
||||
}
|
||||
return items[i].CreatedAt.Before(items[j].CreatedAt)
|
||||
})
|
||||
matchType := "normalized_name"
|
||||
if allProjectNamesExact(items) {
|
||||
matchType = "exact_name"
|
||||
}
|
||||
out = append(out, DuplicateProjectCandidate{MatchType: matchType, NormalizedValue: key, Confidence: 1.0, RecommendedCanonicalID: items[0].ID, Projects: items})
|
||||
if len(out) == limit {
|
||||
return out, len(keys) > limit
|
||||
}
|
||||
}
|
||||
return out, false
|
||||
}
|
||||
|
||||
func duplicateThoughtCandidates(thoughts []thoughttypes.Thought, limit int) ([]DuplicateThoughtCandidate, bool) {
|
||||
groups := map[string][]thoughttypes.Thought{}
|
||||
for _, thought := range thoughts {
|
||||
key := normalizeDuplicateText(thought.Content)
|
||||
if key != "" {
|
||||
groups[key] = append(groups[key], thought)
|
||||
}
|
||||
}
|
||||
keys := sortedDuplicateKeys(groups)
|
||||
out := make([]DuplicateThoughtCandidate, 0, min(len(keys), limit))
|
||||
for _, key := range keys {
|
||||
items := groups[key]
|
||||
sort.SliceStable(items, func(i, j int) bool { return items[i].CreatedAt.Before(items[j].CreatedAt) })
|
||||
matchType := "normalized_content"
|
||||
if allThoughtContentExact(items) {
|
||||
matchType = "exact_content"
|
||||
}
|
||||
out = append(out, DuplicateThoughtCandidate{MatchType: matchType, NormalizedValue: key, Confidence: 1.0, RecommendedCanonicalID: items[0].GUID, Thoughts: items})
|
||||
if len(out) == limit {
|
||||
return out, len(keys) > limit
|
||||
}
|
||||
}
|
||||
return out, false
|
||||
}
|
||||
|
||||
func metadataCleanupCandidates(thoughts []thoughttypes.Thought, limit int) ([]MetadataCleanupCandidate, bool) {
|
||||
groups := map[string]map[string]int{}
|
||||
add := func(field, value string) {
|
||||
trimmed := strings.TrimSpace(value)
|
||||
key := normalizeDuplicateText(trimmed)
|
||||
if key == "" || trimmed == "" {
|
||||
return
|
||||
}
|
||||
bucketKey := field + "\x00" + key
|
||||
if groups[bucketKey] == nil {
|
||||
groups[bucketKey] = map[string]int{}
|
||||
}
|
||||
groups[bucketKey][trimmed]++
|
||||
}
|
||||
for _, thought := range thoughts {
|
||||
add("type", thought.Metadata.Type)
|
||||
add("source", thought.Metadata.Source)
|
||||
for _, topic := range thought.Metadata.Topics {
|
||||
add("topics", topic)
|
||||
}
|
||||
for _, person := range thought.Metadata.People {
|
||||
add("people", person)
|
||||
}
|
||||
}
|
||||
|
||||
keys := make([]string, 0, len(groups))
|
||||
for key, values := range groups {
|
||||
if len(values) > 1 {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
}
|
||||
sort.Strings(keys)
|
||||
|
||||
out := make([]MetadataCleanupCandidate, 0, min(len(keys), limit))
|
||||
for _, key := range keys {
|
||||
parts := strings.SplitN(key, "\x00", 2)
|
||||
values, occurrences, recommended := valuesByCount(groups[key])
|
||||
out = append(out, MetadataCleanupCandidate{
|
||||
Field: parts[0],
|
||||
NormalizedValue: parts[1],
|
||||
Values: values,
|
||||
Occurrences: occurrences,
|
||||
RecommendedValue: recommended,
|
||||
RecommendedAction: "preview bulk metadata remap before applying; no changes made by audit",
|
||||
})
|
||||
if len(out) == limit {
|
||||
return out, len(keys) > limit
|
||||
}
|
||||
}
|
||||
return out, false
|
||||
}
|
||||
|
||||
func sortedDuplicateKeys[T any](groups map[string][]T) []string {
|
||||
keys := make([]string, 0, len(groups))
|
||||
for key, items := range groups {
|
||||
if len(items) > 1 {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
}
|
||||
sort.Strings(keys)
|
||||
return keys
|
||||
}
|
||||
|
||||
func valuesByCount(counts map[string]int) ([]string, int, string) {
|
||||
values := make([]string, 0, len(counts))
|
||||
occurrences := 0
|
||||
for value, count := range counts {
|
||||
values = append(values, value)
|
||||
occurrences += count
|
||||
}
|
||||
sort.Slice(values, func(i, j int) bool {
|
||||
if counts[values[i]] != counts[values[j]] {
|
||||
return counts[values[i]] > counts[values[j]]
|
||||
}
|
||||
return values[i] < values[j]
|
||||
})
|
||||
return values, occurrences, values[0]
|
||||
}
|
||||
|
||||
func normalizeDuplicateText(value string) string {
|
||||
return duplicateWhitespace.ReplaceAllString(strings.ToLower(strings.TrimSpace(value)), " ")
|
||||
}
|
||||
|
||||
func allProjectNamesExact(projects []thoughttypes.ProjectSummary) bool {
|
||||
if len(projects) == 0 {
|
||||
return false
|
||||
}
|
||||
first := strings.TrimSpace(projects[0].Name)
|
||||
for _, project := range projects[1:] {
|
||||
if strings.TrimSpace(project.Name) != first {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func allThoughtContentExact(thoughts []thoughttypes.Thought) bool {
|
||||
if len(thoughts) == 0 {
|
||||
return false
|
||||
}
|
||||
first := strings.TrimSpace(thoughts[0].Content)
|
||||
for _, thought := range thoughts[1:] {
|
||||
if strings.TrimSpace(thought.Content) != first {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
@@ -0,0 +1,76 @@
|
||||
package tools
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
thoughttypes "git.warky.dev/wdevs/amcs/internal/types"
|
||||
)
|
||||
|
||||
func TestBuildDuplicateAuditGroupsProjectsAndThoughtsSafely(t *testing.T) {
|
||||
now := time.Date(2026, 4, 12, 20, 5, 0, 0, time.UTC)
|
||||
projects := []thoughttypes.ProjectSummary{
|
||||
{Project: thoughttypes.Project{ID: uuid.MustParse("11111111-1111-1111-1111-111111111111"), NumericID: 1, Name: "AMCS", CreatedAt: now}, ThoughtCount: 3},
|
||||
{Project: thoughttypes.Project{ID: uuid.MustParse("22222222-2222-2222-2222-222222222222"), NumericID: 2, Name: "amcs ", CreatedAt: now.Add(time.Hour)}, ThoughtCount: 1},
|
||||
{Project: thoughttypes.Project{ID: uuid.MustParse("33333333-3333-3333-3333-333333333333"), NumericID: 3, Name: "other", CreatedAt: now}, ThoughtCount: 1},
|
||||
}
|
||||
thoughts := []thoughttypes.Thought{
|
||||
{ID: 10, GUID: uuid.MustParse("aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"), Content: "Same text", Metadata: thoughttypes.ThoughtMetadata{Type: "Note", Topics: []string{"Go", "go"}, People: []string{"Alice"}}, CreatedAt: now},
|
||||
{ID: 11, GUID: uuid.MustParse("bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"), Content: " same text ", Metadata: thoughttypes.ThoughtMetadata{Type: "note", Topics: []string{"go"}, People: []string{"alice"}}, CreatedAt: now.Add(time.Hour)},
|
||||
{ID: 12, GUID: uuid.MustParse("cccccccc-cccc-cccc-cccc-cccccccccccc"), Content: "unique", Metadata: thoughttypes.ThoughtMetadata{Type: "Task", Topics: []string{"Tasks"}}, CreatedAt: now},
|
||||
}
|
||||
|
||||
report := buildDuplicateAuditReport(projects, thoughts, DuplicateAuditInput{})
|
||||
|
||||
if len(report.ProjectCandidates) != 1 {
|
||||
t.Fatalf("project candidates = %d, want 1: %#v", len(report.ProjectCandidates), report.ProjectCandidates)
|
||||
}
|
||||
projectCandidate := report.ProjectCandidates[0]
|
||||
if projectCandidate.MatchType != "normalized_name" || projectCandidate.NormalizedValue != "amcs" || len(projectCandidate.Projects) != 2 {
|
||||
t.Fatalf("project candidate mismatch: %#v", projectCandidate)
|
||||
}
|
||||
if projectCandidate.RecommendedCanonicalID != projects[0].ID {
|
||||
t.Fatalf("recommended project = %s, want %s", projectCandidate.RecommendedCanonicalID, projects[0].ID)
|
||||
}
|
||||
|
||||
if len(report.ThoughtCandidates) != 1 {
|
||||
t.Fatalf("thought candidates = %d, want 1: %#v", len(report.ThoughtCandidates), report.ThoughtCandidates)
|
||||
}
|
||||
thoughtCandidate := report.ThoughtCandidates[0]
|
||||
if thoughtCandidate.MatchType != "normalized_content" || thoughtCandidate.NormalizedValue != "same text" || len(thoughtCandidate.Thoughts) != 2 {
|
||||
t.Fatalf("thought candidate mismatch: %#v", thoughtCandidate)
|
||||
}
|
||||
if thoughtCandidate.RecommendedCanonicalID != thoughts[0].GUID {
|
||||
t.Fatalf("recommended thought = %s, want %s", thoughtCandidate.RecommendedCanonicalID, thoughts[0].GUID)
|
||||
}
|
||||
|
||||
if len(report.MetadataCandidates) != 3 {
|
||||
t.Fatalf("metadata candidates = %d, want 3: %#v", len(report.MetadataCandidates), report.MetadataCandidates)
|
||||
}
|
||||
if report.Summary.ProjectCandidateGroups != 1 || report.Summary.ThoughtCandidateGroups != 1 || report.Summary.MetadataCandidateGroups != 3 {
|
||||
t.Fatalf("summary mismatch: %#v", report.Summary)
|
||||
}
|
||||
if report.Summary.DryRun != true {
|
||||
t.Fatalf("dry run = false, want true")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildDuplicateAuditRespectsLimit(t *testing.T) {
|
||||
projects := []thoughttypes.ProjectSummary{
|
||||
{Project: thoughttypes.Project{ID: uuid.MustParse("11111111-1111-1111-1111-111111111111"), Name: "Alpha"}},
|
||||
{Project: thoughttypes.Project{ID: uuid.MustParse("22222222-2222-2222-2222-222222222222"), Name: "alpha"}},
|
||||
{Project: thoughttypes.Project{ID: uuid.MustParse("33333333-3333-3333-3333-333333333333"), Name: "Beta"}},
|
||||
{Project: thoughttypes.Project{ID: uuid.MustParse("44444444-4444-4444-4444-444444444444"), Name: "beta"}},
|
||||
}
|
||||
|
||||
report := buildDuplicateAuditReport(projects, nil, DuplicateAuditInput{Limit: 1})
|
||||
|
||||
if len(report.ProjectCandidates) != 1 {
|
||||
t.Fatalf("project candidates = %d, want 1", len(report.ProjectCandidates))
|
||||
}
|
||||
if report.Summary.Truncated != true {
|
||||
t.Fatalf("truncated = false, want true")
|
||||
}
|
||||
}
|
||||
@@ -46,6 +46,7 @@ type RelatedThought struct {
|
||||
|
||||
type RelatedOutput struct {
|
||||
Related []RelatedThought `json:"related"`
|
||||
RetrievalMode string `json:"retrieval_mode,omitempty"`
|
||||
}
|
||||
|
||||
func NewLinksTool(db *store.DB, embeddings *ai.EmbeddingRunner, search config.SearchConfig) *LinksTool {
|
||||
@@ -117,11 +118,13 @@ func (t *LinksTool) Related(ctx context.Context, _ *mcp.CallToolRequest, in Rela
|
||||
includeSemantic = *in.IncludeSemantic
|
||||
}
|
||||
|
||||
var retrievalMode string
|
||||
if includeSemantic {
|
||||
semantic, err := semanticSearch(ctx, t.store, t.embeddings, t.search, thought.Content, t.search.DefaultLimit, t.search.DefaultThreshold, thought.ProjectID, &thought.GUID)
|
||||
semantic, mode, err := semanticSearch(ctx, t.store, t.embeddings, t.search, thought.Content, t.search.DefaultLimit, t.search.DefaultThreshold, thought.ProjectID, &thought.GUID)
|
||||
if err != nil {
|
||||
return nil, RelatedOutput{}, err
|
||||
}
|
||||
retrievalMode = mode
|
||||
for _, item := range semantic {
|
||||
key := fmt.Sprint(item.ID)
|
||||
if _, ok := seen[key]; ok {
|
||||
@@ -138,5 +141,5 @@ func (t *LinksTool) Related(ctx context.Context, _ *mcp.CallToolRequest, in Rela
|
||||
}
|
||||
}
|
||||
|
||||
return nil, RelatedOutput{Related: related}, nil
|
||||
return nil, RelatedOutput{Related: related, RetrievalMode: retrievalMode}, nil
|
||||
}
|
||||
|
||||
@@ -29,6 +29,7 @@ type RecallInput struct {
|
||||
type RecallOutput struct {
|
||||
Context string `json:"context"`
|
||||
Items []ContextItem `json:"items"`
|
||||
RetrievalMode string `json:"retrieval_mode,omitempty"`
|
||||
}
|
||||
|
||||
func NewRecallTool(db *store.DB, embeddings *ai.EmbeddingRunner, search config.SearchConfig, sessions *session.ActiveProjects) *RecallTool {
|
||||
@@ -53,7 +54,7 @@ func (t *RecallTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in Re
|
||||
projectID = &project.NumericID
|
||||
}
|
||||
|
||||
semantic, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, t.search.DefaultThreshold, projectID, nil)
|
||||
semantic, mode, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, t.search.DefaultThreshold, projectID, nil)
|
||||
if err != nil {
|
||||
return nil, RecallOutput{}, err
|
||||
}
|
||||
@@ -102,5 +103,6 @@ func (t *RecallTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in Re
|
||||
return nil, RecallOutput{
|
||||
Context: formatContextBlock(header, lines),
|
||||
Items: items,
|
||||
RetrievalMode: mode,
|
||||
}, nil
|
||||
}
|
||||
|
||||
@@ -11,10 +11,16 @@ import (
|
||||
thoughttypes "git.warky.dev/wdevs/amcs/internal/types"
|
||||
)
|
||||
|
||||
const (
|
||||
RetrievalModeSemantic = "semantic"
|
||||
RetrievalModeText = "text"
|
||||
)
|
||||
|
||||
// semanticSearch runs vector similarity search if embeddings exist for the
|
||||
// primary embedding model in the given scope, otherwise falls back to Postgres
|
||||
// full-text search. Search always uses the primary model so query vectors
|
||||
// match rows stored under the primary model name.
|
||||
// It returns the results and the retrieval mode used ("semantic" or "text").
|
||||
func semanticSearch(
|
||||
ctx context.Context,
|
||||
db *store.DB,
|
||||
@@ -25,20 +31,22 @@ func semanticSearch(
|
||||
threshold float64,
|
||||
projectID *int64,
|
||||
excludeID *uuid.UUID,
|
||||
) ([]thoughttypes.SearchResult, error) {
|
||||
) ([]thoughttypes.SearchResult, string, error) {
|
||||
model := embeddings.PrimaryModel()
|
||||
hasEmbeddings, err := db.HasEmbeddingsForModel(ctx, model, projectID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return nil, "", err
|
||||
}
|
||||
|
||||
if hasEmbeddings {
|
||||
embedding, err := embeddings.EmbedPrimary(ctx, query)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return nil, "", err
|
||||
}
|
||||
return db.SearchSimilarThoughts(ctx, embedding, model, threshold, limit, projectID, excludeID)
|
||||
results, err := db.SearchSimilarThoughts(ctx, embedding, model, threshold, limit, projectID, excludeID)
|
||||
return results, RetrievalModeSemantic, err
|
||||
}
|
||||
|
||||
return db.SearchThoughtsText(ctx, query, limit, projectID, excludeID)
|
||||
results, err := db.SearchThoughtsText(ctx, query, limit, projectID, excludeID)
|
||||
return results, RetrievalModeText, err
|
||||
}
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
package tools
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestRetrievalModeConstants(t *testing.T) {
|
||||
if RetrievalModeSemantic != "semantic" {
|
||||
t.Fatalf("RetrievalModeSemantic = %q, want %q", RetrievalModeSemantic, "semantic")
|
||||
}
|
||||
if RetrievalModeText != "text" {
|
||||
t.Fatalf("RetrievalModeText = %q, want %q", RetrievalModeText, "text")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSearchOutputIncludesRetrievalMode(t *testing.T) {
|
||||
out := SearchOutput{RetrievalMode: RetrievalModeSemantic}
|
||||
if out.RetrievalMode != RetrievalModeSemantic {
|
||||
t.Fatalf("SearchOutput.RetrievalMode = %q, want %q", out.RetrievalMode, RetrievalModeSemantic)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRelatedOutputIncludesRetrievalMode(t *testing.T) {
|
||||
out := RelatedOutput{RetrievalMode: RetrievalModeText}
|
||||
if out.RetrievalMode != RetrievalModeText {
|
||||
t.Fatalf("RelatedOutput.RetrievalMode = %q, want %q", out.RetrievalMode, RetrievalModeText)
|
||||
}
|
||||
// retrieval_mode is omitempty — empty string means semantic search was skipped
|
||||
out2 := RelatedOutput{}
|
||||
if out2.RetrievalMode != "" {
|
||||
t.Fatalf("RelatedOutput.RetrievalMode = %q, want empty when include_semantic is false", out2.RetrievalMode)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSummarizeOutputIncludesRetrievalModeOnlyWhenQueryUsed(t *testing.T) {
|
||||
withQuery := SummarizeOutput{Summary: "s", Count: 1, RetrievalMode: RetrievalModeSemantic}
|
||||
if withQuery.RetrievalMode != RetrievalModeSemantic {
|
||||
t.Fatalf("SummarizeOutput.RetrievalMode = %q, want %q", withQuery.RetrievalMode, RetrievalModeSemantic)
|
||||
}
|
||||
withoutQuery := SummarizeOutput{Summary: "s", Count: 1}
|
||||
if withoutQuery.RetrievalMode != "" {
|
||||
t.Fatalf("SummarizeOutput.RetrievalMode = %q, want empty when no query", withoutQuery.RetrievalMode)
|
||||
}
|
||||
}
|
||||
|
||||
func TestProjectContextOutputIncludesRetrievalModeOnlyWhenQueryUsed(t *testing.T) {
|
||||
withQuery := ProjectContextOutput{RetrievalMode: RetrievalModeText}
|
||||
if withQuery.RetrievalMode != RetrievalModeText {
|
||||
t.Fatalf("ProjectContextOutput.RetrievalMode = %q, want %q", withQuery.RetrievalMode, RetrievalModeText)
|
||||
}
|
||||
withoutQuery := ProjectContextOutput{}
|
||||
if withoutQuery.RetrievalMode != "" {
|
||||
t.Fatalf("ProjectContextOutput.RetrievalMode = %q, want empty when no query", withoutQuery.RetrievalMode)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecallOutputIncludesRetrievalMode(t *testing.T) {
|
||||
out := RecallOutput{RetrievalMode: RetrievalModeSemantic}
|
||||
if out.RetrievalMode != RetrievalModeSemantic {
|
||||
t.Fatalf("RecallOutput.RetrievalMode = %q, want %q", out.RetrievalMode, RetrievalModeSemantic)
|
||||
}
|
||||
}
|
||||
@@ -29,6 +29,7 @@ type SearchInput struct {
|
||||
|
||||
type SearchOutput struct {
|
||||
Results []thoughttypes.SearchResult `json:"results"`
|
||||
RetrievalMode string `json:"retrieval_mode,omitempty"`
|
||||
}
|
||||
|
||||
func NewSearchTool(db *store.DB, embeddings *ai.EmbeddingRunner, search config.SearchConfig, sessions *session.ActiveProjects) *SearchTool {
|
||||
@@ -55,10 +56,10 @@ func (t *SearchTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in Se
|
||||
_ = t.store.TouchProject(ctx, project.NumericID)
|
||||
}
|
||||
|
||||
results, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, threshold, projectID, nil)
|
||||
results, mode, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, threshold, projectID, nil)
|
||||
if err != nil {
|
||||
return nil, SearchOutput{}, err
|
||||
}
|
||||
|
||||
return nil, SearchOutput{Results: results}, nil
|
||||
return nil, SearchOutput{Results: results, RetrievalMode: mode}, nil
|
||||
}
|
||||
|
||||
@@ -30,6 +30,7 @@ type SummarizeInput struct {
|
||||
type SummarizeOutput struct {
|
||||
Summary string `json:"summary"`
|
||||
Count int `json:"count"`
|
||||
RetrievalMode string `json:"retrieval_mode,omitempty"`
|
||||
}
|
||||
|
||||
func NewSummarizeTool(db *store.DB, embeddings *ai.EmbeddingRunner, metadata *ai.MetadataRunner, search config.SearchConfig, sessions *session.ActiveProjects) *SummarizeTool {
|
||||
@@ -47,15 +48,17 @@ func (t *SummarizeTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in
|
||||
lines := make([]string, 0, limit)
|
||||
count := 0
|
||||
|
||||
var retrievalMode string
|
||||
if query != "" {
|
||||
var projectID *int64
|
||||
if project != nil {
|
||||
projectID = &project.NumericID
|
||||
}
|
||||
results, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, t.search.DefaultThreshold, projectID, nil)
|
||||
results, mode, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, t.search.DefaultThreshold, projectID, nil)
|
||||
if err != nil {
|
||||
return nil, SummarizeOutput{}, err
|
||||
}
|
||||
retrievalMode = mode
|
||||
for i, result := range results {
|
||||
lines = append(lines, thoughtContextLine(i, result.Content, result.Metadata, result.Similarity))
|
||||
}
|
||||
@@ -85,5 +88,5 @@ func (t *SummarizeTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in
|
||||
_ = t.store.TouchProject(ctx, project.NumericID)
|
||||
}
|
||||
|
||||
return nil, SummarizeOutput{Summary: summary, Count: count}, nil
|
||||
return nil, SummarizeOutput{Summary: summary, Count: count, RetrievalMode: retrievalMode}, nil
|
||||
}
|
||||
|
||||
@@ -14,12 +14,20 @@ type ThoughtMetadata struct {
|
||||
Type string `json:"type"`
|
||||
Source string `json:"source"`
|
||||
Attachments []ThoughtAttachment `json:"attachments,omitempty"`
|
||||
Webhook *WebhookMetadata `json:"webhook,omitempty"`
|
||||
MetadataStatus string `json:"metadata_status,omitempty"`
|
||||
MetadataUpdatedAt string `json:"metadata_updated_at,omitempty"`
|
||||
MetadataLastAttemptedAt string `json:"metadata_last_attempted_at,omitempty"`
|
||||
MetadataError string `json:"metadata_error,omitempty"`
|
||||
}
|
||||
|
||||
type WebhookMetadata struct {
|
||||
ReceivedAt string `json:"received_at"`
|
||||
IDempotencyKey string `json:"idempotency_key,omitempty"`
|
||||
ExternalID string `json:"external_id,omitempty"`
|
||||
SourceMetadata map[string]any `json:"source_metadata,omitempty"`
|
||||
}
|
||||
|
||||
type ThoughtAttachment struct {
|
||||
FileID uuid.UUID `json:"file_id"`
|
||||
Name string `json:"name"`
|
||||
|
||||
@@ -110,6 +110,13 @@ CREATE SEQUENCE IF NOT EXISTS public.identity_thought_links_id
|
||||
START 1
|
||||
CACHE 1;
|
||||
|
||||
CREATE SEQUENCE IF NOT EXISTS public.identity_thought_learning_links_id
|
||||
INCREMENT 1
|
||||
MINVALUE 1
|
||||
MAXVALUE 9223372036854775807
|
||||
START 1
|
||||
CACHE 1;
|
||||
|
||||
CREATE SEQUENCE IF NOT EXISTS public.identity_embeddings_id
|
||||
INCREMENT 1
|
||||
MINVALUE 1
|
||||
@@ -332,6 +339,7 @@ CREATE TABLE IF NOT EXISTS public.thoughts (
|
||||
id bigserial NOT NULL,
|
||||
metadata jsonb DEFAULT '{}'::jsonb,
|
||||
project_id bigint,
|
||||
tenant_key text,
|
||||
updated_at timestamptz DEFAULT now()
|
||||
);
|
||||
|
||||
@@ -341,7 +349,8 @@ CREATE TABLE IF NOT EXISTS public.projects (
|
||||
guid uuid NOT NULL DEFAULT gen_random_uuid(),
|
||||
id bigserial NOT NULL,
|
||||
last_active_at timestamptz DEFAULT now(),
|
||||
name text NOT NULL
|
||||
name text NOT NULL,
|
||||
tenant_key text
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS public.thought_links (
|
||||
@@ -352,6 +361,14 @@ CREATE TABLE IF NOT EXISTS public.thought_links (
|
||||
to_id bigint NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS public.thought_learning_links (
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
id bigserial NOT NULL,
|
||||
learning_id bigint NOT NULL,
|
||||
relation text NOT NULL DEFAULT 'source',
|
||||
thought_id bigint NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS public.embeddings (
|
||||
created_at timestamptz DEFAULT now(),
|
||||
dim integer NOT NULL,
|
||||
@@ -375,6 +392,7 @@ CREATE TABLE IF NOT EXISTS public.stored_files (
|
||||
project_id bigint,
|
||||
sha256 text NOT NULL,
|
||||
size_bytes bigint NOT NULL,
|
||||
tenant_key text,
|
||||
thought_id bigint,
|
||||
updated_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
@@ -390,6 +408,7 @@ CREATE TABLE IF NOT EXISTS public.chat_histories (
|
||||
project_id bigint,
|
||||
session_id text NOT NULL,
|
||||
summary text,
|
||||
tenant_key text,
|
||||
title text,
|
||||
updated_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
@@ -424,6 +443,7 @@ CREATE TABLE IF NOT EXISTS public.learnings (
|
||||
summary text NOT NULL,
|
||||
supersedes_learning_id bigint,
|
||||
tags text[] NOT NULL DEFAULT '{}',
|
||||
tenant_key text,
|
||||
updated_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
@@ -450,6 +470,7 @@ CREATE TABLE IF NOT EXISTS public.plans (
|
||||
status text NOT NULL DEFAULT 'draft',
|
||||
supersedes_plan_id bigint,
|
||||
tags text[] NOT NULL DEFAULT '{}',
|
||||
tenant_key text,
|
||||
title text NOT NULL,
|
||||
updated_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
@@ -1552,6 +1573,19 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'thoughts'
|
||||
AND column_name = 'tenant_key'
|
||||
) THEN
|
||||
ALTER TABLE public.thoughts ADD COLUMN tenant_key text;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
@@ -1643,6 +1677,19 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'projects'
|
||||
AND column_name = 'tenant_key'
|
||||
) THEN
|
||||
ALTER TABLE public.projects ADD COLUMN tenant_key text;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
@@ -1708,6 +1755,71 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'thought_learning_links'
|
||||
AND column_name = 'created_at'
|
||||
) THEN
|
||||
ALTER TABLE public.thought_learning_links ADD COLUMN created_at timestamptz NOT NULL DEFAULT now();
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'thought_learning_links'
|
||||
AND column_name = 'id'
|
||||
) THEN
|
||||
ALTER TABLE public.thought_learning_links ADD COLUMN id bigserial NOT NULL;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'thought_learning_links'
|
||||
AND column_name = 'learning_id'
|
||||
) THEN
|
||||
ALTER TABLE public.thought_learning_links ADD COLUMN learning_id bigint NOT NULL;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'thought_learning_links'
|
||||
AND column_name = 'relation'
|
||||
) THEN
|
||||
ALTER TABLE public.thought_learning_links ADD COLUMN relation text NOT NULL DEFAULT 'source';
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'thought_learning_links'
|
||||
AND column_name = 'thought_id'
|
||||
) THEN
|
||||
ALTER TABLE public.thought_learning_links ADD COLUMN thought_id bigint NOT NULL;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
@@ -1955,6 +2067,19 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'stored_files'
|
||||
AND column_name = 'tenant_key'
|
||||
) THEN
|
||||
ALTER TABLE public.stored_files ADD COLUMN tenant_key text;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
@@ -2111,6 +2236,19 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'chat_histories'
|
||||
AND column_name = 'tenant_key'
|
||||
) THEN
|
||||
ALTER TABLE public.chat_histories ADD COLUMN tenant_key text;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
@@ -2475,6 +2613,19 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'learnings'
|
||||
AND column_name = 'tenant_key'
|
||||
) THEN
|
||||
ALTER TABLE public.learnings ADD COLUMN tenant_key text;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
@@ -2735,6 +2886,19 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'plans'
|
||||
AND column_name = 'tenant_key'
|
||||
) THEN
|
||||
ALTER TABLE public.plans ADD COLUMN tenant_key text;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
@@ -5177,6 +5341,29 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
BEGIN
|
||||
SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
|
||||
INTO current_type
|
||||
FROM pg_attribute a
|
||||
JOIN pg_class t ON t.oid = a.attrelid
|
||||
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||
WHERE n.nspname = 'public'
|
||||
AND t.relname = 'thoughts'
|
||||
AND a.attname = 'tenant_key'
|
||||
AND a.attnum > 0
|
||||
AND NOT a.attisdropped;
|
||||
|
||||
IF current_type IS NOT NULL
|
||||
AND current_type <> ALL(ARRAY['text']) THEN
|
||||
ALTER TABLE public.thoughts
|
||||
ALTER COLUMN tenant_key TYPE text USING tenant_key::text;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
@@ -5338,6 +5525,29 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
BEGIN
|
||||
SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
|
||||
INTO current_type
|
||||
FROM pg_attribute a
|
||||
JOIN pg_class t ON t.oid = a.attrelid
|
||||
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||
WHERE n.nspname = 'public'
|
||||
AND t.relname = 'projects'
|
||||
AND a.attname = 'tenant_key'
|
||||
AND a.attnum > 0
|
||||
AND NOT a.attisdropped;
|
||||
|
||||
IF current_type IS NOT NULL
|
||||
AND current_type <> ALL(ARRAY['text']) THEN
|
||||
ALTER TABLE public.projects
|
||||
ALTER COLUMN tenant_key TYPE text USING tenant_key::text;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
@@ -5453,6 +5663,121 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
BEGIN
|
||||
SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
|
||||
INTO current_type
|
||||
FROM pg_attribute a
|
||||
JOIN pg_class t ON t.oid = a.attrelid
|
||||
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||
WHERE n.nspname = 'public'
|
||||
AND t.relname = 'thought_learning_links'
|
||||
AND a.attname = 'created_at'
|
||||
AND a.attnum > 0
|
||||
AND NOT a.attisdropped;
|
||||
|
||||
IF current_type IS NOT NULL
|
||||
AND current_type <> ALL(ARRAY['timestamptz', 'timestamp with time zone']) THEN
|
||||
ALTER TABLE public.thought_learning_links
|
||||
ALTER COLUMN created_at TYPE timestamptz USING created_at::timestamptz;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
BEGIN
|
||||
SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
|
||||
INTO current_type
|
||||
FROM pg_attribute a
|
||||
JOIN pg_class t ON t.oid = a.attrelid
|
||||
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||
WHERE n.nspname = 'public'
|
||||
AND t.relname = 'thought_learning_links'
|
||||
AND a.attname = 'id'
|
||||
AND a.attnum > 0
|
||||
AND NOT a.attisdropped;
|
||||
|
||||
IF current_type IS NOT NULL
|
||||
AND current_type <> ALL(ARRAY['bigint']) THEN
|
||||
ALTER TABLE public.thought_learning_links
|
||||
ALTER COLUMN id TYPE bigint USING id::bigint;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
BEGIN
|
||||
SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
|
||||
INTO current_type
|
||||
FROM pg_attribute a
|
||||
JOIN pg_class t ON t.oid = a.attrelid
|
||||
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||
WHERE n.nspname = 'public'
|
||||
AND t.relname = 'thought_learning_links'
|
||||
AND a.attname = 'learning_id'
|
||||
AND a.attnum > 0
|
||||
AND NOT a.attisdropped;
|
||||
|
||||
IF current_type IS NOT NULL
|
||||
AND current_type <> ALL(ARRAY['bigint']) THEN
|
||||
ALTER TABLE public.thought_learning_links
|
||||
ALTER COLUMN learning_id TYPE bigint USING learning_id::bigint;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
BEGIN
|
||||
SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
|
||||
INTO current_type
|
||||
FROM pg_attribute a
|
||||
JOIN pg_class t ON t.oid = a.attrelid
|
||||
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||
WHERE n.nspname = 'public'
|
||||
AND t.relname = 'thought_learning_links'
|
||||
AND a.attname = 'relation'
|
||||
AND a.attnum > 0
|
||||
AND NOT a.attisdropped;
|
||||
|
||||
IF current_type IS NOT NULL
|
||||
AND current_type <> ALL(ARRAY['text']) THEN
|
||||
ALTER TABLE public.thought_learning_links
|
||||
ALTER COLUMN relation TYPE text USING relation::text;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
BEGIN
|
||||
SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
|
||||
INTO current_type
|
||||
FROM pg_attribute a
|
||||
JOIN pg_class t ON t.oid = a.attrelid
|
||||
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||
WHERE n.nspname = 'public'
|
||||
AND t.relname = 'thought_learning_links'
|
||||
AND a.attname = 'thought_id'
|
||||
AND a.attnum > 0
|
||||
AND NOT a.attisdropped;
|
||||
|
||||
IF current_type IS NOT NULL
|
||||
AND current_type <> ALL(ARRAY['bigint']) THEN
|
||||
ALTER TABLE public.thought_learning_links
|
||||
ALTER COLUMN thought_id TYPE bigint USING thought_id::bigint;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
@@ -5890,6 +6215,29 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
BEGIN
|
||||
SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
|
||||
INTO current_type
|
||||
FROM pg_attribute a
|
||||
JOIN pg_class t ON t.oid = a.attrelid
|
||||
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||
WHERE n.nspname = 'public'
|
||||
AND t.relname = 'stored_files'
|
||||
AND a.attname = 'tenant_key'
|
||||
AND a.attnum > 0
|
||||
AND NOT a.attisdropped;
|
||||
|
||||
IF current_type IS NOT NULL
|
||||
AND current_type <> ALL(ARRAY['text']) THEN
|
||||
ALTER TABLE public.stored_files
|
||||
ALTER COLUMN tenant_key TYPE text USING tenant_key::text;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
@@ -6166,6 +6514,29 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
BEGIN
|
||||
SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
|
||||
INTO current_type
|
||||
FROM pg_attribute a
|
||||
JOIN pg_class t ON t.oid = a.attrelid
|
||||
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||
WHERE n.nspname = 'public'
|
||||
AND t.relname = 'chat_histories'
|
||||
AND a.attname = 'tenant_key'
|
||||
AND a.attnum > 0
|
||||
AND NOT a.attisdropped;
|
||||
|
||||
IF current_type IS NOT NULL
|
||||
AND current_type <> ALL(ARRAY['text']) THEN
|
||||
ALTER TABLE public.chat_histories
|
||||
ALTER COLUMN tenant_key TYPE text USING tenant_key::text;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
@@ -6810,6 +7181,29 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
BEGIN
|
||||
SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
|
||||
INTO current_type
|
||||
FROM pg_attribute a
|
||||
JOIN pg_class t ON t.oid = a.attrelid
|
||||
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||
WHERE n.nspname = 'public'
|
||||
AND t.relname = 'learnings'
|
||||
AND a.attname = 'tenant_key'
|
||||
AND a.attnum > 0
|
||||
AND NOT a.attisdropped;
|
||||
|
||||
IF current_type IS NOT NULL
|
||||
AND current_type <> ALL(ARRAY['text']) THEN
|
||||
ALTER TABLE public.learnings
|
||||
ALTER COLUMN tenant_key TYPE text USING tenant_key::text;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
@@ -7270,6 +7664,29 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
BEGIN
|
||||
SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
|
||||
INTO current_type
|
||||
FROM pg_attribute a
|
||||
JOIN pg_class t ON t.oid = a.attrelid
|
||||
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||
WHERE n.nspname = 'public'
|
||||
AND t.relname = 'plans'
|
||||
AND a.attname = 'tenant_key'
|
||||
AND a.attnum > 0
|
||||
AND NOT a.attisdropped;
|
||||
|
||||
IF current_type IS NOT NULL
|
||||
AND current_type <> ALL(ARRAY['text']) THEN
|
||||
ALTER TABLE public.plans
|
||||
ALTER COLUMN tenant_key TYPE text USING tenant_key::text;
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_type text;
|
||||
@@ -9035,6 +9452,50 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_pk_name text;
|
||||
current_pk_matches boolean := false;
|
||||
BEGIN
|
||||
SELECT tc.constraint_name,
|
||||
COALESCE(
|
||||
ARRAY(
|
||||
SELECT a.attname::text
|
||||
FROM pg_constraint c
|
||||
JOIN pg_class t ON t.oid = c.conrelid
|
||||
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||
JOIN unnest(c.conkey) WITH ORDINALITY AS cols(attnum, ord)
|
||||
ON TRUE
|
||||
JOIN pg_attribute a
|
||||
ON a.attrelid = t.oid
|
||||
AND a.attnum = cols.attnum
|
||||
WHERE c.contype = 'p'
|
||||
AND n.nspname = 'public'
|
||||
AND t.relname = 'thought_learning_links'
|
||||
ORDER BY cols.ord
|
||||
),
|
||||
ARRAY[]::text[]
|
||||
) = ARRAY['id']
|
||||
INTO current_pk_name, current_pk_matches
|
||||
FROM information_schema.table_constraints tc
|
||||
WHERE tc.table_schema = 'public'
|
||||
AND tc.table_name = 'thought_learning_links'
|
||||
AND tc.constraint_type = 'PRIMARY KEY';
|
||||
|
||||
IF current_pk_name IS NOT NULL
|
||||
AND NOT current_pk_matches
|
||||
AND current_pk_name IN ('thought_learning_links_pkey', 'public_thought_learning_links_pkey') THEN
|
||||
EXECUTE 'ALTER TABLE public.thought_learning_links DROP CONSTRAINT ' || quote_ident(current_pk_name) || ' CASCADE';
|
||||
END IF;
|
||||
|
||||
-- Add the desired primary key only when no matching primary key already exists.
|
||||
IF current_pk_name IS NULL
|
||||
OR (NOT current_pk_matches AND current_pk_name IN ('thought_learning_links_pkey', 'public_thought_learning_links_pkey')) THEN
|
||||
ALTER TABLE public.thought_learning_links ADD CONSTRAINT pk_public_thought_learning_links PRIMARY KEY (id);
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
current_pk_name text;
|
||||
@@ -9708,9 +10169,18 @@ CREATE INDEX IF NOT EXISTS idx_project_personas_project_id_persona_id
|
||||
CREATE INDEX IF NOT EXISTS idx_arc_stage_parts_stage_id_part_id
|
||||
ON public.arc_stage_parts USING btree (stage_id, part_id);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_thoughts_tenant_key_project_id
|
||||
ON public.thoughts USING btree (tenant_key, project_id);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS uidx_projects_tenant_key_name
|
||||
ON public.projects USING btree (tenant_key, name);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_thought_links_from_id_to_id_relation
|
||||
ON public.thought_links USING btree (from_id, to_id, relation);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS uidx_thought_learning_links_thought_id_learning_id
|
||||
ON public.thought_learning_links USING btree (thought_id, learning_id);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS uidx_embeddings_thought_id_model
|
||||
ON public.embeddings USING btree (thought_id, model);
|
||||
|
||||
@@ -9865,19 +10335,6 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.table_constraints
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'projects'
|
||||
AND constraint_name = 'ukey_projects_name'
|
||||
) THEN
|
||||
ALTER TABLE public.projects ADD CONSTRAINT ukey_projects_name UNIQUE (name);
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
@@ -10328,6 +10785,38 @@ BEGIN
|
||||
END IF;
|
||||
END;
|
||||
$$;DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.table_constraints
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'thought_learning_links'
|
||||
AND constraint_name = 'fk_thought_learning_links_learning_id'
|
||||
) THEN
|
||||
ALTER TABLE public.thought_learning_links
|
||||
ADD CONSTRAINT fk_thought_learning_links_learning_id
|
||||
FOREIGN KEY (learning_id)
|
||||
REFERENCES public.learnings (id)
|
||||
ON DELETE NO ACTION
|
||||
ON UPDATE NO ACTION;
|
||||
END IF;
|
||||
END;
|
||||
$$;DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.table_constraints
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'thought_learning_links'
|
||||
AND constraint_name = 'fk_thought_learning_links_thought_id'
|
||||
) THEN
|
||||
ALTER TABLE public.thought_learning_links
|
||||
ADD CONSTRAINT fk_thought_learning_links_thought_id
|
||||
FOREIGN KEY (thought_id)
|
||||
REFERENCES public.thoughts (id)
|
||||
ON DELETE NO ACTION
|
||||
ON UPDATE NO ACTION;
|
||||
END IF;
|
||||
END;
|
||||
$$;DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM information_schema.table_constraints
|
||||
@@ -10912,15 +11401,15 @@ BEGIN
|
||||
IF EXISTS (
|
||||
SELECT 1 FROM pg_class c
|
||||
INNER JOIN pg_namespace n ON n.oid = c.relnamespace
|
||||
WHERE c.relname = 'identity_persona_arc_id'
|
||||
WHERE c.relname = 'identity_persona_arc_persona_id'
|
||||
AND n.nspname = 'public'
|
||||
AND c.relkind = 'S'
|
||||
) THEN
|
||||
SELECT COALESCE(MAX(id), 0) + 1
|
||||
SELECT COALESCE(MAX(persona_id), 0) + 1
|
||||
FROM public.persona_arc
|
||||
INTO m_cnt;
|
||||
|
||||
PERFORM setval('public.identity_persona_arc_id'::regclass, m_cnt);
|
||||
PERFORM setval('public.identity_persona_arc_persona_id'::regclass, m_cnt);
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
@@ -10982,6 +11471,25 @@ BEGIN
|
||||
END;
|
||||
$$;
|
||||
DO $$
|
||||
DECLARE
|
||||
m_cnt bigint;
|
||||
BEGIN
|
||||
IF EXISTS (
|
||||
SELECT 1 FROM pg_class c
|
||||
INNER JOIN pg_namespace n ON n.oid = c.relnamespace
|
||||
WHERE c.relname = 'identity_thought_learning_links_id'
|
||||
AND n.nspname = 'public'
|
||||
AND c.relkind = 'S'
|
||||
) THEN
|
||||
SELECT COALESCE(MAX(id), 0) + 1
|
||||
FROM public.thought_learning_links
|
||||
INTO m_cnt;
|
||||
|
||||
PERFORM setval('public.identity_thought_learning_links_id'::regclass, m_cnt);
|
||||
END IF;
|
||||
END;
|
||||
$$;
|
||||
DO $$
|
||||
DECLARE
|
||||
m_cnt bigint;
|
||||
BEGIN
|
||||
@@ -11295,5 +11803,6 @@ $$;
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
-- Per-user tenancy: authenticate key/client id becomes an opaque tenant boundary.
|
||||
-- Existing rows remain in the legacy unscoped tenant (NULL) for single-tenant installs.
|
||||
|
||||
alter table projects add column if not exists tenant_key text;
|
||||
alter table thoughts add column if not exists tenant_key text;
|
||||
alter table stored_files add column if not exists tenant_key text;
|
||||
alter table learnings add column if not exists tenant_key text;
|
||||
alter table plans add column if not exists tenant_key text;
|
||||
alter table chat_histories add column if not exists tenant_key text;
|
||||
|
||||
-- Project names are now unique inside a tenant rather than globally.
|
||||
alter table projects drop constraint if exists ukey_projects_name;
|
||||
alter table projects drop constraint if exists projects_name_key;
|
||||
drop index if exists projects_name_key;
|
||||
create unique index if not exists projects_tenant_key_name_idx
|
||||
on projects (coalesce(tenant_key, ''), name);
|
||||
|
||||
create index if not exists projects_tenant_key_idx on projects (tenant_key);
|
||||
create index if not exists thoughts_tenant_key_idx on thoughts (tenant_key);
|
||||
create index if not exists thoughts_tenant_key_project_id_idx on thoughts (tenant_key, project_id);
|
||||
create index if not exists stored_files_tenant_key_idx on stored_files (tenant_key);
|
||||
create index if not exists learnings_tenant_key_idx on learnings (tenant_key);
|
||||
create index if not exists plans_tenant_key_idx on plans (tenant_key);
|
||||
create index if not exists chat_histories_tenant_key_idx on chat_histories (tenant_key);
|
||||
+13
-1
@@ -6,16 +6,28 @@ Table thoughts {
|
||||
created_at timestamptz [default: `now()`]
|
||||
updated_at timestamptz [default: `now()`]
|
||||
project_id bigint [ref: > projects.id]
|
||||
tenant_key text
|
||||
archived_at timestamptz
|
||||
|
||||
indexes {
|
||||
tenant_key
|
||||
(tenant_key, project_id)
|
||||
}
|
||||
}
|
||||
|
||||
Table projects {
|
||||
id bigserial [pk]
|
||||
guid uuid [unique, not null, default: `gen_random_uuid()`]
|
||||
name text [unique, not null]
|
||||
name text [not null]
|
||||
description text
|
||||
tenant_key text
|
||||
created_at timestamptz [default: `now()`]
|
||||
last_active_at timestamptz [default: `now()`]
|
||||
|
||||
indexes {
|
||||
(tenant_key, name) [unique]
|
||||
tenant_key
|
||||
}
|
||||
}
|
||||
|
||||
Table thought_links {
|
||||
|
||||
@@ -3,6 +3,7 @@ Table stored_files {
|
||||
guid uuid [unique, not null, default: `gen_random_uuid()`]
|
||||
thought_id bigint [ref: > thoughts.id]
|
||||
project_id bigint [ref: > projects.id]
|
||||
tenant_key text
|
||||
name text [not null]
|
||||
media_type text [not null]
|
||||
kind text [not null, default: 'file']
|
||||
@@ -16,6 +17,7 @@ Table stored_files {
|
||||
indexes {
|
||||
thought_id
|
||||
project_id
|
||||
tenant_key
|
||||
sha256
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ Table chat_histories {
|
||||
channel text
|
||||
agent_id text
|
||||
project_id bigint [ref: > projects.id]
|
||||
tenant_key text
|
||||
messages jsonb [not null, default: `'[]'`]
|
||||
summary text
|
||||
metadata jsonb [not null, default: `'{}'`]
|
||||
@@ -15,6 +16,7 @@ Table chat_histories {
|
||||
indexes {
|
||||
session_id
|
||||
project_id
|
||||
tenant_key
|
||||
channel
|
||||
agent_id
|
||||
created_at
|
||||
@@ -46,6 +48,7 @@ Table learnings {
|
||||
source_type text
|
||||
source_ref text
|
||||
project_id bigint [ref: > projects.id]
|
||||
tenant_key text
|
||||
related_thought_id bigint [ref: > thoughts.id]
|
||||
related_skill_id bigint [ref: > agent_skills.id]
|
||||
reviewed_by text
|
||||
@@ -58,6 +61,7 @@ Table learnings {
|
||||
|
||||
indexes {
|
||||
project_id
|
||||
tenant_key
|
||||
category
|
||||
area
|
||||
status
|
||||
|
||||
@@ -6,6 +6,7 @@ Table plans {
|
||||
status text [not null, default: 'draft'] // draft, active, blocked, completed, cancelled, superseded
|
||||
priority text [not null, default: 'medium'] // low, medium, high, critical
|
||||
project_id bigint [ref: > projects.id]
|
||||
tenant_key text
|
||||
owner text
|
||||
due_date timestamptz
|
||||
completed_at timestamptz
|
||||
@@ -18,6 +19,7 @@ Table plans {
|
||||
|
||||
indexes {
|
||||
project_id
|
||||
tenant_key
|
||||
status
|
||||
priority
|
||||
owner
|
||||
|
||||
Reference in New Issue
Block a user