Author SHA1 Message Date
warkanum e459775740 feat: add webhook thought ingestion
CI / build-and-test (push) Successful in 3m37s
CI / build-and-test (pull_request) Successful in 3m33s
2026-07-15 04:27:09 +02:00
warkanum 5718685c40 Merge pull request 'Expose retrieval mode in query responses' (#37) from issue-14-expose-retrieval-mode into main
CI / build-and-test (push) Failing after 57s
Reviewed-on: #37
Reviewed-by: Warky <2+warkanum@noreply@warky.dev>
2026-07-14 14:44:14 +00:00
warkanum 5c899d1635 Merge pull request 'Link thoughts and learnings' (#36) from issue-32-link-thoughts-learnings into main
CI / build-and-test (push) Failing after 1m6s
Reviewed-on: #36
Reviewed-by: Warky <2+warkanum@noreply@warky.dev>
2026-07-14 14:42:05 +00:00
warkanum 81a3470407 feat(tools): link thoughts and learnings
CI / build-and-test (pull_request) Failing after 1m2s
CI / build-and-test (push) Failing after 3m10s
2026-07-14 15:18:57 +02:00
warkanumandClaude Sonnet 4.6 cfc78e0493 feat(tools): expose retrieval_mode in query-based tool responses
CI / build-and-test (push) Failing after 1m14s
CI / build-and-test (pull_request) Failing after 2m11s
Add a retrieval_mode field ("semantic" or "text") to the output of all
five query-driven tools so callers can distinguish vector search from
Postgres full-text fallback without inspecting server logs.

Tools updated: search_thoughts, recall_context, get_project_context,
summarize_thoughts, and related_thoughts (semantic neighbours only;
omitempty so field is absent when include_semantic is false or no query
is provided for the non-mandatory-query tools).

The shared semanticSearch helper now returns the mode as a third return
value. All callers updated; fixes two compile errors left by the
previous worker (summarize.go and links.go were not capturing the new
return).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-14 12:44:53 +02:00
23 changed files with 966 additions and 96 deletions
+14
View File
@@ -34,6 +34,20 @@ jobs:
- name: Tidy modules - name: Tidy modules
run: go mod tidy run: go mod tidy
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: 'lts/*'
- name: Install pnpm
run: npm install -g pnpm
- name: Build UI
run: |
cd ui
pnpm install --frozen-lockfile
pnpm run build
- name: Run tests - name: Run tests
run: go test ./... run: go test ./...
+1
View File
@@ -36,3 +36,4 @@ ui/.svelte-kit/
internal/app/ui/dist/* internal/app/ui/dist/*
!internal/app/ui/dist/placeholder.txt !internal/app/ui/dist/placeholder.txt
.codex .codex
.worktrees/
+33 -1
View File
@@ -74,6 +74,38 @@ The AMCS directory is used to store configuration and code for the Avalon Memory
| `describe_tools` | List all available MCP tools with names, descriptions, categories, and model-authored usage notes; call this at the start of a session to orient yourself | | `describe_tools` | List all available MCP tools with names, descriptions, categories, and model-authored usage notes; call this at the start of a session to orient yourself |
| `annotate_tool` | Persist your own usage notes for a specific tool; notes are returned by `describe_tools` in future sessions | | `annotate_tool` | Persist your own usage notes for a specific tool; notes are returned by `describe_tools` in future sessions |
## Webhook ingestion
External automation can create thoughts without speaking MCP by posting JSON to `POST /webhooks/thoughts`. The endpoint is protected by the same AMCS authentication middleware as MCP and file uploads, so pass one configured API key via `x-brain-key`, an authorization bearer token header, or another enabled auth method.
Example:
```bash
curl -X POST http://localhost:8080/webhooks/thoughts \
-H 'Content-Type: application/json' \
-H 'x-brain-key: <api-key>' \
-H 'Idempotency-Key: n8n-run-123' \
-d '{
"content": "External system observed build failure on main",
"project": "amcs",
"source": "n8n",
"type": "task",
"topics": ["ci", "webhook"],
"metadata": {"workflow": "ci-monitor", "run_id": "123"}
}'
```
Payload fields:
- `content` is required and becomes the thought content.
- `project` is optional; when present it must match an existing AMCS project.
- `source`, `type`, `topics`, `people`, `action_items`, and `dates_mentioned` are normalized into the standard thought metadata schema. Unknown `type` values fall back to `observation`.
- `metadata` or `source_metadata` may contain safe source-specific JSON values; unsupported values and overly deep objects are dropped rather than persisted.
- `idempotency_key` or the `Idempotency-Key` header can be supplied to make repeated webhook deliveries return the existing thought with `duplicate: true`.
- `external_id` is stored under `metadata.webhook.external_id` for source-side traceability.
Successful new ingestion returns `201` with the created thought. Duplicate idempotency-key delivery returns `200` and the previously created thought. Invalid JSON, missing content, missing/unknown projects, or unauthenticated requests are rejected before persistence. Metadata and embedding enrichment are queued after the thought is stored.
## Learnings ## Learnings
Learnings are curated, structured memory records for durable insights you want to keep distinct from raw thoughts. Use them for normalized lessons, decisions, and evidence-backed findings that should be easy to retrieve and review over time. Learnings are curated, structured memory records for durable insights you want to keep distinct from raw thoughts. Use them for normalized lessons, decisions, and evidence-backed findings that should be easy to retrieve and review over time.
@@ -478,7 +510,7 @@ metadata_retry:
include_archived: false include_archived: false
``` ```
**Search fallback**: when no embeddings exist for the active model in scope, `search_thoughts`, `recall_context`, `get_project_context`, `summarize_thoughts`, and `related_thoughts` automatically fall back to Postgres full-text search so results are never silently empty. **Search fallback**: when no embeddings exist for the active model in scope, `search_thoughts`, `recall_context`, `get_project_context`, `summarize_thoughts`, and `related_thoughts` automatically fall back to Postgres full-text search so results are never silently empty. All five tools include a `retrieval_mode` field in their response (`"semantic"` or `"text"`) so callers can see which path was taken.
## Client Setup ## Client Setup
+24 -22
View File
@@ -194,28 +194,29 @@ func routes(logger *slog.Logger, cfg *config.Config, info buildinfo.Info, db *st
adminActions := newAdminActions(backfillTool, enrichmentRetryer, logger) adminActions := newAdminActions(backfillTool, enrichmentRetryer, logger)
toolSet := mcpserver.ToolSet{ toolSet := mcpserver.ToolSet{
Capture: tools.NewCaptureTool(db, embeddings, cfg.Capture, activeProjects, enrichmentRetryer, backfillTool), Capture: tools.NewCaptureTool(db, embeddings, cfg.Capture, activeProjects, enrichmentRetryer, backfillTool),
Search: tools.NewSearchTool(db, embeddings, cfg.Search, activeProjects), Search: tools.NewSearchTool(db, embeddings, cfg.Search, activeProjects),
List: tools.NewListTool(db, cfg.Search, activeProjects), List: tools.NewListTool(db, cfg.Search, activeProjects),
Stats: tools.NewStatsTool(db), Stats: tools.NewStatsTool(db),
Get: tools.NewGetTool(db), Get: tools.NewGetTool(db),
Update: tools.NewUpdateTool(db, embeddings, metadata, cfg.Capture, logger), Update: tools.NewUpdateTool(db, embeddings, metadata, cfg.Capture, logger),
Delete: tools.NewDeleteTool(db), Delete: tools.NewDeleteTool(db),
Archive: tools.NewArchiveTool(db), Archive: tools.NewArchiveTool(db),
Projects: tools.NewProjectsTool(db, activeProjects), Projects: tools.NewProjectsTool(db, activeProjects),
Version: tools.NewVersionTool(cfg.MCP.ServerName, info), Version: tools.NewVersionTool(cfg.MCP.ServerName, info),
Learnings: tools.NewLearningsTool(db, activeProjects, cfg.Search), Learnings: tools.NewLearningsTool(db, activeProjects, cfg.Search),
Plans: tools.NewPlansTool(db, activeProjects, cfg.Search), Plans: tools.NewPlansTool(db, activeProjects, cfg.Search),
ProjectPersonas: tools.NewProjectPersonasTool(db, activeProjects), ProjectPersonas: tools.NewProjectPersonasTool(db, activeProjects),
WorldModel: tools.NewWorldModelTool(db, activeProjects), WorldModel: tools.NewWorldModelTool(db, activeProjects),
Context: tools.NewContextTool(db, embeddings, cfg.Search, activeProjects), ThoughtLearningLinks: tools.NewThoughtLearningLinksTool(db),
Recall: tools.NewRecallTool(db, embeddings, cfg.Search, activeProjects), Context: tools.NewContextTool(db, embeddings, cfg.Search, activeProjects),
Summarize: tools.NewSummarizeTool(db, embeddings, metadata, cfg.Search, activeProjects), Recall: tools.NewRecallTool(db, embeddings, cfg.Search, activeProjects),
Links: tools.NewLinksTool(db, embeddings, cfg.Search), Summarize: tools.NewSummarizeTool(db, embeddings, metadata, cfg.Search, activeProjects),
Files: filesTool, Links: tools.NewLinksTool(db, embeddings, cfg.Search),
Backfill: backfillTool, Files: filesTool,
Reparse: tools.NewReparseMetadataTool(db, bgMetadata, cfg.Capture, activeProjects, logger), Backfill: backfillTool,
RetryMetadata: tools.NewRetryEnrichmentTool(enrichmentRetryer), Reparse: tools.NewReparseMetadataTool(db, bgMetadata, cfg.Capture, activeProjects, logger),
RetryMetadata: tools.NewRetryEnrichmentTool(enrichmentRetryer),
//Maintenance: tools.NewMaintenanceTool(db), //Maintenance: tools.NewMaintenanceTool(db),
Skills: tools.NewSkillsTool(db, activeProjects), Skills: tools.NewSkillsTool(db, activeProjects),
Personas: tools.NewAgentPersonasTool(db), Personas: tools.NewAgentPersonasTool(db),
@@ -237,6 +238,7 @@ func routes(logger *slog.Logger, cfg *config.Config, info buildinfo.Info, db *st
} }
mux.Handle("/files", authMiddleware(fileHandler(filesTool))) mux.Handle("/files", authMiddleware(fileHandler(filesTool)))
mux.Handle("/files/{id}", authMiddleware(fileHandler(filesTool))) mux.Handle("/files/{id}", authMiddleware(fileHandler(filesTool)))
mux.Handle("/webhooks/thoughts", authMiddleware(newWebhookThoughtHandler(db, embeddings, cfg.Capture, enrichmentRetryer, backfillTool)))
mux.HandleFunc("/.well-known/oauth-authorization-server", oauthMetadataHandler()) mux.HandleFunc("/.well-known/oauth-authorization-server", oauthMetadataHandler())
mux.HandleFunc("/api/oauth/register", oauthRegisterHandler(dynClients, logger)) mux.HandleFunc("/api/oauth/register", oauthRegisterHandler(dynClients, logger))
mux.HandleFunc("/api/oauth/authorize", oauthAuthorizeHandler(dynClients, authCodes, logger)) mux.HandleFunc("/api/oauth/authorize", oauthAuthorizeHandler(dynClients, authCodes, logger))
+214
View File
@@ -0,0 +1,214 @@
package app
import (
"encoding/json"
"errors"
"net/http"
"strings"
"time"
"git.warky.dev/wdevs/amcs/internal/ai"
"git.warky.dev/wdevs/amcs/internal/config"
"git.warky.dev/wdevs/amcs/internal/metadata"
"git.warky.dev/wdevs/amcs/internal/store"
"git.warky.dev/wdevs/amcs/internal/tools"
thoughttypes "git.warky.dev/wdevs/amcs/internal/types"
)
const maxWebhookBodyBytes = 1 << 20
type webhookThoughtRequest struct {
Content string `json:"content"`
Project string `json:"project,omitempty"`
Source string `json:"source,omitempty"`
Type string `json:"type,omitempty"`
Topics []string `json:"topics,omitempty"`
People []string `json:"people,omitempty"`
ActionItems []string `json:"action_items,omitempty"`
DatesMentioned []string `json:"dates_mentioned,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
SourceMetadata map[string]any `json:"source_metadata,omitempty"`
IDempotencyKey string `json:"idempotency_key,omitempty"`
ExternalID string `json:"external_id,omitempty"`
}
type webhookThoughtResponse struct {
Thought thoughttypes.Thought `json:"thought"`
Duplicate bool `json:"duplicate"`
WebhookMeta thoughttypes.WebhookMetadata `json:"webhook"`
}
type webhookThoughtHandler struct {
store *store.DB
embeddings *ai.EmbeddingRunner
capture config.CaptureConfig
retryer tools.MetadataQueuer
embedRetryer tools.EmbeddingQueuer
}
func newWebhookThoughtHandler(db *store.DB, embeddings *ai.EmbeddingRunner, capture config.CaptureConfig, retryer tools.MetadataQueuer, embedRetryer tools.EmbeddingQueuer) http.Handler {
return &webhookThoughtHandler{store: db, embeddings: embeddings, capture: capture, retryer: retryer, embedRetryer: embedRetryer}
}
func (h *webhookThoughtHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/webhooks/thoughts" {
http.NotFound(w, r)
return
}
if r.Method != http.MethodPost {
w.Header().Set("Allow", http.MethodPost)
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
return
}
r.Body = http.MaxBytesReader(w, r.Body, maxWebhookBodyBytes)
in, err := parseWebhookThoughtRequest(r)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
webhookMeta := buildWebhookMetadata(in, r.Header.Get("Idempotency-Key"), time.Now().UTC())
if webhookMeta.IDempotencyKey != "" {
if existing, err := h.store.GetThoughtByWebhookIDempotencyKey(r.Context(), webhookMeta.IDempotencyKey); err == nil {
writeWebhookThoughtResponse(w, http.StatusOK, webhookThoughtResponse{Thought: existing, Duplicate: true, WebhookMeta: webhookMeta})
return
}
}
projectID, err := h.resolveWebhookProject(r, in.Project)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
thought := thoughttypes.Thought{
Content: strings.TrimSpace(in.Content),
Metadata: normalizeWebhookThoughtMetadata(in, webhookMeta, h.capture),
ProjectID: projectID,
}
created, err := h.store.InsertThought(r.Context(), thought, h.embeddings.PrimaryModel())
if err != nil {
http.Error(w, "insert thought: "+err.Error(), http.StatusInternalServerError)
return
}
if projectID != nil {
_ = h.store.TouchProject(r.Context(), *projectID)
}
if h.retryer != nil {
h.retryer.QueueThought(created.ID)
}
if h.embedRetryer != nil {
h.embedRetryer.QueueThought(r.Context(), created.ID, created.Content)
}
writeWebhookThoughtResponse(w, http.StatusCreated, webhookThoughtResponse{Thought: created, WebhookMeta: webhookMeta})
}
func parseWebhookThoughtRequest(r *http.Request) (webhookThoughtRequest, error) {
if !strings.Contains(r.Header.Get("Content-Type"), "application/json") {
return webhookThoughtRequest{}, errors.New("webhook requires application/json")
}
defer r.Body.Close()
decoder := json.NewDecoder(r.Body)
decoder.DisallowUnknownFields()
var in webhookThoughtRequest
if err := decoder.Decode(&in); err != nil {
return webhookThoughtRequest{}, err
}
if strings.TrimSpace(in.Content) == "" {
return webhookThoughtRequest{}, errors.New("content is required")
}
return in, nil
}
func (h *webhookThoughtHandler) resolveWebhookProject(r *http.Request, projectName string) (*int64, error) {
projectName = strings.TrimSpace(projectName)
if projectName == "" {
return nil, nil
}
project, err := h.store.GetProject(r.Context(), projectName)
if err != nil {
return nil, err
}
return &project.NumericID, nil
}
func buildWebhookMetadata(in webhookThoughtRequest, headerKey string, now time.Time) thoughttypes.WebhookMetadata {
sourceMetadata := in.SourceMetadata
if len(sourceMetadata) == 0 {
sourceMetadata = in.Metadata
}
return thoughttypes.WebhookMetadata{
ReceivedAt: now.Format(time.RFC3339),
IDempotencyKey: firstNonEmpty(in.IDempotencyKey, headerKey),
ExternalID: strings.TrimSpace(in.ExternalID),
SourceMetadata: sanitizeWebhookMetadata(sourceMetadata),
}
}
func normalizeWebhookThoughtMetadata(in webhookThoughtRequest, webhookMeta thoughttypes.WebhookMetadata, capture config.CaptureConfig) thoughttypes.ThoughtMetadata {
return metadata.Normalize(thoughttypes.ThoughtMetadata{
People: in.People,
ActionItems: in.ActionItems,
DatesMentioned: in.DatesMentioned,
Topics: in.Topics,
Type: in.Type,
Source: firstNonEmpty(in.Source, "webhook"),
Webhook: &webhookMeta,
}, capture)
}
func sanitizeWebhookMetadata(in map[string]any) map[string]any {
if len(in) == 0 {
return nil
}
out := make(map[string]any, len(in))
for key, value := range in {
key = strings.TrimSpace(key)
if key == "" {
continue
}
if sanitized, ok := sanitizeWebhookMetadataValue(value, 0); ok {
out[key] = sanitized
}
}
if len(out) == 0 {
return nil
}
return out
}
func sanitizeWebhookMetadataValue(value any, depth int) (any, bool) {
if depth > 3 {
return nil, false
}
switch v := value.(type) {
case nil, bool, float64, string:
return v, true
case []any:
if len(v) > 50 {
v = v[:50]
}
out := make([]any, 0, len(v))
for _, item := range v {
if sanitized, ok := sanitizeWebhookMetadataValue(item, depth+1); ok {
out = append(out, sanitized)
}
}
return out, true
case map[string]any:
if len(v) > 50 {
return nil, false
}
return sanitizeWebhookMetadata(v), true
default:
return nil, false
}
}
func writeWebhookThoughtResponse(w http.ResponseWriter, status int, out webhookThoughtResponse) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
_ = json.NewEncoder(w).Encode(out)
}
+86
View File
@@ -0,0 +1,86 @@
package app
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"git.warky.dev/wdevs/amcs/internal/config"
)
func TestParseWebhookThoughtRequestRequiresJSON(t *testing.T) {
req := httptestRequest("text/plain", `{"content":"hello"}`)
_, err := parseWebhookThoughtRequest(req)
if err == nil {
t.Fatal("expected error for non-json content type")
}
}
func TestParseWebhookThoughtRequestRequiresContent(t *testing.T) {
req := httptestRequest("application/json", `{"source":"n8n"}`)
_, err := parseWebhookThoughtRequest(req)
if err == nil || !strings.Contains(err.Error(), "content is required") {
t.Fatalf("error = %v, want content required", err)
}
}
func TestBuildWebhookMetadataUsesHeaderIdempotencyAndSanitizesMetadata(t *testing.T) {
now := time.Date(2026, 7, 15, 4, 0, 0, 0, time.UTC)
got := buildWebhookMetadata(webhookThoughtRequest{
ExternalID: " ext-1 ",
Metadata: map[string]any{
"service": "n8n",
"unsafe": struct{}{},
"nested": map[string]any{"ok": true},
},
}, " key-1 ", now)
if got.IDempotencyKey != "key-1" {
t.Fatalf("IDempotencyKey = %q, want key-1", got.IDempotencyKey)
}
if got.ExternalID != "ext-1" {
t.Fatalf("ExternalID = %q, want ext-1", got.ExternalID)
}
if got.ReceivedAt != "2026-07-15T04:00:00Z" {
t.Fatalf("ReceivedAt = %q", got.ReceivedAt)
}
if _, ok := got.SourceMetadata["unsafe"]; ok {
t.Fatal("unsafe metadata value was not removed")
}
if got.SourceMetadata["service"] != "n8n" {
t.Fatalf("service metadata = %#v", got.SourceMetadata["service"])
}
}
func TestNormalizeWebhookThoughtMetadata(t *testing.T) {
webhookMeta := buildWebhookMetadata(webhookThoughtRequest{IDempotencyKey: "abc"}, "", time.Date(2026, 7, 15, 4, 0, 0, 0, time.UTC))
got := normalizeWebhookThoughtMetadata(webhookThoughtRequest{
Source: "github",
Type: "task",
Topics: []string{"ci", "ci", ""},
People: []string{" Sam "},
}, webhookMeta, config.CaptureConfig{})
if got.Source != "github" {
t.Fatalf("Source = %q, want github", got.Source)
}
if got.Type != "task" {
t.Fatalf("Type = %q, want task", got.Type)
}
if len(got.Topics) != 1 || got.Topics[0] != "ci" {
t.Fatalf("Topics = %#v, want [ci]", got.Topics)
}
if got.Webhook == nil || got.Webhook.IDempotencyKey != "abc" {
t.Fatalf("Webhook = %#v, want idempotency key abc", got.Webhook)
}
}
func httptestRequest(contentType, body string) *http.Request {
req := httptest.NewRequest(http.MethodPost, "/webhooks/thoughts", strings.NewReader(body))
req.Header.Set("Content-Type", contentType)
return req
}
+42 -8
View File
@@ -38,14 +38,15 @@ type ToolSet struct {
Reparse *tools.ReparseMetadataTool Reparse *tools.ReparseMetadataTool
RetryMetadata *tools.RetryEnrichmentTool RetryMetadata *tools.RetryEnrichmentTool
//Maintenance *tools.MaintenanceTool //Maintenance *tools.MaintenanceTool
Skills *tools.SkillsTool Skills *tools.SkillsTool
Personas *tools.AgentPersonasTool Personas *tools.AgentPersonasTool
ChatHistory *tools.ChatHistoryTool ChatHistory *tools.ChatHistoryTool
Describe *tools.DescribeTool Describe *tools.DescribeTool
Learnings *tools.LearningsTool Learnings *tools.LearningsTool
Plans *tools.PlansTool Plans *tools.PlansTool
ProjectPersonas *tools.ProjectPersonasTool ProjectPersonas *tools.ProjectPersonasTool
WorldModel *tools.WorldModelTool WorldModel *tools.WorldModelTool
ThoughtLearningLinks *tools.ThoughtLearningLinksTool
} }
// Handlers groups the HTTP handlers produced for an MCP server instance. // Handlers groups the HTTP handlers produced for an MCP server instance.
@@ -91,6 +92,7 @@ func NewHandlers(cfg config.MCPConfig, logger *slog.Logger, toolSet ToolSet, onS
registerProjectTools, registerProjectTools,
registerWorldModelTools, registerWorldModelTools,
registerLearningTools, registerLearningTools,
registerThoughtLearningLinkTools,
registerPlanTools, registerPlanTools,
registerFileTools, registerFileTools,
registerMaintenanceTools, registerMaintenanceTools,
@@ -308,6 +310,34 @@ func registerLearningTools(server *mcp.Server, logger *slog.Logger, toolSet Tool
return nil return nil
} }
func registerThoughtLearningLinkTools(server *mcp.Server, logger *slog.Logger, toolSet ToolSet) error {
if err := addTool(server, logger, &mcp.Tool{
Name: "link_thought_learning",
Description: "Create or update an explicit association between a raw thought and a curated learning.",
}, toolSet.ThoughtLearningLinks.Link); err != nil {
return err
}
if err := addTool(server, logger, &mcp.Tool{
Name: "unlink_thought_learning",
Description: "Remove an explicit association between a thought and a learning.",
}, toolSet.ThoughtLearningLinks.Unlink); err != nil {
return err
}
if err := addTool(server, logger, &mcp.Tool{
Name: "get_thought_learnings",
Description: "List curated learnings explicitly linked to a thought.",
}, toolSet.ThoughtLearningLinks.GetThoughtLearnings); err != nil {
return err
}
if err := addTool(server, logger, &mcp.Tool{
Name: "get_learning_thoughts",
Description: "List raw thoughts explicitly linked to a learning.",
}, toolSet.ThoughtLearningLinks.GetLearningThoughts); err != nil {
return err
}
return nil
}
func registerPlanTools(server *mcp.Server, logger *slog.Logger, toolSet ToolSet) error { func registerPlanTools(server *mcp.Server, logger *slog.Logger, toolSet ToolSet) error {
if err := addTool(server, logger, &mcp.Tool{ if err := addTool(server, logger, &mcp.Tool{
Name: "create_plan", Name: "create_plan",
@@ -755,6 +785,10 @@ func BuildToolCatalog() []tools.ToolEntry {
{Name: "add_learning", Description: "Create a curated learning record distinct from raw thoughts.", Category: "projects"}, {Name: "add_learning", Description: "Create a curated learning record distinct from raw thoughts.", Category: "projects"},
{Name: "get_learning", Description: "Retrieve a structured learning by id.", Category: "projects"}, {Name: "get_learning", Description: "Retrieve a structured learning by id.", Category: "projects"},
{Name: "list_learnings", Description: "List structured learnings with optional project, category, area, status, priority, tag, and text filters.", Category: "projects"}, {Name: "list_learnings", Description: "List structured learnings with optional project, category, area, status, priority, tag, and text filters.", Category: "projects"},
{Name: "link_thought_learning", Description: "Create or update a lightweight explicit association between a raw thought and a curated learning.", Category: "projects"},
{Name: "unlink_thought_learning", Description: "Remove a lightweight explicit association between a thought and a learning.", Category: "projects"},
{Name: "get_thought_learnings", Description: "List curated learnings explicitly associated with a thought.", Category: "projects"},
{Name: "get_learning_thoughts", Description: "List raw source thoughts explicitly associated with a learning.", Category: "projects"},
// plans // plans
{Name: "create_plan", Description: "Create a structured plan with status, priority, owner, due date, and optional project link.", Category: "plans"}, {Name: "create_plan", Description: "Create a structured plan with status, priority, owner, due date, and optional project link.", Category: "plans"},
@@ -189,30 +189,31 @@ func TestStreamableHTTPReturnsStructuredToolErrors(t *testing.T) {
func streamableTestToolSet() ToolSet { func streamableTestToolSet() ToolSet {
return ToolSet{ return ToolSet{
Version: tools.NewVersionTool("test", buildinfo.Info{Version: "0.0.1", TagName: "v0.0.1", Commit: "test", BuildDate: "2026-03-31T00:00:00Z"}), Version: tools.NewVersionTool("test", buildinfo.Info{Version: "0.0.1", TagName: "v0.0.1", Commit: "test", BuildDate: "2026-03-31T00:00:00Z"}),
Capture: new(tools.CaptureTool), Capture: new(tools.CaptureTool),
Search: new(tools.SearchTool), Search: new(tools.SearchTool),
List: new(tools.ListTool), List: new(tools.ListTool),
Stats: new(tools.StatsTool), Stats: new(tools.StatsTool),
Get: new(tools.GetTool), Get: new(tools.GetTool),
Update: new(tools.UpdateTool), Update: new(tools.UpdateTool),
Delete: new(tools.DeleteTool), Delete: new(tools.DeleteTool),
Archive: new(tools.ArchiveTool), Archive: new(tools.ArchiveTool),
Projects: new(tools.ProjectsTool), Projects: new(tools.ProjectsTool),
Context: new(tools.ContextTool), Context: new(tools.ContextTool),
Recall: new(tools.RecallTool), Recall: new(tools.RecallTool),
Summarize: new(tools.SummarizeTool), Summarize: new(tools.SummarizeTool),
Links: new(tools.LinksTool), Links: new(tools.LinksTool),
Files: new(tools.FilesTool), Files: new(tools.FilesTool),
Backfill: new(tools.BackfillTool), Backfill: new(tools.BackfillTool),
Reparse: new(tools.ReparseMetadataTool), Reparse: new(tools.ReparseMetadataTool),
RetryMetadata: new(tools.RetryEnrichmentTool), RetryMetadata: new(tools.RetryEnrichmentTool),
Skills: new(tools.SkillsTool), Skills: new(tools.SkillsTool),
ChatHistory: new(tools.ChatHistoryTool), ChatHistory: new(tools.ChatHistoryTool),
Describe: new(tools.DescribeTool), Describe: new(tools.DescribeTool),
Learnings: new(tools.LearningsTool), Learnings: new(tools.LearningsTool),
Plans: new(tools.PlansTool), Plans: new(tools.PlansTool),
ProjectPersonas: new(tools.ProjectPersonasTool), ProjectPersonas: new(tools.ProjectPersonasTool),
WorldModel: new(tools.WorldModelTool), WorldModel: new(tools.WorldModelTool),
ThoughtLearningLinks: new(tools.ThoughtLearningLinksTool),
} }
} }
+22
View File
@@ -53,6 +53,7 @@ func Normalize(in thoughttypes.ThoughtMetadata, capture config.CaptureConfig) th
Type: normalizeType(in.Type), Type: normalizeType(in.Type),
Source: normalizeSource(in.Source), Source: normalizeSource(in.Source),
Attachments: normalizeAttachments(in.Attachments), Attachments: normalizeAttachments(in.Attachments),
Webhook: normalizeWebhook(in.Webhook),
MetadataStatus: normalizeMetadataStatus(in.MetadataStatus), MetadataStatus: normalizeMetadataStatus(in.MetadataStatus),
MetadataUpdatedAt: strings.TrimSpace(in.MetadataUpdatedAt), MetadataUpdatedAt: strings.TrimSpace(in.MetadataUpdatedAt),
MetadataLastAttemptedAt: strings.TrimSpace(in.MetadataLastAttemptedAt), MetadataLastAttemptedAt: strings.TrimSpace(in.MetadataLastAttemptedAt),
@@ -201,10 +202,31 @@ func Merge(base, patch thoughttypes.ThoughtMetadata, capture config.CaptureConfi
if len(patch.Attachments) > 0 { if len(patch.Attachments) > 0 {
merged.Attachments = append(append([]thoughttypes.ThoughtAttachment{}, merged.Attachments...), patch.Attachments...) merged.Attachments = append(append([]thoughttypes.ThoughtAttachment{}, merged.Attachments...), patch.Attachments...)
} }
if patch.Webhook != nil {
merged.Webhook = patch.Webhook
}
return Normalize(merged, capture) return Normalize(merged, capture)
} }
func normalizeWebhook(value *thoughttypes.WebhookMetadata) *thoughttypes.WebhookMetadata {
if value == nil {
return nil
}
out := &thoughttypes.WebhookMetadata{
ReceivedAt: strings.TrimSpace(value.ReceivedAt),
IDempotencyKey: strings.TrimSpace(value.IDempotencyKey),
ExternalID: strings.TrimSpace(value.ExternalID),
}
if len(value.SourceMetadata) > 0 {
out.SourceMetadata = value.SourceMetadata
}
if out.ReceivedAt == "" && out.IDempotencyKey == "" && out.ExternalID == "" && len(out.SourceMetadata) == 0 {
return nil
}
return out
}
func normalizeAttachments(values []thoughttypes.ThoughtAttachment) []thoughttypes.ThoughtAttachment { func normalizeAttachments(values []thoughttypes.ThoughtAttachment) []thoughttypes.ThoughtAttachment {
seen := make(map[string]struct{}, len(values)) seen := make(map[string]struct{}, len(values))
result := make([]thoughttypes.ThoughtAttachment, 0, len(values)) result := make([]thoughttypes.ThoughtAttachment, 0, len(values))
+148
View File
@@ -0,0 +1,148 @@
package store
import (
"context"
"fmt"
"github.com/google/uuid"
"git.warky.dev/wdevs/amcs/internal/generatedmodels"
thoughttypes "git.warky.dev/wdevs/amcs/internal/types"
)
// LinkThoughtLearning creates or updates an association between a thought (by GUID) and a learning (by numeric ID).
// If the pair already exists, the relation label is updated.
func (db *DB) LinkThoughtLearning(ctx context.Context, thoughtGUID uuid.UUID, learningID int64, relation string) error {
_, err := db.pool.Exec(ctx, `
insert into thought_learning_links (thought_id, learning_id, relation)
select t.id, $2, $3
from thoughts t
where t.guid = $1
on conflict (thought_id, learning_id) do update set relation = excluded.relation
`, thoughtGUID, learningID, relation)
if err != nil {
return fmt.Errorf("link thought learning: %w", err)
}
return nil
}
// UnlinkThoughtLearning removes the association between a thought (by GUID) and a learning (by numeric ID).
func (db *DB) UnlinkThoughtLearning(ctx context.Context, thoughtGUID uuid.UUID, learningID int64) error {
_, err := db.pool.Exec(ctx, `
delete from thought_learning_links
where thought_id = (select id from thoughts where guid = $1)
and learning_id = $2
`, thoughtGUID, learningID)
if err != nil {
return fmt.Errorf("unlink thought learning: %w", err)
}
return nil
}
// GetLinkedLearnings returns all learnings explicitly associated with a thought (by GUID).
func (db *DB) GetLinkedLearnings(ctx context.Context, thoughtGUID uuid.UUID) ([]thoughttypes.LinkedLearning, error) {
rows, err := db.pool.Query(ctx, `
select l.id, l.guid, l.summary, l.details, l.category, l.area, l.status, l.priority, l.confidence,
l.action_required, l.source_type, l.source_ref, l.project_id, l.related_thought_id,
l.related_skill_id, l.reviewed_by, l.reviewed_at, l.duplicate_of_learning_id,
l.supersedes_learning_id, l.tags::text[], l.created_at, l.updated_at,
tll.relation, tll.created_at as linked_at
from thought_learning_links tll
join learnings l on l.id = tll.learning_id
where tll.thought_id = (select id from thoughts where guid = $1)
order by tll.created_at desc
`, thoughtGUID)
if err != nil {
return nil, fmt.Errorf("query linked learnings: %w", err)
}
defer rows.Close()
items := make([]thoughttypes.LinkedLearning, 0)
for rows.Next() {
var tags []string
var linked thoughttypes.LinkedLearning
var m generatedmodels.ModelPublicLearnings
if err := rows.Scan(
&m.ID,
&m.GUID,
&m.Summary,
&m.Details,
&m.Category,
&m.Area,
&m.Status,
&m.Priority,
&m.Confidence,
&m.ActionRequired,
&m.SourceType,
&m.SourceRef,
&m.ProjectID,
&m.RelatedThoughtID,
&m.RelatedSkillID,
&m.ReviewedBy,
&m.ReviewedAt,
&m.DuplicateOfLearningID,
&m.SupersedesLearningID,
&tags,
&m.CreatedAt,
&m.UpdatedAt,
&linked.Relation,
&linked.CreatedAt,
); err != nil {
return nil, fmt.Errorf("scan linked learning: %w", err)
}
linked.Learning = learningFromModel(m, tags)
items = append(items, linked)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("iterate linked learnings: %w", err)
}
return items, nil
}
// GetLinkedThoughtsForLearning returns all thoughts explicitly associated with a learning (by numeric ID).
func (db *DB) GetLinkedThoughtsForLearning(ctx context.Context, learningID int64) ([]thoughttypes.LinkedLearningThought, error) {
rows, err := db.pool.Query(ctx, `
select t.id, t.guid, t.content, t.metadata, t.project_id, t.archived_at, t.created_at, t.updated_at,
tll.relation, tll.created_at as linked_at
from thought_learning_links tll
join thoughts t on t.id = tll.thought_id
where tll.learning_id = $1
order by tll.created_at desc
`, learningID)
if err != nil {
return nil, fmt.Errorf("query linked thoughts for learning: %w", err)
}
defer rows.Close()
items := make([]thoughttypes.LinkedLearningThought, 0)
for rows.Next() {
var linked thoughttypes.LinkedLearningThought
var m generatedmodels.ModelPublicThoughts
if err := rows.Scan(
&m.ID,
&m.GUID,
&m.Content,
&m.Metadata,
&m.ProjectID,
&m.ArchivedAt,
&m.CreatedAt,
&m.UpdatedAt,
&linked.Relation,
&linked.CreatedAt,
); err != nil {
return nil, fmt.Errorf("scan linked thought for learning: %w", err)
}
thought, err := thoughtFromModel(m)
if err != nil {
return nil, fmt.Errorf("map linked thought for learning: %w", err)
}
linked.Thought = thought
items = append(items, linked)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("iterate linked thoughts for learning: %w", err)
}
return items, nil
}
+16
View File
@@ -68,6 +68,22 @@ func (db *DB) InsertThought(ctx context.Context, thought thoughttypes.Thought, e
return created, nil return created, nil
} }
func (db *DB) GetThoughtByWebhookIDempotencyKey(ctx context.Context, key string) (thoughttypes.Thought, error) {
row := db.pool.QueryRow(ctx, `
select id, guid, content, metadata, project_id, archived_at, created_at, updated_at
from thoughts
where metadata->'webhook'->>'idempotency_key' = $1
order by created_at desc
limit 1
`, strings.TrimSpace(key))
var model generatedmodels.ModelPublicThoughts
if err := row.Scan(&model.ID, &model.GUID, &model.Content, &model.Metadata, &model.ProjectID, &model.ArchivedAt, &model.CreatedAt, &model.UpdatedAt); err != nil {
return thoughttypes.Thought{}, err
}
return thoughtFromModel(model)
}
func (db *DB) SearchThoughts(ctx context.Context, embedding []float32, embeddingModel string, threshold float64, limit int, filter map[string]any) ([]thoughttypes.SearchResult, error) { func (db *DB) SearchThoughts(ctx context.Context, embedding []float32, embeddingModel string, threshold float64, limit int, filter map[string]any) ([]thoughttypes.SearchResult, error) {
filterJSON, err := json.Marshal(filter) filterJSON, err := json.Marshal(filter)
if err != nil { if err != nil {
+11 -7
View File
@@ -36,9 +36,10 @@ type ContextItem struct {
} }
type ProjectContextOutput struct { type ProjectContextOutput struct {
Project thoughttypes.Project `json:"project"` Project thoughttypes.Project `json:"project"`
Context string `json:"context"` Context string `json:"context"`
Items []ContextItem `json:"items"` Items []ContextItem `json:"items"`
RetrievalMode string `json:"retrieval_mode,omitempty"`
} }
func NewContextTool(db *store.DB, embeddings *ai.EmbeddingRunner, search config.SearchConfig, sessions *session.ActiveProjects) *ContextTool { func NewContextTool(db *store.DB, embeddings *ai.EmbeddingRunner, search config.SearchConfig, sessions *session.ActiveProjects) *ContextTool {
@@ -70,12 +71,14 @@ func (t *ContextTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in P
}) })
} }
var retrievalMode string
query := strings.TrimSpace(in.Query) query := strings.TrimSpace(in.Query)
if query != "" { if query != "" {
semantic, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, t.search.DefaultThreshold, &project.NumericID, nil) semantic, mode, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, t.search.DefaultThreshold, &project.NumericID, nil)
if err != nil { if err != nil {
return nil, ProjectContextOutput{}, err return nil, ProjectContextOutput{}, err
} }
retrievalMode = mode
for _, result := range semantic { for _, result := range semantic {
key := fmt.Sprint(result.ID) key := fmt.Sprint(result.ID)
if _, ok := seen[key]; ok { if _, ok := seen[key]; ok {
@@ -100,8 +103,9 @@ func (t *ContextTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in P
_ = t.store.TouchProject(ctx, project.NumericID) _ = t.store.TouchProject(ctx, project.NumericID)
return nil, ProjectContextOutput{ return nil, ProjectContextOutput{
Project: *project, Project: *project,
Context: contextBlock, Context: contextBlock,
Items: items, Items: items,
RetrievalMode: retrievalMode,
}, nil }, nil
} }
+6 -3
View File
@@ -45,7 +45,8 @@ type RelatedThought struct {
} }
type RelatedOutput struct { type RelatedOutput struct {
Related []RelatedThought `json:"related"` Related []RelatedThought `json:"related"`
RetrievalMode string `json:"retrieval_mode,omitempty"`
} }
func NewLinksTool(db *store.DB, embeddings *ai.EmbeddingRunner, search config.SearchConfig) *LinksTool { func NewLinksTool(db *store.DB, embeddings *ai.EmbeddingRunner, search config.SearchConfig) *LinksTool {
@@ -117,11 +118,13 @@ func (t *LinksTool) Related(ctx context.Context, _ *mcp.CallToolRequest, in Rela
includeSemantic = *in.IncludeSemantic includeSemantic = *in.IncludeSemantic
} }
var retrievalMode string
if includeSemantic { if includeSemantic {
semantic, err := semanticSearch(ctx, t.store, t.embeddings, t.search, thought.Content, t.search.DefaultLimit, t.search.DefaultThreshold, thought.ProjectID, &thought.GUID) semantic, mode, err := semanticSearch(ctx, t.store, t.embeddings, t.search, thought.Content, t.search.DefaultLimit, t.search.DefaultThreshold, thought.ProjectID, &thought.GUID)
if err != nil { if err != nil {
return nil, RelatedOutput{}, err return nil, RelatedOutput{}, err
} }
retrievalMode = mode
for _, item := range semantic { for _, item := range semantic {
key := fmt.Sprint(item.ID) key := fmt.Sprint(item.ID)
if _, ok := seen[key]; ok { if _, ok := seen[key]; ok {
@@ -138,5 +141,5 @@ func (t *LinksTool) Related(ctx context.Context, _ *mcp.CallToolRequest, in Rela
} }
} }
return nil, RelatedOutput{Related: related}, nil return nil, RelatedOutput{Related: related, RetrievalMode: retrievalMode}, nil
} }
+7 -5
View File
@@ -27,8 +27,9 @@ type RecallInput struct {
} }
type RecallOutput struct { type RecallOutput struct {
Context string `json:"context"` Context string `json:"context"`
Items []ContextItem `json:"items"` Items []ContextItem `json:"items"`
RetrievalMode string `json:"retrieval_mode,omitempty"`
} }
func NewRecallTool(db *store.DB, embeddings *ai.EmbeddingRunner, search config.SearchConfig, sessions *session.ActiveProjects) *RecallTool { func NewRecallTool(db *store.DB, embeddings *ai.EmbeddingRunner, search config.SearchConfig, sessions *session.ActiveProjects) *RecallTool {
@@ -53,7 +54,7 @@ func (t *RecallTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in Re
projectID = &project.NumericID projectID = &project.NumericID
} }
semantic, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, t.search.DefaultThreshold, projectID, nil) semantic, mode, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, t.search.DefaultThreshold, projectID, nil)
if err != nil { if err != nil {
return nil, RecallOutput{}, err return nil, RecallOutput{}, err
} }
@@ -100,7 +101,8 @@ func (t *RecallTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in Re
} }
return nil, RecallOutput{ return nil, RecallOutput{
Context: formatContextBlock(header, lines), Context: formatContextBlock(header, lines),
Items: items, Items: items,
RetrievalMode: mode,
}, nil }, nil
} }
+13 -5
View File
@@ -11,10 +11,16 @@ import (
thoughttypes "git.warky.dev/wdevs/amcs/internal/types" thoughttypes "git.warky.dev/wdevs/amcs/internal/types"
) )
const (
RetrievalModeSemantic = "semantic"
RetrievalModeText = "text"
)
// semanticSearch runs vector similarity search if embeddings exist for the // semanticSearch runs vector similarity search if embeddings exist for the
// primary embedding model in the given scope, otherwise falls back to Postgres // primary embedding model in the given scope, otherwise falls back to Postgres
// full-text search. Search always uses the primary model so query vectors // full-text search. Search always uses the primary model so query vectors
// match rows stored under the primary model name. // match rows stored under the primary model name.
// It returns the results and the retrieval mode used ("semantic" or "text").
func semanticSearch( func semanticSearch(
ctx context.Context, ctx context.Context,
db *store.DB, db *store.DB,
@@ -25,20 +31,22 @@ func semanticSearch(
threshold float64, threshold float64,
projectID *int64, projectID *int64,
excludeID *uuid.UUID, excludeID *uuid.UUID,
) ([]thoughttypes.SearchResult, error) { ) ([]thoughttypes.SearchResult, string, error) {
model := embeddings.PrimaryModel() model := embeddings.PrimaryModel()
hasEmbeddings, err := db.HasEmbeddingsForModel(ctx, model, projectID) hasEmbeddings, err := db.HasEmbeddingsForModel(ctx, model, projectID)
if err != nil { if err != nil {
return nil, err return nil, "", err
} }
if hasEmbeddings { if hasEmbeddings {
embedding, err := embeddings.EmbedPrimary(ctx, query) embedding, err := embeddings.EmbedPrimary(ctx, query)
if err != nil { if err != nil {
return nil, err return nil, "", err
} }
return db.SearchSimilarThoughts(ctx, embedding, model, threshold, limit, projectID, excludeID) results, err := db.SearchSimilarThoughts(ctx, embedding, model, threshold, limit, projectID, excludeID)
return results, RetrievalModeSemantic, err
} }
return db.SearchThoughtsText(ctx, query, limit, projectID, excludeID) results, err := db.SearchThoughtsText(ctx, query, limit, projectID, excludeID)
return results, RetrievalModeText, err
} }
+60
View File
@@ -0,0 +1,60 @@
package tools
import "testing"
func TestRetrievalModeConstants(t *testing.T) {
if RetrievalModeSemantic != "semantic" {
t.Fatalf("RetrievalModeSemantic = %q, want %q", RetrievalModeSemantic, "semantic")
}
if RetrievalModeText != "text" {
t.Fatalf("RetrievalModeText = %q, want %q", RetrievalModeText, "text")
}
}
func TestSearchOutputIncludesRetrievalMode(t *testing.T) {
out := SearchOutput{RetrievalMode: RetrievalModeSemantic}
if out.RetrievalMode != RetrievalModeSemantic {
t.Fatalf("SearchOutput.RetrievalMode = %q, want %q", out.RetrievalMode, RetrievalModeSemantic)
}
}
func TestRelatedOutputIncludesRetrievalMode(t *testing.T) {
out := RelatedOutput{RetrievalMode: RetrievalModeText}
if out.RetrievalMode != RetrievalModeText {
t.Fatalf("RelatedOutput.RetrievalMode = %q, want %q", out.RetrievalMode, RetrievalModeText)
}
// retrieval_mode is omitempty — empty string means semantic search was skipped
out2 := RelatedOutput{}
if out2.RetrievalMode != "" {
t.Fatalf("RelatedOutput.RetrievalMode = %q, want empty when include_semantic is false", out2.RetrievalMode)
}
}
func TestSummarizeOutputIncludesRetrievalModeOnlyWhenQueryUsed(t *testing.T) {
withQuery := SummarizeOutput{Summary: "s", Count: 1, RetrievalMode: RetrievalModeSemantic}
if withQuery.RetrievalMode != RetrievalModeSemantic {
t.Fatalf("SummarizeOutput.RetrievalMode = %q, want %q", withQuery.RetrievalMode, RetrievalModeSemantic)
}
withoutQuery := SummarizeOutput{Summary: "s", Count: 1}
if withoutQuery.RetrievalMode != "" {
t.Fatalf("SummarizeOutput.RetrievalMode = %q, want empty when no query", withoutQuery.RetrievalMode)
}
}
func TestProjectContextOutputIncludesRetrievalModeOnlyWhenQueryUsed(t *testing.T) {
withQuery := ProjectContextOutput{RetrievalMode: RetrievalModeText}
if withQuery.RetrievalMode != RetrievalModeText {
t.Fatalf("ProjectContextOutput.RetrievalMode = %q, want %q", withQuery.RetrievalMode, RetrievalModeText)
}
withoutQuery := ProjectContextOutput{}
if withoutQuery.RetrievalMode != "" {
t.Fatalf("ProjectContextOutput.RetrievalMode = %q, want empty when no query", withoutQuery.RetrievalMode)
}
}
func TestRecallOutputIncludesRetrievalMode(t *testing.T) {
out := RecallOutput{RetrievalMode: RetrievalModeSemantic}
if out.RetrievalMode != RetrievalModeSemantic {
t.Fatalf("RecallOutput.RetrievalMode = %q, want %q", out.RetrievalMode, RetrievalModeSemantic)
}
}
+4 -3
View File
@@ -28,7 +28,8 @@ type SearchInput struct {
} }
type SearchOutput struct { type SearchOutput struct {
Results []thoughttypes.SearchResult `json:"results"` Results []thoughttypes.SearchResult `json:"results"`
RetrievalMode string `json:"retrieval_mode,omitempty"`
} }
func NewSearchTool(db *store.DB, embeddings *ai.EmbeddingRunner, search config.SearchConfig, sessions *session.ActiveProjects) *SearchTool { func NewSearchTool(db *store.DB, embeddings *ai.EmbeddingRunner, search config.SearchConfig, sessions *session.ActiveProjects) *SearchTool {
@@ -55,10 +56,10 @@ func (t *SearchTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in Se
_ = t.store.TouchProject(ctx, project.NumericID) _ = t.store.TouchProject(ctx, project.NumericID)
} }
results, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, threshold, projectID, nil) results, mode, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, threshold, projectID, nil)
if err != nil { if err != nil {
return nil, SearchOutput{}, err return nil, SearchOutput{}, err
} }
return nil, SearchOutput{Results: results}, nil return nil, SearchOutput{Results: results, RetrievalMode: mode}, nil
} }
+7 -4
View File
@@ -28,8 +28,9 @@ type SummarizeInput struct {
} }
type SummarizeOutput struct { type SummarizeOutput struct {
Summary string `json:"summary"` Summary string `json:"summary"`
Count int `json:"count"` Count int `json:"count"`
RetrievalMode string `json:"retrieval_mode,omitempty"`
} }
func NewSummarizeTool(db *store.DB, embeddings *ai.EmbeddingRunner, metadata *ai.MetadataRunner, search config.SearchConfig, sessions *session.ActiveProjects) *SummarizeTool { func NewSummarizeTool(db *store.DB, embeddings *ai.EmbeddingRunner, metadata *ai.MetadataRunner, search config.SearchConfig, sessions *session.ActiveProjects) *SummarizeTool {
@@ -47,15 +48,17 @@ func (t *SummarizeTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in
lines := make([]string, 0, limit) lines := make([]string, 0, limit)
count := 0 count := 0
var retrievalMode string
if query != "" { if query != "" {
var projectID *int64 var projectID *int64
if project != nil { if project != nil {
projectID = &project.NumericID projectID = &project.NumericID
} }
results, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, t.search.DefaultThreshold, projectID, nil) results, mode, err := semanticSearch(ctx, t.store, t.embeddings, t.search, query, limit, t.search.DefaultThreshold, projectID, nil)
if err != nil { if err != nil {
return nil, SummarizeOutput{}, err return nil, SummarizeOutput{}, err
} }
retrievalMode = mode
for i, result := range results { for i, result := range results {
lines = append(lines, thoughtContextLine(i, result.Content, result.Metadata, result.Similarity)) lines = append(lines, thoughtContextLine(i, result.Content, result.Metadata, result.Similarity))
} }
@@ -85,5 +88,5 @@ func (t *SummarizeTool) Handle(ctx context.Context, req *mcp.CallToolRequest, in
_ = t.store.TouchProject(ctx, project.NumericID) _ = t.store.TouchProject(ctx, project.NumericID)
} }
return nil, SummarizeOutput{Summary: summary, Count: count}, nil return nil, SummarizeOutput{Summary: summary, Count: count, RetrievalMode: retrievalMode}, nil
} }
+152
View File
@@ -0,0 +1,152 @@
package tools
import (
"context"
"strconv"
"strings"
"github.com/modelcontextprotocol/go-sdk/mcp"
"git.warky.dev/wdevs/amcs/internal/store"
thoughttypes "git.warky.dev/wdevs/amcs/internal/types"
)
type ThoughtLearningLinksTool struct {
store *store.DB
}
type LinkThoughtLearningInput struct {
ThoughtID string `json:"thought_id" jsonschema:"UUID of the thought to link"`
LearningID int64 `json:"learning_id" jsonschema:"numeric id of the learning to link"`
Relation string `json:"relation,omitempty" jsonschema:"relationship label, e.g. source, derived_from, related; defaults to source"`
}
type LinkThoughtLearningOutput struct {
Linked bool `json:"linked"`
}
type UnlinkThoughtLearningInput struct {
ThoughtID string `json:"thought_id" jsonschema:"UUID of the thought"`
LearningID int64 `json:"learning_id" jsonschema:"numeric id of the learning"`
}
type UnlinkThoughtLearningOutput struct {
Unlinked bool `json:"unlinked"`
}
type GetThoughtLearningsInput struct {
ThoughtID string `json:"thought_id" jsonschema:"UUID of the thought"`
}
type GetThoughtLearningsOutput struct {
Learnings []thoughttypes.LinkedLearning `json:"learnings"`
}
type GetLearningThoughtsInput struct {
LearningID int64 `json:"learning_id" jsonschema:"numeric id of the learning"`
}
type GetLearningThoughtsOutput struct {
Thoughts []thoughttypes.LinkedLearningThought `json:"thoughts"`
}
func NewThoughtLearningLinksTool(db *store.DB) *ThoughtLearningLinksTool {
return &ThoughtLearningLinksTool{store: db}
}
func (t *ThoughtLearningLinksTool) Link(ctx context.Context, _ *mcp.CallToolRequest, in LinkThoughtLearningInput) (*mcp.CallToolResult, LinkThoughtLearningOutput, error) {
if err := t.ensureConfigured(); err != nil {
return nil, LinkThoughtLearningOutput{}, err
}
thoughtGUID, err := parseUUID(in.ThoughtID)
if err != nil {
return nil, LinkThoughtLearningOutput{}, err
}
if in.LearningID <= 0 {
return nil, LinkThoughtLearningOutput{}, errRequiredField("learning_id")
}
relation := strings.TrimSpace(in.Relation)
if relation == "" {
relation = "source"
}
if _, err := t.store.GetThought(ctx, thoughtGUID); err != nil {
return nil, LinkThoughtLearningOutput{}, errEntityNotFound("thought", "thought_id", in.ThoughtID)
}
if _, err := t.store.GetLearning(ctx, in.LearningID); err != nil {
return nil, LinkThoughtLearningOutput{}, errEntityNotFound("learning", "learning_id", strconv.FormatInt(in.LearningID, 10))
}
if err := t.store.LinkThoughtLearning(ctx, thoughtGUID, in.LearningID, relation); err != nil {
return nil, LinkThoughtLearningOutput{}, err
}
return nil, LinkThoughtLearningOutput{Linked: true}, nil
}
func (t *ThoughtLearningLinksTool) Unlink(ctx context.Context, _ *mcp.CallToolRequest, in UnlinkThoughtLearningInput) (*mcp.CallToolResult, UnlinkThoughtLearningOutput, error) {
if err := t.ensureConfigured(); err != nil {
return nil, UnlinkThoughtLearningOutput{}, err
}
thoughtGUID, err := parseUUID(in.ThoughtID)
if err != nil {
return nil, UnlinkThoughtLearningOutput{}, err
}
if in.LearningID <= 0 {
return nil, UnlinkThoughtLearningOutput{}, errRequiredField("learning_id")
}
if err := t.store.UnlinkThoughtLearning(ctx, thoughtGUID, in.LearningID); err != nil {
return nil, UnlinkThoughtLearningOutput{}, err
}
return nil, UnlinkThoughtLearningOutput{Unlinked: true}, nil
}
func (t *ThoughtLearningLinksTool) GetThoughtLearnings(ctx context.Context, _ *mcp.CallToolRequest, in GetThoughtLearningsInput) (*mcp.CallToolResult, GetThoughtLearningsOutput, error) {
if err := t.ensureConfigured(); err != nil {
return nil, GetThoughtLearningsOutput{}, err
}
thoughtGUID, err := parseUUID(in.ThoughtID)
if err != nil {
return nil, GetThoughtLearningsOutput{}, err
}
if _, err := t.store.GetThought(ctx, thoughtGUID); err != nil {
return nil, GetThoughtLearningsOutput{}, errEntityNotFound("thought", "thought_id", in.ThoughtID)
}
learnings, err := t.store.GetLinkedLearnings(ctx, thoughtGUID)
if err != nil {
return nil, GetThoughtLearningsOutput{}, err
}
return nil, GetThoughtLearningsOutput{Learnings: learnings}, nil
}
func (t *ThoughtLearningLinksTool) GetLearningThoughts(ctx context.Context, _ *mcp.CallToolRequest, in GetLearningThoughtsInput) (*mcp.CallToolResult, GetLearningThoughtsOutput, error) {
if err := t.ensureConfigured(); err != nil {
return nil, GetLearningThoughtsOutput{}, err
}
if in.LearningID <= 0 {
return nil, GetLearningThoughtsOutput{}, errRequiredField("learning_id")
}
if _, err := t.store.GetLearning(ctx, in.LearningID); err != nil {
return nil, GetLearningThoughtsOutput{}, errEntityNotFound("learning", "learning_id", strconv.FormatInt(in.LearningID, 10))
}
thoughts, err := t.store.GetLinkedThoughtsForLearning(ctx, in.LearningID)
if err != nil {
return nil, GetLearningThoughtsOutput{}, err
}
return nil, GetLearningThoughtsOutput{Thoughts: thoughts}, nil
}
func (t *ThoughtLearningLinksTool) ensureConfigured() error {
if t == nil || t.store == nil {
return errInvalidInput("thought learning links tool is not configured")
}
return nil
}
+19
View File
@@ -39,3 +39,22 @@ type LinkedThought struct {
Direction string `json:"direction"` Direction string `json:"direction"`
CreatedAt time.Time `json:"created_at"` CreatedAt time.Time `json:"created_at"`
} }
type ThoughtLearningLink struct {
ThoughtID int64 `json:"thought_id"`
LearningID int64 `json:"learning_id"`
Relation string `json:"relation"`
CreatedAt time.Time `json:"created_at"`
}
type LinkedLearning struct {
Learning Learning `json:"learning"`
Relation string `json:"relation"`
CreatedAt time.Time `json:"created_at"`
}
type LinkedLearningThought struct {
Thought Thought `json:"thought"`
Relation string `json:"relation"`
CreatedAt time.Time `json:"created_at"`
}
+21 -13
View File
@@ -14,12 +14,20 @@ type ThoughtMetadata struct {
Type string `json:"type"` Type string `json:"type"`
Source string `json:"source"` Source string `json:"source"`
Attachments []ThoughtAttachment `json:"attachments,omitempty"` Attachments []ThoughtAttachment `json:"attachments,omitempty"`
Webhook *WebhookMetadata `json:"webhook,omitempty"`
MetadataStatus string `json:"metadata_status,omitempty"` MetadataStatus string `json:"metadata_status,omitempty"`
MetadataUpdatedAt string `json:"metadata_updated_at,omitempty"` MetadataUpdatedAt string `json:"metadata_updated_at,omitempty"`
MetadataLastAttemptedAt string `json:"metadata_last_attempted_at,omitempty"` MetadataLastAttemptedAt string `json:"metadata_last_attempted_at,omitempty"`
MetadataError string `json:"metadata_error,omitempty"` MetadataError string `json:"metadata_error,omitempty"`
} }
type WebhookMetadata struct {
ReceivedAt string `json:"received_at"`
IDempotencyKey string `json:"idempotency_key,omitempty"`
ExternalID string `json:"external_id,omitempty"`
SourceMetadata map[string]any `json:"source_metadata,omitempty"`
}
type ThoughtAttachment struct { type ThoughtAttachment struct {
FileID uuid.UUID `json:"file_id"` FileID uuid.UUID `json:"file_id"`
Name string `json:"name"` Name string `json:"name"`
@@ -30,19 +38,19 @@ type ThoughtAttachment struct {
} }
type StoredFile struct { type StoredFile struct {
ID int64 `json:"id"` ID int64 `json:"id"`
GUID uuid.UUID `json:"guid"` GUID uuid.UUID `json:"guid"`
ThoughtID *int64 `json:"thought_id,omitempty"` ThoughtID *int64 `json:"thought_id,omitempty"`
ProjectID *int64 `json:"project_id,omitempty"` ProjectID *int64 `json:"project_id,omitempty"`
Name string `json:"name"` Name string `json:"name"`
MediaType string `json:"media_type"` MediaType string `json:"media_type"`
Kind string `json:"kind"` Kind string `json:"kind"`
Encoding string `json:"encoding"` Encoding string `json:"encoding"`
SizeBytes int64 `json:"size_bytes"` SizeBytes int64 `json:"size_bytes"`
SHA256 string `json:"sha256"` SHA256 string `json:"sha256"`
Content []byte `json:"-"` Content []byte `json:"-"`
CreatedAt time.Time `json:"created_at"` CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"` UpdatedAt time.Time `json:"updated_at"`
} }
type StoredFileFilter struct { type StoredFileFilter struct {
+26
View File
@@ -0,0 +1,26 @@
-- Many-to-many join table between thoughts and learnings.
-- Allows a learning to reference multiple source thoughts and a thought to
-- expose multiple related learnings, queryable in both directions.
CREATE SEQUENCE IF NOT EXISTS public.identity_thought_learning_links_id
INCREMENT 1
MINVALUE 1
MAXVALUE 9223372036854775807
START 1
CACHE 1;
CREATE TABLE IF NOT EXISTS public.thought_learning_links (
id bigint NOT NULL DEFAULT nextval('public.identity_thought_learning_links_id'),
thought_id bigint NOT NULL REFERENCES public.thoughts(id) ON DELETE CASCADE,
learning_id bigint NOT NULL REFERENCES public.learnings(id) ON DELETE CASCADE,
relation text NOT NULL DEFAULT 'source',
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT thought_learning_links_pkey PRIMARY KEY (id),
CONSTRAINT thought_learning_links_unique UNIQUE (thought_id, learning_id)
);
CREATE INDEX IF NOT EXISTS idx_thought_learning_links_thought_id
ON public.thought_learning_links (thought_id);
CREATE INDEX IF NOT EXISTS idx_thought_learning_links_learning_id
ON public.thought_learning_links (learning_id);
+14
View File
@@ -32,6 +32,20 @@ Table thought_links {
} }
} }
Table thought_learning_links {
id bigserial [pk]
thought_id bigint [not null, ref: > thoughts.id]
learning_id bigint [not null, ref: > learnings.id]
relation text [not null, default: `'source'`]
created_at timestamptz [not null, default: `now()`]
indexes {
(thought_id, learning_id) [unique]
thought_id
learning_id
}
}
Table embeddings { Table embeddings {
id bigserial [pk] id bigserial [pk]
guid uuid [unique, not null, default: `gen_random_uuid()`] guid uuid [unique, not null, default: `gen_random_uuid()`]