feat: implement file upload handler and related functionality
- Added file upload handler to process both multipart and raw file uploads. - Implemented parsing logic for upload requests, including handling file metadata. - Introduced SaveFileDecodedInput structure for handling decoded file uploads. - Created unit tests for file upload parsing and validation. feat: add metadata retry configuration and functionality - Introduced MetadataRetryConfig to the application configuration. - Implemented MetadataRetryer to handle retrying metadata extraction for thoughts. - Added new tool for retrying failed metadata extractions. - Updated thought metadata structure to include status and timestamps for metadata processing. fix: enhance metadata normalization and error handling - Updated metadata normalization functions to track status and errors. - Improved handling of metadata extraction failures during thought updates and captures. - Ensured that metadata status is correctly set during various operations. refactor: streamline file saving logic in FilesTool - Refactored Save method in FilesTool to utilize new SaveDecoded method. - Simplified project and thought ID resolution logic during file saving.
This commit is contained in:
49
README.md
49
README.md
@@ -46,6 +46,7 @@ A Go MCP server for capturing and retrieving thoughts, memory, and project conte
|
||||
| `list_files` | Browse stored files by thought, project, or kind |
|
||||
| `backfill_embeddings` | Generate missing embeddings for stored thoughts |
|
||||
| `reparse_thought_metadata` | Re-extract and normalize metadata for stored thoughts |
|
||||
| `retry_failed_metadata` | Retry metadata extraction for thoughts still pending or failed |
|
||||
|
||||
## Configuration
|
||||
|
||||
@@ -115,6 +116,24 @@ Run `reparse_thought_metadata` to fix stale or inconsistent metadata by re-extra
|
||||
- If extraction fails for a thought, existing metadata is normalized and written only if it changes
|
||||
- Metadata reparse runs in parallel (4 workers); one failure does not abort the run
|
||||
|
||||
## Failed Metadata Retry
|
||||
|
||||
`capture_thought` now stores the thought even when metadata extraction times out or fails. Those thoughts are marked with `metadata_status: "pending"` and retried in the background. Use `retry_failed_metadata` to sweep any thoughts still marked `pending` or `failed`.
|
||||
|
||||
```json
|
||||
{
|
||||
"project": "optional-project-name",
|
||||
"limit": 100,
|
||||
"include_archived": false,
|
||||
"older_than_days": 1,
|
||||
"dry_run": false
|
||||
}
|
||||
```
|
||||
|
||||
- `dry_run: true` scans only and does not call metadata extraction or write updates
|
||||
- successful retries mark the thought metadata as `complete` and clear the last error
|
||||
- failed retries update the retry markers so the daily sweep can pick them up again later
|
||||
|
||||
## File Storage
|
||||
|
||||
Use `save_file` to persist binary files as base64. Files can optionally be linked to a memory by passing `thought_id`, which also adds an attachment reference to that thought's metadata.
|
||||
@@ -148,6 +167,27 @@ List files for a thought or project with:
|
||||
}
|
||||
```
|
||||
|
||||
AMCS also supports direct authenticated HTTP uploads to `/files` for clients that want to stream file bodies instead of base64-encoding them into an MCP tool call.
|
||||
|
||||
Multipart upload:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/files \
|
||||
-H "x-brain-key: <key>" \
|
||||
-F "file=@./diagram.png" \
|
||||
-F "project=amcs" \
|
||||
-F "kind=image"
|
||||
```
|
||||
|
||||
Raw body upload:
|
||||
|
||||
```bash
|
||||
curl -X POST "http://localhost:8080/files?project=amcs&name=meeting-notes.pdf" \
|
||||
-H "x-brain-key: <key>" \
|
||||
-H "Content-Type: application/pdf" \
|
||||
--data-binary @./meeting-notes.pdf
|
||||
```
|
||||
|
||||
**Automatic backfill** (optional, config-gated):
|
||||
|
||||
```yaml
|
||||
@@ -160,6 +200,15 @@ backfill:
|
||||
include_archived: false
|
||||
```
|
||||
|
||||
```yaml
|
||||
metadata_retry:
|
||||
enabled: true
|
||||
run_on_startup: true # retry failed metadata once on server start
|
||||
interval: "24h" # retry pending/failed metadata daily
|
||||
max_per_run: 100
|
||||
include_archived: false
|
||||
```
|
||||
|
||||
**Search fallback**: when no embeddings exist for the active model in scope, `search_thoughts`, `recall_context`, `get_project_context`, `summarize_thoughts`, and `related_thoughts` automatically fall back to Postgres full-text search so results are never silently empty.
|
||||
|
||||
## Client Setup
|
||||
|
||||
Reference in New Issue
Block a user