Bun's ExcludeColumn errors with "can't find column" for scanonly fields
because they are not in the table's writable fields. Filter the exclude
list to writable bun fields so models with scanonly buffers can insert
and update again. Add tests for the adapter and reflection.
Read-merge-update wrote every model column back, so GENERATED ALWAYS
columns failed with SQLSTATE 428C9. Add a bun 'generated' tag option,
reflection.NonWritableColumns/RemoveNonWritableColumns, and apply them
in resolvespec, restheadspec, websocketspec, mqttspec, resolvemcp and
the nested CUD processor. Add ExcludeColumn to InsertQuery/UpdateQuery
for model-based writes.
- PgSQLSelectQuery implements common.WhereGrouper so x-custom-sql-or
is grouped with the client's own conditions on the pgx adapter too.
- Add a container test (opt-in via RESOLVESPEC_TEST_CONTAINERS=1) that
starts PostgreSQL with podman or docker and checks the hardening
against a real database: parenthesis escape, pg_sleep, catalog
subquery, stacked statements, x-custom-sql-or grouping and the
legacy behaviour when hardening is switched off.
Add a `hardening` config section (RESOLVESPEC_HARDENING_*) so each
fix can be switched off to restore the previous behaviour:
- cors_strict_origins: only reflect origins listed in
cors.allowed_origins / server URLs, with credentials; `*` never
sends credentials; fix shared-slice append of expose headers.
- sort_strict: join aliases must match `alias.identifier` (empty alias
no longer matches everything); sort expressions reject dangerous
functions/catalogs; cql* columns must be identifier-safe.
- sql_strict: client raw-SQL fragments must have balanced parens and
quotes, no comments/`;`/`$$`, DML keywords, dangerous functions or
system catalogs; a rejected fragment now fails closed ("(1=0)")
instead of dropping the filter. Subqueries stay allowed unless
sql_block_subqueries is set.
- x-custom-sql-or is grouped together with the client's own
conditions (new optional WhereGrouper, implemented for bun and gorm)
so it can no longer OR past server-side filters.
Tools: list_tables, describe_table, select_table, insert_into_table, update_table,
delete_from_table, list_functions, call_function. Visibility follows the model rules.
Filter-based update/delete require filters (never dropped silently), cap the matched rows
(MaxWriteRows), support dry_run, and need a single-use confirm token bound to caller, table,
filters, data and the matched rows. RegisterFunction adds Go-callback and SQL-procedure
functions run in a transaction with BeforeCall/AfterCall hooks. Per-model tools and
resources are removed.
fix(pgsql): UPDATE with SET and a multi-placeholder WHERE renumbered the WHERE parameters
wrongly ($1, $2 became $3, $2); shift them in one pass.
pkg/security no longer contains SQL. Every provider calls a store interface
from lookup, implemented by a procedure backend (Postgres stored procedures,
the default there) and a direct backend (dialect-driven SQL for postgres,
sqlite, mysql and mssql with configurable table and column names).
- add sectypes, lookup, lookup/{dialect,procedure,direct,backends,ddl,conformance}
- split totp and providers sub packages out of the core package
- replace SQLNames/TableNames/QueryMode with lookup.Config (see breaking_changes.md)
- direct backend now covers column/row security and API-key login
- move txsettings SQL to lookup.ApplyTxSettings; remove password.go
- move schema scripts under lookup/, add reference DDL per dialect
- add a shared conformance suite; run it on sqlite, and on Postgres in a
podman/docker container (RESOLVESPEC_TEST_CONTAINERS=1)
- fix procedure schema bugs found on real Postgres: duplicate p_data
parameter, JSON null arrays, expires_at timezone casts, passkey list
GROUP BY, missing resolvespec_passkey_login; accept zone-less timestamps
Hooks fire from Connection.Close() once per connection. ConnectionManager
Shutdown now closes connections outside its lock so hooks can call back
into the manager. Update the single-transaction audit plan status.
* Throttle async session-activity writes to once per token per minute
* Add singleflight to session lookups, keystore validation and
column/row security loads to stop cold-cache stampedes
* Preload security rules in BeforeHandle (restheadspec, resolvespec) so
they no longer need a second connection while the read tx is open
* Add pkg/dbtrace: opt-in per-request DB call counting and pool logging
(db_trace.* config, RESOLVESPEC_DB_TRACE_* env), wired into testserver
* Add tests for load dedup, activity throttle and dbtrace
Update handlers skipped empty strings and nulls, so a client could not
blank or null out a column. Every key present in the payload now
overwrites the stored value, including "" and null.
- add common.MergeUpdateValues and use it in resolvespec and restheadspec
- add Handler.SetDisallowNulls to skip null values (""still overwrites)
- websocketspec and mqttspec now write only the keys present in the
payload via SetMap instead of updating the whole zeroed model, which
clobbered absent fields
Requesting a JSON sub-field (e.g. jsonvalue->'product'->>'cost') that has
no matching bun scanonly field on the model made the whole read fail with
"bun: ModelX does not have column Y", since bun scans SELECT results
straight into the typed model struct.
Add reflection.HasColumn to check whether the model can actually receive
a given column (including scanonly fields, walking embedded structs), and
gate the JSON select-column expression on it in ApplySelectColumns
(shared by websocketspec/mqttspec) and the resolvespec/restheadspec
handlers. When there's no scan target, drop just that column with a
warning instead of erroring the whole request.
Support column references that traverse into JSON/JSONB values —
data->>'x', data#>>'{a,b}', data->'a'->>'b', and the dotted data.a.b
shorthand — in SELECT column lists, WHERE filters and ORDER BY, across
the restheadspec, resolvespec, websocketspec and mqttspec handlers.
- pkg/common/json_column.go: canonical ParseColumnRef + ColumnRef.SQL()
builder. JSON path segments are bound as a single ?::text[] parameter,
never interpolated; cast targets are whitelisted via NormalizeCastTarget.
- pkg/common/json_condition.go: shared entry points mirroring
BuildSpatialCondition - ResolveJSONColumnExpr (select/sort),
BuildJSONFilterCondition (where, full operator set; infers ::numeric for
ordered comparisons on numeric values when no explicit cast is given),
and the ApplySelectColumns helper.
- pkg/reflection.IsJSONColumn / pkg/spectypes.IsJSONType: disambiguate the
dotted shorthand (data.city is JSON only when the base is a JSON column).
- pkg/common/validation.go: ColumnValidator accepts JSON tokens.
- Handlers: thread model through the filter call chains and wire the
select/sort paths.
funcspec (raw-SQL string builder, no param binding or model) and the
FetchRowNumber raw-SQL builders are left as follow-ups, as is OpenAPI
reporting of JSON sub-field columns.
* Implemented 'contains' operator using BuildArrayOverlapCondition for real array containment.
* Updated documentation to clarify differences between text-cast ILIKE and array overlap.
* Reflect request origin for Access-Control-Allow-Origin
* Set Vary header for caching based on origin
* Allow specific headers from preflight requests
* Enable credentials only for specific origins
* Add support for BETWEEN-aware AND detection
* Ensure AND inside single-quoted strings does not cause splits
* Update tests to cover new BETWEEN and quote scenarios
* added checks for valid _request values in single and multiple relations
* introduced isValidNestedRequest function to encapsulate validation logic
fix(crud): expand operation handling for nested CUD
* added "add" to insert operations and "modify" to update operations
* included "remove" in delete operations
* adjust handling of "all" filter to consider filtered columns
fix(function_api): improve variable substitution in SQL queries
* add safeSubstituteVar for context-aware value sanitization
Add reflection.IsEmptyValue to detect nil, empty string, and zero numbers.
Use it in recursive CUD processing to skip update/delete when the primary
key is absent, logging a warning instead of proceeding with an invalid operation.
Frontend clients are sensitive to 204 No Content responses; always return 200
with an empty array/object and rely on X-No-Data-Found header to signal absence
of records.
Also treat "change" as an alias for "update" in recursive CUD processing.
* Add support for self-referential models in GetForeignKeyColumn
* Update comments for clarity on foreign key resolution strategies
* Introduce selfRefItem struct for testing self-referential behavior
* Change return type to []string for composite keys
* Adjust related logic in injectForeignKeys method
* Update tests to validate new behavior for composite foreign keys
Hein
warkanum