Re-fetch, BeforeScan and AfterUpdate/AfterCreate now run on a short
transaction that fires OnTxBegin. Existence selects inside the first
transaction use tx instead of the pool.
* Throttle async session-activity writes to once per token per minute
* Add singleflight to session lookups, keystore validation and
column/row security loads to stop cold-cache stampedes
* Preload security rules in BeforeHandle (restheadspec, resolvespec) so
they no longer need a second connection while the read tx is open
* Add pkg/dbtrace: opt-in per-request DB call counting and pool logging
(db_trace.* config, RESOLVESPEC_DB_TRACE_* env), wired into testserver
* Add tests for load dedup, activity throttle and dbtrace
* Introduced ApplicationName field to identify clients in DSN
* Set default ApplicationName to "ResolveSpec"
* Updated tests for ApplicationName handling in DSN
Update handlers skipped empty strings and nulls, so a client could not
blank or null out a column. Every key present in the payload now
overwrites the stored value, including "" and null.
- add common.MergeUpdateValues and use it in resolvespec and restheadspec
- add Handler.SetDisallowNulls to skip null values (""still overwrites)
- websocketspec and mqttspec now write only the keys present in the
payload via SetMap instead of updating the whole zeroed model, which
clobbered absent fields
Counts every request placed in a wait queue regardless of outcome, so the
total ever queued no longer has to be summed from queued, timeout and
canceled.
ClientQueue limits concurrent requests per client (X-Client-Id, then
Authorization, session, IP) and queues the rest first-in-first-out, with
bounded depth, max wait and idle eviction. Chain composes it with auth for
the Setup*Routes middleware slot. Exports burst, wait and depth metrics.
The test server uses it with a limit of 10.
Add MaxOpenConnections to provider and connection stats, exposed as the
max state of dbmanager_connection_pool_size. Count physical dials through a
counting driver connector and export dbmanager_connections_opened_total;
existing_db pools derive an approximate count from sql.DBStats.
Verify bcrypt passwords in Direct mode and the shipped procedures, hash on
register/reset, ignore client-supplied roles and level at registration and
drop the password from the jwt_login payload. Legacy cleartext upgrade is
opt-in. Row security templates now bind the user as a parameter, validate
identifiers, attach via common.SelectQuery and fail the request if the
filter cannot be attached. ApplyColumnSecurity and GetRowSecurityTemplate
convert panics to errors and the hooks fail closed. Update audit status.
Default to TLS export with Insecure/TLSConfig/Headers options, parent-based
ratio sampling (default 0.1), and no query string or Host in span attributes.
Name spans by route template, record status and panics (re-raised), guard the
tracer with atomic.Pointer, reject double init, add init timeout and attribute
length limit, and move to semconv v1.26.0. Add tracing.insecure and
tracing.sample_rate config keys and tests.
Load column/row security without holding locks across provider calls,
honour pOverwrite with a 30s TTL and pruning, cap tokens per
Authorization header, detach session activity updates with a timeout,
stop OAuth2 cleanup goroutines via Close, move nil-map checks inside
locks, and drop O(n^2) string building. Update audit status.
Replace try-lock/sleep scheme with blocking locks, add ErrModelNotFound/
ErrModelExists/ErrInvalidModel sentinels, make RegisterModelWithRules
atomic, snapshot in IterateModels, guard defaultRegistry access, cap the
pointer-unwrap depth, and recover panics in callbacks and reflection.
Security hooks now allow-by-default only on ErrModelNotFound. Add tests.
Redact and rate-limit error tracker fan-out, cap panic stack capture,
sanitise stdlib fallback output, strip contexts in Info/Debug, sync the
replaced logger, add Sync, UpdateLoggerE and CatchPanicRethrow, cache
the PID, and add tests.
Make cache-write failures non-fatal in GetOrSet/Remember, replace the
unsynchronised default cache with an atomic pointer, fix tag-index leaks
and write-lock-on-read in the memory provider, add a janitor, closed
state and byte copies, hash/namespace memcache keys with CAS tag index,
gate Clear behind AllowFlush, return ErrNotFound without the key, and
allowlist Redis stats. Mark audit status.
Add a test-race target and run test-unit over ./pkg/...
Production races:
- logger: guard Logger/errorTracker with an RWMutex
- security: copy UserContext for the async session-activity goroutine
and track it with a WaitGroup
Test fixes:
- eventbroker, websocketspec: use atomics for state shared with workers
- security: wait for async activity updates before touching sqlmock
- mqttspec: build full HookContext, set SubscriptionID, pin the
in-memory SQLite to one connection
Update the cross-cutting audit (X1) with status and findings.
Guard viper with an RWMutex, make the singleton race-free and stop
NewManager replacing the global (add SetConfigManager), write saved
configs 0600, search CWD last, add ConfigFileUsed and Config.Validate,
nil-safe PathsConfig.Set, confine PathsConfig.Join, bound GetIPs DNS
lookup, and drop dead Unmarshal in SetConfig. Mark audit status.
Implements the fixes from audit/pkg/dbmanager.audit.md.
- Stop closing the shared *sql.DB to recover from errors. Adapter
factories and the health checker no longer call Reconnect; Reconnect is
atomic and operator-only.
- Postgres uses a custom driver.Connector: Reconnect retires pooled
connections by generation without closing the pool, so held Bun/GORM
handles keep working. Verified against a live server restart.
- Add TCP keepalive, TCP_USER_TIMEOUT, a bounded reuse ping and
statement_timeout as a runtime parameter; drop the 2 min timeout floor.
- Health check pings without holding the connection lock.
- Listener: single goroutine pair, bounded Close without UNLISTEN, and
serialised use of the pgx connection (fixes conn busy and a close race).
- Fix Connect/Close/Connect/Close panic, idempotent Connect, dial outside
the manager lock, clean up on partial failure.
- SQLite: pin :memory: to one connection, pragmas via DSN.
- Escape credentials in Postgres/MSSQL/Mongo DSNs; sslmode defaults to
prefer. Wire retry settings, publish metrics, fix logger calls.
- NewConnectionFromDB: Close is a no-op with a warning (caller owns the
pool); Reconnect only pings.
- Document correct usage in the README; mark the audit with what was done.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Requesting a JSON sub-field (e.g. jsonvalue->'product'->>'cost') that has
no matching bun scanonly field on the model made the whole read fail with
"bun: ModelX does not have column Y", since bun scans SELECT results
straight into the typed model struct.
Add reflection.HasColumn to check whether the model can actually receive
a given column (including scanonly fields, walking embedded structs), and
gate the JSON select-column expression on it in ApplySelectColumns
(shared by websocketspec/mqttspec) and the resolvespec/restheadspec
handlers. When there's no scan target, drop just that column with a
warning instead of erroring the whole request.
An Exclude entry only ever matches requests that already fall under
its rule's URLPrefix, so one written without that prefix (e.g.
"/health" on a rule for "/api") silently never triggered. Validate
that each Exclude entry itself starts with the rule's URLPrefix,
failing NewService instead of accepting a no-op config.
Rule.Exclude lists path prefixes that should never be proxied by that
rule, even though they fall under its URLPrefix. A request matching an
Exclude prefix is treated as a non-match for that rule: matching
continues against other configured rules, falling back to the
caller-supplied handler if none apply. Lets a catch-all "/" rule proxy
everything except carved-out paths like "/health".
Adds pkg/server/quickproxy: longest-prefix rule matching over
net/http/httputil.ReverseProxy, falling back to a caller-supplied
handler when the upstream is unreachable or returns 404. Any other
upstream response streams through unchanged. All HTTP methods are
proxied, with a configurable global dial/response-header timeout
(quickproxy.WithTimeout, default 10s).
GoCore-side wiring (config field, webserver2/proxy.go, server.go
route ordering) is tracked separately in that repo.
Same class of bug as the restheadspec/resolvespec fix: these handlers
unconditionally rendered CAST(col AS TEXT) LIKE/ILIKE for every column,
which flips a citext column to case-sensitive matching and defeats a
citext index. Thread the model through to buildFilterCondition/applyFilters
so reflection.IsCitextColumn can skip the cast for citext columns.
resolvemcp's eq/neq/gt/lt paths never cast (they never had the
restheadspec-style reflect.Kind cast heuristic), so this only touches
LIKE/ILIKE. funcspec is unaffected: it has no Go struct model to check
against (colname/value come straight from SQL function parameters).
reflect.Type.Kind() on spectypes.SqlNull[T] wrappers (SqlInt16/32/64,
SqlFloat64, SqlBool, SqlString, and embedders like SqlTimeStamp) always
reports reflect.Struct, never the wrapped T. ValidateAndAdjustFilterForColumnType
treated those as "complex" columns and forced CAST(col AS TEXT) on eq/gt/lt
filters, e.g. CAST(atdetail.rid_parent AS TEXT) = '90446096', which can't
use the index on rid_parent.
Add spectypes.UnwrapKind to see through SqlNull wrappers to the underlying
Kind, and use it in GetColumnTypeFromModel so numeric/string SqlNull columns
are recognized correctly and compared natively.
Also stop unconditionally casting to TEXT for LIKE/ILIKE and add
reflection.IsCitextColumn: citext columns are already case-insensitive, so
casting them to TEXT flips to case-sensitive matching and defeats a citext
index.
SqlTime previously treated 00:00:00 as equivalent to null, which incorrectly
dropped legitimate midnight time values on marshal/unmarshal.
Also adds test coverage for SqlBool, generic SqlNull conversion helpers
(Int64/Float64/Bool/Time/UUID), FromString edge cases, NewSql, and the
SqlDate/SqlTimeStamp zero-value/sentinel handling.
Support column references that traverse into JSON/JSONB values —
data->>'x', data#>>'{a,b}', data->'a'->>'b', and the dotted data.a.b
shorthand — in SELECT column lists, WHERE filters and ORDER BY, across
the restheadspec, resolvespec, websocketspec and mqttspec handlers.
- pkg/common/json_column.go: canonical ParseColumnRef + ColumnRef.SQL()
builder. JSON path segments are bound as a single ?::text[] parameter,
never interpolated; cast targets are whitelisted via NormalizeCastTarget.
- pkg/common/json_condition.go: shared entry points mirroring
BuildSpatialCondition - ResolveJSONColumnExpr (select/sort),
BuildJSONFilterCondition (where, full operator set; infers ::numeric for
ordered comparisons on numeric values when no explicit cast is given),
and the ApplySelectColumns helper.
- pkg/reflection.IsJSONColumn / pkg/spectypes.IsJSONType: disambiguate the
dotted shorthand (data.city is JSON only when the base is a JSON column).
- pkg/common/validation.go: ColumnValidator accepts JSON tokens.
- Handlers: thread model through the filter call chains and wire the
select/sort paths.
funcspec (raw-SQL string builder, no param binding or model) and the
FetchRowNumber raw-SQL builders are left as follow-ups, as is OpenAPI
reporting of JSON sub-field columns.
MarshalJSON and Value now emit time.RFC3339 instead of a naive
YYYY-MM-DDTHH:MM:SS layout, so timezone offset is preserved in JSON
and DB writes. UnmarshalJSON zero-check made offset-agnostic.
warkanum
Hein
