mirror of
https://github.com/bitechdev/ResolveSpec.git
synced 2026-09-10 18:32:35 +00:00
Compare commits
12
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e957753ce4 | ||
|
|
206edd4bfd | ||
|
|
f841d58c59 | ||
|
|
cbac47e052 | ||
|
|
3d4f6faa8e | ||
|
|
f259df1258 | ||
|
|
798bb47e71 | ||
|
|
105a5e1b87 | ||
|
|
a68cf83be6 | ||
|
|
dab4940ace | ||
|
|
c7178e0a2b | ||
|
|
0261f121e8 |
@@ -1,5 +1,7 @@
|
|||||||
.PHONY: test test-unit test-integration docker-up docker-down clean
|
.PHONY: test test-unit test-integration docker-up docker-down clean
|
||||||
|
|
||||||
|
GOLANGCI_LINT := $(shell go env GOPATH)/bin/golangci-lint
|
||||||
|
|
||||||
# Run all unit tests
|
# Run all unit tests
|
||||||
test-unit:
|
test-unit:
|
||||||
@echo "Running unit tests..."
|
@echo "Running unit tests..."
|
||||||
@@ -49,7 +51,9 @@ release-version: ## Create and push a release with specific version (use: make r
|
|||||||
|
|
||||||
lint: ## Run linter
|
lint: ## Run linter
|
||||||
@echo "Running linter..."
|
@echo "Running linter..."
|
||||||
@if command -v golangci-lint > /dev/null; then \
|
@if [ -x "$(GOLANGCI_LINT)" ]; then \
|
||||||
|
"$(GOLANGCI_LINT)" run --config=.golangci.json; \
|
||||||
|
elif command -v golangci-lint > /dev/null; then \
|
||||||
golangci-lint run --config=.golangci.json; \
|
golangci-lint run --config=.golangci.json; \
|
||||||
else \
|
else \
|
||||||
echo "golangci-lint not installed. Install with: go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest"; \
|
echo "golangci-lint not installed. Install with: go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest"; \
|
||||||
@@ -58,7 +62,9 @@ lint: ## Run linter
|
|||||||
|
|
||||||
lintfix: ## Run linter
|
lintfix: ## Run linter
|
||||||
@echo "Running linter..."
|
@echo "Running linter..."
|
||||||
@if command -v golangci-lint > /dev/null; then \
|
@if [ -x "$(GOLANGCI_LINT)" ]; then \
|
||||||
|
"$(GOLANGCI_LINT)" run --config=.golangci.json --fix; \
|
||||||
|
elif command -v golangci-lint > /dev/null; then \
|
||||||
golangci-lint run --config=.golangci.json --fix; \
|
golangci-lint run --config=.golangci.json --fix; \
|
||||||
else \
|
else \
|
||||||
echo "golangci-lint not installed. Install with: go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest"; \
|
echo "golangci-lint not installed. Install with: go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest"; \
|
||||||
|
|||||||
@@ -135,6 +135,73 @@ For complete documentation including setup, headers, lifecycle hooks, cursor pag
|
|||||||
|
|
||||||
For detailed examples of reading data, cursor pagination, recursive CRUD operations, filtering, sorting, and more, see [pkg/resolvespec/README.md](pkg/resolvespec/README.md).
|
For detailed examples of reading data, cursor pagination, recursive CRUD operations, filtering, sorting, and more, see [pkg/resolvespec/README.md](pkg/resolvespec/README.md).
|
||||||
|
|
||||||
|
## PostGIS & Vector (PostgreSQL only)
|
||||||
|
|
||||||
|
First-class support for PostGIS geometry/geography and pgvector columns in `resolvespec` + `restheadspec`. No extra dependencies. On non-Postgres databases the spatial/vector operators simply don't match.
|
||||||
|
|
||||||
|
### Column types (`pkg/spectypes`)
|
||||||
|
|
||||||
|
| Go type | SQL type | Wire / JSON |
|
||||||
|
|--------------------|--------------|--------------------------------------------------------|
|
||||||
|
| `SqlGeometry` | `geometry` | JSON in/out = **GeoJSON**; also accepts EWKT / hex-EWKB |
|
||||||
|
| `SqlGeography` | `geography` | same as `SqlGeometry` |
|
||||||
|
| `SqlVector` | `vector` | `[]float32` ⇄ `[1,2,3]` |
|
||||||
|
| `SqlHalfVector` | `halfvec` | `[]float32` ⇄ `[1,2,3]` |
|
||||||
|
| `SqlSparseVector` | `sparsevec` | `{"dim":8,"indices":[1,4],"values":[0.5,0.2]}` |
|
||||||
|
| `SqlBitVector` | `bit`/`varbit` | bool array or `"1011"` string |
|
||||||
|
|
||||||
|
- Geometry `Value()` emits `SRID=<n>;<WKT>` (PostGIS implicit text→geometry cast; no wrapper function needed).
|
||||||
|
- Declare dimensioned types with a tag: `gorm:"type:vector(1536)"` — the tag wins over the canonical name in metadata/OpenAPI.
|
||||||
|
- Metadata endpoint and OpenAPI schema report `geometry`/`vector`/`halfvec`/`sparsevec`/`bit`.
|
||||||
|
|
||||||
|
### Spatial filter operators
|
||||||
|
|
||||||
|
`value` is a geometry (GeoJSON object, EWKT string, or hex-EWKB) unless noted.
|
||||||
|
|
||||||
|
| Operator | Value shape |
|
||||||
|
|----------|-------------|
|
||||||
|
| `st_intersects`, `st_contains`, `st_within`, `st_covers`, `st_coveredby`, `st_overlaps`, `st_touches`, `st_crosses`, `st_equals`, `st_disjoint` | geometry |
|
||||||
|
| `st_dwithin` | `{"geom": <geometry>, "distance": <meters>}` |
|
||||||
|
| `bbox` (alias `&&`) | geometry, or `{"bbox":[minx,miny,maxx,maxy],"srid":4326}` |
|
||||||
|
|
||||||
|
### Vector similarity filter operators
|
||||||
|
|
||||||
|
| Operator | pgvector op | Value shape |
|
||||||
|
|----------|-------------|-------------|
|
||||||
|
| `l2_within` / `euclidean_within` | `<->` | `{"vector":[...], "distance": <n>}` |
|
||||||
|
| `cosine_within` | `<=>` | same (also `"lt"`/`"lte"`/`"gt"`/`"gte"` instead of `"distance"`) |
|
||||||
|
| `ip_within` / `inner_within` | `<#>` | same |
|
||||||
|
|
||||||
|
### KNN search (ordering + distance column)
|
||||||
|
|
||||||
|
**resolvespec** — `options.vector_search`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{ "options": { "vector_search": {
|
||||||
|
"column": "embedding",
|
||||||
|
"vector": [0.1, 0.2, 0.3],
|
||||||
|
"metric": "cosine",
|
||||||
|
"as": "_distance",
|
||||||
|
"direction": "asc"
|
||||||
|
}}}
|
||||||
|
```
|
||||||
|
|
||||||
|
Orders rows by distance; when `as` is set, returns the distance as an extra column (all model columns are auto-selected).
|
||||||
|
`metric`: `l2` (default) | `cosine` | `ip`.
|
||||||
|
|
||||||
|
**restheadspec** — headers:
|
||||||
|
|
||||||
|
```HTTP
|
||||||
|
X-Vector-Search-embedding: cosine
|
||||||
|
X-Vector-Search-Vector: [0.1,0.2,0.3]
|
||||||
|
X-Vector-Search-As: _distance
|
||||||
|
X-Vector-Search-Dir: asc
|
||||||
|
```
|
||||||
|
|
||||||
|
Spatial/vector filters via headers: `X-SpatialFilter-<col>` / `X-VectorFilter-<col>` with a JSON operator object, e.g.
|
||||||
|
`X-SpatialFilter-geom: {"op":"st_dwithin","geom":"SRID=4326;POINT(0 0)","distance":1000}`
|
||||||
|
(optional `"logic":"or"`).
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
```Shell
|
```Shell
|
||||||
|
|||||||
@@ -33,13 +33,13 @@ require (
|
|||||||
github.com/uptrace/bun/driver/sqliteshim v1.2.16
|
github.com/uptrace/bun/driver/sqliteshim v1.2.16
|
||||||
github.com/uptrace/bunrouter v1.0.23
|
github.com/uptrace/bunrouter v1.0.23
|
||||||
go.mongodb.org/mongo-driver v1.17.9
|
go.mongodb.org/mongo-driver v1.17.9
|
||||||
go.opentelemetry.io/otel v1.43.0
|
go.opentelemetry.io/otel v1.44.0
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.43.0
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.43.0
|
||||||
go.opentelemetry.io/otel/sdk v1.43.0
|
go.opentelemetry.io/otel/sdk v1.44.0
|
||||||
go.opentelemetry.io/otel/trace v1.43.0
|
go.opentelemetry.io/otel/trace v1.44.0
|
||||||
go.uber.org/zap v1.28.0
|
go.uber.org/zap v1.28.0
|
||||||
golang.org/x/crypto v0.51.0
|
golang.org/x/crypto v0.55.0
|
||||||
golang.org/x/oauth2 v0.36.0
|
golang.org/x/oauth2 v0.36.0
|
||||||
golang.org/x/time v0.15.0
|
golang.org/x/time v0.15.0
|
||||||
gorm.io/driver/postgres v1.6.0
|
gorm.io/driver/postgres v1.6.0
|
||||||
@@ -137,22 +137,21 @@ require (
|
|||||||
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
|
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
|
||||||
github.com/yusufpapurcu/wmi v1.2.4 // indirect
|
github.com/yusufpapurcu/wmi v1.2.4 // indirect
|
||||||
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
|
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
|
||||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 // indirect
|
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
|
||||||
go.opentelemetry.io/otel/metric v1.43.0 // indirect
|
go.opentelemetry.io/otel/metric v1.44.0 // indirect
|
||||||
go.opentelemetry.io/proto/otlp v1.10.0 // indirect
|
go.opentelemetry.io/proto/otlp v1.10.0 // indirect
|
||||||
go.uber.org/atomic v1.11.0 // indirect
|
go.uber.org/atomic v1.11.0 // indirect
|
||||||
go.uber.org/multierr v1.11.0 // indirect
|
go.uber.org/multierr v1.11.0 // indirect
|
||||||
go.yaml.in/yaml/v2 v2.4.4 // indirect
|
go.yaml.in/yaml/v2 v2.4.4 // indirect
|
||||||
go.yaml.in/yaml/v3 v3.0.4 // indirect
|
go.yaml.in/yaml/v3 v3.0.4 // indirect
|
||||||
golang.org/x/mod v0.36.0 // indirect
|
golang.org/x/mod v0.38.0 // indirect
|
||||||
golang.org/x/net v0.54.0 // indirect
|
golang.org/x/net v0.58.0 // indirect
|
||||||
golang.org/x/sync v0.20.0 // indirect
|
golang.org/x/sync v0.22.0 // indirect
|
||||||
golang.org/x/sys v0.44.0 // indirect
|
golang.org/x/sys v0.47.0 // indirect
|
||||||
golang.org/x/text v0.37.0 // indirect
|
golang.org/x/text v0.41.0 // indirect
|
||||||
golang.org/x/tools v0.45.0 // indirect
|
google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa // indirect
|
||||||
google.golang.org/genproto/googleapis/api v0.0.0-20260519071638-aa98bba5eb94 // indirect
|
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa // indirect
|
||||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260519071638-aa98bba5eb94 // indirect
|
google.golang.org/grpc v1.83.2 // indirect
|
||||||
google.golang.org/grpc v1.81.1 // indirect
|
|
||||||
google.golang.org/protobuf v1.36.11 // indirect
|
google.golang.org/protobuf v1.36.11 // indirect
|
||||||
gopkg.in/yaml.v3 v3.0.1 // indirect
|
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||||
modernc.org/libc v1.72.3 // indirect
|
modernc.org/libc v1.72.3 // indirect
|
||||||
|
|||||||
@@ -348,24 +348,24 @@ go.mongodb.org/mongo-driver v1.17.9 h1:IexDdCuuNJ3BHrELgBlyaH9p60JXAvdzWR128q+U5
|
|||||||
go.mongodb.org/mongo-driver v1.17.9/go.mod h1:LlOhpH5NUEfhxcAwG0UEkMqwYcc4JU18gtCdGudk/tQ=
|
go.mongodb.org/mongo-driver v1.17.9/go.mod h1:LlOhpH5NUEfhxcAwG0UEkMqwYcc4JU18gtCdGudk/tQ=
|
||||||
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
|
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
|
||||||
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
|
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
|
||||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 h1:jq9TW8u3so/bN+JPT166wjOI6/vQPF6Xe7nMNIltagk=
|
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus=
|
||||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0/go.mod h1:p8pYQP+m5XfbZm9fxtSKAbM6oIllS7s2AfxrChvc7iw=
|
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q=
|
||||||
go.opentelemetry.io/otel v1.43.0 h1:mYIM03dnh5zfN7HautFE4ieIig9amkNANT+xcVxAj9I=
|
go.opentelemetry.io/otel v1.44.0 h1:JjwHmHpA4iZ3wBxluu2fbbE7j4kqlE8jXyAyPXH7HqU=
|
||||||
go.opentelemetry.io/otel v1.43.0/go.mod h1:JuG+u74mvjvcm8vj8pI5XiHy1zDeoCS2LB1spIq7Ay0=
|
go.opentelemetry.io/otel v1.44.0/go.mod h1:BMgjTHL9WPRlRjL2oZCBTL4whCGtXch2H4BhOPIAyYc=
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 h1:88Y4s2C8oTui1LGM6bTWkw0ICGcOLCAI5l6zsD1j20k=
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 h1:88Y4s2C8oTui1LGM6bTWkw0ICGcOLCAI5l6zsD1j20k=
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0/go.mod h1:Vl1/iaggsuRlrHf/hfPJPvVag77kKyvrLeD10kpMl+A=
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0/go.mod h1:Vl1/iaggsuRlrHf/hfPJPvVag77kKyvrLeD10kpMl+A=
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.43.0 h1:RAE+JPfvEmvy+0LzyUA25/SGawPwIUbZ6u0Wug54sLc=
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.43.0 h1:RAE+JPfvEmvy+0LzyUA25/SGawPwIUbZ6u0Wug54sLc=
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.43.0/go.mod h1:AGmbycVGEsRx9mXMZ75CsOyhSP6MFIcj/6dnG+vhVjk=
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.43.0/go.mod h1:AGmbycVGEsRx9mXMZ75CsOyhSP6MFIcj/6dnG+vhVjk=
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.19.0 h1:IeMeyr1aBvBiPVYihXIaeIZba6b8E1bYp7lbdxK8CQg=
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.19.0 h1:IeMeyr1aBvBiPVYihXIaeIZba6b8E1bYp7lbdxK8CQg=
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.19.0/go.mod h1:oVdCUtjq9MK9BlS7TtucsQwUcXcymNiEDjgDD2jMtZU=
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.19.0/go.mod h1:oVdCUtjq9MK9BlS7TtucsQwUcXcymNiEDjgDD2jMtZU=
|
||||||
go.opentelemetry.io/otel/metric v1.43.0 h1:d7638QeInOnuwOONPp4JAOGfbCEpYb+K6DVWvdxGzgM=
|
go.opentelemetry.io/otel/metric v1.44.0 h1:1w0gILTcHdr3YI+ixLyjemwrVnsMURbTZFrSYCdDdmc=
|
||||||
go.opentelemetry.io/otel/metric v1.43.0/go.mod h1:RDnPtIxvqlgO8GRW18W6Z/4P462ldprJtfxHxyKd2PY=
|
go.opentelemetry.io/otel/metric v1.44.0/go.mod h1:8O7hanEPBNgEMmybD3s2VBKcgWOCsA6tzHBPODAiquo=
|
||||||
go.opentelemetry.io/otel/sdk v1.43.0 h1:pi5mE86i5rTeLXqoF/hhiBtUNcrAGHLKQdhg4h4V9Dg=
|
go.opentelemetry.io/otel/sdk v1.44.0 h1:nHYwb9lK+fJPU/dnT6s7W7Z8itMWyqrnVfbheVYrZ58=
|
||||||
go.opentelemetry.io/otel/sdk v1.43.0/go.mod h1:P+IkVU3iWukmiit/Yf9AWvpyRDlUeBaRg6Y+C58QHzg=
|
go.opentelemetry.io/otel/sdk v1.44.0/go.mod h1:Osuydd3Se74nqjAKxid74N5eC+jfEqfTegHRnq58oK0=
|
||||||
go.opentelemetry.io/otel/sdk/metric v1.43.0 h1:S88dyqXjJkuBNLeMcVPRFXpRw2fuwdvfCGLEo89fDkw=
|
go.opentelemetry.io/otel/sdk/metric v1.44.0 h1:3LlKgI+VjbVsjNRFZJZAJ30WjXC5VkNRks6si09iEfI=
|
||||||
go.opentelemetry.io/otel/sdk/metric v1.43.0/go.mod h1:C/RJtwSEJ5hzTiUz5pXF1kILHStzb9zFlIEe85bhj6A=
|
go.opentelemetry.io/otel/sdk/metric v1.44.0/go.mod h1:5B5pMARnXxKhltooO4xUuCBorl65a4EpnTalObqOigA=
|
||||||
go.opentelemetry.io/otel/trace v1.43.0 h1:BkNrHpup+4k4w+ZZ86CZoHHEkohws8AY+WTX09nk+3A=
|
go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/6gtIk=
|
||||||
go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLhe+8WVFxiFff0=
|
go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE=
|
||||||
go.opentelemetry.io/proto/otlp v1.10.0 h1:IQRWgT5srOCYfiWnpqUYz9CVmbO8bFmKcwYxpuCSL2g=
|
go.opentelemetry.io/proto/otlp v1.10.0 h1:IQRWgT5srOCYfiWnpqUYz9CVmbO8bFmKcwYxpuCSL2g=
|
||||||
go.opentelemetry.io/proto/otlp v1.10.0/go.mod h1:/CV4QoCR/S9yaPj8utp3lvQPoqMtxXdzn7ozvvozVqk=
|
go.opentelemetry.io/proto/otlp v1.10.0/go.mod h1:/CV4QoCR/S9yaPj8utp3lvQPoqMtxXdzn7ozvvozVqk=
|
||||||
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
|
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
|
||||||
@@ -392,16 +392,16 @@ golang.org/x/crypto v0.21.0/go.mod h1:0BP7YvVV9gBbVKyeTG0Gyn+gZm94bibOW5BjDEYAOM
|
|||||||
golang.org/x/crypto v0.22.0/go.mod h1:vr6Su+7cTlO45qkww3VDJlzDn0ctJvRgYbC2NvXHt+M=
|
golang.org/x/crypto v0.22.0/go.mod h1:vr6Su+7cTlO45qkww3VDJlzDn0ctJvRgYbC2NvXHt+M=
|
||||||
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
|
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
|
||||||
golang.org/x/crypto v0.24.0/go.mod h1:Z1PMYSOR5nyMcyAVAIQSKCDwalqy85Aqn1x3Ws4L5DM=
|
golang.org/x/crypto v0.24.0/go.mod h1:Z1PMYSOR5nyMcyAVAIQSKCDwalqy85Aqn1x3Ws4L5DM=
|
||||||
golang.org/x/crypto v0.51.0 h1:IBPXwPfKxY7cWQZ38ZCIRPI50YLeevDLlLnyC5wRGTI=
|
golang.org/x/crypto v0.55.0 h1:+KWHjbgOaAQ66dh/YlkZKHlz9ZUlq61AFirAR9ntP8M=
|
||||||
golang.org/x/crypto v0.51.0/go.mod h1:8AdwkbraGNABw2kOX6YFPs3WM22XqI4EXEd8g+x7Oc8=
|
golang.org/x/crypto v0.55.0/go.mod h1:uq0V9dE/fzQuJtbnL+2EhWOE63vo164FY8xqEnV9xis=
|
||||||
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
|
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
|
||||||
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
|
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
|
||||||
golang.org/x/mod v0.9.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
|
golang.org/x/mod v0.9.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
|
||||||
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
|
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
|
||||||
golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
|
golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
|
||||||
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
|
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
|
||||||
golang.org/x/mod v0.36.0 h1:JJjpVx6myfUsUdAzZuOSTTmRE0PfZeNWzzvKrP7amb4=
|
golang.org/x/mod v0.38.0 h1:MECBjubtXD7yj4HrhIUcywNaGeNVUdfVnxmPajOk4yk=
|
||||||
golang.org/x/mod v0.36.0/go.mod h1:moc6ELqsWcOw5Ef3xVprK5ul/MvtVvkIXLziUOICjUQ=
|
golang.org/x/mod v0.38.0/go.mod h1:V6Xz0pq8TQ3dGqVQ1FVHuelZpAL0uNhSkk9ogYP3c40=
|
||||||
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||||
golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||||
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
|
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
|
||||||
@@ -419,8 +419,8 @@ golang.org/x/net v0.22.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
|
|||||||
golang.org/x/net v0.24.0/go.mod h1:2Q7sJY5mzlzWjKtYUEXSlBWCdyaioyXzRB2RtU8KVE8=
|
golang.org/x/net v0.24.0/go.mod h1:2Q7sJY5mzlzWjKtYUEXSlBWCdyaioyXzRB2RtU8KVE8=
|
||||||
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
|
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
|
||||||
golang.org/x/net v0.26.0/go.mod h1:5YKkiSynbBIh3p6iOc/vibscux0x38BZDkn8sCUPxHE=
|
golang.org/x/net v0.26.0/go.mod h1:5YKkiSynbBIh3p6iOc/vibscux0x38BZDkn8sCUPxHE=
|
||||||
golang.org/x/net v0.54.0 h1:2zJIZAxAHV/OHCDTCOHAYehQzLfSXuf/5SoL/Dv6w/w=
|
golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To=
|
||||||
golang.org/x/net v0.54.0/go.mod h1:Sj4oj8jK6XmHpBZU/zWHw3BV3abl4Kvi+Ut7cQcY+cQ=
|
golang.org/x/net v0.58.0/go.mod h1:YwCddHnFlT7eLQqVprV19OnhLGtc5xOKgE0RyqgfWAU=
|
||||||
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
|
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
|
||||||
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
|
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
|
||||||
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||||
@@ -430,8 +430,8 @@ golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
|
|||||||
golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
|
golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
|
||||||
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
|
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
|
||||||
golang.org/x/sync v0.9.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
|
golang.org/x/sync v0.9.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
|
||||||
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
|
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
|
||||||
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||||
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||||
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||||
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||||
@@ -455,8 +455,8 @@ golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
|||||||
golang.org/x/sys v0.19.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
golang.org/x/sys v0.19.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||||
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||||
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||||
golang.org/x/sys v0.44.0 h1:ildZl3J4uzeKP07r2F++Op7E9B29JRUy+a27EibtBTQ=
|
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
|
||||||
golang.org/x/sys v0.44.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||||
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
|
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
|
||||||
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
|
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
|
||||||
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
|
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
|
||||||
@@ -472,8 +472,8 @@ golang.org/x/term v0.18.0/go.mod h1:ILwASektA3OnRv7amZ1xhE/KTR+u50pbXfZ03+6Nx58=
|
|||||||
golang.org/x/term v0.19.0/go.mod h1:2CuTdWZ7KHSQwUzKva0cbMg6q2DMI3Mmxp+gKJbskEk=
|
golang.org/x/term v0.19.0/go.mod h1:2CuTdWZ7KHSQwUzKva0cbMg6q2DMI3Mmxp+gKJbskEk=
|
||||||
golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
|
golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
|
||||||
golang.org/x/term v0.21.0/go.mod h1:ooXLefLobQVslOqselCNF4SxFAaoS6KujMbsGzSDmX0=
|
golang.org/x/term v0.21.0/go.mod h1:ooXLefLobQVslOqselCNF4SxFAaoS6KujMbsGzSDmX0=
|
||||||
golang.org/x/term v0.43.0 h1:S4RLU2sB31O/NCl+zFN9Aru9A/Cq2aqKpTZJ6B+DwT4=
|
golang.org/x/term v0.45.0 h1:NwWyBmoJCbfTHpxrWoZ9C6/VxOf7ic219I8xZZFdrf0=
|
||||||
golang.org/x/term v0.43.0/go.mod h1:lrhlHNdQJHO+1qVYiHfFKVuVioJIheAc3fBSMFYEIsk=
|
golang.org/x/term v0.45.0/go.mod h1:9aqxs0blBcrm/n0L9QW0aRVD+ktan8ssZromtqJC43w=
|
||||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||||
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||||
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
|
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
|
||||||
@@ -488,8 +488,8 @@ golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
|
|||||||
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
|
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
|
||||||
golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
|
golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
|
||||||
golang.org/x/text v0.20.0/go.mod h1:D4IsuqiFMhST5bX19pQ9ikHC2GsaKyk/oF+pn3ducp4=
|
golang.org/x/text v0.20.0/go.mod h1:D4IsuqiFMhST5bX19pQ9ikHC2GsaKyk/oF+pn3ducp4=
|
||||||
golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
|
golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
|
||||||
golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=
|
golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
|
||||||
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
|
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
|
||||||
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
|
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
|
||||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||||
@@ -498,18 +498,18 @@ golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc
|
|||||||
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
|
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
|
||||||
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
|
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
|
||||||
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
|
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
|
||||||
golang.org/x/tools v0.45.0 h1:18qN3FAooORvApf5XjCXgsuayZOEtXf6JK18I3+ONa8=
|
golang.org/x/tools v0.48.0 h1:3+hClM1aLL5mjMKm5ovokw9epgRXPuu2tILgismM6RE=
|
||||||
golang.org/x/tools v0.45.0/go.mod h1:LuUGqqaXcXMEFEruIVJVm5mgDD8vww/z/SR1gQ4uE/0=
|
golang.org/x/tools v0.48.0/go.mod h1:08xX0orndb/F7jJxGDicx061tyd5pcMto75YMAXr6lk=
|
||||||
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||||
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||||
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
|
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
|
||||||
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
|
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
|
||||||
google.golang.org/genproto/googleapis/api v0.0.0-20260519071638-aa98bba5eb94 h1:DddG61lE5LkX6144z22i0gma9BMBs5aZ9B8lZLobxyw=
|
google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa h1:Kjn0N0tCrDgiAFW+lGO4JZ3ck44CehvJQMAwj9QF0G8=
|
||||||
google.golang.org/genproto/googleapis/api v0.0.0-20260519071638-aa98bba5eb94/go.mod h1:1dCETSCY2YKZNXQE3h4fun3TYwF5p8jejRKZgfWAgAY=
|
google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa/go.mod h1:q4lMZS6kskjT5HvCPrnnypcDPVJqT/f4nfxmkE7gryY=
|
||||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260519071638-aa98bba5eb94 h1:eZCjr/aAF8c5ccm5pb6T4EXgIei5MlAAPWPJk+5ArfY=
|
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa h1:mZHHdPZl0dbGHCflZgAq/Q468DWVFcU2whhB2KAo8fk=
|
||||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260519071638-aa98bba5eb94/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
|
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
|
||||||
google.golang.org/grpc v1.81.1 h1:VnnIIZ88UzOOKLukQi+ImGz8O1Wdp8nAGGnvOfEIWQQ=
|
google.golang.org/grpc v1.83.2 h1:EManeRomTObA0BU7I8vXgg/78uE5MJ9M8B39EX2WscU=
|
||||||
google.golang.org/grpc v1.81.1/go.mod h1:xGH9GfzOyMTGIOXBJmXt+BX/V0kcdQbdcuwQ/zNw42I=
|
google.golang.org/grpc v1.83.2/go.mod h1:YPI1hK3kDked6iHvgX3tR0y+nX/qpMFKhPgFsokw1S8=
|
||||||
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
|
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
|
||||||
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
||||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
|
|||||||
@@ -669,10 +669,7 @@ func (b *BunSelectQuery) PreloadRelation(relation string, apply ...func(common.S
|
|||||||
b.query = b.query.Relation(relation, func(sq *bun.SelectQuery) *bun.SelectQuery {
|
b.query = b.query.Relation(relation, func(sq *bun.SelectQuery) *bun.SelectQuery {
|
||||||
defer func() {
|
defer func() {
|
||||||
if r := recover(); r != nil {
|
if r := recover(); r != nil {
|
||||||
err := logger.HandlePanic("BunSelectQuery.PreloadRelation", r)
|
_ = logger.HandlePanic("BunSelectQuery.PreloadRelation", r)
|
||||||
if err != nil {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}()
|
}()
|
||||||
if len(apply) == 0 {
|
if len(apply) == 0 {
|
||||||
|
|||||||
@@ -82,6 +82,20 @@ type queryMetricsBunUser struct {
|
|||||||
Name string `bun:"name"`
|
Name string `bun:"name"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type queryMetricsBunParent struct {
|
||||||
|
bun.BaseModel `bun:"table:metrics_bun_parents"`
|
||||||
|
ID int64 `bun:"id,pk,autoincrement"`
|
||||||
|
Name string `bun:"name"`
|
||||||
|
Children []queryMetricsBunChild `bun:"rel:has-many,join:id=parent_id"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type queryMetricsBunChild struct {
|
||||||
|
bun.BaseModel `bun:"table:metrics_bun_children"`
|
||||||
|
ID int64 `bun:"id,pk,autoincrement"`
|
||||||
|
ParentID int64 `bun:"parent_id"`
|
||||||
|
Name string `bun:"name"`
|
||||||
|
}
|
||||||
|
|
||||||
func TestPgSQLAdapterRecordsSchemaEntityTableMetrics(t *testing.T) {
|
func TestPgSQLAdapterRecordsSchemaEntityTableMetrics(t *testing.T) {
|
||||||
db, mock, err := sqlmock.New()
|
db, mock, err := sqlmock.New()
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
@@ -346,3 +360,35 @@ func TestBunAdapterRecordsEntityAndTableMetrics(t *testing.T) {
|
|||||||
assert.Equal(t, "query_metrics_bun_user", calls[0].entity)
|
assert.Equal(t, "query_metrics_bun_user", calls[0].entity)
|
||||||
assert.Equal(t, "metrics_bun_users", calls[0].table)
|
assert.Equal(t, "metrics_bun_users", calls[0].table)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestBunSelectQueryScanModelSupportsHasManyPreload(t *testing.T) {
|
||||||
|
sqldb, err := sql.Open(sqliteshim.ShimName, "file::memory:?cache=shared")
|
||||||
|
require.NoError(t, err)
|
||||||
|
defer sqldb.Close()
|
||||||
|
|
||||||
|
db := bun.NewDB(sqldb, sqlitedialect.New())
|
||||||
|
defer db.Close()
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
_, err = db.NewCreateTable().Model((*queryMetricsBunParent)(nil)).IfNotExists().Exec(ctx)
|
||||||
|
require.NoError(t, err)
|
||||||
|
_, err = db.NewCreateTable().Model((*queryMetricsBunChild)(nil)).IfNotExists().Exec(ctx)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
parent := &queryMetricsBunParent{Name: "parent"}
|
||||||
|
_, err = db.NewInsert().Model(parent).Exec(ctx)
|
||||||
|
require.NoError(t, err)
|
||||||
|
_, err = db.NewInsert().Model(&queryMetricsBunChild{ParentID: parent.ID, Name: "child"}).Exec(ctx)
|
||||||
|
require.NoError(t, err)
|
||||||
|
|
||||||
|
adapter := NewBunAdapter(db)
|
||||||
|
var parents []queryMetricsBunParent
|
||||||
|
err = adapter.NewSelect().Model(&parents).PreloadRelation("Children").Scan(ctx, &parents)
|
||||||
|
require.ErrorContains(t, err, "use Model instead of the dest parameter in Scan")
|
||||||
|
|
||||||
|
parents = nil
|
||||||
|
err = adapter.NewSelect().Model(&parents).PreloadRelation("Children").ScanModel(ctx)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, parents, 1)
|
||||||
|
require.Len(t, parents[0].Children, 1)
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,402 @@
|
|||||||
|
package common
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// This file implements a single canonical parser + SQL builder for column
|
||||||
|
// references that traverse into JSON / JSONB values. It is used by SELECT,
|
||||||
|
// WHERE (filter) and ORDER BY handling so that all three treat JSON access
|
||||||
|
// consistently and safely.
|
||||||
|
//
|
||||||
|
// Supported input syntaxes (all PostgreSQL-oriented):
|
||||||
|
//
|
||||||
|
// data->>'city' arrow chain, text extraction
|
||||||
|
// data->'addr'->>'city' nested arrow chain
|
||||||
|
// data->2->>'name' arrow chain with array index
|
||||||
|
// data#>>'{addr,city}' hash-path, text extraction
|
||||||
|
// data#>'{addr,city}' hash-path, jsonb result
|
||||||
|
// data.addr.city dotted shorthand (Ambiguous: caller must
|
||||||
|
// confirm "data" is a JSON column)
|
||||||
|
// data->>'age'::int trailing cast (whitelisted targets only)
|
||||||
|
// (data->>'city') AS city parenthesised, with output alias
|
||||||
|
//
|
||||||
|
// JSON path segments are never interpolated into SQL: SQL() emits a `#>>` /
|
||||||
|
// `#>` operator with the path bound as a single `text[]` parameter.
|
||||||
|
|
||||||
|
// ColumnRef is a parsed reference to a (possibly JSON-traversing) column.
|
||||||
|
type ColumnRef struct {
|
||||||
|
// Base is the bare base column name, e.g. "data". Always a simple
|
||||||
|
// identifier ([A-Za-z_][A-Za-z0-9_]*); qualified names are rejected.
|
||||||
|
Base string
|
||||||
|
// Path is the JSON key / array-index path, e.g. ["address", "city"].
|
||||||
|
// Empty for a plain column reference.
|
||||||
|
Path []string
|
||||||
|
// AsText is true when the final extraction should yield text (->> / #>>)
|
||||||
|
// rather than jsonb (-> / #>).
|
||||||
|
AsText bool
|
||||||
|
// Cast is a normalised SQL type name to cast the whole expression to
|
||||||
|
// (e.g. "integer", "numeric", "timestamptz"), or "" for no cast.
|
||||||
|
Cast string
|
||||||
|
// Alias is a validated output identifier for `AS <alias>`, or "".
|
||||||
|
Alias string
|
||||||
|
// Ambiguous is true when Path was produced from the dotted "a.b.c"
|
||||||
|
// shorthand. The caller MUST verify that Base is a JSON column
|
||||||
|
// (reflection.IsJSONColumn) before treating this as a JSON expression,
|
||||||
|
// otherwise "a.b" is an ordinary table-qualified column.
|
||||||
|
Ambiguous bool
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
reSimpleIdent = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
|
||||||
|
reSimpleSegment = regexp.MustCompile(`^[A-Za-z0-9_]+$`)
|
||||||
|
reAliasSuffix = regexp.MustCompile(`(?i)\s+AS\s+("?[A-Za-z_][A-Za-z0-9_]*"?)\s*$`)
|
||||||
|
reArrowStep = regexp.MustCompile(`^\s*(->>|->)\s*(?:'((?:[^']|'')*)'|(\d+))\s*`)
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
maxJSONPathDepth = 32
|
||||||
|
maxJSONSegmentSize = 128
|
||||||
|
)
|
||||||
|
|
||||||
|
// castAliases maps accepted cast spellings to their canonical PostgreSQL type.
|
||||||
|
var castAliases = map[string]string{
|
||||||
|
"int": "integer",
|
||||||
|
"int4": "integer",
|
||||||
|
"integer": "integer",
|
||||||
|
"int2": "smallint",
|
||||||
|
"smallint": "smallint",
|
||||||
|
"int8": "bigint",
|
||||||
|
"bigint": "bigint",
|
||||||
|
"numeric": "numeric",
|
||||||
|
"decimal": "numeric",
|
||||||
|
"real": "real",
|
||||||
|
"float4": "real",
|
||||||
|
"float": "double precision",
|
||||||
|
"float8": "double precision",
|
||||||
|
"double precision": "double precision",
|
||||||
|
"bool": "boolean",
|
||||||
|
"boolean": "boolean",
|
||||||
|
"text": "text",
|
||||||
|
"varchar": "text",
|
||||||
|
"uuid": "uuid",
|
||||||
|
"date": "date",
|
||||||
|
"time": "time",
|
||||||
|
"timestamp": "timestamp",
|
||||||
|
"timestamptz": "timestamptz",
|
||||||
|
"json": "json",
|
||||||
|
"jsonb": "jsonb",
|
||||||
|
}
|
||||||
|
|
||||||
|
// NormalizeCastTarget returns the canonical PostgreSQL type name for a
|
||||||
|
// user-supplied cast spelling, and whether it is on the allowlist.
|
||||||
|
func NormalizeCastTarget(s string) (string, bool) {
|
||||||
|
c, ok := castAliases[strings.ToLower(strings.TrimSpace(s))]
|
||||||
|
return c, ok
|
||||||
|
}
|
||||||
|
|
||||||
|
// ParseColumnRef parses a column token that traverses into a JSON value.
|
||||||
|
//
|
||||||
|
// ok is true only when the token carries JSON traversal syntax (arrow chain,
|
||||||
|
// hash-path, or dotted shorthand with at least one sub-key). For a plain
|
||||||
|
// column name — with or without an alias/cast — ok is false and the caller
|
||||||
|
// should handle the token the way it did before.
|
||||||
|
//
|
||||||
|
// When ok is true and ref.Ambiguous is true, the caller must confirm that
|
||||||
|
// ref.Base is a JSON column before using ref.SQL; otherwise the dotted token
|
||||||
|
// is an ordinary "table.column" reference.
|
||||||
|
func ParseColumnRef(raw string) (ColumnRef, bool) {
|
||||||
|
expr := strings.TrimSpace(raw)
|
||||||
|
if expr == "" {
|
||||||
|
return ColumnRef{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
var ref ColumnRef
|
||||||
|
|
||||||
|
// 1. Trailing `AS <alias>`.
|
||||||
|
if m := reAliasSuffix.FindStringSubmatch(expr); m != nil {
|
||||||
|
ref.Alias = strings.Trim(m[1], `"`)
|
||||||
|
expr = strings.TrimSpace(expr[:len(expr)-len(m[0])])
|
||||||
|
if expr == "" {
|
||||||
|
return ColumnRef{}, false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 2. Trailing `::<type>` cast (take the last `::` in the string).
|
||||||
|
if idx := strings.LastIndex(expr, "::"); idx != -1 {
|
||||||
|
candidate := strings.TrimSpace(expr[idx+2:])
|
||||||
|
if canonical, allowed := NormalizeCastTarget(candidate); allowed {
|
||||||
|
ref.Cast = canonical
|
||||||
|
expr = strings.TrimSpace(expr[:idx])
|
||||||
|
} else if candidate != "" && looksLikeCastTail(candidate) {
|
||||||
|
// An explicit but unsupported cast target — reject rather than
|
||||||
|
// silently dropping it.
|
||||||
|
return ColumnRef{}, false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 3. One layer of wrapping parentheses: "(expr)" -> "expr".
|
||||||
|
if wrapped, ok := stripWrappingParens(expr); ok {
|
||||||
|
expr = strings.TrimSpace(wrapped)
|
||||||
|
if expr == "" {
|
||||||
|
return ColumnRef{}, false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 4. Parse the core expression.
|
||||||
|
switch {
|
||||||
|
case strings.Contains(expr, "#>>") || strings.Contains(expr, "#>"):
|
||||||
|
if !parseHashPath(expr, &ref) {
|
||||||
|
return ColumnRef{}, false
|
||||||
|
}
|
||||||
|
case strings.Contains(expr, "->"):
|
||||||
|
if !parseArrowChain(expr, &ref) {
|
||||||
|
return ColumnRef{}, false
|
||||||
|
}
|
||||||
|
case strings.Contains(expr, "."):
|
||||||
|
if !parseDottedPath(expr, &ref) {
|
||||||
|
return ColumnRef{}, false
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
// Plain column — nothing JSON about it.
|
||||||
|
return ColumnRef{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
if !validateRef(&ref) {
|
||||||
|
return ColumnRef{}, false
|
||||||
|
}
|
||||||
|
return ref, true
|
||||||
|
}
|
||||||
|
|
||||||
|
// SQL renders the reference as a parameterised SQL expression plus its args.
|
||||||
|
// tableAlias, when non-empty, qualifies the base column (each dot-separated
|
||||||
|
// part is quoted independently, so "public.users" -> `"public"."users"`).
|
||||||
|
func (r ColumnRef) SQL(tableAlias string) (expr string, args []interface{}) {
|
||||||
|
base := quoteQualifiedIdent(r.Base)
|
||||||
|
if tableAlias != "" {
|
||||||
|
base = quoteQualifiedIdent(tableAlias) + "." + QuoteIdent(r.Base)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(r.Path) == 0 {
|
||||||
|
if r.Cast != "" {
|
||||||
|
return fmt.Sprintf("(%s)::%s", base, r.Cast), nil
|
||||||
|
}
|
||||||
|
return base, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
op := "#>"
|
||||||
|
if r.AsText {
|
||||||
|
op = "#>>"
|
||||||
|
}
|
||||||
|
expr = fmt.Sprintf("(%s %s ?::text[])", base, op)
|
||||||
|
args = []interface{}{pgTextArrayLiteral(r.Path)}
|
||||||
|
|
||||||
|
if r.Cast != "" {
|
||||||
|
expr = fmt.Sprintf("(%s)::%s", expr, r.Cast)
|
||||||
|
}
|
||||||
|
return expr, args
|
||||||
|
}
|
||||||
|
|
||||||
|
// OutputAlias returns the alias to use for this reference in a SELECT list:
|
||||||
|
// the explicit alias when given, otherwise a deterministic name derived from
|
||||||
|
// the base column and path (e.g. "data_address_city").
|
||||||
|
func (r ColumnRef) OutputAlias() string {
|
||||||
|
if r.Alias != "" {
|
||||||
|
return r.Alias
|
||||||
|
}
|
||||||
|
if len(r.Path) == 0 {
|
||||||
|
return r.Base
|
||||||
|
}
|
||||||
|
parts := make([]string, 0, len(r.Path)+1)
|
||||||
|
parts = append(parts, r.Base)
|
||||||
|
for _, p := range r.Path {
|
||||||
|
parts = append(parts, sanitizeAliasPart(p))
|
||||||
|
}
|
||||||
|
return strings.Join(parts, "_")
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── parsing helpers ─────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
func parseHashPath(expr string, ref *ColumnRef) bool {
|
||||||
|
op := "#>>"
|
||||||
|
ref.AsText = true
|
||||||
|
if !strings.Contains(expr, "#>>") {
|
||||||
|
op = "#>"
|
||||||
|
ref.AsText = false
|
||||||
|
}
|
||||||
|
parts := strings.SplitN(expr, op, 2)
|
||||||
|
if len(parts) != 2 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
ref.Base = strings.TrimSpace(parts[0])
|
||||||
|
|
||||||
|
rhs := strings.TrimSpace(parts[1])
|
||||||
|
// Expect a single-quoted array literal: '{a,b,c}'
|
||||||
|
if len(rhs) < 2 || rhs[0] != '\'' || rhs[len(rhs)-1] != '\'' {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
rhs = rhs[1 : len(rhs)-1]
|
||||||
|
rhs = strings.TrimSpace(rhs)
|
||||||
|
rhs = strings.TrimPrefix(rhs, "{")
|
||||||
|
rhs = strings.TrimSuffix(rhs, "}")
|
||||||
|
if strings.TrimSpace(rhs) == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, seg := range strings.Split(rhs, ",") {
|
||||||
|
seg = strings.TrimSpace(seg)
|
||||||
|
seg = strings.Trim(seg, `"`)
|
||||||
|
if seg == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
ref.Path = append(ref.Path, seg)
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseArrowChain(expr string, ref *ColumnRef) bool {
|
||||||
|
arrowIdx := strings.Index(expr, "->")
|
||||||
|
if arrowIdx <= 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
ref.Base = strings.TrimSpace(expr[:arrowIdx])
|
||||||
|
|
||||||
|
rest := expr[arrowIdx:]
|
||||||
|
for strings.TrimSpace(rest) != "" {
|
||||||
|
m := reArrowStep.FindStringSubmatch(rest)
|
||||||
|
if m == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
ref.AsText = m[1] == "->>"
|
||||||
|
if m[3] != "" {
|
||||||
|
// unquoted array index
|
||||||
|
ref.Path = append(ref.Path, m[3])
|
||||||
|
} else {
|
||||||
|
// quoted key; unescape doubled single quotes
|
||||||
|
ref.Path = append(ref.Path, strings.ReplaceAll(m[2], "''", "'"))
|
||||||
|
}
|
||||||
|
rest = rest[len(m[0]):]
|
||||||
|
}
|
||||||
|
return len(ref.Path) > 0
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseDottedPath(expr string, ref *ColumnRef) bool {
|
||||||
|
segs := strings.Split(expr, ".")
|
||||||
|
if len(segs) < 2 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for i, s := range segs {
|
||||||
|
s = strings.TrimSpace(s)
|
||||||
|
if !reSimpleSegment.MatchString(s) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if i == 0 {
|
||||||
|
ref.Base = s
|
||||||
|
} else {
|
||||||
|
ref.Path = append(ref.Path, s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ref.AsText = true
|
||||||
|
ref.Ambiguous = true
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateRef(ref *ColumnRef) bool {
|
||||||
|
if !reSimpleIdent.MatchString(ref.Base) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if len(ref.Path) == 0 || len(ref.Path) > maxJSONPathDepth {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, seg := range ref.Path {
|
||||||
|
if seg == "" || len(seg) > maxJSONSegmentSize || strings.ContainsRune(seg, 0) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ref.Alias != "" && !reSimpleIdent.MatchString(ref.Alias) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// looksLikeCastTail reports whether s is plausibly meant as a `::type` target
|
||||||
|
// (letters/digits/spaces only) rather than, say, part of a JSON operator.
|
||||||
|
func looksLikeCastTail(s string) bool {
|
||||||
|
for _, r := range s {
|
||||||
|
isCastChar := r == ' ' || r == '_' ||
|
||||||
|
(r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9')
|
||||||
|
if !isCastChar {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// stripWrappingParens removes one layer of parentheses when they wrap the whole
|
||||||
|
// expression, e.g. "(a->>'b')" -> "a->>'b'". It respects single-quoted strings.
|
||||||
|
func stripWrappingParens(s string) (string, bool) {
|
||||||
|
s = strings.TrimSpace(s)
|
||||||
|
if len(s) < 2 || s[0] != '(' || s[len(s)-1] != ')' {
|
||||||
|
return s, false
|
||||||
|
}
|
||||||
|
depth := 0
|
||||||
|
inQuote := false
|
||||||
|
for i := 0; i < len(s); i++ {
|
||||||
|
c := s[i]
|
||||||
|
switch {
|
||||||
|
case c == '\'':
|
||||||
|
inQuote = !inQuote
|
||||||
|
case inQuote:
|
||||||
|
// skip
|
||||||
|
case c == '(':
|
||||||
|
depth++
|
||||||
|
case c == ')':
|
||||||
|
depth--
|
||||||
|
if depth == 0 && i != len(s)-1 {
|
||||||
|
// closing paren is not the last char -> not a full wrap
|
||||||
|
return s, false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if depth != 0 {
|
||||||
|
return s, false
|
||||||
|
}
|
||||||
|
return s[1 : len(s)-1], true
|
||||||
|
}
|
||||||
|
|
||||||
|
// pgTextArrayLiteral builds a PostgreSQL text[] array literal ("{a,b,c}") from
|
||||||
|
// path segments, quoting and escaping any segment that is not a bare word.
|
||||||
|
func pgTextArrayLiteral(segs []string) string {
|
||||||
|
escaper := strings.NewReplacer(`\`, `\\`, `"`, `\"`)
|
||||||
|
parts := make([]string, len(segs))
|
||||||
|
for i, s := range segs {
|
||||||
|
if reSimpleSegment.MatchString(s) {
|
||||||
|
parts[i] = s
|
||||||
|
} else {
|
||||||
|
parts[i] = `"` + escaper.Replace(s) + `"`
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "{" + strings.Join(parts, ",") + "}"
|
||||||
|
}
|
||||||
|
|
||||||
|
// quoteQualifiedIdent quotes each dot-separated part of an identifier.
|
||||||
|
func quoteQualifiedIdent(ident string) string {
|
||||||
|
parts := strings.Split(ident, ".")
|
||||||
|
for i, p := range parts {
|
||||||
|
parts[i] = QuoteIdent(p)
|
||||||
|
}
|
||||||
|
return strings.Join(parts, ".")
|
||||||
|
}
|
||||||
|
|
||||||
|
func sanitizeAliasPart(s string) string {
|
||||||
|
var b strings.Builder
|
||||||
|
for _, r := range s {
|
||||||
|
if (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') || r == '_' {
|
||||||
|
b.WriteRune(r)
|
||||||
|
} else {
|
||||||
|
b.WriteRune('_')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return b.String()
|
||||||
|
}
|
||||||
@@ -0,0 +1,274 @@
|
|||||||
|
package common
|
||||||
|
|
||||||
|
import (
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestParseColumnRef_Valid(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
input string
|
||||||
|
base string
|
||||||
|
path []string
|
||||||
|
asText bool
|
||||||
|
cast string
|
||||||
|
alias string
|
||||||
|
ambiguous bool
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "arrow text extraction",
|
||||||
|
input: "data->>'city'",
|
||||||
|
base: "data", path: []string{"city"}, asText: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "arrow whitespace tolerant",
|
||||||
|
input: "data ->> 'city'",
|
||||||
|
base: "data", path: []string{"city"}, asText: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "nested arrow chain",
|
||||||
|
input: "data->'address'->>'city'",
|
||||||
|
base: "data", path: []string{"address", "city"}, asText: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "arrow jsonb result",
|
||||||
|
input: "data->'address'",
|
||||||
|
base: "data", path: []string{"address"}, asText: false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "arrow array index",
|
||||||
|
input: "items->0->>'name'",
|
||||||
|
base: "items", path: []string{"0", "name"}, asText: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "hash path text",
|
||||||
|
input: "data#>>'{address,city}'",
|
||||||
|
base: "data", path: []string{"address", "city"}, asText: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "hash path jsonb",
|
||||||
|
input: "data#>'{address,city}'",
|
||||||
|
base: "data", path: []string{"address", "city"}, asText: false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "dotted shorthand",
|
||||||
|
input: "data.address.city",
|
||||||
|
base: "data", path: []string{"address", "city"}, asText: true, ambiguous: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "trailing cast",
|
||||||
|
input: "data->>'age'::int",
|
||||||
|
base: "data", path: []string{"age"}, asText: true, cast: "integer",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "cast normalises",
|
||||||
|
input: "data->>'ts'::timestamptz",
|
||||||
|
base: "data", path: []string{"ts"}, asText: true, cast: "timestamptz",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "parenthesised with alias",
|
||||||
|
input: "(data->>'city') AS city_name",
|
||||||
|
base: "data", path: []string{"city"}, asText: true, alias: "city_name",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "paren wrap and cast",
|
||||||
|
input: "(data->>'age')::numeric",
|
||||||
|
base: "data", path: []string{"age"}, asText: true, cast: "numeric",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "quoted key with spaces",
|
||||||
|
input: "data->>'key with space'",
|
||||||
|
base: "data", path: []string{"key with space"}, asText: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "quoted key with escaped quote",
|
||||||
|
input: "data->>'o''brien'",
|
||||||
|
base: "data", path: []string{"o'brien"}, asText: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "relation column is ambiguous json",
|
||||||
|
input: "orders.total",
|
||||||
|
base: "orders", path: []string{"total"}, asText: true, ambiguous: true,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range tests {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
ref, ok := ParseColumnRef(tc.input)
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("ParseColumnRef(%q) returned ok=false", tc.input)
|
||||||
|
}
|
||||||
|
if ref.Base != tc.base {
|
||||||
|
t.Errorf("Base = %q, want %q", ref.Base, tc.base)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(ref.Path, tc.path) {
|
||||||
|
t.Errorf("Path = %#v, want %#v", ref.Path, tc.path)
|
||||||
|
}
|
||||||
|
if ref.AsText != tc.asText {
|
||||||
|
t.Errorf("AsText = %v, want %v", ref.AsText, tc.asText)
|
||||||
|
}
|
||||||
|
if ref.Cast != tc.cast {
|
||||||
|
t.Errorf("Cast = %q, want %q", ref.Cast, tc.cast)
|
||||||
|
}
|
||||||
|
if ref.Alias != tc.alias {
|
||||||
|
t.Errorf("Alias = %q, want %q", ref.Alias, tc.alias)
|
||||||
|
}
|
||||||
|
if ref.Ambiguous != tc.ambiguous {
|
||||||
|
t.Errorf("Ambiguous = %v, want %v", ref.Ambiguous, tc.ambiguous)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseColumnRef_NotJSON(t *testing.T) {
|
||||||
|
// These must return ok=false so callers fall back to their normal handling.
|
||||||
|
inputs := []string{
|
||||||
|
"",
|
||||||
|
" ",
|
||||||
|
"name",
|
||||||
|
"data",
|
||||||
|
"created_at",
|
||||||
|
"(id)",
|
||||||
|
}
|
||||||
|
for _, in := range inputs {
|
||||||
|
if ref, ok := ParseColumnRef(in); ok {
|
||||||
|
t.Errorf("ParseColumnRef(%q) = %+v, ok=true; want ok=false", in, ref)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseColumnRef_Rejected(t *testing.T) {
|
||||||
|
// Malformed or unsafe tokens must be rejected outright.
|
||||||
|
inputs := []string{
|
||||||
|
"data->>'x'::bogus", // cast not on allowlist
|
||||||
|
"data->>'x' AS 1bad", // invalid alias
|
||||||
|
"(data->>'a') OR (x->>'b')", // not a single wrapped expr
|
||||||
|
"data->>'x'); DROP TABLE users; --", // injection attempt
|
||||||
|
"data->b", // unquoted non-numeric key
|
||||||
|
"data->>''", // empty key
|
||||||
|
"data#>>'{}'", // empty hash path
|
||||||
|
"data#>>address", // hash path not a quoted literal
|
||||||
|
"weird col->>'x'", // base not an identifier
|
||||||
|
"data.address.city.but.way.too...deep.", // trailing dot -> empty segment
|
||||||
|
}
|
||||||
|
for _, in := range inputs {
|
||||||
|
if ref, ok := ParseColumnRef(in); ok {
|
||||||
|
t.Errorf("ParseColumnRef(%q) = %+v, ok=true; want rejected", in, ref)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestColumnRef_SQL(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
ref ColumnRef
|
||||||
|
alias string
|
||||||
|
wantExpr string
|
||||||
|
wantArgs []interface{}
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "text extraction qualified",
|
||||||
|
ref: ColumnRef{Base: "data", Path: []string{"address", "city"}, AsText: true},
|
||||||
|
alias: "u",
|
||||||
|
wantExpr: `("u"."data" #>> ?::text[])`,
|
||||||
|
wantArgs: []interface{}{"{address,city}"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "jsonb extraction unqualified",
|
||||||
|
ref: ColumnRef{Base: "data", Path: []string{"a"}, AsText: false},
|
||||||
|
alias: "",
|
||||||
|
wantExpr: `("data" #> ?::text[])`,
|
||||||
|
wantArgs: []interface{}{"{a}"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "with cast",
|
||||||
|
ref: ColumnRef{Base: "data", Path: []string{"age"}, AsText: true, Cast: "integer"},
|
||||||
|
alias: "t",
|
||||||
|
wantExpr: `(("t"."data" #>> ?::text[]))::integer`,
|
||||||
|
wantArgs: []interface{}{"{age}"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "schema qualified alias",
|
||||||
|
ref: ColumnRef{Base: "data", Path: []string{"k"}, AsText: true},
|
||||||
|
alias: "public.users",
|
||||||
|
wantExpr: `("public"."users"."data" #>> ?::text[])`,
|
||||||
|
wantArgs: []interface{}{"{k}"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "key needing quoting",
|
||||||
|
ref: ColumnRef{Base: "data", Path: []string{"key with space", `ev"il`}, AsText: true},
|
||||||
|
alias: "",
|
||||||
|
wantExpr: `("data" #>> ?::text[])`,
|
||||||
|
wantArgs: []interface{}{`{"key with space","ev\"il"}`},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range tests {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
expr, args := tc.ref.SQL(tc.alias)
|
||||||
|
if expr != tc.wantExpr {
|
||||||
|
t.Errorf("expr = %q, want %q", expr, tc.wantExpr)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(args, tc.wantArgs) {
|
||||||
|
t.Errorf("args = %#v, want %#v", args, tc.wantArgs)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestColumnRef_SQL_RoundTrip(t *testing.T) {
|
||||||
|
ref, ok := ParseColumnRef("profile->'contact'->>'email'")
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("parse failed")
|
||||||
|
}
|
||||||
|
expr, args := ref.SQL("customers")
|
||||||
|
wantExpr := `("customers"."profile" #>> ?::text[])`
|
||||||
|
if expr != wantExpr {
|
||||||
|
t.Errorf("expr = %q, want %q", expr, wantExpr)
|
||||||
|
}
|
||||||
|
if len(args) != 1 || args[0] != "{contact,email}" {
|
||||||
|
t.Errorf("args = %#v, want [{contact,email}]", args)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestColumnRef_OutputAlias(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
ref ColumnRef
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{ColumnRef{Base: "data", Path: []string{"address", "city"}, AsText: true}, "data_address_city"},
|
||||||
|
{ColumnRef{Base: "data", Path: []string{"city"}, Alias: "city"}, "city"},
|
||||||
|
{ColumnRef{Base: "data", Path: []string{"weird key"}}, "data_weird_key"},
|
||||||
|
{ColumnRef{Base: "data"}, "data"},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
if got := c.ref.OutputAlias(); got != c.want {
|
||||||
|
t.Errorf("OutputAlias(%+v) = %q, want %q", c.ref, got, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNormalizeCastTarget(t *testing.T) {
|
||||||
|
ok := map[string]string{
|
||||||
|
"int": "integer",
|
||||||
|
"INT": "integer",
|
||||||
|
" bigint ": "bigint",
|
||||||
|
"decimal": "numeric",
|
||||||
|
"float8": "double precision",
|
||||||
|
"bool": "boolean",
|
||||||
|
"timestamptz": "timestamptz",
|
||||||
|
"uuid": "uuid",
|
||||||
|
}
|
||||||
|
for in, want := range ok {
|
||||||
|
got, allowed := NormalizeCastTarget(in)
|
||||||
|
if !allowed || got != want {
|
||||||
|
t.Errorf("NormalizeCastTarget(%q) = %q, %v; want %q, true", in, got, allowed, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, in := range []string{"", "regclass", "int; drop", "text[]"} {
|
||||||
|
if got, allowed := NormalizeCastTarget(in); allowed {
|
||||||
|
t.Errorf("NormalizeCastTarget(%q) = %q, true; want not allowed", in, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,209 @@
|
|||||||
|
package common
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/reflection"
|
||||||
|
)
|
||||||
|
|
||||||
|
// This file wires the canonical JSON column parser (json_column.go) into the
|
||||||
|
// three query-building paths that every spec handler shares: SELECT column
|
||||||
|
// lists, WHERE filters and ORDER BY. The helpers here are the single place
|
||||||
|
// those paths call so that JSON access is resolved (and made injection-safe)
|
||||||
|
// identically everywhere. They mirror the style of BuildSpatialCondition /
|
||||||
|
// BuildVectorCondition: a boolean ok result tells the caller whether the token
|
||||||
|
// was a JSON reference it should take over, otherwise the caller keeps its
|
||||||
|
// existing (non-JSON) behaviour.
|
||||||
|
|
||||||
|
// jsonComparisonOps are the operators for which a JSON text extraction should be
|
||||||
|
// cast to a concrete type when the value looks numeric — otherwise "10" < "9".
|
||||||
|
var jsonComparisonOps = map[string]bool{
|
||||||
|
"gt": true, "greater_than": true, ">": true,
|
||||||
|
"gte": true, "greater_than_equals": true, "ge": true, ">=": true,
|
||||||
|
"lt": true, "less_than": true, "<": true,
|
||||||
|
"lte": true, "less_than_equals": true, "le": true, "<=": true,
|
||||||
|
"between": true, "between_inclusive": true,
|
||||||
|
}
|
||||||
|
|
||||||
|
// ResolveJSONColumnRef parses token and, when it is a usable JSON reference for
|
||||||
|
// model, returns the parsed ColumnRef. For the dotted "a.b" shorthand (which is
|
||||||
|
// otherwise indistinguishable from a table-qualified column) ok is true only
|
||||||
|
// when model confirms the base is a JSON column.
|
||||||
|
func ResolveJSONColumnRef(model interface{}, token string) (ColumnRef, bool) {
|
||||||
|
ref, ok := ParseColumnRef(token)
|
||||||
|
if !ok {
|
||||||
|
return ColumnRef{}, false
|
||||||
|
}
|
||||||
|
if ref.Ambiguous && !reflection.IsJSONColumn(model, ref.Base) {
|
||||||
|
return ColumnRef{}, false
|
||||||
|
}
|
||||||
|
return ref, true
|
||||||
|
}
|
||||||
|
|
||||||
|
// IsJSONColumnToken reports whether token is a JSON reference this package can
|
||||||
|
// resolve for model (arrow/hash syntax always; dotted shorthand only when the
|
||||||
|
// base is a JSON column).
|
||||||
|
func IsJSONColumnToken(model interface{}, token string) bool {
|
||||||
|
_, ok := ResolveJSONColumnRef(model, token)
|
||||||
|
return ok
|
||||||
|
}
|
||||||
|
|
||||||
|
// ResolveJSONColumnExpr resolves a raw column token that traverses into a JSON
|
||||||
|
// value into a parameterised SQL expression plus its args and a deterministic
|
||||||
|
// output alias. ok is false when the token is not a JSON reference, in which
|
||||||
|
// case the caller should handle it the way it did before.
|
||||||
|
//
|
||||||
|
// tableAlias, when non-empty, qualifies the base column.
|
||||||
|
func ResolveJSONColumnExpr(model interface{}, tableAlias, token string) (expr string, args []interface{}, alias string, ok bool) {
|
||||||
|
ref, ok := ResolveJSONColumnRef(model, token)
|
||||||
|
if !ok {
|
||||||
|
return "", nil, "", false
|
||||||
|
}
|
||||||
|
expr, args = ref.SQL(tableAlias)
|
||||||
|
return expr, args, ref.OutputAlias(), true
|
||||||
|
}
|
||||||
|
|
||||||
|
// ApplySelectColumns adds the requested columns to query, resolving any that are
|
||||||
|
// JSON sub-field references (data->>'x', data#>>'{a,b}', or the dotted data.x
|
||||||
|
// shorthand for a JSON column) into safe parameterised expressions with a
|
||||||
|
// deterministic alias. Plain columns are passed through reflection.ExtractSourceColumn
|
||||||
|
// exactly as before. tableAlias, when non-empty, qualifies JSON base columns.
|
||||||
|
func ApplySelectColumns(query SelectQuery, model interface{}, tableAlias string, columns []string) SelectQuery {
|
||||||
|
for _, col := range columns {
|
||||||
|
if expr, args, alias, ok := ResolveJSONColumnExpr(model, tableAlias, col); ok {
|
||||||
|
query = query.ColumnExpr(expr+" AS "+QuoteIdent(alias), args...)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
query = query.Column(reflection.ExtractSourceColumn(col))
|
||||||
|
}
|
||||||
|
return query
|
||||||
|
}
|
||||||
|
|
||||||
|
// BuildJSONFilterCondition builds a complete WHERE condition for a JSON column
|
||||||
|
// token. ok is false when the token is not a JSON reference or the operator is
|
||||||
|
// not one this builder handles (the caller then keeps its existing behaviour).
|
||||||
|
//
|
||||||
|
// The JSON path is always bound as a parameter, never interpolated. When the
|
||||||
|
// reference carries no explicit ::cast and the operator is an ordered
|
||||||
|
// comparison against a numeric value, the extracted text is cast to numeric so
|
||||||
|
// the comparison is numeric rather than lexical.
|
||||||
|
func BuildJSONFilterCondition(model interface{}, tableAlias, token, operator string, value interface{}) (condition string, args []interface{}, ok bool) {
|
||||||
|
ref, ok := ResolveJSONColumnRef(model, token)
|
||||||
|
if !ok {
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
|
||||||
|
op := strings.ToLower(strings.TrimSpace(operator))
|
||||||
|
|
||||||
|
// Infer a cast for ordered comparisons on numeric values so "10" > "9".
|
||||||
|
if ref.Cast == "" && jsonComparisonOps[op] && jsonValueIsNumeric(value) {
|
||||||
|
ref.Cast = "numeric"
|
||||||
|
}
|
||||||
|
|
||||||
|
colExpr, colArgs := ref.SQL(tableAlias)
|
||||||
|
|
||||||
|
// prepend copies the column-expression args (the bound JSON path, and any
|
||||||
|
// others) ahead of the value args so placeholder order matches the SQL.
|
||||||
|
prepend := func(valueArgs ...interface{}) []interface{} {
|
||||||
|
out := make([]interface{}, 0, len(colArgs)+len(valueArgs))
|
||||||
|
out = append(out, colArgs...)
|
||||||
|
out = append(out, valueArgs...)
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
switch op {
|
||||||
|
case "eq", "equals", "=":
|
||||||
|
return fmt.Sprintf("%s = ?", colExpr), prepend(value), true
|
||||||
|
case "neq", "not_equals", "ne", "!=", "<>":
|
||||||
|
return fmt.Sprintf("%s != ?", colExpr), prepend(value), true
|
||||||
|
case "gt", "greater_than", ">":
|
||||||
|
return fmt.Sprintf("%s > ?", colExpr), prepend(value), true
|
||||||
|
case "gte", "greater_than_equals", "ge", ">=":
|
||||||
|
return fmt.Sprintf("%s >= ?", colExpr), prepend(value), true
|
||||||
|
case "lt", "less_than", "<":
|
||||||
|
return fmt.Sprintf("%s < ?", colExpr), prepend(value), true
|
||||||
|
case "lte", "less_than_equals", "le", "<=":
|
||||||
|
return fmt.Sprintf("%s <= ?", colExpr), prepend(value), true
|
||||||
|
case "like":
|
||||||
|
return fmt.Sprintf("%s LIKE ?", colExpr), prepend(value), true
|
||||||
|
case "ilike":
|
||||||
|
return fmt.Sprintf("%s ILIKE ?", colExpr), prepend(value), true
|
||||||
|
case "in":
|
||||||
|
inCond, inArgs := BuildInCondition(colExpr, value)
|
||||||
|
if inCond == "" {
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
return inCond, prepend(inArgs...), true
|
||||||
|
case "between", "between_inclusive":
|
||||||
|
lo, hi, bok := twoBoundValues(value)
|
||||||
|
if !bok {
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
loOp, hiOp := ">", "<"
|
||||||
|
if op == "between_inclusive" {
|
||||||
|
loOp, hiOp = ">=", "<="
|
||||||
|
}
|
||||||
|
// colExpr appears twice, so its bound args (the JSON path) appear twice.
|
||||||
|
betweenArgs := make([]interface{}, 0, 2*len(colArgs)+2)
|
||||||
|
betweenArgs = append(betweenArgs, colArgs...)
|
||||||
|
betweenArgs = append(betweenArgs, lo)
|
||||||
|
betweenArgs = append(betweenArgs, colArgs...)
|
||||||
|
betweenArgs = append(betweenArgs, hi)
|
||||||
|
return fmt.Sprintf("(%s %s ? AND %s %s ?)", colExpr, loOp, colExpr, hiOp), betweenArgs, true
|
||||||
|
case "is_null", "isnull":
|
||||||
|
return fmt.Sprintf("%s IS NULL", colExpr), prepend(), true
|
||||||
|
case "is_not_null", "isnotnull":
|
||||||
|
return fmt.Sprintf("%s IS NOT NULL", colExpr), prepend(), true
|
||||||
|
default:
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// jsonValueIsNumeric reports whether value (or every element of a 2-slice) is a
|
||||||
|
// number or a numeric-looking string.
|
||||||
|
func jsonValueIsNumeric(value interface{}) bool {
|
||||||
|
switch v := value.(type) {
|
||||||
|
case []interface{}:
|
||||||
|
if len(v) == 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, e := range v {
|
||||||
|
if !jsonValueIsNumeric(e) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
case []string:
|
||||||
|
if len(v) == 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, e := range v {
|
||||||
|
if _, ok := toFloat(e); !ok {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
case string:
|
||||||
|
_, ok := toFloat(v)
|
||||||
|
return ok
|
||||||
|
default:
|
||||||
|
_, ok := toFloat(value)
|
||||||
|
return ok
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// twoBoundValues extracts the low/high bounds from a BETWEEN filter value.
|
||||||
|
func twoBoundValues(value interface{}) (lo, hi interface{}, ok bool) {
|
||||||
|
switch v := value.(type) {
|
||||||
|
case []interface{}:
|
||||||
|
if len(v) == 2 {
|
||||||
|
return v[0], v[1], true
|
||||||
|
}
|
||||||
|
case []string:
|
||||||
|
if len(v) == 2 {
|
||||||
|
return v[0], v[1], true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil, nil, false
|
||||||
|
}
|
||||||
@@ -0,0 +1,164 @@
|
|||||||
|
package common
|
||||||
|
|
||||||
|
import (
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/spectypes"
|
||||||
|
)
|
||||||
|
|
||||||
|
type jsonCondModel struct {
|
||||||
|
ID int64 `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Data spectypes.SqlJSONB `json:"data"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveJSONColumnRef_Gate(t *testing.T) {
|
||||||
|
m := jsonCondModel{}
|
||||||
|
|
||||||
|
// Explicit operator syntax needs no model confirmation.
|
||||||
|
if _, ok := ResolveJSONColumnRef(m, "data->>'city'"); !ok {
|
||||||
|
t.Error("arrow syntax should resolve")
|
||||||
|
}
|
||||||
|
// Dotted shorthand on a real JSON column resolves.
|
||||||
|
if ref, ok := ResolveJSONColumnRef(m, "data.city"); !ok || !reflect.DeepEqual(ref.Path, []string{"city"}) {
|
||||||
|
t.Errorf("dotted shorthand on JSON column should resolve, got ok=%v ref=%+v", ok, ref)
|
||||||
|
}
|
||||||
|
// Dotted shorthand on a non-JSON column must NOT be treated as JSON.
|
||||||
|
if _, ok := ResolveJSONColumnRef(m, "name.first"); ok {
|
||||||
|
t.Error("dotted shorthand on non-JSON column must not resolve as JSON")
|
||||||
|
}
|
||||||
|
// Plain columns never resolve.
|
||||||
|
if _, ok := ResolveJSONColumnRef(m, "name"); ok {
|
||||||
|
t.Error("plain column must not resolve")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveJSONColumnExpr(t *testing.T) {
|
||||||
|
m := jsonCondModel{}
|
||||||
|
|
||||||
|
expr, args, alias, ok := ResolveJSONColumnExpr(m, "t", "data->'addr'->>'city'")
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("expected ok")
|
||||||
|
}
|
||||||
|
if expr != `("t"."data" #>> ?::text[])` {
|
||||||
|
t.Errorf("expr = %q", expr)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(args, []interface{}{"{addr,city}"}) {
|
||||||
|
t.Errorf("args = %#v", args)
|
||||||
|
}
|
||||||
|
if alias != "data_addr_city" {
|
||||||
|
t.Errorf("alias = %q", alias)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, _, _, ok := ResolveJSONColumnExpr(m, "t", "name"); ok {
|
||||||
|
t.Error("plain column must not resolve")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildJSONFilterCondition(t *testing.T) {
|
||||||
|
m := jsonCondModel{}
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
token string
|
||||||
|
operator string
|
||||||
|
value interface{}
|
||||||
|
wantCond string
|
||||||
|
wantArgs []interface{}
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "eq stays text", token: "data->>'city'", operator: "eq", value: "LA",
|
||||||
|
wantCond: `("data" #>> ?::text[]) = ?`,
|
||||||
|
wantArgs: []interface{}{"{city}", "LA"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "gt numeric value infers numeric cast", token: "data->>'age'", operator: "gt", value: 18,
|
||||||
|
wantCond: `(("data" #>> ?::text[]))::numeric > ?`,
|
||||||
|
wantArgs: []interface{}{"{age}", 18},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "gt non-numeric value stays text", token: "data->>'name'", operator: "gt", value: "m",
|
||||||
|
wantCond: `("data" #>> ?::text[]) > ?`,
|
||||||
|
wantArgs: []interface{}{"{name}", "m"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "explicit cast is respected for lt", token: "data->>'ts'::timestamptz", operator: "lt", value: "2020-01-01",
|
||||||
|
wantCond: `(("data" #>> ?::text[]))::timestamptz < ?`,
|
||||||
|
wantArgs: []interface{}{"{ts}", "2020-01-01"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "ilike", token: "data->>'city'", operator: "ilike", value: "%la%",
|
||||||
|
wantCond: `("data" #>> ?::text[]) ILIKE ?`,
|
||||||
|
wantArgs: []interface{}{"{city}", "%la%"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "in", token: "data->>'tier'", operator: "in", value: []string{"a", "b"},
|
||||||
|
wantCond: `("data" #>> ?::text[]) IN (?,?)`,
|
||||||
|
wantArgs: []interface{}{"{tier}", "a", "b"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "between numeric", token: "data->>'age'", operator: "between", value: []interface{}{10, 20},
|
||||||
|
wantCond: `((("data" #>> ?::text[]))::numeric > ? AND (("data" #>> ?::text[]))::numeric < ?)`,
|
||||||
|
wantArgs: []interface{}{"{age}", 10, "{age}", 20},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "is_null", token: "data->>'city'", operator: "is_null", value: nil,
|
||||||
|
wantCond: `("data" #>> ?::text[]) IS NULL`,
|
||||||
|
wantArgs: []interface{}{"{city}"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "hash path", token: "data#>>'{a,b}'", operator: "eq", value: "x",
|
||||||
|
wantCond: `("data" #>> ?::text[]) = ?`,
|
||||||
|
wantArgs: []interface{}{"{a,b}", "x"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "dotted shorthand on json column", token: "data.city", operator: "eq", value: "x",
|
||||||
|
wantCond: `("data" #>> ?::text[]) = ?`,
|
||||||
|
wantArgs: []interface{}{"{city}", "x"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range tests {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
cond, args, ok := BuildJSONFilterCondition(m, "", tc.token, tc.operator, tc.value)
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("ok=false for %q", tc.token)
|
||||||
|
}
|
||||||
|
if cond != tc.wantCond {
|
||||||
|
t.Errorf("cond = %q, want %q", cond, tc.wantCond)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(args, tc.wantArgs) {
|
||||||
|
t.Errorf("args = %#v, want %#v", args, tc.wantArgs)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildJSONFilterCondition_NotJSON(t *testing.T) {
|
||||||
|
m := jsonCondModel{}
|
||||||
|
for _, tok := range []string{"name", "id", "name.first"} {
|
||||||
|
if _, _, ok := BuildJSONFilterCondition(m, "", tok, "eq", "x"); ok {
|
||||||
|
t.Errorf("BuildJSONFilterCondition(%q) ok=true, want false", tok)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Unknown operator on a real JSON ref -> caller keeps its own handling.
|
||||||
|
if _, _, ok := BuildJSONFilterCondition(m, "", "data->>'x'", "st_intersects", "y"); ok {
|
||||||
|
t.Error("unknown operator must yield ok=false")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildJSONFilterCondition_QualifiedAndInjectionSafe(t *testing.T) {
|
||||||
|
m := jsonCondModel{}
|
||||||
|
// A hostile key never reaches the SQL string — it is bound in the text[] arg.
|
||||||
|
cond, args, ok := BuildJSONFilterCondition(m, "pub.tbl", "data->>'ev\"il'", "eq", "x")
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("ok=false")
|
||||||
|
}
|
||||||
|
if cond != `("pub"."tbl"."data" #>> ?::text[]) = ?` {
|
||||||
|
t.Errorf("cond = %q", cond)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(args, []interface{}{`{"ev\"il"}`, "x"}) {
|
||||||
|
t.Errorf("args = %#v", args)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,317 @@
|
|||||||
|
package common
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// This file implements PostGIS spatial and pgvector similarity filter operators.
|
||||||
|
// The builders return parameterised SQL fragments (with `?` placeholders) plus
|
||||||
|
// their args, matching the style of BuildInCondition / BuildArrayOverlapCondition.
|
||||||
|
// PostgreSQL only — on other databases these operators simply will not resolve.
|
||||||
|
|
||||||
|
// ── vector similarity ───────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
// VectorOperator maps a metric name to its pgvector distance operator.
|
||||||
|
//
|
||||||
|
// "l2" / "euclidean" / "" -> <->
|
||||||
|
// "cosine" -> <=>
|
||||||
|
// "ip" / "inner" / "dot" -> <#>
|
||||||
|
func VectorOperator(metric string) string {
|
||||||
|
switch strings.ToLower(strings.TrimSpace(metric)) {
|
||||||
|
case "cosine", "cos":
|
||||||
|
return "<=>"
|
||||||
|
case "ip", "inner", "dot", "innerproduct", "inner_product":
|
||||||
|
return "<#>"
|
||||||
|
default:
|
||||||
|
return "<->"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// VectorLiteral converts a vector value into a pgvector literal string
|
||||||
|
// "[1,2,3]". Accepts []float32, []float64, []int, []any (of numbers), or an
|
||||||
|
// already-formatted string.
|
||||||
|
func VectorLiteral(value any) (string, error) {
|
||||||
|
switch v := value.(type) {
|
||||||
|
case string:
|
||||||
|
s := strings.TrimSpace(v)
|
||||||
|
if strings.HasPrefix(s, "[") && strings.HasSuffix(s, "]") {
|
||||||
|
return s, nil
|
||||||
|
}
|
||||||
|
return "", fmt.Errorf("vector literal: malformed string %q", v)
|
||||||
|
case []float32:
|
||||||
|
return floatsToVectorLiteral(len(v), func(i int) float64 { return float64(v[i]) }), nil
|
||||||
|
case []float64:
|
||||||
|
return floatsToVectorLiteral(len(v), func(i int) float64 { return v[i] }), nil
|
||||||
|
case []int:
|
||||||
|
return floatsToVectorLiteral(len(v), func(i int) float64 { return float64(v[i]) }), nil
|
||||||
|
case []any:
|
||||||
|
nums := make([]float64, len(v))
|
||||||
|
for i, e := range v {
|
||||||
|
f, ok := toFloat(e)
|
||||||
|
if !ok {
|
||||||
|
return "", fmt.Errorf("vector literal: element %d is not a number (%T)", i, e)
|
||||||
|
}
|
||||||
|
nums[i] = f
|
||||||
|
}
|
||||||
|
return floatsToVectorLiteral(len(nums), func(i int) float64 { return nums[i] }), nil
|
||||||
|
default:
|
||||||
|
return "", fmt.Errorf("vector literal: unsupported type %T", value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func floatsToVectorLiteral(n int, at func(int) float64) string {
|
||||||
|
parts := make([]string, n)
|
||||||
|
for i := 0; i < n; i++ {
|
||||||
|
parts[i] = strconv.FormatFloat(at(i), 'f', -1, 32)
|
||||||
|
}
|
||||||
|
return "[" + strings.Join(parts, ",") + "]"
|
||||||
|
}
|
||||||
|
|
||||||
|
// BuildVectorCondition builds a pgvector distance-threshold filter.
|
||||||
|
//
|
||||||
|
// operator: "l2_within" | "cosine_within" | "ip_within"
|
||||||
|
// value: {"vector": [...], "distance": <n>}
|
||||||
|
// {"vector": [...], "lt"|"lte"|"gt"|"gte": <n>}
|
||||||
|
//
|
||||||
|
// Produces e.g. `embedding <=> ? < ?` with args [vectorLiteral, threshold].
|
||||||
|
func BuildVectorCondition(column, operator string, value any) (query string, args []interface{}, ok bool) {
|
||||||
|
var op string
|
||||||
|
switch strings.ToLower(operator) {
|
||||||
|
case "l2_within", "l2distance_within", "euclidean_within":
|
||||||
|
op = "<->"
|
||||||
|
case "cosine_within", "cosinedistance_within":
|
||||||
|
op = "<=>"
|
||||||
|
case "ip_within", "inner_within", "negativeinnerproduct_within":
|
||||||
|
op = "<#>"
|
||||||
|
default:
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
|
||||||
|
m, mok := value.(map[string]any)
|
||||||
|
if !mok {
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
lit, err := VectorLiteral(m["vector"])
|
||||||
|
if err != nil {
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
|
||||||
|
cmp := "<"
|
||||||
|
var threshold any
|
||||||
|
if t, ok := m["distance"]; ok {
|
||||||
|
threshold = t
|
||||||
|
} else {
|
||||||
|
for _, k := range []string{"lt", "lte", "gt", "gte"} {
|
||||||
|
if t, ok := m[k]; ok {
|
||||||
|
threshold = t
|
||||||
|
switch k {
|
||||||
|
case "lt":
|
||||||
|
cmp = "<"
|
||||||
|
case "lte":
|
||||||
|
cmp = "<="
|
||||||
|
case "gt":
|
||||||
|
cmp = ">"
|
||||||
|
case "gte":
|
||||||
|
cmp = ">="
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if threshold == nil {
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
f, fok := toFloat(threshold)
|
||||||
|
if !fok {
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
|
||||||
|
return fmt.Sprintf("%s %s ? %s ?", column, op, cmp), []interface{}{lit, f}, true
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── PostGIS spatial ─────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
var spatialPredicates = map[string]string{
|
||||||
|
"st_intersects": "ST_Intersects",
|
||||||
|
"st_contains": "ST_Contains",
|
||||||
|
"st_within": "ST_Within",
|
||||||
|
"st_covers": "ST_Covers",
|
||||||
|
"st_coveredby": "ST_CoveredBy",
|
||||||
|
"st_overlaps": "ST_Overlaps",
|
||||||
|
"st_touches": "ST_Touches",
|
||||||
|
"st_crosses": "ST_Crosses",
|
||||||
|
"st_equals": "ST_Equals",
|
||||||
|
"st_disjoint": "ST_Disjoint",
|
||||||
|
}
|
||||||
|
|
||||||
|
// BuildSpatialCondition builds a PostGIS spatial filter.
|
||||||
|
//
|
||||||
|
// "st_dwithin" value: {"geom": <geojson|ewkt|hex>, "distance": <n>}
|
||||||
|
// "st_intersects" / "st_contains" / "st_within" / "st_covers" /
|
||||||
|
// "st_coveredby" / "st_overlaps" / "st_touches" / "st_crosses" /
|
||||||
|
// "st_equals" / "st_disjoint" value: <geojson|ewkt|hex>
|
||||||
|
// "bbox" (alias "&&") value: <geom> or {"bbox":[minx,miny,maxx,maxy],"srid":4326}
|
||||||
|
func BuildSpatialCondition(column, operator string, value any) (query string, args []interface{}, ok bool) {
|
||||||
|
operator = strings.ToLower(strings.TrimSpace(operator))
|
||||||
|
|
||||||
|
if fn, isPred := spatialPredicates[operator]; isPred {
|
||||||
|
expr, arg, err := geomArgExpr(value)
|
||||||
|
if err != nil {
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("%s(%s, %s)", fn, column, expr), []interface{}{arg}, true
|
||||||
|
}
|
||||||
|
|
||||||
|
switch operator {
|
||||||
|
case "st_dwithin":
|
||||||
|
m, mok := value.(map[string]any)
|
||||||
|
if !mok {
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
expr, arg, err := geomArgExpr(m["geom"])
|
||||||
|
if err != nil {
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
dist, dok := toFloat(m["distance"])
|
||||||
|
if !dok {
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("ST_DWithin(%s, %s, ?)", column, expr), []interface{}{arg, dist}, true
|
||||||
|
|
||||||
|
case "bbox", "&&":
|
||||||
|
if m, mok := value.(map[string]any); mok {
|
||||||
|
if bboxRaw, has := m["bbox"]; has {
|
||||||
|
coords, cok := toFloatSlice(bboxRaw)
|
||||||
|
if !cok || len(coords) != 4 {
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
srid := 4326
|
||||||
|
if s, sok := toFloat(m["srid"]); sok {
|
||||||
|
srid = int(s)
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("%s && ST_MakeEnvelope(?, ?, ?, ?, ?)", column),
|
||||||
|
[]interface{}{coords[0], coords[1], coords[2], coords[3], srid}, true
|
||||||
|
}
|
||||||
|
// fall through: treat the map as a GeoJSON geometry
|
||||||
|
}
|
||||||
|
expr, arg, err := geomArgExpr(value)
|
||||||
|
if err != nil {
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("%s && %s", column, expr), []interface{}{arg}, true
|
||||||
|
}
|
||||||
|
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
|
||||||
|
// geomArgExpr inspects a geometry value and returns the SQL placeholder
|
||||||
|
// expression that turns a bound argument into a geometry, plus that argument.
|
||||||
|
//
|
||||||
|
// GeoJSON object -> "ST_GeomFromGeoJSON(?)", <json string>
|
||||||
|
// hex EWKB -> "?::geometry", <hex string>
|
||||||
|
// WKT / EWKT -> "ST_GeomFromEWKT(?)", <ewkt string>
|
||||||
|
func geomArgExpr(value any) (expr string, arg any, err error) {
|
||||||
|
switch v := value.(type) {
|
||||||
|
case nil:
|
||||||
|
return "", nil, fmt.Errorf("geometry: nil value")
|
||||||
|
case map[string]any:
|
||||||
|
b, mErr := json.Marshal(v)
|
||||||
|
if mErr != nil {
|
||||||
|
return "", nil, mErr
|
||||||
|
}
|
||||||
|
return "ST_GeomFromGeoJSON(?)", string(b), nil
|
||||||
|
case json.RawMessage:
|
||||||
|
return "ST_GeomFromGeoJSON(?)", string(v), nil
|
||||||
|
case []byte:
|
||||||
|
return geomArgExpr(string(v))
|
||||||
|
case string:
|
||||||
|
s := strings.TrimSpace(v)
|
||||||
|
if s == "" {
|
||||||
|
return "", nil, fmt.Errorf("geometry: empty value")
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(s, "{") {
|
||||||
|
return "ST_GeomFromGeoJSON(?)", s, nil
|
||||||
|
}
|
||||||
|
if isHexString(s) {
|
||||||
|
return "?::geometry", s, nil
|
||||||
|
}
|
||||||
|
return "ST_GeomFromEWKT(?)", s, nil
|
||||||
|
default:
|
||||||
|
return "", nil, fmt.Errorf("geometry: unsupported type %T", value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func isHexString(s string) bool {
|
||||||
|
if len(s) < 10 || len(s)%2 != 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
_, err := hex.DecodeString(s)
|
||||||
|
return err == nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func toFloat(v any) (float64, bool) {
|
||||||
|
switch n := v.(type) {
|
||||||
|
case float64:
|
||||||
|
return n, true
|
||||||
|
case float32:
|
||||||
|
return float64(n), true
|
||||||
|
case int:
|
||||||
|
return float64(n), true
|
||||||
|
case int64:
|
||||||
|
return float64(n), true
|
||||||
|
case json.Number:
|
||||||
|
f, err := n.Float64()
|
||||||
|
return f, err == nil
|
||||||
|
case string:
|
||||||
|
f, err := strconv.ParseFloat(strings.TrimSpace(n), 64)
|
||||||
|
return f, err == nil
|
||||||
|
default:
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func toFloatSlice(v any) ([]float64, bool) {
|
||||||
|
switch s := v.(type) {
|
||||||
|
case []float64:
|
||||||
|
return s, true
|
||||||
|
case []any:
|
||||||
|
out := make([]float64, len(s))
|
||||||
|
for i, e := range s {
|
||||||
|
f, ok := toFloat(e)
|
||||||
|
if !ok {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
out[i] = f
|
||||||
|
}
|
||||||
|
return out, true
|
||||||
|
default:
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// IsSpatialOperator reports whether op is a spatial filter operator handled by
|
||||||
|
// BuildSpatialCondition.
|
||||||
|
func IsSpatialOperator(op string) bool {
|
||||||
|
op = strings.ToLower(strings.TrimSpace(op))
|
||||||
|
if _, ok := spatialPredicates[op]; ok {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return op == "st_dwithin" || op == "bbox" || op == "&&"
|
||||||
|
}
|
||||||
|
|
||||||
|
// IsVectorOperator reports whether op is a vector similarity filter operator
|
||||||
|
// handled by BuildVectorCondition.
|
||||||
|
func IsVectorOperator(op string) bool {
|
||||||
|
switch strings.ToLower(strings.TrimSpace(op)) {
|
||||||
|
case "l2_within", "l2distance_within", "euclidean_within",
|
||||||
|
"cosine_within", "cosinedistance_within",
|
||||||
|
"ip_within", "inner_within", "negativeinnerproduct_within":
|
||||||
|
return true
|
||||||
|
default:
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,145 @@
|
|||||||
|
package common
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestVectorOperator(t *testing.T) {
|
||||||
|
cases := map[string]string{
|
||||||
|
"": "<->", "l2": "<->", "euclidean": "<->",
|
||||||
|
"cosine": "<=>", "cos": "<=>",
|
||||||
|
"ip": "<#>", "inner": "<#>", "dot": "<#>",
|
||||||
|
}
|
||||||
|
for in, want := range cases {
|
||||||
|
if got := VectorOperator(in); got != want {
|
||||||
|
t.Errorf("VectorOperator(%q) = %q, want %q", in, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVectorLiteral(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
in any
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{[]float32{1, 2, 3}, "[1,2,3]"},
|
||||||
|
{[]float64{1.5, -2}, "[1.5,-2]"},
|
||||||
|
{[]int{1, 2}, "[1,2]"},
|
||||||
|
{[]any{1.0, 2.0}, "[1,2]"},
|
||||||
|
{"[4,5,6]", "[4,5,6]"},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
got, err := VectorLiteral(c.in)
|
||||||
|
if err != nil || got != c.want {
|
||||||
|
t.Errorf("VectorLiteral(%v) = %q, %v; want %q", c.in, got, err, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := VectorLiteral("not-a-vector"); err == nil {
|
||||||
|
t.Error("expected error for malformed string")
|
||||||
|
}
|
||||||
|
if _, err := VectorLiteral(42); err == nil {
|
||||||
|
t.Error("expected error for unsupported type")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildVectorCondition(t *testing.T) {
|
||||||
|
q, args, ok := BuildVectorCondition("embedding", "cosine_within", map[string]any{
|
||||||
|
"vector": []any{1.0, 2.0, 3.0}, "distance": 0.5,
|
||||||
|
})
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("expected ok")
|
||||||
|
}
|
||||||
|
if q != "embedding <=> ? < ?" {
|
||||||
|
t.Errorf("query = %q", q)
|
||||||
|
}
|
||||||
|
if len(args) != 2 || args[0] != "[1,2,3]" || args[1] != 0.5 {
|
||||||
|
t.Errorf("args = %v", args)
|
||||||
|
}
|
||||||
|
|
||||||
|
// explicit comparator
|
||||||
|
q, _, ok = BuildVectorCondition("v", "l2_within", map[string]any{
|
||||||
|
"vector": []float32{1}, "lte": 2.0,
|
||||||
|
})
|
||||||
|
if !ok || q != "v <-> ? <= ?" {
|
||||||
|
t.Errorf("lte: q=%q ok=%v", q, ok)
|
||||||
|
}
|
||||||
|
|
||||||
|
// unknown operator
|
||||||
|
if _, _, ok := BuildVectorCondition("v", "bogus", map[string]any{}); ok {
|
||||||
|
t.Error("expected not ok for unknown operator")
|
||||||
|
}
|
||||||
|
// missing threshold
|
||||||
|
if _, _, ok := BuildVectorCondition("v", "l2_within", map[string]any{"vector": []float32{1}}); ok {
|
||||||
|
t.Error("expected not ok without threshold")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildSpatialCondition_Predicates(t *testing.T) {
|
||||||
|
q, args, ok := BuildSpatialCondition("geom", "st_intersects", "SRID=4326;POINT(0 0)")
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("expected ok")
|
||||||
|
}
|
||||||
|
if q != "ST_Intersects(geom, ST_GeomFromEWKT(?))" {
|
||||||
|
t.Errorf("query = %q", q)
|
||||||
|
}
|
||||||
|
if len(args) != 1 || args[0] != "SRID=4326;POINT(0 0)" {
|
||||||
|
t.Errorf("args = %v", args)
|
||||||
|
}
|
||||||
|
|
||||||
|
// GeoJSON value
|
||||||
|
q, args, ok = BuildSpatialCondition("geom", "st_contains", map[string]any{
|
||||||
|
"type": "Point", "coordinates": []any{1.0, 2.0},
|
||||||
|
})
|
||||||
|
if !ok || q != "ST_Contains(geom, ST_GeomFromGeoJSON(?))" {
|
||||||
|
t.Errorf("geojson: q=%q ok=%v", q, ok)
|
||||||
|
}
|
||||||
|
if len(args) != 1 {
|
||||||
|
t.Errorf("args = %v", args)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildSpatialCondition_DWithin(t *testing.T) {
|
||||||
|
q, args, ok := BuildSpatialCondition("geom", "st_dwithin", map[string]any{
|
||||||
|
"geom": "SRID=4326;POINT(0 0)", "distance": 1000.0,
|
||||||
|
})
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("expected ok")
|
||||||
|
}
|
||||||
|
if q != "ST_DWithin(geom, ST_GeomFromEWKT(?), ?)" {
|
||||||
|
t.Errorf("query = %q", q)
|
||||||
|
}
|
||||||
|
if len(args) != 2 || args[1] != 1000.0 {
|
||||||
|
t.Errorf("args = %v", args)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildSpatialCondition_BBox(t *testing.T) {
|
||||||
|
q, args, ok := BuildSpatialCondition("geom", "bbox", map[string]any{
|
||||||
|
"bbox": []any{0.0, 0.0, 10.0, 10.0}, "srid": 4326.0,
|
||||||
|
})
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("expected ok")
|
||||||
|
}
|
||||||
|
if q != "geom && ST_MakeEnvelope(?, ?, ?, ?, ?)" {
|
||||||
|
t.Errorf("query = %q", q)
|
||||||
|
}
|
||||||
|
if len(args) != 5 || args[4] != 4326 {
|
||||||
|
t.Errorf("args = %v", args)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIsSpatialAndVectorOperator(t *testing.T) {
|
||||||
|
for _, op := range []string{"st_dwithin", "st_intersects", "bbox", "&&"} {
|
||||||
|
if !IsSpatialOperator(op) {
|
||||||
|
t.Errorf("%q should be spatial", op)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, op := range []string{"l2_within", "cosine_within", "ip_within"} {
|
||||||
|
if !IsVectorOperator(op) {
|
||||||
|
t.Errorf("%q should be vector", op)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if IsSpatialOperator("eq") || IsVectorOperator("eq") {
|
||||||
|
t.Error("eq is neither spatial nor vector")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -42,6 +42,10 @@ type RequestOptions struct {
|
|||||||
CursorBackward string `json:"cursor_backward"`
|
CursorBackward string `json:"cursor_backward"`
|
||||||
FetchRowNumber *string `json:"fetch_row_number"`
|
FetchRowNumber *string `json:"fetch_row_number"`
|
||||||
|
|
||||||
|
// VectorSearch performs a pgvector nearest-neighbour ordering (KNN) and
|
||||||
|
// optionally returns the computed distance as an extra column.
|
||||||
|
VectorSearch *VectorSearchOption `json:"vector_search"`
|
||||||
|
|
||||||
// Join table aliases (used for validation of prefixed columns in filters/sorts)
|
// Join table aliases (used for validation of prefixed columns in filters/sorts)
|
||||||
// Not serialized to JSON as it's internal validation state
|
// Not serialized to JSON as it's internal validation state
|
||||||
JoinAliases []string `json:"-"`
|
JoinAliases []string `json:"-"`
|
||||||
@@ -114,6 +118,17 @@ func ResolveSortColumns(sort []SortOption, pkName string) []SortOption {
|
|||||||
return resolved
|
return resolved
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// VectorSearchOption describes a pgvector KNN search: order rows by the distance
|
||||||
|
// between Column and Vector using Metric, and (when As is set) select that
|
||||||
|
// distance as an additional result column.
|
||||||
|
type VectorSearchOption struct {
|
||||||
|
Column string `json:"column"`
|
||||||
|
Vector []float32 `json:"vector"`
|
||||||
|
Metric string `json:"metric"` // "l2" (default) | "cosine" | "ip"
|
||||||
|
As string `json:"as"` // distance column alias; default "_distance"
|
||||||
|
Direction string `json:"direction"` // "asc" (default) | "desc"
|
||||||
|
}
|
||||||
|
|
||||||
type CustomOperator struct {
|
type CustomOperator struct {
|
||||||
Name string `json:"name"`
|
Name string `json:"name"`
|
||||||
SQL string `json:"sql"`
|
SQL string `json:"sql"`
|
||||||
|
|||||||
@@ -109,6 +109,19 @@ func (v *ColumnValidator) ValidateColumn(column string) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// JSON-traversing references (data->>'x', data#>>'{a,b}', or the dotted
|
||||||
|
// data.x shorthand): validate the base column, and for the ambiguous
|
||||||
|
// dotted form require that the base is actually a JSON column.
|
||||||
|
if ref, isJSON := ParseColumnRef(column); isJSON {
|
||||||
|
if ref.Ambiguous && !reflection.IsJSONColumn(v.model, ref.Base) {
|
||||||
|
return fmt.Errorf("invalid column '%s': '%s' is not a JSON column", column, ref.Base)
|
||||||
|
}
|
||||||
|
if _, exists := v.validColumns[strings.ToLower(ref.Base)]; !exists {
|
||||||
|
return fmt.Errorf("invalid column '%s': column does not exist in model", column)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
// Extract source column name (remove JSON operators like ->> or ->)
|
// Extract source column name (remove JSON operators like ->> or ->)
|
||||||
sourceColumn := reflection.ExtractSourceColumn(column)
|
sourceColumn := reflection.ExtractSourceColumn(column)
|
||||||
|
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"github.com/bitechdev/ResolveSpec/pkg/reflection"
|
"github.com/bitechdev/ResolveSpec/pkg/reflection"
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/spectypes"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestExtractSourceColumn(t *testing.T) {
|
func TestExtractSourceColumn(t *testing.T) {
|
||||||
@@ -124,3 +125,35 @@ func TestValidateColumnWithJSONOperators(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestValidateColumn_JSONPathsAndDottedShorthand(t *testing.T) {
|
||||||
|
type Model struct {
|
||||||
|
ID int64 `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Data spectypes.SqlJSONB `json:"data"`
|
||||||
|
}
|
||||||
|
v := NewColumnValidator(Model{})
|
||||||
|
|
||||||
|
valid := []string{
|
||||||
|
"data->>'city'",
|
||||||
|
"data->'addr'->>'city'",
|
||||||
|
"data#>>'{addr,city}'",
|
||||||
|
"data.addr.city", // dotted shorthand, base is JSON -> allowed
|
||||||
|
"(data->>'age')::int", // cast + paren
|
||||||
|
}
|
||||||
|
for _, c := range valid {
|
||||||
|
if err := v.ValidateColumn(c); err != nil {
|
||||||
|
t.Errorf("ValidateColumn(%q) = %v, want nil", c, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
invalid := []string{
|
||||||
|
"nope->>'city'", // base column does not exist
|
||||||
|
"name.first", // dotted shorthand but 'name' is not a JSON column
|
||||||
|
}
|
||||||
|
for _, c := range invalid {
|
||||||
|
if err := v.ValidateColumn(c); err == nil {
|
||||||
|
t.Errorf("ValidateColumn(%q) = nil, want error", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -51,7 +51,7 @@ func (h *Handler) ParseParameters(r *http.Request) *RequestParameters {
|
|||||||
FieldFilters: make(map[string]string),
|
FieldFilters: make(map[string]string),
|
||||||
SearchFilters: make(map[string]string),
|
SearchFilters: make(map[string]string),
|
||||||
SearchOps: make(map[string]FilterOperator),
|
SearchOps: make(map[string]FilterOperator),
|
||||||
Limit: 20, // Default limit
|
Limit: 100000, // Default limit
|
||||||
Offset: 0, // Default offset
|
Offset: 0, // Default offset
|
||||||
ResponseFormat: "simple", // Default format
|
ResponseFormat: "simple", // Default format
|
||||||
ComplexAPI: false, // Default to simple API
|
ComplexAPI: false, // Default to simple API
|
||||||
|
|||||||
+14
-2
@@ -676,7 +676,7 @@ func (h *Handler) readByID(hookCtx *HookContext) (interface{}, error) {
|
|||||||
|
|
||||||
// Apply columns
|
// Apply columns
|
||||||
if hookCtx.Options != nil && len(hookCtx.Options.Columns) > 0 {
|
if hookCtx.Options != nil && len(hookCtx.Options.Columns) > 0 {
|
||||||
query = query.Column(hookCtx.Options.Columns...)
|
query = common.ApplySelectColumns(query, hookCtx.Model, "", hookCtx.Options.Columns)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Apply preloads (simplified)
|
// Apply preloads (simplified)
|
||||||
@@ -714,6 +714,10 @@ func (h *Handler) readMultiple(hookCtx *HookContext) (data interface{}, metadata
|
|||||||
if hookCtx.Options != nil {
|
if hookCtx.Options != nil {
|
||||||
// Apply filters
|
// Apply filters
|
||||||
for _, filter := range hookCtx.Options.Filters {
|
for _, filter := range hookCtx.Options.Filters {
|
||||||
|
if cond, jargs, ok := common.BuildJSONFilterCondition(hookCtx.Model, "", filter.Column, filter.Operator, filter.Value); ok {
|
||||||
|
query = query.Where(cond, jargs...)
|
||||||
|
continue
|
||||||
|
}
|
||||||
op := strings.ToLower(filter.Operator)
|
op := strings.ToLower(filter.Operator)
|
||||||
if op == "like" || op == "ilike" {
|
if op == "like" || op == "ilike" {
|
||||||
query = query.Where(fmt.Sprintf("CAST(%s AS TEXT) %s ?", filter.Column, h.getOperatorSQL(filter.Operator)), filter.Value)
|
query = query.Where(fmt.Sprintf("CAST(%s AS TEXT) %s ?", filter.Column, h.getOperatorSQL(filter.Operator)), filter.Value)
|
||||||
@@ -728,6 +732,10 @@ func (h *Handler) readMultiple(hookCtx *HookContext) (data interface{}, metadata
|
|||||||
if sort.Direction == "desc" {
|
if sort.Direction == "desc" {
|
||||||
direction = "DESC"
|
direction = "DESC"
|
||||||
}
|
}
|
||||||
|
if expr, jargs, _, ok := common.ResolveJSONColumnExpr(hookCtx.Model, "", sort.Column); ok {
|
||||||
|
query = query.OrderExpr(fmt.Sprintf("%s %s", expr, direction), jargs...)
|
||||||
|
continue
|
||||||
|
}
|
||||||
query = query.Order(fmt.Sprintf("%s %s", sort.Column, direction))
|
query = query.Order(fmt.Sprintf("%s %s", sort.Column, direction))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -746,7 +754,7 @@ func (h *Handler) readMultiple(hookCtx *HookContext) (data interface{}, metadata
|
|||||||
|
|
||||||
// Apply columns
|
// Apply columns
|
||||||
if len(hookCtx.Options.Columns) > 0 {
|
if len(hookCtx.Options.Columns) > 0 {
|
||||||
query = query.Column(hookCtx.Options.Columns...)
|
query = common.ApplySelectColumns(query, hookCtx.Model, "", hookCtx.Options.Columns)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -772,6 +780,10 @@ func (h *Handler) readMultiple(hookCtx *HookContext) (data interface{}, metadata
|
|||||||
countQuery := h.db.NewSelect().Model(hookCtx.ModelPtr).Table(hookCtx.TableName)
|
countQuery := h.db.NewSelect().Model(hookCtx.ModelPtr).Table(hookCtx.TableName)
|
||||||
if hookCtx.Options != nil {
|
if hookCtx.Options != nil {
|
||||||
for _, filter := range hookCtx.Options.Filters {
|
for _, filter := range hookCtx.Options.Filters {
|
||||||
|
if cond, jargs, ok := common.BuildJSONFilterCondition(hookCtx.Model, "", filter.Column, filter.Operator, filter.Value); ok {
|
||||||
|
countQuery = countQuery.Where(cond, jargs...)
|
||||||
|
continue
|
||||||
|
}
|
||||||
op := strings.ToLower(filter.Operator)
|
op := strings.ToLower(filter.Operator)
|
||||||
if op == "like" || op == "ilike" {
|
if op == "like" || op == "ilike" {
|
||||||
countQuery = countQuery.Where(fmt.Sprintf("CAST(%s AS TEXT) %s ?", filter.Column, h.getOperatorSQL(filter.Operator)), filter.Value)
|
countQuery = countQuery.Where(fmt.Sprintf("CAST(%s AS TEXT) %s ?", filter.Column, h.getOperatorSQL(filter.Operator)), filter.Value)
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"github.com/bitechdev/ResolveSpec/pkg/modelregistry"
|
"github.com/bitechdev/ResolveSpec/pkg/modelregistry"
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/spectypes"
|
||||||
)
|
)
|
||||||
|
|
||||||
// OpenAPISpec represents the OpenAPI 3.0 specification structure
|
// OpenAPISpec represents the OpenAPI 3.0 specification structure
|
||||||
@@ -440,6 +441,28 @@ func (g *Generator) generatePropertySchema(field reflect.StructField) *Schema {
|
|||||||
schema.Description = desc
|
schema.Description = desc
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// spectypes PostGIS / pgvector wrappers get dedicated schemas.
|
||||||
|
if n, ok := spectypes.SQLTypeName(field.Type); ok {
|
||||||
|
switch n {
|
||||||
|
case "geometry", "geography":
|
||||||
|
schema.Type = "object"
|
||||||
|
schema.Format = "geojson"
|
||||||
|
return schema
|
||||||
|
case "vector", "halfvec":
|
||||||
|
schema.Type = "array"
|
||||||
|
schema.Items = &Schema{Type: "number"}
|
||||||
|
schema.Format = "vector"
|
||||||
|
return schema
|
||||||
|
case "sparsevec":
|
||||||
|
schema.Type = "object"
|
||||||
|
schema.Format = "sparsevec"
|
||||||
|
return schema
|
||||||
|
case "bit":
|
||||||
|
schema.Type = "string"
|
||||||
|
return schema
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
switch fieldType.Kind() {
|
switch fieldType.Kind() {
|
||||||
case reflect.String:
|
case reflect.String:
|
||||||
schema.Type = "string"
|
schema.Type = "string"
|
||||||
|
|||||||
@@ -0,0 +1,56 @@
|
|||||||
|
package reflection
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/spectypes"
|
||||||
|
)
|
||||||
|
|
||||||
|
type geoModel struct {
|
||||||
|
ID int64 `json:"id"`
|
||||||
|
Location spectypes.SqlGeometry `json:"location"`
|
||||||
|
Area spectypes.SqlGeography `json:"area"`
|
||||||
|
Embedding spectypes.SqlVector `json:"embedding"`
|
||||||
|
HalfEmb spectypes.SqlHalfVector `json:"half_emb"`
|
||||||
|
Name spectypes.SqlString `json:"name"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGetColumnSQLTypeName(t *testing.T) {
|
||||||
|
m := geoModel{}
|
||||||
|
cases := map[string]string{
|
||||||
|
"location": "geometry",
|
||||||
|
"area": "geography",
|
||||||
|
"embedding": "vector",
|
||||||
|
"half_emb": "halfvec",
|
||||||
|
"name": "text",
|
||||||
|
}
|
||||||
|
for col, want := range cases {
|
||||||
|
got, ok := GetColumnSQLTypeName(m, col)
|
||||||
|
if !ok || got != want {
|
||||||
|
t.Errorf("GetColumnSQLTypeName(%q) = %q, %v; want %q", col, got, ok, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, ok := GetColumnSQLTypeName(m, "id"); ok {
|
||||||
|
t.Error("id is not a spectypes column")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIsSpatialColumn(t *testing.T) {
|
||||||
|
m := geoModel{}
|
||||||
|
if !IsSpatialColumn(m, "location") || !IsSpatialColumn(m, "area") {
|
||||||
|
t.Error("location/area should be spatial")
|
||||||
|
}
|
||||||
|
if IsSpatialColumn(m, "embedding") || IsSpatialColumn(m, "name") {
|
||||||
|
t.Error("embedding/name should not be spatial")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIsVectorColumn(t *testing.T) {
|
||||||
|
m := geoModel{}
|
||||||
|
if !IsVectorColumn(m, "embedding") || !IsVectorColumn(m, "half_emb") {
|
||||||
|
t.Error("embedding/half_emb should be vector")
|
||||||
|
}
|
||||||
|
if IsVectorColumn(m, "location") || IsVectorColumn(m, "name") {
|
||||||
|
t.Error("location/name should not be vector")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,166 @@
|
|||||||
|
package reflection
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/spectypes"
|
||||||
|
)
|
||||||
|
|
||||||
|
// getColumnStructField resolves the struct field that backs colName (matched by
|
||||||
|
// json tag, field name or snake_case), following the same rules as
|
||||||
|
// GetColumnTypeFromModel.
|
||||||
|
func getColumnStructField(model interface{}, colName string) (reflect.StructField, bool) {
|
||||||
|
if model == nil {
|
||||||
|
return reflect.StructField{}, false
|
||||||
|
}
|
||||||
|
sourceColName := ExtractSourceColumn(colName)
|
||||||
|
|
||||||
|
modelType := reflect.TypeOf(model)
|
||||||
|
for modelType != nil && modelType.Kind() == reflect.Pointer {
|
||||||
|
modelType = modelType.Elem()
|
||||||
|
}
|
||||||
|
if modelType == nil || modelType.Kind() != reflect.Struct {
|
||||||
|
return reflect.StructField{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
for i := 0; i < modelType.NumField(); i++ {
|
||||||
|
field := modelType.Field(i)
|
||||||
|
|
||||||
|
if jsonTag := field.Tag.Get("json"); jsonTag != "" {
|
||||||
|
if name := jsonTagName(jsonTag); name == sourceColName {
|
||||||
|
return field, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if equalFold(field.Name, sourceColName) {
|
||||||
|
return field, true
|
||||||
|
}
|
||||||
|
if ToSnakeCase(field.Name) == sourceColName {
|
||||||
|
return field, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return reflect.StructField{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
// getColumnFieldType resolves the reflect.Type of the struct field that backs
|
||||||
|
// colName (matched by json tag, field name or snake_case), following the same
|
||||||
|
// rules as GetColumnTypeFromModel.
|
||||||
|
func getColumnFieldType(model interface{}, colName string) (reflect.Type, bool) {
|
||||||
|
f, ok := getColumnStructField(model, colName)
|
||||||
|
if !ok {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
return f.Type, true
|
||||||
|
}
|
||||||
|
|
||||||
|
func jsonTagName(tag string) string {
|
||||||
|
for i := 0; i < len(tag); i++ {
|
||||||
|
if tag[i] == ',' {
|
||||||
|
return tag[:i]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return tag
|
||||||
|
}
|
||||||
|
|
||||||
|
func equalFold(a, b string) bool {
|
||||||
|
if len(a) != len(b) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for i := 0; i < len(a); i++ {
|
||||||
|
ca, cb := a[i], b[i]
|
||||||
|
if 'A' <= ca && ca <= 'Z' {
|
||||||
|
ca += 'a' - 'A'
|
||||||
|
}
|
||||||
|
if 'A' <= cb && cb <= 'Z' {
|
||||||
|
cb += 'a' - 'A'
|
||||||
|
}
|
||||||
|
if ca != cb {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetColumnSQLTypeName returns the canonical PostgreSQL type name for a column
|
||||||
|
// backed by a spectypes wrapper (e.g. "geometry", "vector", "jsonb"), or
|
||||||
|
// ("", false) if the column is not found or not a spectypes type.
|
||||||
|
func GetColumnSQLTypeName(model interface{}, colName string) (string, bool) {
|
||||||
|
t, ok := getColumnFieldType(model, colName)
|
||||||
|
if !ok {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return spectypes.SQLTypeName(t)
|
||||||
|
}
|
||||||
|
|
||||||
|
// IsSpatialColumn reports whether colName is backed by a PostGIS
|
||||||
|
// geometry/geography wrapper.
|
||||||
|
func IsSpatialColumn(model interface{}, colName string) bool {
|
||||||
|
t, ok := getColumnFieldType(model, colName)
|
||||||
|
return ok && spectypes.IsSpatialType(t)
|
||||||
|
}
|
||||||
|
|
||||||
|
// IsVectorColumn reports whether colName is backed by a pgvector wrapper.
|
||||||
|
func IsVectorColumn(model interface{}, colName string) bool {
|
||||||
|
t, ok := getColumnFieldType(model, colName)
|
||||||
|
return ok && spectypes.IsVectorType(t)
|
||||||
|
}
|
||||||
|
|
||||||
|
var rawMessageType = reflect.TypeOf(json.RawMessage(nil))
|
||||||
|
|
||||||
|
// IsJSONColumn reports whether colName is backed by a JSON/JSONB column on the
|
||||||
|
// model. It recognises the spectypes SqlJSONB wrapper, encoding/json.RawMessage,
|
||||||
|
// map-typed fields, and fields carrying a bun/gorm `type:json` / `type:jsonb`
|
||||||
|
// tag. colName should be a bare column name (callers pass the parsed base column
|
||||||
|
// of a JSON path, not the full "col->>'x'" expression).
|
||||||
|
func IsJSONColumn(model interface{}, colName string) bool {
|
||||||
|
f, ok := getColumnStructField(model, colName)
|
||||||
|
if !ok {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
ft := f.Type
|
||||||
|
for ft != nil && ft.Kind() == reflect.Pointer {
|
||||||
|
ft = ft.Elem()
|
||||||
|
}
|
||||||
|
if ft == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
if spectypes.IsJSONType(ft) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if ft == rawMessageType {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if ft.Kind() == reflect.Map {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if tagDeclaresJSON(f.Tag.Get("bun")) || tagDeclaresJSON(f.Tag.Get("gorm")) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// tagDeclaresJSON reports whether an ORM struct tag declares a json/jsonb column
|
||||||
|
// type, e.g. `bun:"meta,type:jsonb"` or `gorm:"column:meta;type:json"`.
|
||||||
|
func tagDeclaresJSON(tag string) bool {
|
||||||
|
if tag == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, part := range strings.FieldsFunc(tag, func(r rune) bool {
|
||||||
|
return r == ',' || r == ';' || r == ' '
|
||||||
|
}) {
|
||||||
|
value, found := strings.CutPrefix(strings.TrimSpace(part), "type:")
|
||||||
|
if !found {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
value = strings.ToLower(strings.TrimSpace(value))
|
||||||
|
// Match "json" and "jsonb", including parametrised forms just in case.
|
||||||
|
if value == "json" || value == "jsonb" ||
|
||||||
|
strings.HasPrefix(value, "json(") || strings.HasPrefix(value, "jsonb(") {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
package reflection
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/spectypes"
|
||||||
|
)
|
||||||
|
|
||||||
|
type jsonColModel struct {
|
||||||
|
ID int64 `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Meta spectypes.SqlJSONB `json:"meta"`
|
||||||
|
Raw json.RawMessage `json:"raw"`
|
||||||
|
Attrs map[string]interface{} `json:"attrs"`
|
||||||
|
Config []byte `json:"config" bun:"config,type:jsonb"`
|
||||||
|
Settings string `json:"settings" gorm:"column:settings;type:json"`
|
||||||
|
Blob []byte `json:"blob"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIsJSONColumn(t *testing.T) {
|
||||||
|
m := jsonColModel{}
|
||||||
|
|
||||||
|
jsonCols := []string{"meta", "raw", "attrs", "config", "settings"}
|
||||||
|
for _, c := range jsonCols {
|
||||||
|
if !IsJSONColumn(m, c) {
|
||||||
|
t.Errorf("expected %q to be a JSON column", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
notJSON := []string{"id", "name", "blob", "missing"}
|
||||||
|
for _, c := range notJSON {
|
||||||
|
if IsJSONColumn(m, c) {
|
||||||
|
t.Errorf("expected %q NOT to be a JSON column", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if IsJSONColumn(nil, "meta") {
|
||||||
|
t.Error("nil model must not report JSON columns")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTagDeclaresJSON(t *testing.T) {
|
||||||
|
cases := map[string]bool{
|
||||||
|
"config,type:jsonb": true,
|
||||||
|
"column:settings;type:json": true,
|
||||||
|
"col,type:text": false,
|
||||||
|
"column:name": false,
|
||||||
|
"": false,
|
||||||
|
"col,type:jsonb,notnull": true,
|
||||||
|
"column:x;type:varchar(255)": false,
|
||||||
|
"col , type:json": true,
|
||||||
|
}
|
||||||
|
for tag, want := range cases {
|
||||||
|
if got := tagDeclaresJSON(tag); got != want {
|
||||||
|
t.Errorf("tagDeclaresJSON(%q) = %v; want %v", tag, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -340,18 +340,28 @@ func (h *Handler) executeRead(ctx context.Context, schema, entity, id string, op
|
|||||||
|
|
||||||
var data interface{}
|
var data interface{}
|
||||||
if id != "" {
|
if id != "" {
|
||||||
singleResult := reflect.New(modelType).Interface()
|
pkName := reflection.GetPrimaryKeyName(model)
|
||||||
pkName := reflection.GetPrimaryKeyName(singleResult)
|
|
||||||
query = query.Where(fmt.Sprintf("%s = ?", common.QuoteIdent(pkName)), id)
|
query = query.Where(fmt.Sprintf("%s = ?", common.QuoteIdent(pkName)), id)
|
||||||
if err := query.Scan(ctx, singleResult); err != nil {
|
// Scan through the model configured on the query. Bun rejects Scan with
|
||||||
|
// a destination when the query preloads a has-many relation.
|
||||||
|
if err := query.ScanModel(ctx); err != nil {
|
||||||
if err == sql.ErrNoRows {
|
if err == sql.ErrNoRows {
|
||||||
return nil, nil, fmt.Errorf("record not found")
|
return nil, nil, fmt.Errorf("record not found")
|
||||||
}
|
}
|
||||||
return nil, nil, fmt.Errorf("query error: %w", err)
|
return nil, nil, fmt.Errorf("query error: %w", err)
|
||||||
}
|
}
|
||||||
data = singleResult
|
|
||||||
|
// The configured model is a slice so the same query construction works
|
||||||
|
// for both collection and single-record reads. Extract its one result.
|
||||||
|
scannedResults := reflect.ValueOf(modelPtr).Elem()
|
||||||
|
if scannedResults.Len() == 0 {
|
||||||
|
return nil, nil, fmt.Errorf("record not found")
|
||||||
|
}
|
||||||
|
data = scannedResults.Index(0).Interface()
|
||||||
} else {
|
} else {
|
||||||
if err := query.Scan(ctx, modelPtr); err != nil {
|
// Use the model already configured on the query. This is required by
|
||||||
|
// Bun whenever the query includes a has-many preload.
|
||||||
|
if err := query.ScanModel(ctx); err != nil {
|
||||||
return nil, nil, fmt.Errorf("query error: %w", err)
|
return nil, nil, fmt.Errorf("query error: %w", err)
|
||||||
}
|
}
|
||||||
data = reflect.ValueOf(modelPtr).Elem().Interface()
|
data = reflect.ValueOf(modelPtr).Elem().Interface()
|
||||||
|
|||||||
+31
-3
@@ -85,13 +85,19 @@ func buildModelInfo(schema, entity string, model interface{}) modelInfo {
|
|||||||
|
|
||||||
// Skip relation fields (slice or user-defined struct that isn't time.Time).
|
// Skip relation fields (slice or user-defined struct that isn't time.Time).
|
||||||
fieldType, found := modelType.FieldByName(d.Name)
|
fieldType, found := modelType.FieldByName(d.Name)
|
||||||
|
var unwrappedType reflect.Type
|
||||||
|
isSQLType := false
|
||||||
if found {
|
if found {
|
||||||
ft := fieldType.Type
|
ft := fieldType.Type
|
||||||
if ft.Kind() == reflect.Pointer {
|
if sqlType, ok := unwrapSQLType(ft); ok {
|
||||||
|
unwrappedType = sqlType
|
||||||
|
ft = sqlType
|
||||||
|
isSQLType = true
|
||||||
|
} else if ft.Kind() == reflect.Pointer {
|
||||||
ft = ft.Elem()
|
ft = ft.Elem()
|
||||||
}
|
}
|
||||||
isUserStruct := ft.Kind() == reflect.Struct && ft.Name() != "Time" && ft.PkgPath() != ""
|
isUserStruct := ft.Kind() == reflect.Struct && ft.Name() != "Time" && ft.PkgPath() != ""
|
||||||
if ft.Kind() == reflect.Slice || isUserStruct {
|
if !isSQLType && (ft.Kind() == reflect.Slice || isUserStruct) {
|
||||||
info.relationNames = append(info.relationNames, jsonName)
|
info.relationNames = append(info.relationNames, jsonName)
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
@@ -104,6 +110,9 @@ func buildModelInfo(schema, entity string, model interface{}) modelInfo {
|
|||||||
|
|
||||||
// Derive Go type name, unwrapping pointer if needed.
|
// Derive Go type name, unwrapping pointer if needed.
|
||||||
goType := d.DataType
|
goType := d.DataType
|
||||||
|
if isSQLType {
|
||||||
|
goType = unwrappedType.Name()
|
||||||
|
}
|
||||||
if goType == "" && found {
|
if goType == "" && found {
|
||||||
ft := fieldType.Type
|
ft := fieldType.Type
|
||||||
for ft.Kind() == reflect.Pointer {
|
for ft.Kind() == reflect.Pointer {
|
||||||
@@ -125,7 +134,7 @@ func buildModelInfo(schema, entity string, model interface{}) modelInfo {
|
|||||||
isPrimary: isPrimary,
|
isPrimary: isPrimary,
|
||||||
isUnique: d.SQLKey == "unique" || d.SQLKey == "uniqueindex",
|
isUnique: d.SQLKey == "unique" || d.SQLKey == "uniqueindex",
|
||||||
isFK: d.SQLKey == "foreign_key",
|
isFK: d.SQLKey == "foreign_key",
|
||||||
nullable: d.Nullable,
|
nullable: isSQLType || d.Nullable,
|
||||||
}
|
}
|
||||||
info.columns = append(info.columns, ci)
|
info.columns = append(info.columns, ci)
|
||||||
}
|
}
|
||||||
@@ -134,6 +143,25 @@ func buildModelInfo(schema, entity string, model interface{}) modelInfo {
|
|||||||
return info
|
return info
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// unwrapSQLType returns the value type wrapped by a spectypes SQL value. These
|
||||||
|
// types are scalar columns even when their Go representation is a struct or a
|
||||||
|
// slice (for example, SqlNull[string] and SqlJSONB).
|
||||||
|
func unwrapSQLType(t reflect.Type) (reflect.Type, bool) {
|
||||||
|
for t.Kind() == reflect.Pointer {
|
||||||
|
t = t.Elem()
|
||||||
|
}
|
||||||
|
|
||||||
|
if t.PkgPath() != "github.com/bitechdev/ResolveSpec/pkg/spectypes" {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
if t.Kind() == reflect.Struct {
|
||||||
|
if value, ok := t.FieldByName("Val"); ok {
|
||||||
|
return value.Type, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return t, true
|
||||||
|
}
|
||||||
|
|
||||||
// fieldJSONName returns the JSON tag name for a struct field, falling back to the field name.
|
// fieldJSONName returns the JSON tag name for a struct field, falling back to the field name.
|
||||||
func fieldJSONName(modelType reflect.Type, fieldName string) string {
|
func fieldJSONName(modelType reflect.Type, fieldName string) string {
|
||||||
field, ok := modelType.FieldByName(fieldName)
|
field, ok := modelType.FieldByName(fieldName)
|
||||||
|
|||||||
@@ -0,0 +1,34 @@
|
|||||||
|
package resolvemcp
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/spectypes"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestBuildModelInfo_UnwrapsSQLTypes(t *testing.T) {
|
||||||
|
type related struct {
|
||||||
|
ID int64 `json:"id"`
|
||||||
|
}
|
||||||
|
type model struct {
|
||||||
|
Name spectypes.SqlString `gorm:"column:name" json:"name"`
|
||||||
|
Metadata spectypes.SqlJSONB `gorm:"column:metadata;type:jsonb" json:"metadata"`
|
||||||
|
Related related `json:"related"`
|
||||||
|
}
|
||||||
|
|
||||||
|
info := buildModelInfo("public", "models", model{})
|
||||||
|
columns := make(map[string]columnInfo, len(info.columns))
|
||||||
|
for _, column := range info.columns {
|
||||||
|
columns[column.jsonName] = column
|
||||||
|
}
|
||||||
|
|
||||||
|
if column, ok := columns["name"]; !ok || column.goType != "string" || !column.nullable {
|
||||||
|
t.Errorf("expected name SQL wrapper column, got %+v", column)
|
||||||
|
}
|
||||||
|
if column, ok := columns["metadata"]; !ok || !column.nullable {
|
||||||
|
t.Errorf("expected metadata SQL wrapper column, got %+v", column)
|
||||||
|
}
|
||||||
|
if len(info.relationNames) != 1 || info.relationNames[0] != "related" {
|
||||||
|
t.Errorf("expected only related to be a relation, got %v", info.relationNames)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -87,11 +87,48 @@ func TestBuildFilterCondition(t *testing.T) {
|
|||||||
expectedCondition: "",
|
expectedCondition: "",
|
||||||
expectedArgsCount: 0,
|
expectedArgsCount: 0,
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
name: "st_dwithin spatial operator",
|
||||||
|
filter: common.FilterOption{
|
||||||
|
Column: "geom",
|
||||||
|
Operator: "st_dwithin",
|
||||||
|
Value: map[string]any{
|
||||||
|
"geom": "SRID=4326;POINT(0 0)",
|
||||||
|
"distance": 1000.0,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
expectedCondition: "ST_DWithin(geom, ST_GeomFromEWKT(?), ?)",
|
||||||
|
expectedArgsCount: 2,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "st_intersects spatial operator",
|
||||||
|
filter: common.FilterOption{
|
||||||
|
Column: "geom",
|
||||||
|
Operator: "st_intersects",
|
||||||
|
Value: "SRID=4326;POLYGON((0 0,1 0,1 1,0 1,0 0))",
|
||||||
|
LogicOperator: "",
|
||||||
|
},
|
||||||
|
expectedCondition: "ST_Intersects(geom, ST_GeomFromEWKT(?))",
|
||||||
|
expectedArgsCount: 1,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "l2_within vector operator",
|
||||||
|
filter: common.FilterOption{
|
||||||
|
Column: "embedding",
|
||||||
|
Operator: "l2_within",
|
||||||
|
Value: map[string]any{
|
||||||
|
"vector": []any{1.0, 2.0, 3.0},
|
||||||
|
"distance": 0.5,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
expectedCondition: "embedding <-> ? < ?",
|
||||||
|
expectedArgsCount: 2,
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, tt := range tests {
|
for _, tt := range tests {
|
||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
condition, args := h.buildFilterCondition(tt.filter)
|
condition, args := h.buildFilterCondition(tt.filter, nil)
|
||||||
|
|
||||||
if condition != tt.expectedCondition {
|
if condition != tt.expectedCondition {
|
||||||
t.Errorf("Expected condition '%s', got '%s'", tt.expectedCondition, condition)
|
t.Errorf("Expected condition '%s', got '%s'", tt.expectedCondition, condition)
|
||||||
|
|||||||
+156
-26
@@ -16,6 +16,7 @@ import (
|
|||||||
"github.com/bitechdev/ResolveSpec/pkg/common"
|
"github.com/bitechdev/ResolveSpec/pkg/common"
|
||||||
"github.com/bitechdev/ResolveSpec/pkg/logger"
|
"github.com/bitechdev/ResolveSpec/pkg/logger"
|
||||||
"github.com/bitechdev/ResolveSpec/pkg/reflection"
|
"github.com/bitechdev/ResolveSpec/pkg/reflection"
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/spectypes"
|
||||||
)
|
)
|
||||||
|
|
||||||
// FallbackHandler is a function that handles requests when no model is found
|
// FallbackHandler is a function that handles requests when no model is found
|
||||||
@@ -332,8 +333,13 @@ func (h *Handler) handleRead(ctx context.Context, w common.ResponseWriter, id st
|
|||||||
query = query.Table(tableName)
|
query = query.Table(tableName)
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(options.Columns) == 0 && (len(options.ComputedColumns) > 0) {
|
vectorSearchActive := options.VectorSearch != nil &&
|
||||||
logger.Debug("Populating options.Columns with all model columns since computed columns are additions")
|
options.VectorSearch.Column != "" && len(options.VectorSearch.Vector) > 0
|
||||||
|
|
||||||
|
if len(options.Columns) == 0 &&
|
||||||
|
(len(options.ComputedColumns) > 0 ||
|
||||||
|
(vectorSearchActive && options.VectorSearch.As != "")) {
|
||||||
|
logger.Debug("Populating options.Columns with all model columns since computed/vector columns are additions")
|
||||||
options.Columns = reflection.GetSQLModelColumns(model)
|
options.Columns = reflection.GetSQLModelColumns(model)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -341,6 +347,10 @@ func (h *Handler) handleRead(ctx context.Context, w common.ResponseWriter, id st
|
|||||||
if len(options.Columns) > 0 {
|
if len(options.Columns) > 0 {
|
||||||
logger.Debug("Selecting columns: %v", options.Columns)
|
logger.Debug("Selecting columns: %v", options.Columns)
|
||||||
for _, col := range options.Columns {
|
for _, col := range options.Columns {
|
||||||
|
if expr, jargs, alias, ok := common.ResolveJSONColumnExpr(model, "", col); ok {
|
||||||
|
query = query.ColumnExpr(expr+" AS "+common.QuoteIdent(alias), jargs...)
|
||||||
|
continue
|
||||||
|
}
|
||||||
query = query.Column(reflection.ExtractSourceColumn(col))
|
query = query.Column(reflection.ExtractSourceColumn(col))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -352,6 +362,29 @@ func (h *Handler) handleRead(ctx context.Context, w common.ResponseWriter, id st
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// pgvector KNN search: order by distance to the query vector and,
|
||||||
|
// optionally, return that distance as an extra column. Postgres only.
|
||||||
|
if vectorSearchActive {
|
||||||
|
vs := options.VectorSearch
|
||||||
|
op := common.VectorOperator(vs.Metric)
|
||||||
|
lit, litErr := common.VectorLiteral(vs.Vector)
|
||||||
|
if litErr != nil {
|
||||||
|
logger.Error("Invalid vector search vector: %v", litErr)
|
||||||
|
statusCode, errCode, errMsg = http.StatusBadRequest, "invalid_vector_search", "Invalid vector search vector"
|
||||||
|
return litErr
|
||||||
|
}
|
||||||
|
col := common.QuoteIdent(vs.Column)
|
||||||
|
dir := "ASC"
|
||||||
|
if strings.EqualFold(vs.Direction, "desc") {
|
||||||
|
dir = "DESC"
|
||||||
|
}
|
||||||
|
if vs.As != "" {
|
||||||
|
query = query.ColumnExpr(fmt.Sprintf("(%s %s ?) AS %s", col, op, common.QuoteIdent(vs.As)), lit)
|
||||||
|
}
|
||||||
|
query = query.OrderExpr(fmt.Sprintf("%s %s ? %s", col, op, dir), lit)
|
||||||
|
logger.Debug("Applying vector search on %s (%s)", vs.Column, op)
|
||||||
|
}
|
||||||
|
|
||||||
// Apply preloading
|
// Apply preloading
|
||||||
if len(options.Preload) > 0 {
|
if len(options.Preload) > 0 {
|
||||||
var err error
|
var err error
|
||||||
@@ -364,7 +397,7 @@ func (h *Handler) handleRead(ctx context.Context, w common.ResponseWriter, id st
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Apply filters with proper grouping for OR logic
|
// Apply filters with proper grouping for OR logic
|
||||||
query = h.applyFilters(query, options.Filters)
|
query = h.applyFilters(query, options.Filters, model)
|
||||||
|
|
||||||
// Apply custom operators
|
// Apply custom operators
|
||||||
for _, customOp := range options.CustomOperators {
|
for _, customOp := range options.CustomOperators {
|
||||||
@@ -384,6 +417,10 @@ func (h *Handler) handleRead(ctx context.Context, w common.ResponseWriter, id st
|
|||||||
direction = "DESC"
|
direction = "DESC"
|
||||||
}
|
}
|
||||||
logger.Debug("Applying sort: %s %s", sort.Column, direction)
|
logger.Debug("Applying sort: %s %s", sort.Column, direction)
|
||||||
|
if expr, jargs, _, ok := common.ResolveJSONColumnExpr(model, "", sort.Column); ok {
|
||||||
|
query = query.OrderExpr(fmt.Sprintf("%s %s", expr, direction), jargs...)
|
||||||
|
continue
|
||||||
|
}
|
||||||
query = query.Order(fmt.Sprintf("%s %s", sort.Column, direction))
|
query = query.Order(fmt.Sprintf("%s %s", sort.Column, direction))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -507,7 +544,7 @@ func (h *Handler) handleRead(ctx context.Context, w common.ResponseWriter, id st
|
|||||||
|
|
||||||
// Apply the same filters as the main query
|
// Apply the same filters as the main query
|
||||||
for _, filter := range options.Filters {
|
for _, filter := range options.Filters {
|
||||||
rowNumQuery = h.applyFilter(rowNumQuery, filter)
|
rowNumQuery = h.applyFilter(rowNumQuery, filter, model)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Apply custom operators
|
// Apply custom operators
|
||||||
@@ -565,9 +602,7 @@ func (h *Handler) handleRead(ctx context.Context, w common.ResponseWriter, id st
|
|||||||
logger.Debug("Querying single record with FetchRowNumber ID: %s", targetID)
|
logger.Debug("Querying single record with FetchRowNumber ID: %s", targetID)
|
||||||
}
|
}
|
||||||
|
|
||||||
// For single record, create a new pointer to the struct type
|
pkName := reflection.GetPrimaryKeyName(model)
|
||||||
singleResult := reflect.New(modelType).Interface()
|
|
||||||
pkName := reflection.GetPrimaryKeyName(singleResult)
|
|
||||||
|
|
||||||
query = query.Where(fmt.Sprintf("%s = ?", common.QuoteIdent(pkName)), targetID)
|
query = query.Where(fmt.Sprintf("%s = ?", common.QuoteIdent(pkName)), targetID)
|
||||||
|
|
||||||
@@ -580,12 +615,21 @@ func (h *Handler) handleRead(ctx context.Context, w common.ResponseWriter, id st
|
|||||||
}
|
}
|
||||||
query = hookCtx.Query
|
query = hookCtx.Query
|
||||||
|
|
||||||
if err := query.Scan(ctx, singleResult); err != nil {
|
// Scan through the model configured on the query. Bun rejects Scan with
|
||||||
|
// a destination when the query preloads a has-many relation.
|
||||||
|
if err := query.ScanModel(ctx); err != nil {
|
||||||
logger.Error("Error querying record: %v", err)
|
logger.Error("Error querying record: %v", err)
|
||||||
statusCode, errCode, errMsg = http.StatusInternalServerError, "query_error", "Error executing query"
|
statusCode, errCode, errMsg = http.StatusInternalServerError, "query_error", "Error executing query"
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
result = singleResult
|
|
||||||
|
// The configured model is a slice so the same query construction works
|
||||||
|
// for both collection and single-record reads. Extract its one result.
|
||||||
|
scannedResults := reflect.ValueOf(modelPtr).Elem()
|
||||||
|
if scannedResults.Len() == 0 {
|
||||||
|
return sql.ErrNoRows
|
||||||
|
}
|
||||||
|
result = scannedResults.Index(0).Interface()
|
||||||
} else {
|
} else {
|
||||||
logger.Debug("Querying multiple records")
|
logger.Debug("Querying multiple records")
|
||||||
|
|
||||||
@@ -598,8 +642,9 @@ func (h *Handler) handleRead(ctx context.Context, w common.ResponseWriter, id st
|
|||||||
}
|
}
|
||||||
query = hookCtx.Query
|
query = hookCtx.Query
|
||||||
|
|
||||||
// Use the modelPtr already created and set on the query
|
// Use the model already configured on the query. This is required by
|
||||||
if err := query.Scan(ctx, modelPtr); err != nil {
|
// Bun whenever the query includes a has-many preload.
|
||||||
|
if err := query.ScanModel(ctx); err != nil {
|
||||||
logger.Error("Error querying records: %v", err)
|
logger.Error("Error querying records: %v", err)
|
||||||
statusCode, errCode, errMsg = http.StatusInternalServerError, "query_error", "Error executing query"
|
statusCode, errCode, errMsg = http.StatusInternalServerError, "query_error", "Error executing query"
|
||||||
return err
|
return err
|
||||||
@@ -1792,7 +1837,7 @@ func (h *Handler) handleDelete(ctx context.Context, w common.ResponseWriter, id
|
|||||||
// applyFilters applies all filters with proper grouping for OR logic
|
// applyFilters applies all filters with proper grouping for OR logic
|
||||||
// Groups consecutive OR filters together to ensure proper query precedence
|
// Groups consecutive OR filters together to ensure proper query precedence
|
||||||
// Example: [A, B(OR), C(OR), D(AND)] => WHERE (A OR B OR C) AND D
|
// Example: [A, B(OR), C(OR), D(AND)] => WHERE (A OR B OR C) AND D
|
||||||
func (h *Handler) applyFilters(query common.SelectQuery, filters []common.FilterOption) common.SelectQuery {
|
func (h *Handler) applyFilters(query common.SelectQuery, filters []common.FilterOption, model interface{}) common.SelectQuery {
|
||||||
if len(filters) == 0 {
|
if len(filters) == 0 {
|
||||||
return query
|
return query
|
||||||
}
|
}
|
||||||
@@ -1812,11 +1857,11 @@ func (h *Handler) applyFilters(query common.SelectQuery, filters []common.Filter
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Apply the OR group as a single grouped WHERE clause
|
// Apply the OR group as a single grouped WHERE clause
|
||||||
query = h.applyFilterGroup(query, orGroup)
|
query = h.applyFilterGroup(query, orGroup, model)
|
||||||
i = j
|
i = j
|
||||||
} else {
|
} else {
|
||||||
// Single filter with AND logic (or first filter)
|
// Single filter with AND logic (or first filter)
|
||||||
condition, args := h.buildFilterCondition(filters[i])
|
condition, args := h.buildFilterCondition(filters[i], model)
|
||||||
if condition != "" {
|
if condition != "" {
|
||||||
query = query.Where(condition, args...)
|
query = query.Where(condition, args...)
|
||||||
}
|
}
|
||||||
@@ -1829,7 +1874,7 @@ func (h *Handler) applyFilters(query common.SelectQuery, filters []common.Filter
|
|||||||
|
|
||||||
// applyFilterGroup applies a group of filters that should be OR'd together
|
// applyFilterGroup applies a group of filters that should be OR'd together
|
||||||
// Always wraps them in parentheses and applies as a single WHERE clause
|
// Always wraps them in parentheses and applies as a single WHERE clause
|
||||||
func (h *Handler) applyFilterGroup(query common.SelectQuery, filters []common.FilterOption) common.SelectQuery {
|
func (h *Handler) applyFilterGroup(query common.SelectQuery, filters []common.FilterOption, model interface{}) common.SelectQuery {
|
||||||
if len(filters) == 0 {
|
if len(filters) == 0 {
|
||||||
return query
|
return query
|
||||||
}
|
}
|
||||||
@@ -1839,7 +1884,7 @@ func (h *Handler) applyFilterGroup(query common.SelectQuery, filters []common.Fi
|
|||||||
var args []interface{}
|
var args []interface{}
|
||||||
|
|
||||||
for _, filter := range filters {
|
for _, filter := range filters {
|
||||||
condition, filterArgs := h.buildFilterCondition(filter)
|
condition, filterArgs := h.buildFilterCondition(filter, model)
|
||||||
if condition != "" {
|
if condition != "" {
|
||||||
conditions = append(conditions, condition)
|
conditions = append(conditions, condition)
|
||||||
args = append(args, filterArgs...)
|
args = append(args, filterArgs...)
|
||||||
@@ -1860,11 +1905,18 @@ func (h *Handler) applyFilterGroup(query common.SelectQuery, filters []common.Fi
|
|||||||
return query.Where(groupedCondition, args...)
|
return query.Where(groupedCondition, args...)
|
||||||
}
|
}
|
||||||
|
|
||||||
// buildFilterCondition builds a filter condition and returns it with args
|
// buildFilterCondition builds a filter condition and returns it with args.
|
||||||
func (h *Handler) buildFilterCondition(filter common.FilterOption) (conditionString string, conditionArgs []interface{}) {
|
// model, when non-nil, lets JSON sub-field references (data->>'x', data#>>'{a,b}',
|
||||||
|
// or the dotted data.x shorthand for a JSON column) resolve to a safe,
|
||||||
|
// parameterised expression before the ordinary operator handling below.
|
||||||
|
func (h *Handler) buildFilterCondition(filter common.FilterOption, model interface{}) (conditionString string, conditionArgs []interface{}) {
|
||||||
var condition string
|
var condition string
|
||||||
var args []interface{}
|
var args []interface{}
|
||||||
|
|
||||||
|
if cond, jargs, ok := common.BuildJSONFilterCondition(model, "", filter.Column, filter.Operator, filter.Value); ok {
|
||||||
|
return cond, jargs
|
||||||
|
}
|
||||||
|
|
||||||
switch filter.Operator {
|
switch filter.Operator {
|
||||||
case "eq", "=":
|
case "eq", "=":
|
||||||
condition = fmt.Sprintf("%s = ?", filter.Column)
|
condition = fmt.Sprintf("%s = ?", filter.Column)
|
||||||
@@ -1901,19 +1953,40 @@ func (h *Handler) buildFilterCondition(filter common.FilterOption) (conditionStr
|
|||||||
return "", nil
|
return "", nil
|
||||||
}
|
}
|
||||||
default:
|
default:
|
||||||
return "", nil
|
if common.IsSpatialOperator(filter.Operator) {
|
||||||
|
q, a, ok := common.BuildSpatialCondition(filter.Column, filter.Operator, filter.Value)
|
||||||
|
if !ok {
|
||||||
|
return "", nil
|
||||||
|
}
|
||||||
|
condition, args = q, a
|
||||||
|
} else if common.IsVectorOperator(filter.Operator) {
|
||||||
|
q, a, ok := common.BuildVectorCondition(filter.Column, filter.Operator, filter.Value)
|
||||||
|
if !ok {
|
||||||
|
return "", nil
|
||||||
|
}
|
||||||
|
condition, args = q, a
|
||||||
|
} else {
|
||||||
|
return "", nil
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return condition, args
|
return condition, args
|
||||||
}
|
}
|
||||||
|
|
||||||
func (h *Handler) applyFilter(query common.SelectQuery, filter common.FilterOption) common.SelectQuery {
|
func (h *Handler) applyFilter(query common.SelectQuery, filter common.FilterOption, model interface{}) common.SelectQuery {
|
||||||
// Determine which method to use based on LogicOperator
|
// Determine which method to use based on LogicOperator
|
||||||
useOrLogic := strings.EqualFold(filter.LogicOperator, "OR")
|
useOrLogic := strings.EqualFold(filter.LogicOperator, "OR")
|
||||||
|
|
||||||
var condition string
|
var condition string
|
||||||
var args []interface{}
|
var args []interface{}
|
||||||
|
|
||||||
|
if cond, jargs, ok := common.BuildJSONFilterCondition(model, "", filter.Column, filter.Operator, filter.Value); ok {
|
||||||
|
if useOrLogic {
|
||||||
|
return query.WhereOr(cond, jargs...)
|
||||||
|
}
|
||||||
|
return query.Where(cond, jargs...)
|
||||||
|
}
|
||||||
|
|
||||||
switch filter.Operator {
|
switch filter.Operator {
|
||||||
case "eq", "=":
|
case "eq", "=":
|
||||||
condition = fmt.Sprintf("%s = ?", filter.Column)
|
condition = fmt.Sprintf("%s = ?", filter.Column)
|
||||||
@@ -1950,7 +2023,21 @@ func (h *Handler) applyFilter(query common.SelectQuery, filter common.FilterOpti
|
|||||||
return query
|
return query
|
||||||
}
|
}
|
||||||
default:
|
default:
|
||||||
return query
|
if common.IsSpatialOperator(filter.Operator) {
|
||||||
|
q, a, ok := common.BuildSpatialCondition(filter.Column, filter.Operator, filter.Value)
|
||||||
|
if !ok {
|
||||||
|
return query
|
||||||
|
}
|
||||||
|
condition, args = q, a
|
||||||
|
} else if common.IsVectorOperator(filter.Operator) {
|
||||||
|
q, a, ok := common.BuildVectorCondition(filter.Column, filter.Operator, filter.Value)
|
||||||
|
if !ok {
|
||||||
|
return query
|
||||||
|
}
|
||||||
|
condition, args = q, a
|
||||||
|
} else {
|
||||||
|
return query
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Apply filter with appropriate logic operator
|
// Apply filter with appropriate logic operator
|
||||||
@@ -2073,16 +2160,34 @@ func (h *Handler) generateMetadata(schema, entity string, model interface{}) *co
|
|||||||
jsonName = field.Name
|
jsonName = field.Name
|
||||||
}
|
}
|
||||||
|
|
||||||
if field.Type.Kind() == reflect.Slice ||
|
columnField := field
|
||||||
(field.Type.Kind() == reflect.Struct && field.Type.Name() != "Time") {
|
isSQLType := false
|
||||||
|
if unwrappedType, ok := unwrapSQLType(field.Type); ok {
|
||||||
|
columnField.Type = unwrappedType
|
||||||
|
isSQLType = true
|
||||||
|
}
|
||||||
|
|
||||||
|
if !isSQLType && (columnField.Type.Kind() == reflect.Slice ||
|
||||||
|
(columnField.Type.Kind() == reflect.Struct && columnField.Type.Name() != "Time")) {
|
||||||
metadata.Relations = append(metadata.Relations, jsonName)
|
metadata.Relations = append(metadata.Relations, jsonName)
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
|
colTypeStr := getColumnType(columnField)
|
||||||
|
// Fill the gap for spectypes wrappers whose Go representation (struct or
|
||||||
|
// slice) has no obvious SQL mapping — PostGIS geometry/geography and
|
||||||
|
// pgvector vector/halfvec/sparsevec/bit. A gorm `type:` tag still wins
|
||||||
|
// (dimensioned types like vector(1536)).
|
||||||
|
if colTypeStr == "unknown" && !strings.Contains(field.Tag.Get("gorm"), "type:") {
|
||||||
|
if n, ok := spectypes.SQLTypeName(field.Type); ok {
|
||||||
|
colTypeStr = n
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
column := common.Column{
|
column := common.Column{
|
||||||
Name: jsonName,
|
Name: jsonName,
|
||||||
Type: getColumnType(field),
|
Type: colTypeStr,
|
||||||
IsNullable: isNullable(field),
|
IsNullable: isSQLType || isNullable(field),
|
||||||
IsPrimary: strings.Contains(gormTag, "primaryKey"),
|
IsPrimary: strings.Contains(gormTag, "primaryKey"),
|
||||||
IsUnique: strings.Contains(gormTag, "unique") || strings.Contains(gormTag, "uniqueIndex"),
|
IsUnique: strings.Contains(gormTag, "unique") || strings.Contains(gormTag, "uniqueIndex"),
|
||||||
HasIndex: strings.Contains(gormTag, "index") || strings.Contains(gormTag, "uniqueIndex"),
|
HasIndex: strings.Contains(gormTag, "index") || strings.Contains(gormTag, "uniqueIndex"),
|
||||||
@@ -2173,6 +2278,31 @@ func getColumnType(field reflect.StructField) string {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// unwrapSQLType returns the value type wrapped by a spectypes SQL value. These
|
||||||
|
// types represent columns, even when their Go representation is a struct or a
|
||||||
|
// slice (for example, SqlNull[string] and SqlJSONB).
|
||||||
|
func unwrapSQLType(t reflect.Type) (reflect.Type, bool) {
|
||||||
|
for t.Kind() == reflect.Pointer {
|
||||||
|
t = t.Elem()
|
||||||
|
}
|
||||||
|
|
||||||
|
if t.PkgPath() != "github.com/bitechdev/ResolveSpec/pkg/spectypes" {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
|
||||||
|
// SqlNull aliases and the date/time wrappers expose their actual value via
|
||||||
|
// Val. FieldByName also resolves Val through the embedded SqlNull field.
|
||||||
|
if t.Kind() == reflect.Struct {
|
||||||
|
if value, ok := t.FieldByName("Val"); ok {
|
||||||
|
return value.Type, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// SqlJSONB has no wrapper field, but remains a scalar SQL value rather than
|
||||||
|
// a relation.
|
||||||
|
return t, true
|
||||||
|
}
|
||||||
|
|
||||||
func isNullable(field reflect.StructField) bool {
|
func isNullable(field reflect.StructField) bool {
|
||||||
// Check if it's a pointer type
|
// Check if it's a pointer type
|
||||||
if field.Type.Kind() == reflect.Pointer {
|
if field.Type.Kind() == reflect.Pointer {
|
||||||
@@ -2286,7 +2416,7 @@ func (h *Handler) applyPreloads(model interface{}, query common.SelectQuery, pre
|
|||||||
|
|
||||||
if len(preload.Filters) > 0 {
|
if len(preload.Filters) > 0 {
|
||||||
for _, filter := range preload.Filters {
|
for _, filter := range preload.Filters {
|
||||||
sq = h.applyFilter(sq, filter)
|
sq = h.applyFilter(sq, filter, nil)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if len(preload.Sort) > 0 {
|
if len(preload.Sort) > 0 {
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"github.com/bitechdev/ResolveSpec/pkg/common"
|
"github.com/bitechdev/ResolveSpec/pkg/common"
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/spectypes"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestNewHandler(t *testing.T) {
|
func TestNewHandler(t *testing.T) {
|
||||||
@@ -202,6 +203,48 @@ func TestGetColumnType(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestGenerateMetadata_UnwrapsSQLTypes(t *testing.T) {
|
||||||
|
type related struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
}
|
||||||
|
type model struct {
|
||||||
|
Name spectypes.SqlString `json:"name"`
|
||||||
|
Count spectypes.SqlInt64 `json:"count"`
|
||||||
|
CreatedAt spectypes.SqlTimeStamp `json:"created_at"`
|
||||||
|
Metadata spectypes.SqlJSONB `json:"metadata" gorm:"type:jsonb"`
|
||||||
|
Related related `json:"related"`
|
||||||
|
}
|
||||||
|
|
||||||
|
metadata := NewHandler(nil, nil).generateMetadata("public", "models", model{})
|
||||||
|
columns := make(map[string]common.Column, len(metadata.Columns))
|
||||||
|
for _, column := range metadata.Columns {
|
||||||
|
columns[column.Name] = column
|
||||||
|
}
|
||||||
|
|
||||||
|
for name, wantType := range map[string]string{
|
||||||
|
"name": "string",
|
||||||
|
"count": "bigint",
|
||||||
|
"created_at": "timestamp",
|
||||||
|
"metadata": "jsonb",
|
||||||
|
} {
|
||||||
|
column, ok := columns[name]
|
||||||
|
if !ok {
|
||||||
|
t.Errorf("expected %q to be a metadata column", name)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if column.Type != wantType {
|
||||||
|
t.Errorf("%q: expected type %q, got %q", name, wantType, column.Type)
|
||||||
|
}
|
||||||
|
if !column.IsNullable {
|
||||||
|
t.Errorf("%q: expected SQL wrapper to be nullable", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(metadata.Relations) != 1 || metadata.Relations[0] != "related" {
|
||||||
|
t.Errorf("expected only related to be a relation, got %v", metadata.Relations)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestIsNullable(t *testing.T) {
|
func TestIsNullable(t *testing.T) {
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
name string
|
name string
|
||||||
|
|||||||
@@ -0,0 +1,153 @@
|
|||||||
|
package resolvespec
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/common"
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/spectypes"
|
||||||
|
)
|
||||||
|
|
||||||
|
// jsonColModel has a real JSONB column so the dotted "data.x" shorthand is
|
||||||
|
// recognised as JSON access.
|
||||||
|
type jsonColModel struct {
|
||||||
|
ID int64 `json:"id" bun:"id,pk"`
|
||||||
|
Name string `json:"name" bun:"name"`
|
||||||
|
Data spectypes.SqlJSONB `json:"data" bun:"data"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type jsonCapCall struct {
|
||||||
|
method string
|
||||||
|
query string
|
||||||
|
args []interface{}
|
||||||
|
}
|
||||||
|
|
||||||
|
// jsonCapQuery records the string + args of the calls the handler makes.
|
||||||
|
type jsonCapQuery struct {
|
||||||
|
calls []jsonCapCall
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *jsonCapQuery) rec(method, query string, args []interface{}) common.SelectQuery {
|
||||||
|
m.calls = append(m.calls, jsonCapCall{method: method, query: query, args: args})
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *jsonCapQuery) Model(interface{}) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) Table(string) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) Column(cols ...string) common.SelectQuery {
|
||||||
|
for _, c := range cols {
|
||||||
|
m.rec("Column", c, nil)
|
||||||
|
}
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) ColumnExpr(q string, args ...interface{}) common.SelectQuery {
|
||||||
|
return m.rec("ColumnExpr", q, args)
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) Where(q string, args ...interface{}) common.SelectQuery {
|
||||||
|
return m.rec("Where", q, args)
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) WhereOr(q string, args ...interface{}) common.SelectQuery {
|
||||||
|
return m.rec("WhereOr", q, args)
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) Join(string, ...interface{}) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) LeftJoin(string, ...interface{}) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) Preload(string, ...interface{}) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) PreloadRelation(string, ...func(common.SelectQuery) common.SelectQuery) common.SelectQuery {
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) JoinRelation(string, ...func(common.SelectQuery) common.SelectQuery) common.SelectQuery {
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) Order(o string) common.SelectQuery { return m.rec("Order", o, nil) }
|
||||||
|
func (m *jsonCapQuery) OrderExpr(o string, args ...interface{}) common.SelectQuery {
|
||||||
|
return m.rec("OrderExpr", o, args)
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) Limit(int) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) Offset(int) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) Group(string) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) Having(string, ...interface{}) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) Scan(context.Context, interface{}) error { return nil }
|
||||||
|
func (m *jsonCapQuery) ScanModel(context.Context) error { return nil }
|
||||||
|
func (m *jsonCapQuery) Count(context.Context) (int, error) { return 0, nil }
|
||||||
|
func (m *jsonCapQuery) Exists(context.Context) (bool, error) { return false, nil }
|
||||||
|
|
||||||
|
func (m *jsonCapQuery) only(t *testing.T) jsonCapCall {
|
||||||
|
t.Helper()
|
||||||
|
if len(m.calls) != 1 {
|
||||||
|
t.Fatalf("expected exactly 1 recorded call, got %d: %+v", len(m.calls), m.calls)
|
||||||
|
}
|
||||||
|
return m.calls[0]
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildFilterCondition_JSONColumn(t *testing.T) {
|
||||||
|
h := &Handler{}
|
||||||
|
model := jsonColModel{}
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
filter common.FilterOption
|
||||||
|
wantCond string
|
||||||
|
wantArgs []interface{}
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "arrow syntax eq stays text",
|
||||||
|
filter: common.FilterOption{Column: "data->>'city'", Operator: "eq", Value: "LA"},
|
||||||
|
wantCond: `("data" #>> ?::text[]) = ?`,
|
||||||
|
wantArgs: []interface{}{"{city}", "LA"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "dotted shorthand numeric cast inference",
|
||||||
|
filter: common.FilterOption{Column: "data.age", Operator: "gt", Value: 18},
|
||||||
|
wantCond: `(("data" #>> ?::text[]))::numeric > ?`,
|
||||||
|
wantArgs: []interface{}{"{age}", 18},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "hash path with explicit cast",
|
||||||
|
filter: common.FilterOption{Column: "data#>>'{a,b}'::int", Operator: "lte", Value: "5"},
|
||||||
|
wantCond: `(("data" #>> ?::text[]))::integer <= ?`,
|
||||||
|
wantArgs: []interface{}{"{a,b}", "5"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range tests {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
cond, args := h.buildFilterCondition(tc.filter, model)
|
||||||
|
if cond != tc.wantCond {
|
||||||
|
t.Fatalf("cond = %q, want %q", cond, tc.wantCond)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(args, tc.wantArgs) {
|
||||||
|
t.Fatalf("args = %#v, want %#v", args, tc.wantArgs)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// Non-JSON column falls through to ordinary handling.
|
||||||
|
cond, _ := h.buildFilterCondition(common.FilterOption{Column: "name", Operator: "eq", Value: "x"}, model)
|
||||||
|
if cond != "name = ?" {
|
||||||
|
t.Fatalf("non-JSON cond = %q", cond)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Without a model the dotted shorthand must NOT be treated as JSON.
|
||||||
|
cond, _ = h.buildFilterCondition(common.FilterOption{Column: "data.age", Operator: "eq", Value: "x"}, nil)
|
||||||
|
if cond != "data.age = ?" {
|
||||||
|
t.Fatalf("nil-model dotted cond = %q, want ordinary handling", cond)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestApplyFilter_JSONColumn(t *testing.T) {
|
||||||
|
h := &Handler{}
|
||||||
|
model := jsonColModel{}
|
||||||
|
|
||||||
|
q := &jsonCapQuery{}
|
||||||
|
h.applyFilter(q, common.FilterOption{
|
||||||
|
Column: "data->>'tier'", Operator: "in", Value: []string{"a", "b"}, LogicOperator: "OR",
|
||||||
|
}, model)
|
||||||
|
c := q.only(t)
|
||||||
|
if c.method != "WhereOr" || c.query != `("data" #>> ?::text[]) IN (?,?)` {
|
||||||
|
t.Fatalf("call = %+v", c)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(c.args, []interface{}{"{tier}", "a", "b"}) {
|
||||||
|
t.Fatalf("args = %#v", c.args)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"github.com/bitechdev/ResolveSpec/pkg/common"
|
"github.com/bitechdev/ResolveSpec/pkg/common"
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/spectypes"
|
||||||
)
|
)
|
||||||
|
|
||||||
// detailTestModel is a simple model with gorm column/type tags for detail format tests.
|
// detailTestModel is a simple model with gorm column/type tags for detail format tests.
|
||||||
@@ -207,3 +208,30 @@ func TestBuildDetailFields_SkipsRelations(t *testing.T) {
|
|||||||
t.Errorf("expected 2 scalar fields (id, name), got %d", len(fields))
|
t.Errorf("expected 2 scalar fields (id, name), got %d", len(fields))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestBuildDetailFields_UnwrapsSQLTypes(t *testing.T) {
|
||||||
|
type model struct {
|
||||||
|
Name spectypes.SqlString `gorm:"column:name" json:"name"`
|
||||||
|
CreatedAt spectypes.SqlTimeStamp `gorm:"column:created_at" json:"created_at"`
|
||||||
|
Metadata spectypes.SqlJSONB `gorm:"column:metadata;type:jsonb" json:"metadata"`
|
||||||
|
}
|
||||||
|
|
||||||
|
fields := (&Handler{}).buildDetailFields(model{})
|
||||||
|
byName := make(map[string]string, len(fields))
|
||||||
|
for _, field := range fields {
|
||||||
|
byName[field.Name] = field.DataType
|
||||||
|
if !field.Nullable {
|
||||||
|
t.Errorf("%q: expected SQL wrapper to be nullable", field.Name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for name, want := range map[string]string{
|
||||||
|
"name": "string",
|
||||||
|
"created_at": "unknown",
|
||||||
|
"metadata": "unknown",
|
||||||
|
} {
|
||||||
|
if got := byName[name]; got != want {
|
||||||
|
t.Errorf("%q: expected type %q, got %q", name, want, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
+155
-26
@@ -419,7 +419,12 @@ func (h *Handler) handleRead(ctx context.Context, w common.ResponseWriter, id st
|
|||||||
|
|
||||||
// If we have computed columns/expressions but options.Columns is empty,
|
// If we have computed columns/expressions but options.Columns is empty,
|
||||||
// populate it with all model columns first since computed columns are additions
|
// populate it with all model columns first since computed columns are additions
|
||||||
if len(options.Columns) == 0 && (len(options.ComputedQL) > 0 || len(options.ComputedColumns) > 0) {
|
vectorSearchActive := options.VectorSearch != nil &&
|
||||||
|
options.VectorSearch.Column != "" && len(options.VectorSearch.Vector) > 0
|
||||||
|
|
||||||
|
if len(options.Columns) == 0 &&
|
||||||
|
(len(options.ComputedQL) > 0 || len(options.ComputedColumns) > 0 ||
|
||||||
|
(vectorSearchActive && options.VectorSearch.As != "")) {
|
||||||
logger.Debug("Populating options.Columns with all model columns since computed columns are additions")
|
logger.Debug("Populating options.Columns with all model columns since computed columns are additions")
|
||||||
options.Columns = reflection.GetSQLModelColumns(model)
|
options.Columns = reflection.GetSQLModelColumns(model)
|
||||||
}
|
}
|
||||||
@@ -466,12 +471,42 @@ func (h *Handler) handleRead(ctx context.Context, w common.ResponseWriter, id st
|
|||||||
// Apply column selection
|
// Apply column selection
|
||||||
if len(options.Columns) > 0 {
|
if len(options.Columns) > 0 {
|
||||||
logger.Debug("Selecting columns: %v", options.Columns)
|
logger.Debug("Selecting columns: %v", options.Columns)
|
||||||
|
selectAlias := reflection.ExtractTableNameOnly(tableName)
|
||||||
for _, col := range options.Columns {
|
for _, col := range options.Columns {
|
||||||
|
// JSON sub-field selection (data->>'x', data.x, data#>>'{a,b}'):
|
||||||
|
// emit a parameterised expression aliased to a stable name.
|
||||||
|
if expr, jargs, alias, ok := common.ResolveJSONColumnExpr(model, selectAlias, col); ok {
|
||||||
|
query = query.ColumnExpr(expr+" AS "+common.QuoteIdent(alias), jargs...)
|
||||||
|
continue
|
||||||
|
}
|
||||||
query = query.Column(reflection.ExtractSourceColumn(col))
|
query = query.Column(reflection.ExtractSourceColumn(col))
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// pgvector KNN search: order by distance to the query vector and,
|
||||||
|
// optionally, return that distance as an extra column. Postgres only.
|
||||||
|
if vectorSearchActive {
|
||||||
|
vs := options.VectorSearch
|
||||||
|
op := common.VectorOperator(vs.Metric)
|
||||||
|
lit, litErr := common.VectorLiteral(vs.Vector)
|
||||||
|
if litErr != nil {
|
||||||
|
logger.Error("Invalid vector search vector: %v", litErr)
|
||||||
|
statusCode, errCode, errMsg = http.StatusBadRequest, "invalid_vector_search", "Invalid vector search vector"
|
||||||
|
return litErr
|
||||||
|
}
|
||||||
|
col := common.QuoteIdent(vs.Column)
|
||||||
|
dir := "ASC"
|
||||||
|
if strings.EqualFold(vs.Direction, "desc") {
|
||||||
|
dir = "DESC"
|
||||||
|
}
|
||||||
|
if vs.As != "" {
|
||||||
|
query = query.ColumnExpr(fmt.Sprintf("(%s %s ?) AS %s", col, op, common.QuoteIdent(vs.As)), lit)
|
||||||
|
}
|
||||||
|
query = query.OrderExpr(fmt.Sprintf("%s %s ? %s", col, op, dir), lit)
|
||||||
|
logger.Debug("Applying vector search on %s (%s)", vs.Column, op)
|
||||||
|
}
|
||||||
|
|
||||||
// Apply expand (Just expand to Preload for now)
|
// Apply expand (Just expand to Preload for now)
|
||||||
for _, expand := range options.Expand {
|
for _, expand := range options.Expand {
|
||||||
logger.Debug("Applying expand: %s", expand.Relation)
|
logger.Debug("Applying expand: %s", expand.Relation)
|
||||||
@@ -582,12 +617,12 @@ func (h *Handler) handleRead(ctx context.Context, w common.ResponseWriter, id st
|
|||||||
|
|
||||||
// Apply the OR group as a single grouped condition
|
// Apply the OR group as a single grouped condition
|
||||||
logger.Debug("Applying OR filter group with %d conditions", len(orFilters))
|
logger.Debug("Applying OR filter group with %d conditions", len(orFilters))
|
||||||
query = h.applyOrFilterGroup(query, orFilters, orCastInfo, tableName)
|
query = h.applyOrFilterGroup(query, orFilters, orCastInfo, tableName, model)
|
||||||
i = j
|
i = j
|
||||||
} else {
|
} else {
|
||||||
// Single AND filter - apply normally
|
// Single AND filter - apply normally
|
||||||
logger.Debug("Applying filter: %s %s %v (needsCast=%v, logic=%s)", filter.Column, filter.Operator, filter.Value, castInfo.NeedsCast, logicOp)
|
logger.Debug("Applying filter: %s %s %v (needsCast=%v, logic=%s)", filter.Column, filter.Operator, filter.Value, castInfo.NeedsCast, logicOp)
|
||||||
query = h.applyFilter(query, *filter, tableName, castInfo.NeedsCast, logicOp)
|
query = h.applyFilter(query, *filter, tableName, castInfo.NeedsCast, logicOp, model)
|
||||||
i++
|
i++
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -687,8 +722,12 @@ func (h *Handler) handleRead(ctx context.Context, w common.ResponseWriter, id st
|
|||||||
}
|
}
|
||||||
logger.Debug("Applying sort: %s %s", sort.Column, direction)
|
logger.Debug("Applying sort: %s %s", sort.Column, direction)
|
||||||
|
|
||||||
// Check if it's an expression (enclosed in brackets) - use directly without quoting
|
// JSON sub-field reference (data->>'x', data#>>'{a,b}', or dotted
|
||||||
if strings.HasPrefix(sort.Column, "(") && strings.HasSuffix(sort.Column, ")") {
|
// shorthand when the base is a JSON column) - resolve to a safe
|
||||||
|
// parameterised expression before the generic branches.
|
||||||
|
if expr, jargs, _, ok := common.ResolveJSONColumnExpr(model, tableAlias, sort.Column); ok {
|
||||||
|
query = query.OrderExpr(fmt.Sprintf("%s %s", expr, direction), jargs...)
|
||||||
|
} else if strings.HasPrefix(sort.Column, "(") && strings.HasSuffix(sort.Column, ")") {
|
||||||
// For expressions, pass as raw SQL to prevent auto-quoting
|
// For expressions, pass as raw SQL to prevent auto-quoting
|
||||||
query = query.OrderExpr(fmt.Sprintf("%s %s", sort.Column, direction))
|
query = query.OrderExpr(fmt.Sprintf("%s %s", sort.Column, direction))
|
||||||
} else if strings.Contains(sort.Column, ".") {
|
} else if strings.Contains(sort.Column, ".") {
|
||||||
@@ -1035,7 +1074,7 @@ func (h *Handler) applyPreloadWithRecursion(query common.SelectQuery, preload co
|
|||||||
// Apply filters
|
// Apply filters
|
||||||
if len(preload.Filters) > 0 {
|
if len(preload.Filters) > 0 {
|
||||||
for _, filter := range preload.Filters {
|
for _, filter := range preload.Filters {
|
||||||
sq = h.applyFilter(sq, filter, "", false, "AND")
|
sq = h.applyFilter(sq, filter, "", false, "AND", nil)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -2260,16 +2299,11 @@ func (h *Handler) qualifyColumnName(columnName, fullTableName string) string {
|
|||||||
return fmt.Sprintf("%s.%s", tableOnly, columnName)
|
return fmt.Sprintf("%s.%s", tableOnly, columnName)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (h *Handler) applyFilter(query common.SelectQuery, filter common.FilterOption, tableName string, needsCast bool, logicOp string) common.SelectQuery {
|
func (h *Handler) applyFilter(query common.SelectQuery, filter common.FilterOption, tableName string, needsCast bool, logicOp string, model interface{}) common.SelectQuery {
|
||||||
// Qualify the column name with table name if not already qualified
|
// Qualify the column name with table name if not already qualified
|
||||||
rawQualifiedColumn := h.qualifyColumnName(filter.Column, tableName)
|
rawQualifiedColumn := h.qualifyColumnName(filter.Column, tableName)
|
||||||
qualifiedColumn := rawQualifiedColumn
|
qualifiedColumn := rawQualifiedColumn
|
||||||
|
|
||||||
// Apply casting to text if needed for non-numeric columns or non-numeric values
|
|
||||||
if needsCast {
|
|
||||||
qualifiedColumn = fmt.Sprintf("CAST(%s AS TEXT)", rawQualifiedColumn)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Helper function to apply the correct Where method based on logic operator
|
// Helper function to apply the correct Where method based on logic operator
|
||||||
applyWhere := func(condition string, args ...interface{}) common.SelectQuery {
|
applyWhere := func(condition string, args ...interface{}) common.SelectQuery {
|
||||||
if logicOp == "OR" {
|
if logicOp == "OR" {
|
||||||
@@ -2278,6 +2312,19 @@ func (h *Handler) applyFilter(query common.SelectQuery, filter common.FilterOpti
|
|||||||
return query.Where(condition, args...)
|
return query.Where(condition, args...)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// JSON sub-field access (data->>'x', data#>>'{a,b}', or the dotted data.x
|
||||||
|
// shorthand when "data" is a JSON column): resolve to a safe, parameterised
|
||||||
|
// expression before the ordinary column handling below.
|
||||||
|
tableAlias := reflection.ExtractTableNameOnly(tableName)
|
||||||
|
if cond, jargs, ok := common.BuildJSONFilterCondition(model, tableAlias, filter.Column, filter.Operator, filter.Value); ok {
|
||||||
|
return applyWhere(cond, jargs...)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Apply casting to text if needed for non-numeric columns or non-numeric values
|
||||||
|
if needsCast {
|
||||||
|
qualifiedColumn = fmt.Sprintf("CAST(%s AS TEXT)", rawQualifiedColumn)
|
||||||
|
}
|
||||||
|
|
||||||
switch strings.ToLower(filter.Operator) {
|
switch strings.ToLower(filter.Operator) {
|
||||||
case "eq", "equals":
|
case "eq", "equals":
|
||||||
return applyWhere(fmt.Sprintf("%s = ?", qualifiedColumn), filter.Value)
|
return applyWhere(fmt.Sprintf("%s = ?", qualifiedColumn), filter.Value)
|
||||||
@@ -2330,6 +2377,18 @@ func (h *Handler) applyFilter(query common.SelectQuery, filter common.FilterOpti
|
|||||||
colName := h.qualifyColumnName(filter.Column, tableName)
|
colName := h.qualifyColumnName(filter.Column, tableName)
|
||||||
return applyWhere(fmt.Sprintf("(%s IS NOT NULL AND %s != '')", colName, colName))
|
return applyWhere(fmt.Sprintf("(%s IS NOT NULL AND %s != '')", colName, colName))
|
||||||
default:
|
default:
|
||||||
|
if common.IsSpatialOperator(filter.Operator) {
|
||||||
|
if cond, sargs, ok := common.BuildSpatialCondition(rawQualifiedColumn, filter.Operator, filter.Value); ok {
|
||||||
|
return applyWhere(cond, sargs...)
|
||||||
|
}
|
||||||
|
return query
|
||||||
|
}
|
||||||
|
if common.IsVectorOperator(filter.Operator) {
|
||||||
|
if cond, vargs, ok := common.BuildVectorCondition(rawQualifiedColumn, filter.Operator, filter.Value); ok {
|
||||||
|
return applyWhere(cond, vargs...)
|
||||||
|
}
|
||||||
|
return query
|
||||||
|
}
|
||||||
logger.Warn("Unknown filter operator: %s, defaulting to equals", filter.Operator)
|
logger.Warn("Unknown filter operator: %s, defaulting to equals", filter.Operator)
|
||||||
return applyWhere(fmt.Sprintf("%s = ?", qualifiedColumn), filter.Value)
|
return applyWhere(fmt.Sprintf("%s = ?", qualifiedColumn), filter.Value)
|
||||||
}
|
}
|
||||||
@@ -2337,16 +2396,25 @@ func (h *Handler) applyFilter(query common.SelectQuery, filter common.FilterOpti
|
|||||||
|
|
||||||
// applyOrFilterGroup applies a group of OR filters as a single grouped condition
|
// applyOrFilterGroup applies a group of OR filters as a single grouped condition
|
||||||
// This ensures OR conditions are properly grouped with parentheses to prevent OR logic from escaping
|
// This ensures OR conditions are properly grouped with parentheses to prevent OR logic from escaping
|
||||||
func (h *Handler) applyOrFilterGroup(query common.SelectQuery, filters []*common.FilterOption, castInfo []ColumnCastInfo, tableName string) common.SelectQuery {
|
func (h *Handler) applyOrFilterGroup(query common.SelectQuery, filters []*common.FilterOption, castInfo []ColumnCastInfo, tableName string, model interface{}) common.SelectQuery {
|
||||||
if len(filters) == 0 {
|
if len(filters) == 0 {
|
||||||
return query
|
return query
|
||||||
}
|
}
|
||||||
|
|
||||||
|
tableAlias := reflection.ExtractTableNameOnly(tableName)
|
||||||
|
|
||||||
// Build individual filter conditions
|
// Build individual filter conditions
|
||||||
conditions := []string{}
|
conditions := []string{}
|
||||||
args := []interface{}{}
|
args := []interface{}{}
|
||||||
|
|
||||||
for i, filter := range filters {
|
for i, filter := range filters {
|
||||||
|
// JSON sub-field access: resolve to a safe parameterised condition first.
|
||||||
|
if cond, jargs, ok := common.BuildJSONFilterCondition(model, tableAlias, filter.Column, filter.Operator, filter.Value); ok {
|
||||||
|
conditions = append(conditions, cond)
|
||||||
|
args = append(args, jargs...)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
// Qualify the column name with table name if not already qualified
|
// Qualify the column name with table name if not already qualified
|
||||||
rawQualifiedColumn := h.qualifyColumnName(filter.Column, tableName)
|
rawQualifiedColumn := h.qualifyColumnName(filter.Column, tableName)
|
||||||
qualifiedColumn := rawQualifiedColumn
|
qualifiedColumn := rawQualifiedColumn
|
||||||
@@ -2429,6 +2497,20 @@ func (h *Handler) buildFilterCondition(qualifiedColumn string, filter *common.Fi
|
|||||||
colName := h.qualifyColumnName(filter.Column, tableName)
|
colName := h.qualifyColumnName(filter.Column, tableName)
|
||||||
return fmt.Sprintf("(%s IS NOT NULL AND %s != '')", colName, colName), nil
|
return fmt.Sprintf("(%s IS NOT NULL AND %s != '')", colName, colName), nil
|
||||||
default:
|
default:
|
||||||
|
if common.IsSpatialOperator(filter.Operator) {
|
||||||
|
rawCol := h.qualifyColumnName(filter.Column, tableName)
|
||||||
|
if cond, sargs, ok := common.BuildSpatialCondition(rawCol, filter.Operator, filter.Value); ok {
|
||||||
|
return cond, sargs
|
||||||
|
}
|
||||||
|
return "", nil
|
||||||
|
}
|
||||||
|
if common.IsVectorOperator(filter.Operator) {
|
||||||
|
rawCol := h.qualifyColumnName(filter.Column, tableName)
|
||||||
|
if cond, vargs, ok := common.BuildVectorCondition(rawCol, filter.Operator, filter.Value); ok {
|
||||||
|
return cond, vargs
|
||||||
|
}
|
||||||
|
return "", nil
|
||||||
|
}
|
||||||
logger.Warn("Unknown filter operator: %s, defaulting to equals", filter.Operator)
|
logger.Warn("Unknown filter operator: %s, defaulting to equals", filter.Operator)
|
||||||
return fmt.Sprintf("%s = ?", qualifiedColumn), []interface{}{filter.Value}
|
return fmt.Sprintf("%s = ?", qualifiedColumn), []interface{}{filter.Value}
|
||||||
}
|
}
|
||||||
@@ -2551,10 +2633,18 @@ func (h *Handler) generateMetadata(schema, entity string, model interface{}) *co
|
|||||||
jsonName = field.Name
|
jsonName = field.Name
|
||||||
}
|
}
|
||||||
|
|
||||||
// Check if this is a relation field (slice or struct, but not time.Time)
|
columnType := field.Type
|
||||||
if field.Type.Kind() == reflect.Slice ||
|
isSQLType := false
|
||||||
(field.Type.Kind() == reflect.Struct && field.Type.Name() != "Time") ||
|
if unwrappedType, ok := unwrapSQLType(field.Type); ok {
|
||||||
(field.Type.Kind() == reflect.Pointer && field.Type.Elem().Kind() == reflect.Struct && field.Type.Elem().Name() != "Time") {
|
columnType = unwrappedType
|
||||||
|
isSQLType = true
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if this is a relation field (slice or struct, but not time.Time).
|
||||||
|
// spectypes SQL values are columns even when their Go representation is a
|
||||||
|
// struct or a slice.
|
||||||
|
if !isSQLType && (columnType.Kind() == reflect.Slice ||
|
||||||
|
(columnType.Kind() == reflect.Struct && columnType.Name() != "Time")) {
|
||||||
metadata.Relations = append(metadata.Relations, jsonName)
|
metadata.Relations = append(metadata.Relations, jsonName)
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
@@ -2575,8 +2665,8 @@ func (h *Handler) generateMetadata(schema, entity string, model interface{}) *co
|
|||||||
|
|
||||||
column := common.Column{
|
column := common.Column{
|
||||||
Name: columnName,
|
Name: columnName,
|
||||||
Type: h.getColumnType(field.Type),
|
Type: h.getColumnType(columnType),
|
||||||
IsNullable: h.isNullable(field),
|
IsNullable: isSQLType || h.isNullable(field),
|
||||||
IsPrimary: strings.Contains(gormTag, "primaryKey") || strings.Contains(gormTag, "primary_key"),
|
IsPrimary: strings.Contains(gormTag, "primaryKey") || strings.Contains(gormTag, "primary_key"),
|
||||||
IsUnique: strings.Contains(gormTag, "unique"),
|
IsUnique: strings.Contains(gormTag, "unique"),
|
||||||
HasIndex: strings.Contains(gormTag, "index"),
|
HasIndex: strings.Contains(gormTag, "index"),
|
||||||
@@ -2607,6 +2697,27 @@ func (h *Handler) getColumnType(t reflect.Type) string {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// unwrapSQLType returns the value type wrapped by a spectypes SQL value. These
|
||||||
|
// types represent columns, even when their Go representation is a struct or a
|
||||||
|
// slice (for example, SqlNull[string] and SqlJSONB).
|
||||||
|
func unwrapSQLType(t reflect.Type) (reflect.Type, bool) {
|
||||||
|
for t.Kind() == reflect.Pointer {
|
||||||
|
t = t.Elem()
|
||||||
|
}
|
||||||
|
|
||||||
|
if t.PkgPath() != "github.com/bitechdev/ResolveSpec/pkg/spectypes" {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
|
||||||
|
if t.Kind() == reflect.Struct {
|
||||||
|
if value, ok := t.FieldByName("Val"); ok {
|
||||||
|
return value.Type, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return t, true
|
||||||
|
}
|
||||||
|
|
||||||
func (h *Handler) isNullable(field reflect.StructField) bool {
|
func (h *Handler) isNullable(field reflect.StructField) bool {
|
||||||
return field.Type.Kind() == reflect.Pointer
|
return field.Type.Kind() == reflect.Pointer
|
||||||
}
|
}
|
||||||
@@ -2705,13 +2816,18 @@ func (h *Handler) buildDetailFields(model interface{}) []reflection.ModelFieldDe
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
// Skip relation fields (slices, structs that aren't time.Time, ptrs to struct)
|
// Skip relation fields (slices and structs that aren't time.Time). spectypes
|
||||||
|
// SQL values are columns, not relations.
|
||||||
ft := field.Type
|
ft := field.Type
|
||||||
if ft.Kind() == reflect.Pointer {
|
isSQLType := false
|
||||||
|
if unwrappedType, ok := unwrapSQLType(ft); ok {
|
||||||
|
ft = unwrappedType
|
||||||
|
isSQLType = true
|
||||||
|
} else if ft.Kind() == reflect.Pointer {
|
||||||
ft = ft.Elem()
|
ft = ft.Elem()
|
||||||
}
|
}
|
||||||
if ft.Kind() == reflect.Slice ||
|
if !isSQLType && (ft.Kind() == reflect.Slice ||
|
||||||
(ft.Kind() == reflect.Struct && ft.Name() != "Time") {
|
(ft.Kind() == reflect.Struct && ft.Name() != "Time")) {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -2739,7 +2855,7 @@ func (h *Handler) buildDetailFields(model interface{}) []reflection.ModelFieldDe
|
|||||||
sqlKey = "unique"
|
sqlKey = "unique"
|
||||||
}
|
}
|
||||||
|
|
||||||
nullable := field.Type.Kind() == reflect.Pointer
|
nullable := isSQLType || field.Type.Kind() == reflect.Pointer
|
||||||
if strings.Contains(gormLower, "not null") {
|
if strings.Contains(gormLower, "not null") {
|
||||||
nullable = false
|
nullable = false
|
||||||
} else if strings.Contains(gormLower, "nullable") || strings.Contains(gormLower, ",null") {
|
} else if strings.Contains(gormLower, "nullable") || strings.Contains(gormLower, ",null") {
|
||||||
@@ -2748,7 +2864,7 @@ func (h *Handler) buildDetailFields(model interface{}) []reflection.ModelFieldDe
|
|||||||
|
|
||||||
fields = append(fields, reflection.ModelFieldDetail{
|
fields = append(fields, reflection.ModelFieldDetail{
|
||||||
Name: jsonName,
|
Name: jsonName,
|
||||||
DataType: h.getColumnType(field.Type),
|
DataType: h.getColumnType(ft),
|
||||||
SQLName: sqlName,
|
SQLName: sqlName,
|
||||||
SQLDataType: sqlDataType,
|
SQLDataType: sqlDataType,
|
||||||
SQLKey: sqlKey,
|
SQLKey: sqlKey,
|
||||||
@@ -2814,8 +2930,21 @@ func (h *Handler) sendFormattedResponse(w common.ResponseWriter, data interface{
|
|||||||
switch options.ResponseFormat {
|
switch options.ResponseFormat {
|
||||||
case "simple":
|
case "simple":
|
||||||
// Simple format: just return the data array
|
// Simple format: just return the data array
|
||||||
|
jsonData, err := json.Marshal(data)
|
||||||
|
if err != nil {
|
||||||
|
logger.Error("Failed to marshal JSON response: %v", err)
|
||||||
|
w.WriteHeader(http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if string(jsonData) == "null" {
|
||||||
|
if options.SingleRecordAsObject {
|
||||||
|
jsonData = []byte("{}")
|
||||||
|
} else {
|
||||||
|
jsonData = []byte("[]")
|
||||||
|
}
|
||||||
|
}
|
||||||
w.WriteHeader(http.StatusOK)
|
w.WriteHeader(http.StatusOK)
|
||||||
if err := w.WriteJSON(data); err != nil {
|
if _, err := w.Write(jsonData); err != nil {
|
||||||
logger.Error("Failed to write JSON response: %v", err)
|
logger.Error("Failed to write JSON response: %v", err)
|
||||||
}
|
}
|
||||||
case "syncfusion":
|
case "syncfusion":
|
||||||
|
|||||||
@@ -182,6 +182,22 @@ func (h *Handler) parseOptionsFromHeaders(r common.Request, model interface{}) E
|
|||||||
h.parseSearchOp(&options, key, decodedValue, "AND")
|
h.parseSearchOp(&options, key, decodedValue, "AND")
|
||||||
case strings.HasPrefix(key, "x-searchcols"):
|
case strings.HasPrefix(key, "x-searchcols"):
|
||||||
options.SearchColumns = h.parseCommaSeparated(decodedValue)
|
options.SearchColumns = h.parseCommaSeparated(decodedValue)
|
||||||
|
case strings.HasPrefix(key, "x-spatialfilter-"):
|
||||||
|
h.parseGeoFilter(&options, key, "x-spatialfilter-", decodedValue)
|
||||||
|
case strings.HasPrefix(key, "x-vectorfilter-"):
|
||||||
|
h.parseGeoFilter(&options, key, "x-vectorfilter-", decodedValue)
|
||||||
|
|
||||||
|
// pgvector KNN search
|
||||||
|
case key == "x-vector-search-vector":
|
||||||
|
h.ensureVectorSearch(&options).Vector = parseFloat32List(decodedValue)
|
||||||
|
case key == "x-vector-search-as":
|
||||||
|
h.ensureVectorSearch(&options).As = decodedValue
|
||||||
|
case key == "x-vector-search-dir":
|
||||||
|
h.ensureVectorSearch(&options).Direction = decodedValue
|
||||||
|
case strings.HasPrefix(key, "x-vector-search-"):
|
||||||
|
vs := h.ensureVectorSearch(&options)
|
||||||
|
vs.Column = strings.TrimPrefix(key, "x-vector-search-")
|
||||||
|
vs.Metric = decodedValue
|
||||||
case strings.HasPrefix(key, "x-custom-sql-w"):
|
case strings.HasPrefix(key, "x-custom-sql-w"):
|
||||||
if options.CustomSQLWhere != "" {
|
if options.CustomSQLWhere != "" {
|
||||||
options.CustomSQLWhere = fmt.Sprintf("%s AND (%s)", options.CustomSQLWhere, decodedValue)
|
options.CustomSQLWhere = fmt.Sprintf("%s AND (%s)", options.CustomSQLWhere, decodedValue)
|
||||||
@@ -309,6 +325,83 @@ func (h *Handler) parseOptionsFromHeaders(r common.Request, model interface{}) E
|
|||||||
return options
|
return options
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ensureVectorSearch returns the options' VectorSearchOption, allocating it on
|
||||||
|
// first use.
|
||||||
|
func (h *Handler) ensureVectorSearch(options *ExtendedRequestOptions) *common.VectorSearchOption {
|
||||||
|
if options.VectorSearch == nil {
|
||||||
|
options.VectorSearch = &common.VectorSearchOption{}
|
||||||
|
}
|
||||||
|
return options.VectorSearch
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseFloat32List parses a JSON array ("[1,2,3]") or comma-separated list into
|
||||||
|
// a []float32.
|
||||||
|
func parseFloat32List(value string) []float32 {
|
||||||
|
value = strings.TrimSpace(value)
|
||||||
|
if value == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var raw []float64
|
||||||
|
if err := json.Unmarshal([]byte(value), &raw); err == nil {
|
||||||
|
out := make([]float32, len(raw))
|
||||||
|
for i, f := range raw {
|
||||||
|
out[i] = float32(f)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
parts := strings.Split(strings.Trim(value, "[]"), ",")
|
||||||
|
out := make([]float32, 0, len(parts))
|
||||||
|
for _, p := range parts {
|
||||||
|
f, err := strconv.ParseFloat(strings.TrimSpace(p), 32)
|
||||||
|
if err != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
out = append(out, float32(f))
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseGeoFilter parses an x-spatialfilter-<col> / x-vectorfilter-<col> header.
|
||||||
|
// The value is a JSON object: {"op":"st_dwithin","geom":...,"distance":...} or
|
||||||
|
// {"op":"st_intersects","value":<geojson>}. An optional "logic":"or" controls
|
||||||
|
// how the filter combines with the previous one.
|
||||||
|
func (h *Handler) parseGeoFilter(options *ExtendedRequestOptions, key, prefix, value string) {
|
||||||
|
col := strings.TrimPrefix(key, prefix)
|
||||||
|
if col == "" || strings.TrimSpace(value) == "" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var raw map[string]interface{}
|
||||||
|
if err := json.Unmarshal([]byte(value), &raw); err != nil {
|
||||||
|
logger.Warn("Invalid %s%s filter JSON: %v", prefix, col, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
op, _ := raw["op"].(string)
|
||||||
|
if op == "" {
|
||||||
|
logger.Warn("%s%s filter missing \"op\"", prefix, col)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
logicOp := "AND"
|
||||||
|
if lo, ok := raw["logic"].(string); ok && strings.EqualFold(lo, "or") {
|
||||||
|
logicOp = "OR"
|
||||||
|
}
|
||||||
|
|
||||||
|
var fv interface{}
|
||||||
|
if v, ok := raw["value"]; ok {
|
||||||
|
fv = v
|
||||||
|
} else {
|
||||||
|
delete(raw, "op")
|
||||||
|
delete(raw, "logic")
|
||||||
|
fv = raw
|
||||||
|
}
|
||||||
|
|
||||||
|
options.Filters = append(options.Filters, common.FilterOption{
|
||||||
|
Column: col,
|
||||||
|
Operator: op,
|
||||||
|
Value: fv,
|
||||||
|
LogicOperator: logicOp,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
// parseSelectFields parses x-select-fields header
|
// parseSelectFields parses x-select-fields header
|
||||||
func (h *Handler) parseSelectFields(options *ExtendedRequestOptions, value string) {
|
func (h *Handler) parseSelectFields(options *ExtendedRequestOptions, value string) {
|
||||||
if value == "" {
|
if value == "" {
|
||||||
@@ -1365,6 +1458,14 @@ func (h *Handler) ValidateAndAdjustFilterForColumnType(filter *common.FilterOpti
|
|||||||
return ColumnCastInfo{NeedsCast: false, IsNumericType: false}
|
return ColumnCastInfo{NeedsCast: false, IsNumericType: false}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Never cast geometry/geography or pgvector columns to TEXT — spatial and
|
||||||
|
// vector operators need the native column type. Also bypass when the
|
||||||
|
// operator itself is spatial/vector (e.g. st_dwithin, l2_within).
|
||||||
|
if common.IsSpatialOperator(filter.Operator) || common.IsVectorOperator(filter.Operator) ||
|
||||||
|
reflection.IsSpatialColumn(model, filter.Column) || reflection.IsVectorColumn(model, filter.Column) {
|
||||||
|
return ColumnCastInfo{NeedsCast: false, IsNumericType: false}
|
||||||
|
}
|
||||||
|
|
||||||
colType := reflection.GetColumnTypeFromModel(model, filter.Column)
|
colType := reflection.GetColumnTypeFromModel(model, filter.Column)
|
||||||
if colType == reflect.Invalid {
|
if colType == reflect.Invalid {
|
||||||
// Column not found in model, no casting needed
|
// Column not found in model, no casting needed
|
||||||
|
|||||||
@@ -0,0 +1,149 @@
|
|||||||
|
package restheadspec
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/common"
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/spectypes"
|
||||||
|
)
|
||||||
|
|
||||||
|
// jsonColModel exercises the JSON-column wiring: Data is a real JSONB column so
|
||||||
|
// the dotted "data.x" shorthand is recognised as JSON access.
|
||||||
|
type jsonColModel struct {
|
||||||
|
ID int64 `json:"id" bun:"id,pk"`
|
||||||
|
Name string `json:"name" bun:"name"`
|
||||||
|
Data spectypes.SqlJSONB `json:"data" bun:"data"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// jsonCapQuery is a minimal common.SelectQuery that records the string + args of
|
||||||
|
// the calls the handler makes so a test can assert on them.
|
||||||
|
type jsonCapQuery struct {
|
||||||
|
calls []jsonCapCall
|
||||||
|
}
|
||||||
|
|
||||||
|
type jsonCapCall struct {
|
||||||
|
method string
|
||||||
|
query string
|
||||||
|
args []interface{}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *jsonCapQuery) rec(method, query string, args []interface{}) common.SelectQuery {
|
||||||
|
m.calls = append(m.calls, jsonCapCall{method: method, query: query, args: args})
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *jsonCapQuery) Model(interface{}) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) Table(string) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) Column(cols ...string) common.SelectQuery {
|
||||||
|
for _, c := range cols {
|
||||||
|
m.rec("Column", c, nil)
|
||||||
|
}
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) ColumnExpr(q string, args ...interface{}) common.SelectQuery {
|
||||||
|
return m.rec("ColumnExpr", q, args)
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) Where(q string, args ...interface{}) common.SelectQuery {
|
||||||
|
return m.rec("Where", q, args)
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) WhereOr(q string, args ...interface{}) common.SelectQuery {
|
||||||
|
return m.rec("WhereOr", q, args)
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) WhereIn(col string, values interface{}) common.SelectQuery {
|
||||||
|
return m.rec("WhereIn", col, []interface{}{values})
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) Order(o string) common.SelectQuery { return m.rec("Order", o, nil) }
|
||||||
|
func (m *jsonCapQuery) OrderExpr(o string, args ...interface{}) common.SelectQuery {
|
||||||
|
return m.rec("OrderExpr", o, args)
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) Limit(int) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) Offset(int) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) Join(string, ...interface{}) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) LeftJoin(string, ...interface{}) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) Group(string) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) Having(string, ...interface{}) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) Preload(string, ...interface{}) common.SelectQuery { return m }
|
||||||
|
func (m *jsonCapQuery) PreloadRelation(string, ...func(common.SelectQuery) common.SelectQuery) common.SelectQuery {
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) JoinRelation(string, ...func(common.SelectQuery) common.SelectQuery) common.SelectQuery {
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
func (m *jsonCapQuery) Scan(context.Context, interface{}) error { return nil }
|
||||||
|
func (m *jsonCapQuery) ScanModel(context.Context) error { return nil }
|
||||||
|
func (m *jsonCapQuery) Count(context.Context) (int, error) { return 0, nil }
|
||||||
|
func (m *jsonCapQuery) Exists(context.Context) (bool, error) { return false, nil }
|
||||||
|
func (m *jsonCapQuery) GetUnderlyingQuery() interface{} { return nil }
|
||||||
|
func (m *jsonCapQuery) GetModel() interface{} { return nil }
|
||||||
|
|
||||||
|
func (m *jsonCapQuery) only(t *testing.T) jsonCapCall {
|
||||||
|
t.Helper()
|
||||||
|
if len(m.calls) != 1 {
|
||||||
|
t.Fatalf("expected exactly 1 recorded call, got %d: %+v", len(m.calls), m.calls)
|
||||||
|
}
|
||||||
|
return m.calls[0]
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestApplyFilter_JSONColumn(t *testing.T) {
|
||||||
|
h := &Handler{}
|
||||||
|
model := jsonColModel{}
|
||||||
|
|
||||||
|
t.Run("arrow syntax eq", func(t *testing.T) {
|
||||||
|
q := &jsonCapQuery{}
|
||||||
|
h.applyFilter(q, common.FilterOption{
|
||||||
|
Column: "data->>'city'", Operator: "eq", Value: "LA",
|
||||||
|
}, "public.things", false, "AND", model)
|
||||||
|
c := q.only(t)
|
||||||
|
if c.method != "Where" || c.query != `("things"."data" #>> ?::text[]) = ?` {
|
||||||
|
t.Fatalf("call = %+v", c)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(c.args, []interface{}{"{city}", "LA"}) {
|
||||||
|
t.Fatalf("args = %#v", c.args)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("dotted shorthand with numeric cast inference, OR logic", func(t *testing.T) {
|
||||||
|
q := &jsonCapQuery{}
|
||||||
|
h.applyFilter(q, common.FilterOption{
|
||||||
|
Column: "data.age", Operator: "gt", Value: 18,
|
||||||
|
}, "public.things", false, "OR", model)
|
||||||
|
c := q.only(t)
|
||||||
|
if c.method != "WhereOr" || c.query != `(("things"."data" #>> ?::text[]))::numeric > ?` {
|
||||||
|
t.Fatalf("call = %+v", c)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(c.args, []interface{}{"{age}", 18}) {
|
||||||
|
t.Fatalf("args = %#v", c.args)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("non-JSON column is untouched", func(t *testing.T) {
|
||||||
|
q := &jsonCapQuery{}
|
||||||
|
h.applyFilter(q, common.FilterOption{
|
||||||
|
Column: "name", Operator: "eq", Value: "x",
|
||||||
|
}, "public.things", false, "AND", model)
|
||||||
|
c := q.only(t)
|
||||||
|
if c.query != "things.name = ?" {
|
||||||
|
t.Fatalf("call = %+v", c)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("nil model: explicit syntax still works, dotted does not", func(t *testing.T) {
|
||||||
|
q := &jsonCapQuery{}
|
||||||
|
h.applyFilter(q, common.FilterOption{
|
||||||
|
Column: "data->>'city'", Operator: "eq", Value: "LA",
|
||||||
|
}, "public.things", false, "AND", nil)
|
||||||
|
if c := q.only(t); c.query != `("things"."data" #>> ?::text[]) = ?` {
|
||||||
|
t.Fatalf("explicit call = %+v", c)
|
||||||
|
}
|
||||||
|
|
||||||
|
q2 := &jsonCapQuery{}
|
||||||
|
h.applyFilter(q2, common.FilterOption{
|
||||||
|
Column: "data.city", Operator: "eq", Value: "LA",
|
||||||
|
}, "public.things", false, "AND", nil)
|
||||||
|
if c := q2.only(t); c.query == `("things"."data" #>> ?::text[]) = ?` {
|
||||||
|
t.Fatalf("dotted shorthand should not resolve without a model: %+v", c)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -0,0 +1,124 @@
|
|||||||
|
package restheadspec
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/bitechdev/ResolveSpec/pkg/common"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestBuildFilterCondition_Spatial(t *testing.T) {
|
||||||
|
h := &Handler{}
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
filter common.FilterOption
|
||||||
|
wantCond string
|
||||||
|
wantCount int
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "st_dwithin",
|
||||||
|
filter: common.FilterOption{
|
||||||
|
Column: "geom",
|
||||||
|
Operator: "st_dwithin",
|
||||||
|
Value: map[string]interface{}{
|
||||||
|
"geom": "SRID=4326;POINT(0 0)",
|
||||||
|
"distance": 1000.0,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
wantCond: "ST_DWithin(geom, ST_GeomFromEWKT(?), ?)",
|
||||||
|
wantCount: 2,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "st_intersects",
|
||||||
|
filter: common.FilterOption{
|
||||||
|
Column: "geom",
|
||||||
|
Operator: "st_intersects",
|
||||||
|
Value: "SRID=4326;POINT(0 0)",
|
||||||
|
},
|
||||||
|
wantCond: "ST_Intersects(geom, ST_GeomFromEWKT(?))",
|
||||||
|
wantCount: 1,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "l2_within",
|
||||||
|
filter: common.FilterOption{
|
||||||
|
Column: "embedding",
|
||||||
|
Operator: "l2_within",
|
||||||
|
Value: map[string]interface{}{
|
||||||
|
"vector": []interface{}{1.0, 2.0},
|
||||||
|
"distance": 0.3,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
wantCond: "embedding <-> ? < ?",
|
||||||
|
wantCount: 2,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
f := tt.filter
|
||||||
|
cond, args := h.buildFilterCondition(f.Column, &f, "")
|
||||||
|
if cond != tt.wantCond {
|
||||||
|
t.Errorf("cond = %q, want %q", cond, tt.wantCond)
|
||||||
|
}
|
||||||
|
if len(args) != tt.wantCount {
|
||||||
|
t.Errorf("args = %d, want %d", len(args), tt.wantCount)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseGeoFilter(t *testing.T) {
|
||||||
|
h := &Handler{}
|
||||||
|
options := &ExtendedRequestOptions{}
|
||||||
|
|
||||||
|
h.parseGeoFilter(options, "x-spatialfilter-geom", "x-spatialfilter-",
|
||||||
|
`{"op":"st_dwithin","geom":"SRID=4326;POINT(0 0)","distance":500}`)
|
||||||
|
|
||||||
|
if len(options.Filters) != 1 {
|
||||||
|
t.Fatalf("expected 1 filter, got %d", len(options.Filters))
|
||||||
|
}
|
||||||
|
f := options.Filters[0]
|
||||||
|
if f.Column != "geom" || f.Operator != "st_dwithin" {
|
||||||
|
t.Errorf("filter = %+v", f)
|
||||||
|
}
|
||||||
|
m, ok := f.Value.(map[string]interface{})
|
||||||
|
if !ok || m["distance"] != float64(500) {
|
||||||
|
t.Errorf("value = %v", f.Value)
|
||||||
|
}
|
||||||
|
if _, has := m["op"]; has {
|
||||||
|
t.Error("op should be stripped from value map")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseGeoFilter_ExplicitValue(t *testing.T) {
|
||||||
|
h := &Handler{}
|
||||||
|
options := &ExtendedRequestOptions{}
|
||||||
|
|
||||||
|
h.parseGeoFilter(options, "x-vectorfilter-embedding", "x-vectorfilter-",
|
||||||
|
`{"op":"cosine_within","logic":"or","value":{"vector":[1,2,3],"distance":0.2}}`)
|
||||||
|
|
||||||
|
if len(options.Filters) != 1 {
|
||||||
|
t.Fatalf("expected 1 filter, got %d", len(options.Filters))
|
||||||
|
}
|
||||||
|
f := options.Filters[0]
|
||||||
|
if f.Operator != "cosine_within" || f.LogicOperator != "OR" {
|
||||||
|
t.Errorf("filter = %+v", f)
|
||||||
|
}
|
||||||
|
if _, ok := f.Value.(map[string]interface{}); !ok {
|
||||||
|
t.Errorf("value type = %T", f.Value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseFloat32List(t *testing.T) {
|
||||||
|
got := parseFloat32List("[1,2.5,3]")
|
||||||
|
if len(got) != 3 || got[1] != 2.5 {
|
||||||
|
t.Errorf("json array = %v", got)
|
||||||
|
}
|
||||||
|
got = parseFloat32List("1, 2, 3")
|
||||||
|
if len(got) != 3 || got[2] != 3 {
|
||||||
|
t.Errorf("csv = %v", got)
|
||||||
|
}
|
||||||
|
if parseFloat32List("") != nil {
|
||||||
|
t.Error("empty should be nil")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -551,11 +551,27 @@ func (a *DatabaseAuthenticator) RefreshToken(ctx context.Context, refreshToken s
|
|||||||
return nil, fmt.Errorf("failed to parse user context: %w", err)
|
return nil, fmt.Errorf("failed to parse user context: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
return &LoginResponse{
|
// A resolvespec_refresh_token implementation that issues its own rotating
|
||||||
|
// refresh token (independent of the access/session token) returns it
|
||||||
|
// under claims.refresh_token, since UserContext has no dedicated field
|
||||||
|
// for it. Surface that into LoginResponse.RefreshToken so callers don't
|
||||||
|
// need to reach into User.Claims themselves. claims.expires_in
|
||||||
|
// (seconds) similarly overrides the default access-token ExpiresIn when
|
||||||
|
// the procedure provides a real value. Implementations that don't set
|
||||||
|
// these claims keep today's behavior unchanged (empty RefreshToken,
|
||||||
|
// 24h ExpiresIn default).
|
||||||
|
resp := &LoginResponse{
|
||||||
Token: userCtx.SessionID, // New session token from stored procedure
|
Token: userCtx.SessionID, // New session token from stored procedure
|
||||||
User: &userCtx,
|
User: &userCtx,
|
||||||
ExpiresIn: int64(24 * time.Hour.Seconds()),
|
ExpiresIn: int64(24 * time.Hour.Seconds()),
|
||||||
}, nil
|
}
|
||||||
|
if refreshToken, ok := userCtx.Claims["refresh_token"].(string); ok && refreshToken != "" {
|
||||||
|
resp.RefreshToken = refreshToken
|
||||||
|
}
|
||||||
|
if expiresIn, ok := userCtx.Claims["expires_in"].(float64); ok && expiresIn > 0 {
|
||||||
|
resp.ExpiresIn = int64(expiresIn)
|
||||||
|
}
|
||||||
|
return resp, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// JWTAuthenticator provides JWT token-based authentication
|
// JWTAuthenticator provides JWT token-based authentication
|
||||||
@@ -924,8 +940,8 @@ func (p *DatabaseRowSecurityProvider) GetRowSecurity(ctx context.Context, userRe
|
|||||||
userRef = v.UserID
|
userRef = v.UserID
|
||||||
}
|
}
|
||||||
|
|
||||||
var template string
|
var template sql.NullString
|
||||||
var hasBlock bool
|
var hasBlock sql.NullBool
|
||||||
|
|
||||||
runQuery := func() error {
|
runQuery := func() error {
|
||||||
query := fmt.Sprintf(`SELECT p_template, p_block FROM %s($1, $2, $3)`, p.sqlNames.RowSecurity)
|
query := fmt.Sprintf(`SELECT p_template, p_block FROM %s($1, $2, $3)`, p.sqlNames.RowSecurity)
|
||||||
@@ -945,8 +961,8 @@ func (p *DatabaseRowSecurityProvider) GetRowSecurity(ctx context.Context, userRe
|
|||||||
Schema: schema,
|
Schema: schema,
|
||||||
Tablename: table,
|
Tablename: table,
|
||||||
UserID: userRef,
|
UserID: userRef,
|
||||||
Template: template,
|
Template: template.String,
|
||||||
HasBlock: hasBlock,
|
HasBlock: hasBlock.Bool,
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -793,6 +793,49 @@ func TestDatabaseAuthenticatorRefreshToken(t *testing.T) {
|
|||||||
t.Errorf("unfulfilled expectations: %v", err)
|
t.Errorf("unfulfilled expectations: %v", err)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
|
// A resolvespec_refresh_token implementation that rotates its own
|
||||||
|
// independent refresh token (not just reusing the session/access token)
|
||||||
|
// has nowhere else to put the new refresh token and real access-token
|
||||||
|
// expiry than under UserContext.Claims, since UserContext has no
|
||||||
|
// dedicated fields for either. RefreshToken must surface those claims
|
||||||
|
// keys into LoginResponse.RefreshToken/ExpiresIn rather than silently
|
||||||
|
// dropping them (see the "successful token refresh" case above, which
|
||||||
|
// covers an implementation that has no independent refresh token at all
|
||||||
|
// and gets the 24h default instead).
|
||||||
|
t.Run("surfaces rotated refresh token and expiry from claims", func(t *testing.T) {
|
||||||
|
refreshToken := "refresh-token-abc"
|
||||||
|
|
||||||
|
sessionRows := sqlmock.NewRows([]string{"p_success", "p_error", "p_user"}).
|
||||||
|
AddRow(true, nil, `{"user_id":1,"user_name":"testuser"}`)
|
||||||
|
mock.ExpectQuery(`SELECT p_success, p_error, p_user::text FROM resolvespec_session`).
|
||||||
|
WithArgs(refreshToken, "refresh").
|
||||||
|
WillReturnRows(sessionRows)
|
||||||
|
|
||||||
|
refreshRows := sqlmock.NewRows([]string{"p_success", "p_error", "p_user"}).
|
||||||
|
AddRow(true, nil, `{"user_id":1,"user_name":"testuser","session_id":"new-access-789","claims":{"refresh_token":"new-refresh-def","expires_in":900}}`)
|
||||||
|
mock.ExpectQuery(`SELECT p_success, p_error, p_user::text FROM resolvespec_refresh_token`).
|
||||||
|
WithArgs(refreshToken, sqlmock.AnyArg()).
|
||||||
|
WillReturnRows(refreshRows)
|
||||||
|
|
||||||
|
resp, err := auth.RefreshToken(ctx, refreshToken)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected no error, got %v", err)
|
||||||
|
}
|
||||||
|
if resp.Token != "new-access-789" {
|
||||||
|
t.Errorf("expected token new-access-789, got %s", resp.Token)
|
||||||
|
}
|
||||||
|
if resp.RefreshToken != "new-refresh-def" {
|
||||||
|
t.Errorf("expected rotated refresh token new-refresh-def, got %q", resp.RefreshToken)
|
||||||
|
}
|
||||||
|
if resp.ExpiresIn != 900 {
|
||||||
|
t.Errorf("expected ExpiresIn 900 from claims, got %d", resp.ExpiresIn)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := mock.ExpectationsWereMet(); err != nil {
|
||||||
|
t.Errorf("unfulfilled expectations: %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestDatabaseAuthenticatorReconnectsClosedDBPaths(t *testing.T) {
|
func TestDatabaseAuthenticatorReconnectsClosedDBPaths(t *testing.T) {
|
||||||
|
|||||||
@@ -0,0 +1,239 @@
|
|||||||
|
package spectypes
|
||||||
|
|
||||||
|
import (
|
||||||
|
"database/sql/driver"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ── PostGIS geometry / geography ─────────────────────────────────────────────
|
||||||
|
|
||||||
|
// SqlGeometry is a nullable PostGIS `geometry` column.
|
||||||
|
//
|
||||||
|
// On Scan it accepts the PostGIS default hex-EWKB text output, a raw GeoJSON
|
||||||
|
// object, or a WKT/EWKT string (e.g. when the column is selected via
|
||||||
|
// ST_AsGeoJSON / ST_AsText). Internally it holds a canonical GeoJSON geometry
|
||||||
|
// object plus the SRID.
|
||||||
|
//
|
||||||
|
// On Value it emits `SRID=<n>;<WKT>` text. PostGIS registers an implicit
|
||||||
|
// text -> geometry cast, so parameterised inserts/updates work without wrapping
|
||||||
|
// the placeholder in a constructor function.
|
||||||
|
//
|
||||||
|
// MarshalJSON emits the GeoJSON geometry object (or null).
|
||||||
|
type SqlGeometry struct {
|
||||||
|
GeoJSON json.RawMessage
|
||||||
|
SRID int
|
||||||
|
Valid bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// SqlGeography is identical to SqlGeometry but maps to a PostGIS `geography`
|
||||||
|
// column. Coordinates are always lon/lat and the default SRID is 4326.
|
||||||
|
type SqlGeography struct {
|
||||||
|
SqlGeometry
|
||||||
|
}
|
||||||
|
|
||||||
|
func (g *SqlGeometry) Scan(value any) error {
|
||||||
|
if value == nil {
|
||||||
|
g.Valid = false
|
||||||
|
g.GeoJSON = nil
|
||||||
|
g.SRID = 0
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var s string
|
||||||
|
switch v := value.(type) {
|
||||||
|
case string:
|
||||||
|
s = v
|
||||||
|
case []byte:
|
||||||
|
s = string(v)
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("SqlGeometry: cannot scan type %T", value)
|
||||||
|
}
|
||||||
|
|
||||||
|
s = strings.TrimSpace(s)
|
||||||
|
if s == "" {
|
||||||
|
g.Valid = false
|
||||||
|
g.GeoJSON = nil
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
switch {
|
||||||
|
case strings.HasPrefix(s, "{"):
|
||||||
|
// GeoJSON object.
|
||||||
|
if _, err := geoJSONToGeom([]byte(s)); err != nil {
|
||||||
|
return fmt.Errorf("SqlGeometry: invalid GeoJSON: %w", err)
|
||||||
|
}
|
||||||
|
g.GeoJSON = json.RawMessage(s)
|
||||||
|
g.Valid = true
|
||||||
|
return nil
|
||||||
|
case isHex(s):
|
||||||
|
gj, srid, err := DecodeEWKBHex(s)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("SqlGeometry: %w", err)
|
||||||
|
}
|
||||||
|
g.GeoJSON = gj
|
||||||
|
g.SRID = srid
|
||||||
|
g.Valid = true
|
||||||
|
return nil
|
||||||
|
default:
|
||||||
|
// WKT / EWKT text.
|
||||||
|
srid, wkt := splitEWKT(s)
|
||||||
|
gj, err := wktToGeoJSON(wkt)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("SqlGeometry: %w", err)
|
||||||
|
}
|
||||||
|
g.GeoJSON = gj
|
||||||
|
g.SRID = srid
|
||||||
|
g.Valid = true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (g SqlGeometry) Value() (driver.Value, error) {
|
||||||
|
if !g.Valid || len(g.GeoJSON) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
wkt, err := GeoJSONToWKT(g.GeoJSON)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
srid := g.SRID
|
||||||
|
if srid == 0 {
|
||||||
|
srid = 4326
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("SRID=%d;%s", srid, wkt), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (g SqlGeometry) MarshalJSON() ([]byte, error) {
|
||||||
|
if !g.Valid || len(g.GeoJSON) == 0 {
|
||||||
|
return []byte("null"), nil
|
||||||
|
}
|
||||||
|
return g.GeoJSON, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (g *SqlGeometry) UnmarshalJSON(b []byte) error {
|
||||||
|
s := strings.TrimSpace(string(b))
|
||||||
|
if s == "" || s == "null" {
|
||||||
|
g.Valid = false
|
||||||
|
g.GeoJSON = nil
|
||||||
|
g.SRID = 0
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
if strings.HasPrefix(s, "{") {
|
||||||
|
if _, err := geoJSONToGeom(b); err != nil {
|
||||||
|
return fmt.Errorf("SqlGeometry: invalid GeoJSON: %w", err)
|
||||||
|
}
|
||||||
|
g.GeoJSON = append(json.RawMessage(nil), b...)
|
||||||
|
g.Valid = true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// String value: EWKT / WKT / hex-EWKB.
|
||||||
|
var str string
|
||||||
|
if err := json.Unmarshal(b, &str); err != nil {
|
||||||
|
return fmt.Errorf("SqlGeometry: cannot unmarshal %s", b)
|
||||||
|
}
|
||||||
|
str = strings.TrimSpace(str)
|
||||||
|
if str == "" {
|
||||||
|
g.Valid = false
|
||||||
|
g.GeoJSON = nil
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if isHex(str) {
|
||||||
|
gj, srid, err := DecodeEWKBHex(str)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("SqlGeometry: %w", err)
|
||||||
|
}
|
||||||
|
g.GeoJSON = gj
|
||||||
|
g.SRID = srid
|
||||||
|
g.Valid = true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
srid, wkt := splitEWKT(str)
|
||||||
|
gj, err := wktToGeoJSON(wkt)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("SqlGeometry: %w", err)
|
||||||
|
}
|
||||||
|
g.GeoJSON = gj
|
||||||
|
g.SRID = srid
|
||||||
|
g.Valid = true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// WKT returns the geometry as a plain WKT string (no SRID prefix).
|
||||||
|
func (g SqlGeometry) WKT() string {
|
||||||
|
if !g.Valid || len(g.GeoJSON) == 0 {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
wkt, err := GeoJSONToWKT(g.GeoJSON)
|
||||||
|
if err != nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return wkt
|
||||||
|
}
|
||||||
|
|
||||||
|
// EWKT returns the geometry as `SRID=<n>;<WKT>`.
|
||||||
|
func (g SqlGeometry) EWKT() string {
|
||||||
|
wkt := g.WKT()
|
||||||
|
if wkt == "" {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
srid := g.SRID
|
||||||
|
if srid == 0 {
|
||||||
|
srid = 4326
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("SRID=%d;%s", srid, wkt)
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSqlGeometryFromGeoJSON builds a SqlGeometry from a GeoJSON geometry object.
|
||||||
|
func NewSqlGeometryFromGeoJSON(geojson []byte, srid int) (SqlGeometry, error) {
|
||||||
|
if _, err := geoJSONToGeom(geojson); err != nil {
|
||||||
|
return SqlGeometry{}, err
|
||||||
|
}
|
||||||
|
return SqlGeometry{GeoJSON: append(json.RawMessage(nil), geojson...), SRID: srid, Valid: true}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSqlGeometryFromEWKT builds a SqlGeometry from an EWKT or WKT string.
|
||||||
|
func NewSqlGeometryFromEWKT(ewkt string) (SqlGeometry, error) {
|
||||||
|
srid, wkt := splitEWKT(strings.TrimSpace(ewkt))
|
||||||
|
gj, err := wktToGeoJSON(wkt)
|
||||||
|
if err != nil {
|
||||||
|
return SqlGeometry{}, err
|
||||||
|
}
|
||||||
|
return SqlGeometry{GeoJSON: gj, SRID: srid, Valid: true}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── helpers ─────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
func isHex(s string) bool {
|
||||||
|
if len(s) < 10 || len(s)%2 != 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
_, err := hex.DecodeString(s)
|
||||||
|
return err == nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// splitEWKT separates an optional `SRID=<n>;` prefix from a WKT body.
|
||||||
|
func splitEWKT(s string) (srid int, wkt string) {
|
||||||
|
if strings.HasPrefix(strings.ToUpper(s), "SRID=") {
|
||||||
|
if idx := strings.Index(s, ";"); idx > 0 {
|
||||||
|
if n, err := strconv.Atoi(strings.TrimSpace(s[5:idx])); err == nil {
|
||||||
|
return n, strings.TrimSpace(s[idx+1:])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return 0, s
|
||||||
|
}
|
||||||
|
|
||||||
|
// wktToGeoJSON parses a (subset of) WKT into a GeoJSON geometry object.
|
||||||
|
func wktToGeoJSON(wkt string) ([]byte, error) {
|
||||||
|
g, err := parseWKT(wkt)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return geomToGeoJSON(g)
|
||||||
|
}
|
||||||
@@ -0,0 +1,126 @@
|
|||||||
|
package spectypes
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SRID=4326;POINT (1 2)
|
||||||
|
const pointHexEWKB = "0101000020E6100000000000000000F03F0000000000000040"
|
||||||
|
|
||||||
|
func TestSqlGeometry_ScanHexEWKB(t *testing.T) {
|
||||||
|
var g SqlGeometry
|
||||||
|
if err := g.Scan(pointHexEWKB); err != nil {
|
||||||
|
t.Fatalf("Scan: %v", err)
|
||||||
|
}
|
||||||
|
if !g.Valid || g.SRID != 4326 {
|
||||||
|
t.Fatalf("got Valid=%v SRID=%d", g.Valid, g.SRID)
|
||||||
|
}
|
||||||
|
if !jsonEqual(t, g.GeoJSON, `{"type":"Point","coordinates":[1,2]}`) {
|
||||||
|
t.Errorf("GeoJSON = %s", g.GeoJSON)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSqlGeometry_ScanGeoJSON(t *testing.T) {
|
||||||
|
var g SqlGeometry
|
||||||
|
if err := g.Scan(`{"type":"Point","coordinates":[3,4]}`); err != nil {
|
||||||
|
t.Fatalf("Scan: %v", err)
|
||||||
|
}
|
||||||
|
if !g.Valid {
|
||||||
|
t.Fatal("expected valid")
|
||||||
|
}
|
||||||
|
if !jsonEqual(t, g.GeoJSON, `{"type":"Point","coordinates":[3,4]}`) {
|
||||||
|
t.Errorf("GeoJSON = %s", g.GeoJSON)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSqlGeometry_ScanEWKT(t *testing.T) {
|
||||||
|
var g SqlGeometry
|
||||||
|
if err := g.Scan("SRID=3857;POINT (5 6)"); err != nil {
|
||||||
|
t.Fatalf("Scan: %v", err)
|
||||||
|
}
|
||||||
|
if g.SRID != 3857 {
|
||||||
|
t.Errorf("SRID = %d, want 3857", g.SRID)
|
||||||
|
}
|
||||||
|
if !jsonEqual(t, g.GeoJSON, `{"type":"Point","coordinates":[5,6]}`) {
|
||||||
|
t.Errorf("GeoJSON = %s", g.GeoJSON)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSqlGeometry_Value(t *testing.T) {
|
||||||
|
g, err := NewSqlGeometryFromGeoJSON([]byte(`{"type":"Point","coordinates":[1,2]}`), 4326)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("New: %v", err)
|
||||||
|
}
|
||||||
|
v, err := g.Value()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Value: %v", err)
|
||||||
|
}
|
||||||
|
if v != "SRID=4326;POINT (1 2)" {
|
||||||
|
t.Errorf("Value = %v, want SRID=4326;POINT (1 2)", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSqlGeometry_ValueDefaultsSRID(t *testing.T) {
|
||||||
|
g, _ := NewSqlGeometryFromGeoJSON([]byte(`{"type":"Point","coordinates":[1,2]}`), 0)
|
||||||
|
v, _ := g.Value()
|
||||||
|
if v != "SRID=4326;POINT (1 2)" {
|
||||||
|
t.Errorf("Value = %v", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSqlGeometry_JSON(t *testing.T) {
|
||||||
|
g, _ := NewSqlGeometryFromEWKT("SRID=4326;POINT (1 2)")
|
||||||
|
b, err := json.Marshal(g)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Marshal: %v", err)
|
||||||
|
}
|
||||||
|
if !jsonEqual(t, b, `{"type":"Point","coordinates":[1,2]}`) {
|
||||||
|
t.Errorf("json = %s", b)
|
||||||
|
}
|
||||||
|
|
||||||
|
var back SqlGeometry
|
||||||
|
if err := json.Unmarshal([]byte(`{"type":"Point","coordinates":[7,8]}`), &back); err != nil {
|
||||||
|
t.Fatalf("Unmarshal: %v", err)
|
||||||
|
}
|
||||||
|
if !back.Valid || !jsonEqual(t, back.GeoJSON, `{"type":"Point","coordinates":[7,8]}`) {
|
||||||
|
t.Errorf("unmarshal = %+v", back)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Unmarshal also accepts an EWKT string.
|
||||||
|
var fromStr SqlGeometry
|
||||||
|
if err := json.Unmarshal([]byte(`"SRID=4326;POINT(9 10)"`), &fromStr); err != nil {
|
||||||
|
t.Fatalf("Unmarshal string: %v", err)
|
||||||
|
}
|
||||||
|
if !jsonEqual(t, fromStr.GeoJSON, `{"type":"Point","coordinates":[9,10]}`) {
|
||||||
|
t.Errorf("fromStr = %s", fromStr.GeoJSON)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSqlGeometry_Null(t *testing.T) {
|
||||||
|
var g SqlGeometry
|
||||||
|
if err := g.Scan(nil); err != nil {
|
||||||
|
t.Fatalf("Scan(nil): %v", err)
|
||||||
|
}
|
||||||
|
if g.Valid {
|
||||||
|
t.Error("expected invalid")
|
||||||
|
}
|
||||||
|
v, err := g.Value()
|
||||||
|
if err != nil || v != nil {
|
||||||
|
t.Errorf("Value = %v, %v", v, err)
|
||||||
|
}
|
||||||
|
b, _ := json.Marshal(g)
|
||||||
|
if string(b) != "null" {
|
||||||
|
t.Errorf("json = %s", b)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSqlGeography_Embeds(t *testing.T) {
|
||||||
|
var g SqlGeography
|
||||||
|
if err := g.Scan("SRID=4326;POINT (1 2)"); err != nil {
|
||||||
|
t.Fatalf("Scan: %v", err)
|
||||||
|
}
|
||||||
|
if !g.Valid || !jsonEqual(t, g.GeoJSON, `{"type":"Point","coordinates":[1,2]}`) {
|
||||||
|
t.Errorf("geography scan = %+v", g)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -336,21 +336,21 @@ type (
|
|||||||
SqlUUID = SqlNull[uuid.UUID]
|
SqlUUID = SqlNull[uuid.UUID]
|
||||||
)
|
)
|
||||||
|
|
||||||
// SqlTimeStamp - Timestamp with custom formatting (YYYY-MM-DDTHH:MM:SS).
|
// SqlTimeStamp - Timestamp serialized as RFC3339 with timezone offset.
|
||||||
type SqlTimeStamp struct{ SqlNull[time.Time] }
|
type SqlTimeStamp struct{ SqlNull[time.Time] }
|
||||||
|
|
||||||
func (t SqlTimeStamp) MarshalJSON() ([]byte, error) {
|
func (t SqlTimeStamp) MarshalJSON() ([]byte, error) {
|
||||||
if !t.Valid || t.Val.IsZero() || t.Val.Before(time.Date(0002, 1, 1, 0, 0, 0, 0, time.UTC)) {
|
if !t.Valid || t.Val.IsZero() || t.Val.Before(time.Date(0002, 1, 1, 0, 0, 0, 0, time.UTC)) {
|
||||||
return []byte("null"), nil
|
return []byte("null"), nil
|
||||||
}
|
}
|
||||||
return []byte(fmt.Sprintf(`"%s"`, t.Val.Format("2006-01-02T15:04:05"))), nil
|
return []byte(fmt.Sprintf(`"%s"`, t.Val.Format(time.RFC3339))), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (t *SqlTimeStamp) UnmarshalJSON(b []byte) error {
|
func (t *SqlTimeStamp) UnmarshalJSON(b []byte) error {
|
||||||
if err := t.SqlNull.UnmarshalJSON(b); err != nil {
|
if err := t.SqlNull.UnmarshalJSON(b); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if t.Valid && (t.Val.IsZero() || t.Val.Format("2006-01-02T15:04:05") == "0001-01-01T00:00:00") {
|
if t.Valid && (t.Val.IsZero() || t.Val.Year() <= 1) {
|
||||||
t.Valid = false
|
t.Valid = false
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
@@ -360,7 +360,7 @@ func (t SqlTimeStamp) Value() (driver.Value, error) {
|
|||||||
if !t.Valid || t.Val.IsZero() || t.Val.Before(time.Date(0002, 1, 1, 0, 0, 0, 0, time.UTC)) {
|
if !t.Valid || t.Val.IsZero() || t.Val.Before(time.Date(0002, 1, 1, 0, 0, 0, 0, time.UTC)) {
|
||||||
return nil, nil
|
return nil, nil
|
||||||
}
|
}
|
||||||
return t.Val.Format("2006-01-02T15:04:05"), nil
|
return t.Val.Format(time.RFC3339), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func SqlTimeStampNow() SqlTimeStamp {
|
func SqlTimeStampNow() SqlTimeStamp {
|
||||||
|
|||||||
@@ -178,7 +178,7 @@ func TestSqlTimeStamp_JSON(t *testing.T) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Marshal failed: %v", err)
|
t.Fatalf("Marshal failed: %v", err)
|
||||||
}
|
}
|
||||||
expected := `"2024-01-15T10:30:45"`
|
expected := `"2024-01-15T10:30:45Z"`
|
||||||
if string(data) != expected {
|
if string(data) != expected {
|
||||||
t.Errorf("expected %s, got %s", expected, string(data))
|
t.Errorf("expected %s, got %s", expected, string(data))
|
||||||
}
|
}
|
||||||
@@ -955,4 +955,3 @@ func TestSqlByteArray_Base64_RoundTrip(t *testing.T) {
|
|||||||
t.Errorf("Round-trip failed: expected %v, got %v", original, b3.Val)
|
t.Errorf("Round-trip failed: expected %v, got %v", original, b3.Val)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,308 @@
|
|||||||
|
package spectypes
|
||||||
|
|
||||||
|
import (
|
||||||
|
"database/sql/driver"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// pgvector column types beyond the plain `vector` (SqlVector, in
|
||||||
|
// sql_array_types.go): `halfvec`, `sparsevec` and `bit`.
|
||||||
|
|
||||||
|
// parseVectorLiteral parses a pgvector dense literal `[1,2,3]` into []float32.
|
||||||
|
func parseVectorLiteral(s string) ([]float32, error) {
|
||||||
|
s = strings.TrimSpace(s)
|
||||||
|
if !strings.HasPrefix(s, "[") || !strings.HasSuffix(s, "]") {
|
||||||
|
return nil, fmt.Errorf("not a valid vector literal: %q", s)
|
||||||
|
}
|
||||||
|
inner := strings.TrimSpace(s[1 : len(s)-1])
|
||||||
|
if inner == "" {
|
||||||
|
return []float32{}, nil
|
||||||
|
}
|
||||||
|
parts := strings.Split(inner, ",")
|
||||||
|
out := make([]float32, len(parts))
|
||||||
|
for i, p := range parts {
|
||||||
|
f, err := strconv.ParseFloat(strings.TrimSpace(p), 32)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("vector element %d %q: %w", i, p, err)
|
||||||
|
}
|
||||||
|
out[i] = float32(f)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func formatVectorLiteral(vals []float32) string {
|
||||||
|
parts := make([]string, len(vals))
|
||||||
|
for i, v := range vals {
|
||||||
|
parts[i] = strconv.FormatFloat(float64(v), 'f', -1, 32)
|
||||||
|
}
|
||||||
|
return "[" + strings.Join(parts, ",") + "]"
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── SqlHalfVector ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
// SqlHalfVector is a nullable pgvector `halfvec` (half-precision) column, backed
|
||||||
|
// by []float32. Wire format matches `vector`: `[1,2,3]`.
|
||||||
|
type SqlHalfVector struct {
|
||||||
|
Val []float32
|
||||||
|
Valid bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (v *SqlHalfVector) Scan(value any) error {
|
||||||
|
if value == nil {
|
||||||
|
v.Valid = false
|
||||||
|
v.Val = nil
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var s string
|
||||||
|
switch val := value.(type) {
|
||||||
|
case string:
|
||||||
|
s = val
|
||||||
|
case []byte:
|
||||||
|
s = string(val)
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("SqlHalfVector: cannot scan type %T", value)
|
||||||
|
}
|
||||||
|
parsed, err := parseVectorLiteral(s)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("SqlHalfVector: %w", err)
|
||||||
|
}
|
||||||
|
v.Val = parsed
|
||||||
|
v.Valid = true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (v SqlHalfVector) Value() (driver.Value, error) {
|
||||||
|
if !v.Valid {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
return formatVectorLiteral(v.Val), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (v SqlHalfVector) MarshalJSON() ([]byte, error) {
|
||||||
|
if !v.Valid {
|
||||||
|
return []byte("null"), nil
|
||||||
|
}
|
||||||
|
return json.Marshal(v.Val)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (v *SqlHalfVector) UnmarshalJSON(b []byte) error {
|
||||||
|
if strings.TrimSpace(string(b)) == "null" {
|
||||||
|
v.Valid = false
|
||||||
|
v.Val = nil
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var vals []float32
|
||||||
|
if err := json.Unmarshal(b, &vals); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
v.Val = vals
|
||||||
|
v.Valid = true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewSqlHalfVector(val []float32) SqlHalfVector {
|
||||||
|
return SqlHalfVector{Val: val, Valid: true}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── SqlSparseVector ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
// SqlSparseVector is a nullable pgvector `sparsevec` column. Wire format:
|
||||||
|
// `{1:0.5,4:0.2}/8` (1-based indices). JSON:
|
||||||
|
// `{"dim":8,"indices":[1,4],"values":[0.5,0.2]}`.
|
||||||
|
type SqlSparseVector struct {
|
||||||
|
Dim int
|
||||||
|
Indices []int32
|
||||||
|
Values []float32
|
||||||
|
Valid bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (v *SqlSparseVector) Scan(value any) error {
|
||||||
|
if value == nil {
|
||||||
|
v.Valid = false
|
||||||
|
v.Dim, v.Indices, v.Values = 0, nil, nil
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var s string
|
||||||
|
switch val := value.(type) {
|
||||||
|
case string:
|
||||||
|
s = val
|
||||||
|
case []byte:
|
||||||
|
s = string(val)
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("SqlSparseVector: cannot scan type %T", value)
|
||||||
|
}
|
||||||
|
s = strings.TrimSpace(s)
|
||||||
|
slash := strings.LastIndex(s, "/")
|
||||||
|
if !strings.HasPrefix(s, "{") || slash < 0 || !strings.Contains(s[:slash], "}") {
|
||||||
|
return fmt.Errorf("SqlSparseVector: invalid literal %q", s)
|
||||||
|
}
|
||||||
|
dim, err := strconv.Atoi(strings.TrimSpace(s[slash+1:]))
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("SqlSparseVector: bad dimension: %w", err)
|
||||||
|
}
|
||||||
|
body := strings.TrimSpace(s[1:strings.LastIndex(s, "}")])
|
||||||
|
var idx []int32
|
||||||
|
var vals []float32
|
||||||
|
if body != "" {
|
||||||
|
for _, pair := range strings.Split(body, ",") {
|
||||||
|
kv := strings.SplitN(pair, ":", 2)
|
||||||
|
if len(kv) != 2 {
|
||||||
|
return fmt.Errorf("SqlSparseVector: bad pair %q", pair)
|
||||||
|
}
|
||||||
|
k, err := strconv.Atoi(strings.TrimSpace(kv[0]))
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("SqlSparseVector: bad index %q: %w", kv[0], err)
|
||||||
|
}
|
||||||
|
f, err := strconv.ParseFloat(strings.TrimSpace(kv[1]), 32)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("SqlSparseVector: bad value %q: %w", kv[1], err)
|
||||||
|
}
|
||||||
|
idx = append(idx, int32(k))
|
||||||
|
vals = append(vals, float32(f))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
v.Dim, v.Indices, v.Values, v.Valid = dim, idx, vals, true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (v SqlSparseVector) Value() (driver.Value, error) {
|
||||||
|
if !v.Valid {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
pairs := make([]string, len(v.Indices))
|
||||||
|
for i, k := range v.Indices {
|
||||||
|
val := float32(0)
|
||||||
|
if i < len(v.Values) {
|
||||||
|
val = v.Values[i]
|
||||||
|
}
|
||||||
|
pairs[i] = strconv.Itoa(int(k)) + ":" + strconv.FormatFloat(float64(val), 'f', -1, 32)
|
||||||
|
}
|
||||||
|
return "{" + strings.Join(pairs, ",") + "}/" + strconv.Itoa(v.Dim), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type sparseVectorJSON struct {
|
||||||
|
Dim int `json:"dim"`
|
||||||
|
Indices []int32 `json:"indices"`
|
||||||
|
Values []float32 `json:"values"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (v SqlSparseVector) MarshalJSON() ([]byte, error) {
|
||||||
|
if !v.Valid {
|
||||||
|
return []byte("null"), nil
|
||||||
|
}
|
||||||
|
return json.Marshal(sparseVectorJSON{Dim: v.Dim, Indices: v.Indices, Values: v.Values})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (v *SqlSparseVector) UnmarshalJSON(b []byte) error {
|
||||||
|
if strings.TrimSpace(string(b)) == "null" {
|
||||||
|
v.Valid = false
|
||||||
|
v.Dim, v.Indices, v.Values = 0, nil, nil
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var j sparseVectorJSON
|
||||||
|
if err := json.Unmarshal(b, &j); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
v.Dim, v.Indices, v.Values, v.Valid = j.Dim, j.Indices, j.Values, true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewSqlSparseVector(dim int, indices []int32, values []float32) SqlSparseVector {
|
||||||
|
return SqlSparseVector{Dim: dim, Indices: indices, Values: values, Valid: true}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── SqlBitVector ─────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
// SqlBitVector is a nullable Postgres `bit(n)` / `varbit` column (used by
|
||||||
|
// pgvector for Hamming/Jaccard distance), backed by []bool. Wire format: a
|
||||||
|
// string of '0'/'1' characters. JSON: a bool array.
|
||||||
|
type SqlBitVector struct {
|
||||||
|
Val []bool
|
||||||
|
Valid bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (v *SqlBitVector) Scan(value any) error {
|
||||||
|
if value == nil {
|
||||||
|
v.Valid = false
|
||||||
|
v.Val = nil
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var s string
|
||||||
|
switch val := value.(type) {
|
||||||
|
case string:
|
||||||
|
s = val
|
||||||
|
case []byte:
|
||||||
|
s = string(val)
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("SqlBitVector: cannot scan type %T", value)
|
||||||
|
}
|
||||||
|
s = strings.TrimSpace(s)
|
||||||
|
out := make([]bool, len(s))
|
||||||
|
for i, c := range s {
|
||||||
|
switch c {
|
||||||
|
case '1':
|
||||||
|
out[i] = true
|
||||||
|
case '0':
|
||||||
|
out[i] = false
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("SqlBitVector: invalid bit %q", string(c))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
v.Val = out
|
||||||
|
v.Valid = true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (v SqlBitVector) Value() (driver.Value, error) {
|
||||||
|
if !v.Valid {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
var b strings.Builder
|
||||||
|
b.Grow(len(v.Val))
|
||||||
|
for _, bit := range v.Val {
|
||||||
|
if bit {
|
||||||
|
b.WriteByte('1')
|
||||||
|
} else {
|
||||||
|
b.WriteByte('0')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return b.String(), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (v SqlBitVector) MarshalJSON() ([]byte, error) {
|
||||||
|
if !v.Valid {
|
||||||
|
return []byte("null"), nil
|
||||||
|
}
|
||||||
|
return json.Marshal(v.Val)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (v *SqlBitVector) UnmarshalJSON(b []byte) error {
|
||||||
|
s := strings.TrimSpace(string(b))
|
||||||
|
if s == "null" {
|
||||||
|
v.Valid = false
|
||||||
|
v.Val = nil
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
// Accept both a bool array and a "0101" string.
|
||||||
|
if strings.HasPrefix(s, "\"") {
|
||||||
|
var str string
|
||||||
|
if err := json.Unmarshal(b, &str); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return v.Scan(str)
|
||||||
|
}
|
||||||
|
var vals []bool
|
||||||
|
if err := json.Unmarshal(b, &vals); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
v.Val = vals
|
||||||
|
v.Valid = true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewSqlBitVector(val []bool) SqlBitVector {
|
||||||
|
return SqlBitVector{Val: val, Valid: true}
|
||||||
|
}
|
||||||
@@ -0,0 +1,149 @@
|
|||||||
|
package spectypes
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestSqlHalfVector_RoundTrip(t *testing.T) {
|
||||||
|
v := NewSqlHalfVector([]float32{1, 2.5, -3})
|
||||||
|
|
||||||
|
dv, err := v.Value()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Value: %v", err)
|
||||||
|
}
|
||||||
|
if dv != "[1,2.5,-3]" {
|
||||||
|
t.Errorf("Value = %v, want [1,2.5,-3]", dv)
|
||||||
|
}
|
||||||
|
|
||||||
|
var back SqlHalfVector
|
||||||
|
if err := back.Scan(dv.(string)); err != nil {
|
||||||
|
t.Fatalf("Scan: %v", err)
|
||||||
|
}
|
||||||
|
if !back.Valid || !reflect.DeepEqual(back.Val, v.Val) {
|
||||||
|
t.Errorf("Scan = %+v, want %+v", back, v)
|
||||||
|
}
|
||||||
|
|
||||||
|
b, err := json.Marshal(v)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Marshal: %v", err)
|
||||||
|
}
|
||||||
|
if string(b) != "[1,2.5,-3]" {
|
||||||
|
t.Errorf("json = %s, want [1,2.5,-3]", b)
|
||||||
|
}
|
||||||
|
|
||||||
|
var fromJSON SqlHalfVector
|
||||||
|
if err := json.Unmarshal(b, &fromJSON); err != nil {
|
||||||
|
t.Fatalf("Unmarshal: %v", err)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(fromJSON.Val, v.Val) {
|
||||||
|
t.Errorf("json round-trip = %+v", fromJSON)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSqlHalfVector_Null(t *testing.T) {
|
||||||
|
var v SqlHalfVector
|
||||||
|
if err := v.Scan(nil); err != nil {
|
||||||
|
t.Fatalf("Scan(nil): %v", err)
|
||||||
|
}
|
||||||
|
if v.Valid {
|
||||||
|
t.Error("expected invalid after Scan(nil)")
|
||||||
|
}
|
||||||
|
dv, err := v.Value()
|
||||||
|
if err != nil || dv != nil {
|
||||||
|
t.Errorf("Value = %v, %v; want nil, nil", dv, err)
|
||||||
|
}
|
||||||
|
b, _ := json.Marshal(v)
|
||||||
|
if string(b) != "null" {
|
||||||
|
t.Errorf("json = %s, want null", b)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSqlSparseVector_RoundTrip(t *testing.T) {
|
||||||
|
v := NewSqlSparseVector(8, []int32{1, 4}, []float32{0.5, 0.2})
|
||||||
|
|
||||||
|
dv, err := v.Value()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Value: %v", err)
|
||||||
|
}
|
||||||
|
if dv != "{1:0.5,4:0.2}/8" {
|
||||||
|
t.Errorf("Value = %v, want {1:0.5,4:0.2}/8", dv)
|
||||||
|
}
|
||||||
|
|
||||||
|
var back SqlSparseVector
|
||||||
|
if err := back.Scan(dv.(string)); err != nil {
|
||||||
|
t.Fatalf("Scan: %v", err)
|
||||||
|
}
|
||||||
|
if back.Dim != 8 || !reflect.DeepEqual(back.Indices, []int32{1, 4}) ||
|
||||||
|
!reflect.DeepEqual(back.Values, []float32{0.5, 0.2}) {
|
||||||
|
t.Errorf("Scan = %+v", back)
|
||||||
|
}
|
||||||
|
|
||||||
|
b, err := json.Marshal(v)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Marshal: %v", err)
|
||||||
|
}
|
||||||
|
want := `{"dim":8,"indices":[1,4],"values":[0.5,0.2]}`
|
||||||
|
if string(b) != want {
|
||||||
|
t.Errorf("json = %s, want %s", b, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
var fromJSON SqlSparseVector
|
||||||
|
if err := json.Unmarshal([]byte(want), &fromJSON); err != nil {
|
||||||
|
t.Fatalf("Unmarshal: %v", err)
|
||||||
|
}
|
||||||
|
if fromJSON.Dim != 8 || !fromJSON.Valid {
|
||||||
|
t.Errorf("json round-trip = %+v", fromJSON)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSqlSparseVector_ScanInvalid(t *testing.T) {
|
||||||
|
var v SqlSparseVector
|
||||||
|
for _, s := range []string{"[1,2,3]", "{1:0.5}", "{1:0.5}/x", "bad"} {
|
||||||
|
if err := v.Scan(s); err == nil {
|
||||||
|
t.Errorf("Scan(%q) expected error", s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSqlBitVector_RoundTrip(t *testing.T) {
|
||||||
|
v := NewSqlBitVector([]bool{true, false, true, true})
|
||||||
|
|
||||||
|
dv, err := v.Value()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Value: %v", err)
|
||||||
|
}
|
||||||
|
if dv != "1011" {
|
||||||
|
t.Errorf("Value = %v, want 1011", dv)
|
||||||
|
}
|
||||||
|
|
||||||
|
var back SqlBitVector
|
||||||
|
if err := back.Scan("1011"); err != nil {
|
||||||
|
t.Fatalf("Scan: %v", err)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(back.Val, v.Val) {
|
||||||
|
t.Errorf("Scan = %+v", back)
|
||||||
|
}
|
||||||
|
|
||||||
|
b, _ := json.Marshal(v)
|
||||||
|
if string(b) != "[true,false,true,true]" {
|
||||||
|
t.Errorf("json = %s", b)
|
||||||
|
}
|
||||||
|
|
||||||
|
// JSON also accepts a "0101" string.
|
||||||
|
var fromStr SqlBitVector
|
||||||
|
if err := json.Unmarshal([]byte(`"1011"`), &fromStr); err != nil {
|
||||||
|
t.Fatalf("Unmarshal string: %v", err)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(fromStr.Val, v.Val) {
|
||||||
|
t.Errorf("string json = %+v", fromStr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSqlBitVector_ScanInvalid(t *testing.T) {
|
||||||
|
var v SqlBitVector
|
||||||
|
if err := v.Scan("1021"); err == nil {
|
||||||
|
t.Error("expected error for invalid bit")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
package spectypes
|
||||||
|
|
||||||
|
import (
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// pkgPath is the import path of this package, used to recognise spectypes
|
||||||
|
// wrappers by reflection.
|
||||||
|
const pkgPath = "github.com/bitechdev/ResolveSpec/pkg/spectypes"
|
||||||
|
|
||||||
|
// canonicalSQLNames maps a spectypes wrapper type name to the PostgreSQL type
|
||||||
|
// name it represents. Dimensioned types (vector(1536), geometry(Point,4326))
|
||||||
|
// still need a gorm/bun `type:` tag for the full declaration — this is the
|
||||||
|
// fallback used for metadata and OpenAPI when no tag is present.
|
||||||
|
var canonicalSQLNames = map[string]string{
|
||||||
|
"SqlVector": "vector",
|
||||||
|
"SqlHalfVector": "halfvec",
|
||||||
|
"SqlSparseVector": "sparsevec",
|
||||||
|
"SqlBitVector": "bit",
|
||||||
|
"SqlGeometry": "geometry",
|
||||||
|
"SqlGeography": "geography",
|
||||||
|
"SqlJSONB": "jsonb",
|
||||||
|
"SqlStringArray": "text[]",
|
||||||
|
"SqlInt16Array": "smallint[]",
|
||||||
|
"SqlInt32Array": "integer[]",
|
||||||
|
"SqlInt64Array": "bigint[]",
|
||||||
|
"SqlFloat32Array": "real[]",
|
||||||
|
"SqlFloat64Array": "double precision[]",
|
||||||
|
"SqlBoolArray": "boolean[]",
|
||||||
|
"SqlUUIDArray": "uuid[]",
|
||||||
|
"SqlDate": "date",
|
||||||
|
"SqlTime": "time",
|
||||||
|
"SqlTimeStamp": "timestamp",
|
||||||
|
}
|
||||||
|
|
||||||
|
// sqlNullElemNames maps the element type of a SqlNull[T] alias to a PG type name.
|
||||||
|
var sqlNullElemNames = map[string]string{
|
||||||
|
"int16": "smallint",
|
||||||
|
"int32": "integer",
|
||||||
|
"int64": "bigint",
|
||||||
|
"float64": "double precision",
|
||||||
|
"bool": "boolean",
|
||||||
|
"string": "text",
|
||||||
|
"[]uint8": "bytea",
|
||||||
|
"uuid.UUID": "uuid",
|
||||||
|
"Time": "timestamp",
|
||||||
|
}
|
||||||
|
|
||||||
|
// SQLTypeName returns the canonical PostgreSQL type name for a spectypes wrapper
|
||||||
|
// type, or ("", false) if t is not a recognised spectypes type.
|
||||||
|
func SQLTypeName(t reflect.Type) (string, bool) {
|
||||||
|
for t != nil && t.Kind() == reflect.Pointer {
|
||||||
|
t = t.Elem()
|
||||||
|
}
|
||||||
|
if t == nil || t.PkgPath() != pkgPath {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
|
||||||
|
name := t.Name()
|
||||||
|
if n, ok := canonicalSQLNames[name]; ok {
|
||||||
|
return n, true
|
||||||
|
}
|
||||||
|
|
||||||
|
// SqlNull[T] aliases, e.g. "SqlNull[int16]", "SqlNull[uuid.UUID]".
|
||||||
|
if strings.HasPrefix(name, "SqlNull[") && strings.HasSuffix(name, "]") {
|
||||||
|
elem := name[len("SqlNull[") : len(name)-1]
|
||||||
|
if idx := strings.LastIndex(elem, "."); idx >= 0 {
|
||||||
|
// keep last path segment, e.g. "github.com/google/uuid.UUID" -> "uuid.UUID"
|
||||||
|
if slash := strings.LastIndex(elem[:idx], "/"); slash >= 0 {
|
||||||
|
elem = elem[slash+1:]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if n, ok := sqlNullElemNames[elem]; ok {
|
||||||
|
return n, true
|
||||||
|
}
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
|
||||||
|
// IsSpatialType reports whether t is a PostGIS geometry/geography wrapper.
|
||||||
|
func IsSpatialType(t reflect.Type) bool {
|
||||||
|
n, ok := SQLTypeName(t)
|
||||||
|
return ok && (n == "geometry" || n == "geography")
|
||||||
|
}
|
||||||
|
|
||||||
|
// IsVectorType reports whether t is a pgvector wrapper (vector/halfvec/sparsevec).
|
||||||
|
func IsVectorType(t reflect.Type) bool {
|
||||||
|
n, ok := SQLTypeName(t)
|
||||||
|
return ok && (n == "vector" || n == "halfvec" || n == "sparsevec")
|
||||||
|
}
|
||||||
|
|
||||||
|
// IsJSONType reports whether t is a spectypes JSON/JSONB wrapper.
|
||||||
|
func IsJSONType(t reflect.Type) bool {
|
||||||
|
n, ok := SQLTypeName(t)
|
||||||
|
return ok && (n == "jsonb" || n == "json")
|
||||||
|
}
|
||||||
@@ -0,0 +1,80 @@
|
|||||||
|
package spectypes
|
||||||
|
|
||||||
|
import (
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestSQLTypeName(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
val any
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{SqlVector{}, "vector"},
|
||||||
|
{SqlHalfVector{}, "halfvec"},
|
||||||
|
{SqlSparseVector{}, "sparsevec"},
|
||||||
|
{SqlBitVector{}, "bit"},
|
||||||
|
{SqlGeometry{}, "geometry"},
|
||||||
|
{SqlGeography{}, "geography"},
|
||||||
|
{SqlJSONB{}, "jsonb"},
|
||||||
|
{SqlStringArray{}, "text[]"},
|
||||||
|
{SqlString{}, "text"},
|
||||||
|
{SqlInt64{}, "bigint"},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
got, ok := SQLTypeName(reflect.TypeOf(c.val))
|
||||||
|
if !ok || got != c.want {
|
||||||
|
t.Errorf("SQLTypeName(%T) = %q, %v; want %q", c.val, got, ok, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Pointer is unwrapped.
|
||||||
|
if got, ok := SQLTypeName(reflect.TypeOf(&SqlGeometry{})); !ok || got != "geometry" {
|
||||||
|
t.Errorf("pointer: got %q, %v", got, ok)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Non-spectypes type.
|
||||||
|
if _, ok := SQLTypeName(reflect.TypeOf("")); ok {
|
||||||
|
t.Error("expected false for string")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIsSpatialType(t *testing.T) {
|
||||||
|
if !IsSpatialType(reflect.TypeOf(SqlGeometry{})) {
|
||||||
|
t.Error("SqlGeometry should be spatial")
|
||||||
|
}
|
||||||
|
if !IsSpatialType(reflect.TypeOf(SqlGeography{})) {
|
||||||
|
t.Error("SqlGeography should be spatial")
|
||||||
|
}
|
||||||
|
if IsSpatialType(reflect.TypeOf(SqlVector{})) {
|
||||||
|
t.Error("SqlVector should not be spatial")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIsJSONType(t *testing.T) {
|
||||||
|
if !IsJSONType(reflect.TypeOf(SqlJSONB{})) {
|
||||||
|
t.Error("SqlJSONB should be a JSON type")
|
||||||
|
}
|
||||||
|
if !IsJSONType(reflect.TypeOf(&SqlJSONB{})) {
|
||||||
|
t.Error("*SqlJSONB should be a JSON type (pointer unwrapped)")
|
||||||
|
}
|
||||||
|
for _, v := range []any{SqlGeometry{}, SqlVector{}, SqlString{}, SqlStringArray{}, ""} {
|
||||||
|
if IsJSONType(reflect.TypeOf(v)) {
|
||||||
|
t.Errorf("%T should not be a JSON type", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIsVectorType(t *testing.T) {
|
||||||
|
for _, v := range []any{SqlVector{}, SqlHalfVector{}, SqlSparseVector{}} {
|
||||||
|
if !IsVectorType(reflect.TypeOf(v)) {
|
||||||
|
t.Errorf("%T should be vector", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if IsVectorType(reflect.TypeOf(SqlBitVector{})) {
|
||||||
|
t.Error("SqlBitVector is not a vector type")
|
||||||
|
}
|
||||||
|
if IsVectorType(reflect.TypeOf(SqlGeometry{})) {
|
||||||
|
t.Error("SqlGeometry is not a vector type")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,697 @@
|
|||||||
|
package spectypes
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/binary"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"math"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Minimal self-contained EWKB (PostGIS extended WKB) <-> GeoJSON / WKT codec.
|
||||||
|
// Supports 2D and 3D (Z) geometries of type Point, LineString, Polygon,
|
||||||
|
// MultiPoint, MultiLineString, MultiPolygon and GeometryCollection. The M
|
||||||
|
// dimension is parsed but dropped (GeoJSON has no M). SRID is tracked separately
|
||||||
|
// from the GeoJSON payload (GeoJSON assumes CRS84 / EPSG:4326).
|
||||||
|
|
||||||
|
// EWKB type flag bits (PostGIS).
|
||||||
|
const (
|
||||||
|
ewkbZ = 0x80000000
|
||||||
|
ewkbM = 0x40000000
|
||||||
|
ewkbSRID = 0x20000000
|
||||||
|
)
|
||||||
|
|
||||||
|
// geom is the intermediate geometry representation used by the codec.
|
||||||
|
//
|
||||||
|
// Point -> coord ([]float64, len 2 or 3)
|
||||||
|
// LineString/MultiPt -> line ([][]float64)
|
||||||
|
// Polygon/MultiLine -> poly ([][][]float64)
|
||||||
|
// MultiPolygon -> multi ([][][][]float64)
|
||||||
|
// GeometryCollection -> geoms ([]geom)
|
||||||
|
type geom struct {
|
||||||
|
typ string
|
||||||
|
coord []float64
|
||||||
|
line [][]float64
|
||||||
|
poly [][][]float64
|
||||||
|
multi [][][][]float64
|
||||||
|
geoms []geom
|
||||||
|
}
|
||||||
|
|
||||||
|
// wkbReader consumes an EWKB byte stream.
|
||||||
|
type wkbReader struct {
|
||||||
|
buf []byte
|
||||||
|
pos int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *wkbReader) readByte() (byte, error) {
|
||||||
|
if r.pos >= len(r.buf) {
|
||||||
|
return 0, fmt.Errorf("wkb: unexpected end of input")
|
||||||
|
}
|
||||||
|
b := r.buf[r.pos]
|
||||||
|
r.pos++
|
||||||
|
return b, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *wkbReader) readUint32(bo binary.ByteOrder) (uint32, error) {
|
||||||
|
if r.pos+4 > len(r.buf) {
|
||||||
|
return 0, fmt.Errorf("wkb: unexpected end of input")
|
||||||
|
}
|
||||||
|
v := bo.Uint32(r.buf[r.pos:])
|
||||||
|
r.pos += 4
|
||||||
|
return v, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *wkbReader) readFloat64(bo binary.ByteOrder) (float64, error) {
|
||||||
|
if r.pos+8 > len(r.buf) {
|
||||||
|
return 0, fmt.Errorf("wkb: unexpected end of input")
|
||||||
|
}
|
||||||
|
v := math.Float64frombits(bo.Uint64(r.buf[r.pos:]))
|
||||||
|
r.pos += 8
|
||||||
|
return v, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DecodeEWKBHex decodes a PostGIS hex-EWKB string (the default text
|
||||||
|
// representation of a geometry column) into a GeoJSON geometry object and its
|
||||||
|
// SRID. An SRID of 0 means "unspecified".
|
||||||
|
func DecodeEWKBHex(s string) (geojson []byte, srid int, err error) {
|
||||||
|
s = strings.TrimSpace(s)
|
||||||
|
raw, err := hex.DecodeString(s)
|
||||||
|
if err != nil {
|
||||||
|
return nil, 0, fmt.Errorf("wkb: invalid hex: %w", err)
|
||||||
|
}
|
||||||
|
return DecodeEWKB(raw)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DecodeEWKB decodes raw PostGIS EWKB bytes into a GeoJSON geometry object and
|
||||||
|
// its SRID.
|
||||||
|
func DecodeEWKB(raw []byte) (geojson []byte, srid int, err error) {
|
||||||
|
r := &wkbReader{buf: raw}
|
||||||
|
g, sr, err := readGeom(r)
|
||||||
|
if err != nil {
|
||||||
|
return nil, 0, err
|
||||||
|
}
|
||||||
|
out, err := geomToGeoJSON(g)
|
||||||
|
if err != nil {
|
||||||
|
return nil, 0, err
|
||||||
|
}
|
||||||
|
return out, sr, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func readGeom(r *wkbReader) (geom, int, error) {
|
||||||
|
order, err := r.readByte()
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, 0, err
|
||||||
|
}
|
||||||
|
var bo binary.ByteOrder
|
||||||
|
switch order {
|
||||||
|
case 0:
|
||||||
|
bo = binary.BigEndian
|
||||||
|
case 1:
|
||||||
|
bo = binary.LittleEndian
|
||||||
|
default:
|
||||||
|
return geom{}, 0, fmt.Errorf("wkb: invalid byte order %d", order)
|
||||||
|
}
|
||||||
|
|
||||||
|
rawType, err := r.readUint32(bo)
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, 0, err
|
||||||
|
}
|
||||||
|
hasZ := rawType&ewkbZ != 0
|
||||||
|
hasM := rawType&ewkbM != 0
|
||||||
|
hasSRID := rawType&ewkbSRID != 0
|
||||||
|
baseType := rawType & 0xff
|
||||||
|
|
||||||
|
srid := 0
|
||||||
|
if hasSRID {
|
||||||
|
s, err := r.readUint32(bo)
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, 0, err
|
||||||
|
}
|
||||||
|
srid = int(s)
|
||||||
|
}
|
||||||
|
|
||||||
|
dims := 2
|
||||||
|
if hasZ {
|
||||||
|
dims = 3
|
||||||
|
}
|
||||||
|
// M is consumed but not retained.
|
||||||
|
stride := dims
|
||||||
|
if hasM {
|
||||||
|
stride++
|
||||||
|
}
|
||||||
|
|
||||||
|
readCoord := func() ([]float64, error) {
|
||||||
|
c := make([]float64, 0, dims)
|
||||||
|
for i := 0; i < stride; i++ {
|
||||||
|
v, err := r.readFloat64(bo)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if i < dims {
|
||||||
|
c = append(c, v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return c, nil
|
||||||
|
}
|
||||||
|
readLine := func() ([][]float64, error) {
|
||||||
|
n, err := r.readUint32(bo)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
pts := make([][]float64, n)
|
||||||
|
for i := range pts {
|
||||||
|
pts[i], err = readCoord()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return pts, nil
|
||||||
|
}
|
||||||
|
readPoly := func() ([][][]float64, error) {
|
||||||
|
n, err := r.readUint32(bo)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
rings := make([][][]float64, n)
|
||||||
|
for i := range rings {
|
||||||
|
rings[i], err = readLine()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return rings, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
switch baseType {
|
||||||
|
case 1: // Point
|
||||||
|
c, err := readCoord()
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, 0, err
|
||||||
|
}
|
||||||
|
return geom{typ: "Point", coord: c}, srid, nil
|
||||||
|
case 2: // LineString
|
||||||
|
l, err := readLine()
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, 0, err
|
||||||
|
}
|
||||||
|
return geom{typ: "LineString", line: l}, srid, nil
|
||||||
|
case 3: // Polygon
|
||||||
|
p, err := readPoly()
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, 0, err
|
||||||
|
}
|
||||||
|
return geom{typ: "Polygon", poly: p}, srid, nil
|
||||||
|
case 4, 5, 6: // Multi*
|
||||||
|
n, err := r.readUint32(bo)
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, 0, err
|
||||||
|
}
|
||||||
|
parts := make([]geom, n)
|
||||||
|
for i := range parts {
|
||||||
|
sub, _, err := readGeom(r)
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, 0, err
|
||||||
|
}
|
||||||
|
parts[i] = sub
|
||||||
|
}
|
||||||
|
switch baseType {
|
||||||
|
case 4:
|
||||||
|
pts := make([][]float64, len(parts))
|
||||||
|
for i := range parts {
|
||||||
|
pts[i] = parts[i].coord
|
||||||
|
}
|
||||||
|
return geom{typ: "MultiPoint", line: pts}, srid, nil
|
||||||
|
case 5:
|
||||||
|
lines := make([][][]float64, len(parts))
|
||||||
|
for i := range parts {
|
||||||
|
lines[i] = parts[i].line
|
||||||
|
}
|
||||||
|
return geom{typ: "MultiLineString", poly: lines}, srid, nil
|
||||||
|
default:
|
||||||
|
polys := make([][][][]float64, len(parts))
|
||||||
|
for i := range parts {
|
||||||
|
polys[i] = parts[i].poly
|
||||||
|
}
|
||||||
|
return geom{typ: "MultiPolygon", multi: polys}, srid, nil
|
||||||
|
}
|
||||||
|
case 7: // GeometryCollection
|
||||||
|
n, err := r.readUint32(bo)
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, 0, err
|
||||||
|
}
|
||||||
|
parts := make([]geom, n)
|
||||||
|
for i := range parts {
|
||||||
|
sub, _, err := readGeom(r)
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, 0, err
|
||||||
|
}
|
||||||
|
parts[i] = sub
|
||||||
|
}
|
||||||
|
return geom{typ: "GeometryCollection", geoms: parts}, srid, nil
|
||||||
|
default:
|
||||||
|
return geom{}, 0, fmt.Errorf("wkb: unsupported geometry type %d", baseType)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── GeoJSON ──────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
type geoJSON struct {
|
||||||
|
Type string `json:"type"`
|
||||||
|
Coordinates json.RawMessage `json:"coordinates,omitempty"`
|
||||||
|
Geometries []geoJSON `json:"geometries,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func geomToGeoJSON(g geom) ([]byte, error) {
|
||||||
|
gj, err := geomToGeoJSONStruct(g)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return json.Marshal(gj)
|
||||||
|
}
|
||||||
|
|
||||||
|
func geomToGeoJSONStruct(g geom) (geoJSON, error) {
|
||||||
|
var coords any
|
||||||
|
switch g.typ {
|
||||||
|
case "Point":
|
||||||
|
coords = g.coord
|
||||||
|
case "LineString", "MultiPoint":
|
||||||
|
coords = g.line
|
||||||
|
case "Polygon", "MultiLineString":
|
||||||
|
coords = g.poly
|
||||||
|
case "MultiPolygon":
|
||||||
|
coords = g.multi
|
||||||
|
case "GeometryCollection":
|
||||||
|
subs := make([]geoJSON, len(g.geoms))
|
||||||
|
for i := range g.geoms {
|
||||||
|
s, err := geomToGeoJSONStruct(g.geoms[i])
|
||||||
|
if err != nil {
|
||||||
|
return geoJSON{}, err
|
||||||
|
}
|
||||||
|
subs[i] = s
|
||||||
|
}
|
||||||
|
return geoJSON{Type: "GeometryCollection", Geometries: subs}, nil
|
||||||
|
default:
|
||||||
|
return geoJSON{}, fmt.Errorf("wkb: cannot encode geometry type %q", g.typ)
|
||||||
|
}
|
||||||
|
rc, err := json.Marshal(coords)
|
||||||
|
if err != nil {
|
||||||
|
return geoJSON{}, err
|
||||||
|
}
|
||||||
|
return geoJSON{Type: g.typ, Coordinates: rc}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func geoJSONToGeom(data []byte) (geom, error) {
|
||||||
|
var gj geoJSON
|
||||||
|
if err := json.Unmarshal(data, &gj); err != nil {
|
||||||
|
return geom{}, fmt.Errorf("geojson: %w", err)
|
||||||
|
}
|
||||||
|
return geoJSONStructToGeom(gj)
|
||||||
|
}
|
||||||
|
|
||||||
|
func geoJSONStructToGeom(gj geoJSON) (geom, error) {
|
||||||
|
switch gj.Type {
|
||||||
|
case "Point":
|
||||||
|
var c []float64
|
||||||
|
if err := json.Unmarshal(gj.Coordinates, &c); err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
return geom{typ: "Point", coord: c}, nil
|
||||||
|
case "LineString", "MultiPoint":
|
||||||
|
var l [][]float64
|
||||||
|
if err := json.Unmarshal(gj.Coordinates, &l); err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
return geom{typ: gj.Type, line: l}, nil
|
||||||
|
case "Polygon", "MultiLineString":
|
||||||
|
var p [][][]float64
|
||||||
|
if err := json.Unmarshal(gj.Coordinates, &p); err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
return geom{typ: gj.Type, poly: p}, nil
|
||||||
|
case "MultiPolygon":
|
||||||
|
var m [][][][]float64
|
||||||
|
if err := json.Unmarshal(gj.Coordinates, &m); err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
return geom{typ: gj.Type, multi: m}, nil
|
||||||
|
case "GeometryCollection":
|
||||||
|
subs := make([]geom, len(gj.Geometries))
|
||||||
|
for i, s := range gj.Geometries {
|
||||||
|
g, err := geoJSONStructToGeom(s)
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
subs[i] = g
|
||||||
|
}
|
||||||
|
return geom{typ: "GeometryCollection", geoms: subs}, nil
|
||||||
|
default:
|
||||||
|
return geom{}, fmt.Errorf("geojson: unsupported type %q", gj.Type)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── WKT ──────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
// GeoJSONToWKT converts a GeoJSON geometry object to its WKT representation.
|
||||||
|
func GeoJSONToWKT(geojson []byte) (string, error) {
|
||||||
|
g, err := geoJSONToGeom(geojson)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return geomToWKT(g)
|
||||||
|
}
|
||||||
|
|
||||||
|
func fmtNum(f float64) string {
|
||||||
|
return strconv.FormatFloat(f, 'f', -1, 64)
|
||||||
|
}
|
||||||
|
|
||||||
|
func coordWKT(c []float64) string {
|
||||||
|
parts := make([]string, len(c))
|
||||||
|
for i, v := range c {
|
||||||
|
parts[i] = fmtNum(v)
|
||||||
|
}
|
||||||
|
return strings.Join(parts, " ")
|
||||||
|
}
|
||||||
|
|
||||||
|
func lineWKT(pts [][]float64) string {
|
||||||
|
parts := make([]string, len(pts))
|
||||||
|
for i, p := range pts {
|
||||||
|
parts[i] = coordWKT(p)
|
||||||
|
}
|
||||||
|
return "(" + strings.Join(parts, ", ") + ")"
|
||||||
|
}
|
||||||
|
|
||||||
|
func polyWKT(rings [][][]float64) string {
|
||||||
|
parts := make([]string, len(rings))
|
||||||
|
for i, r := range rings {
|
||||||
|
parts[i] = lineWKT(r)
|
||||||
|
}
|
||||||
|
return "(" + strings.Join(parts, ", ") + ")"
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── WKT parsing ─────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
// parseWKT parses a subset of WKT (2D/3D, no M) into the intermediate geom.
|
||||||
|
func parseWKT(s string) (geom, error) {
|
||||||
|
p := &wktParser{s: s}
|
||||||
|
p.skipSpace()
|
||||||
|
g, err := p.parseGeom()
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
p.skipSpace()
|
||||||
|
if p.pos != len(p.s) {
|
||||||
|
return geom{}, fmt.Errorf("wkt: trailing input %q", p.s[p.pos:])
|
||||||
|
}
|
||||||
|
return g, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type wktParser struct {
|
||||||
|
s string
|
||||||
|
pos int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *wktParser) skipSpace() {
|
||||||
|
for p.pos < len(p.s) && (p.s[p.pos] == ' ' || p.s[p.pos] == '\t' || p.s[p.pos] == '\n' || p.s[p.pos] == '\r') {
|
||||||
|
p.pos++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *wktParser) parseGeom() (geom, error) {
|
||||||
|
p.skipSpace()
|
||||||
|
start := p.pos
|
||||||
|
for p.pos < len(p.s) && (p.s[p.pos] >= 'A' && p.s[p.pos] <= 'Z' || p.s[p.pos] >= 'a' && p.s[p.pos] <= 'z') {
|
||||||
|
p.pos++
|
||||||
|
}
|
||||||
|
kw := strings.ToUpper(p.s[start:p.pos])
|
||||||
|
p.skipSpace()
|
||||||
|
// Optional Z / M / ZM dimension tag — coordinates carry their own arity.
|
||||||
|
if p.pos < len(p.s) && (p.s[p.pos] == 'Z' || p.s[p.pos] == 'M' || p.s[p.pos] == 'z' || p.s[p.pos] == 'm') {
|
||||||
|
for p.pos < len(p.s) && p.s[p.pos] != '(' {
|
||||||
|
p.pos++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
p.skipSpace()
|
||||||
|
|
||||||
|
switch kw {
|
||||||
|
case "POINT":
|
||||||
|
pts, err := p.parseCoordList()
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
if len(pts) != 1 {
|
||||||
|
return geom{}, fmt.Errorf("wkt: POINT needs exactly one coordinate")
|
||||||
|
}
|
||||||
|
return geom{typ: "Point", coord: pts[0]}, nil
|
||||||
|
case "LINESTRING":
|
||||||
|
pts, err := p.parseCoordList()
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
return geom{typ: "LineString", line: pts}, nil
|
||||||
|
case "MULTIPOINT":
|
||||||
|
pts, err := p.parseMultiPoint()
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
return geom{typ: "MultiPoint", line: pts}, nil
|
||||||
|
case "POLYGON":
|
||||||
|
rings, err := p.parseRingList()
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
return geom{typ: "Polygon", poly: rings}, nil
|
||||||
|
case "MULTILINESTRING":
|
||||||
|
lines, err := p.parseRingList()
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
return geom{typ: "MultiLineString", poly: lines}, nil
|
||||||
|
case "MULTIPOLYGON":
|
||||||
|
polys, err := p.parsePolyList()
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
return geom{typ: "MultiPolygon", multi: polys}, nil
|
||||||
|
case "GEOMETRYCOLLECTION":
|
||||||
|
return p.parseCollection()
|
||||||
|
default:
|
||||||
|
return geom{}, fmt.Errorf("wkt: unsupported geometry %q", kw)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *wktParser) expect(c byte) error {
|
||||||
|
p.skipSpace()
|
||||||
|
if p.pos >= len(p.s) || p.s[p.pos] != c {
|
||||||
|
return fmt.Errorf("wkt: expected %q at offset %d", string(c), p.pos)
|
||||||
|
}
|
||||||
|
p.pos++
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *wktParser) peek() byte {
|
||||||
|
p.skipSpace()
|
||||||
|
if p.pos >= len(p.s) {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
return p.s[p.pos]
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseCoordList parses `(x y[, x y]...)`.
|
||||||
|
func (p *wktParser) parseCoordList() ([][]float64, error) {
|
||||||
|
if err := p.expect('('); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var out [][]float64
|
||||||
|
for {
|
||||||
|
c, err := p.parseCoord()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out = append(out, c)
|
||||||
|
if p.peek() == ',' {
|
||||||
|
p.pos++
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if err := p.expect(')'); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *wktParser) parseCoord() ([]float64, error) {
|
||||||
|
p.skipSpace()
|
||||||
|
// Some MULTIPOINT forms wrap each coord in parentheses.
|
||||||
|
wrapped := false
|
||||||
|
if p.peek() == '(' {
|
||||||
|
p.pos++
|
||||||
|
wrapped = true
|
||||||
|
}
|
||||||
|
var nums []float64
|
||||||
|
for {
|
||||||
|
p.skipSpace()
|
||||||
|
start := p.pos
|
||||||
|
for p.pos < len(p.s) {
|
||||||
|
ch := p.s[p.pos]
|
||||||
|
if ch == '-' || ch == '+' || ch == '.' || ch == 'e' || ch == 'E' || (ch >= '0' && ch <= '9') {
|
||||||
|
p.pos++
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if p.pos == start {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
f, err := strconv.ParseFloat(p.s[start:p.pos], 64)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("wkt: bad number %q", p.s[start:p.pos])
|
||||||
|
}
|
||||||
|
nums = append(nums, f)
|
||||||
|
p.skipSpace()
|
||||||
|
if p.pos < len(p.s) && p.s[p.pos] == ' ' {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if wrapped {
|
||||||
|
if err := p.expect(')'); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(nums) < 2 {
|
||||||
|
return nil, fmt.Errorf("wkt: coordinate needs at least 2 numbers")
|
||||||
|
}
|
||||||
|
if len(nums) > 3 {
|
||||||
|
nums = nums[:3]
|
||||||
|
}
|
||||||
|
return nums, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *wktParser) parseMultiPoint() ([][]float64, error) {
|
||||||
|
if err := p.expect('('); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var out [][]float64
|
||||||
|
for {
|
||||||
|
c, err := p.parseCoord()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out = append(out, c)
|
||||||
|
if p.peek() == ',' {
|
||||||
|
p.pos++
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if err := p.expect(')'); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseRingList parses `((x y, ...), (...))`.
|
||||||
|
func (p *wktParser) parseRingList() ([][][]float64, error) {
|
||||||
|
if err := p.expect('('); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var out [][][]float64
|
||||||
|
for {
|
||||||
|
ring, err := p.parseCoordList()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out = append(out, ring)
|
||||||
|
if p.peek() == ',' {
|
||||||
|
p.pos++
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if err := p.expect(')'); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// parsePolyList parses `(((...)), ((...)))`.
|
||||||
|
func (p *wktParser) parsePolyList() ([][][][]float64, error) {
|
||||||
|
if err := p.expect('('); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var out [][][][]float64
|
||||||
|
for {
|
||||||
|
poly, err := p.parseRingList()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out = append(out, poly)
|
||||||
|
if p.peek() == ',' {
|
||||||
|
p.pos++
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if err := p.expect(')'); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *wktParser) parseCollection() (geom, error) {
|
||||||
|
if err := p.expect('('); err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
var subs []geom
|
||||||
|
for {
|
||||||
|
g, err := p.parseGeom()
|
||||||
|
if err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
subs = append(subs, g)
|
||||||
|
if p.peek() == ',' {
|
||||||
|
p.pos++
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if err := p.expect(')'); err != nil {
|
||||||
|
return geom{}, err
|
||||||
|
}
|
||||||
|
return geom{typ: "GeometryCollection", geoms: subs}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func geomToWKT(g geom) (string, error) {
|
||||||
|
switch g.typ {
|
||||||
|
case "Point":
|
||||||
|
return "POINT (" + coordWKT(g.coord) + ")", nil
|
||||||
|
case "LineString":
|
||||||
|
return "LINESTRING " + lineWKT(g.line), nil
|
||||||
|
case "MultiPoint":
|
||||||
|
return "MULTIPOINT " + lineWKT(g.line), nil
|
||||||
|
case "Polygon":
|
||||||
|
return "POLYGON " + polyWKT(g.poly), nil
|
||||||
|
case "MultiLineString":
|
||||||
|
return "MULTILINESTRING " + polyWKT(g.poly), nil
|
||||||
|
case "MultiPolygon":
|
||||||
|
parts := make([]string, len(g.multi))
|
||||||
|
for i, p := range g.multi {
|
||||||
|
parts[i] = polyWKT(p)
|
||||||
|
}
|
||||||
|
return "MULTIPOLYGON (" + strings.Join(parts, ", ") + ")", nil
|
||||||
|
case "GeometryCollection":
|
||||||
|
parts := make([]string, len(g.geoms))
|
||||||
|
for i := range g.geoms {
|
||||||
|
w, err := geomToWKT(g.geoms[i])
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
parts[i] = w
|
||||||
|
}
|
||||||
|
return "GEOMETRYCOLLECTION (" + strings.Join(parts, ", ") + ")", nil
|
||||||
|
default:
|
||||||
|
return "", fmt.Errorf("wkt: cannot encode geometry type %q", g.typ)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,91 @@
|
|||||||
|
package spectypes
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestDecodeEWKBHex(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
hex string
|
||||||
|
wantSRID int
|
||||||
|
wantJSON string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
// SRID=4326;POINT(1 2)
|
||||||
|
name: "point with srid",
|
||||||
|
hex: "0101000020E6100000000000000000F03F0000000000000040",
|
||||||
|
wantSRID: 4326,
|
||||||
|
wantJSON: `{"type":"Point","coordinates":[1,2]}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
// POINT(1 2) no SRID, little endian
|
||||||
|
name: "point no srid",
|
||||||
|
hex: "0101000000000000000000F03F0000000000000040",
|
||||||
|
wantSRID: 0,
|
||||||
|
wantJSON: `{"type":"Point","coordinates":[1,2]}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
// SRID=4326;LINESTRING(0 0, 1 1, 2 2)
|
||||||
|
name: "linestring",
|
||||||
|
hex: "0102000020E610000003000000000000000000000000000000000000000000000000" +
|
||||||
|
"00F03F000000000000F03F00000000000000400000000000000040",
|
||||||
|
wantSRID: 4326,
|
||||||
|
wantJSON: `{"type":"LineString","coordinates":[[0,0],[1,1],[2,2]]}`,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
gj, srid, err := DecodeEWKBHex(tt.hex)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("DecodeEWKBHex: %v", err)
|
||||||
|
}
|
||||||
|
if srid != tt.wantSRID {
|
||||||
|
t.Errorf("srid = %d, want %d", srid, tt.wantSRID)
|
||||||
|
}
|
||||||
|
if !jsonEqual(t, gj, tt.wantJSON) {
|
||||||
|
t.Errorf("geojson = %s, want %s", gj, tt.wantJSON)
|
||||||
|
}
|
||||||
|
// Round-trip through WKT parser.
|
||||||
|
wkt, err := GeoJSONToWKT(gj)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GeoJSONToWKT: %v", err)
|
||||||
|
}
|
||||||
|
gj2, err := wktToGeoJSON(wkt)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("wktToGeoJSON(%q): %v", wkt, err)
|
||||||
|
}
|
||||||
|
if !jsonEqual(t, gj2, tt.wantJSON) {
|
||||||
|
t.Errorf("round-trip geojson = %s, want %s", gj2, tt.wantJSON)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWKTPolygon(t *testing.T) {
|
||||||
|
src := `POLYGON ((0 0, 4 0, 4 4, 0 4, 0 0), (1 1, 2 1, 2 2, 1 2, 1 1))`
|
||||||
|
gj, err := wktToGeoJSON(src)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("wktToGeoJSON: %v", err)
|
||||||
|
}
|
||||||
|
want := `{"type":"Polygon","coordinates":[[[0,0],[4,0],[4,4],[0,4],[0,0]],[[1,1],[2,1],[2,2],[1,2],[1,1]]]}`
|
||||||
|
if !jsonEqual(t, gj, want) {
|
||||||
|
t.Errorf("geojson = %s, want %s", gj, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func jsonEqual(t *testing.T, got []byte, want string) bool {
|
||||||
|
t.Helper()
|
||||||
|
var a, b any
|
||||||
|
if err := json.Unmarshal(got, &a); err != nil {
|
||||||
|
t.Fatalf("unmarshal got %s: %v", got, err)
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal([]byte(want), &b); err != nil {
|
||||||
|
t.Fatalf("unmarshal want %s: %v", want, err)
|
||||||
|
}
|
||||||
|
ab, _ := json.Marshal(a)
|
||||||
|
bb, _ := json.Marshal(b)
|
||||||
|
return string(ab) == string(bb)
|
||||||
|
}
|
||||||
@@ -564,7 +564,7 @@ func (h *Handler) readByID(hookCtx *HookContext) (interface{}, error) {
|
|||||||
|
|
||||||
// Apply columns
|
// Apply columns
|
||||||
if hookCtx.Options != nil && len(hookCtx.Options.Columns) > 0 {
|
if hookCtx.Options != nil && len(hookCtx.Options.Columns) > 0 {
|
||||||
query = query.Column(hookCtx.Options.Columns...)
|
query = common.ApplySelectColumns(query, hookCtx.Model, "", hookCtx.Options.Columns)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Apply preloads (simplified for now)
|
// Apply preloads (simplified for now)
|
||||||
@@ -606,7 +606,7 @@ func (h *Handler) readMultiple(hookCtx *HookContext) (data interface{}, metadata
|
|||||||
// Apply options (simplified implementation)
|
// Apply options (simplified implementation)
|
||||||
if hookCtx.Options != nil {
|
if hookCtx.Options != nil {
|
||||||
// Apply filters with OR grouping support
|
// Apply filters with OR grouping support
|
||||||
query = h.applyFilters(query, hookCtx.Options.Filters)
|
query = h.applyFilters(query, hookCtx.Options.Filters, hookCtx.Model)
|
||||||
|
|
||||||
// Apply sorting
|
// Apply sorting
|
||||||
for _, sort := range hookCtx.Options.Sort {
|
for _, sort := range hookCtx.Options.Sort {
|
||||||
@@ -614,6 +614,10 @@ func (h *Handler) readMultiple(hookCtx *HookContext) (data interface{}, metadata
|
|||||||
if sort.Direction == "desc" {
|
if sort.Direction == "desc" {
|
||||||
direction = "DESC"
|
direction = "DESC"
|
||||||
}
|
}
|
||||||
|
if expr, jargs, _, ok := common.ResolveJSONColumnExpr(hookCtx.Model, "", sort.Column); ok {
|
||||||
|
query = query.OrderExpr(fmt.Sprintf("%s %s", expr, direction), jargs...)
|
||||||
|
continue
|
||||||
|
}
|
||||||
query = query.Order(fmt.Sprintf("%s %s", sort.Column, direction))
|
query = query.Order(fmt.Sprintf("%s %s", sort.Column, direction))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -632,7 +636,7 @@ func (h *Handler) readMultiple(hookCtx *HookContext) (data interface{}, metadata
|
|||||||
|
|
||||||
// Apply columns
|
// Apply columns
|
||||||
if len(hookCtx.Options.Columns) > 0 {
|
if len(hookCtx.Options.Columns) > 0 {
|
||||||
query = query.Column(hookCtx.Options.Columns...)
|
query = common.ApplySelectColumns(query, hookCtx.Model, "", hookCtx.Options.Columns)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -665,7 +669,7 @@ func (h *Handler) readMultiple(hookCtx *HookContext) (data interface{}, metadata
|
|||||||
countQuery := h.db.NewSelect().Model(hookCtx.ModelPtr).Table(hookCtx.TableName)
|
countQuery := h.db.NewSelect().Model(hookCtx.ModelPtr).Table(hookCtx.TableName)
|
||||||
if hookCtx.Options != nil {
|
if hookCtx.Options != nil {
|
||||||
for _, filter := range hookCtx.Options.Filters {
|
for _, filter := range hookCtx.Options.Filters {
|
||||||
cond, args := h.buildFilterCondition(filter)
|
cond, args := h.buildFilterCondition(filter, hookCtx.Model)
|
||||||
if cond != "" {
|
if cond != "" {
|
||||||
countQuery = countQuery.Where(cond, args...)
|
countQuery = countQuery.Where(cond, args...)
|
||||||
}
|
}
|
||||||
@@ -776,7 +780,7 @@ func (h *Handler) getMetadata(schema, entity string, model interface{}) map[stri
|
|||||||
// getOperatorSQL converts filter operator to SQL operator
|
// getOperatorSQL converts filter operator to SQL operator
|
||||||
// applyFilters applies all filters with proper grouping for OR logic
|
// applyFilters applies all filters with proper grouping for OR logic
|
||||||
// Groups consecutive OR filters together to ensure proper query precedence
|
// Groups consecutive OR filters together to ensure proper query precedence
|
||||||
func (h *Handler) applyFilters(query common.SelectQuery, filters []common.FilterOption) common.SelectQuery {
|
func (h *Handler) applyFilters(query common.SelectQuery, filters []common.FilterOption, model interface{}) common.SelectQuery {
|
||||||
if len(filters) == 0 {
|
if len(filters) == 0 {
|
||||||
return query
|
return query
|
||||||
}
|
}
|
||||||
@@ -796,11 +800,11 @@ func (h *Handler) applyFilters(query common.SelectQuery, filters []common.Filter
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Apply the OR group as a single grouped WHERE clause
|
// Apply the OR group as a single grouped WHERE clause
|
||||||
query = h.applyFilterGroup(query, orGroup)
|
query = h.applyFilterGroup(query, orGroup, model)
|
||||||
i = j
|
i = j
|
||||||
} else {
|
} else {
|
||||||
// Single filter with AND logic (or first filter)
|
// Single filter with AND logic (or first filter)
|
||||||
condition, args := h.buildFilterCondition(filters[i])
|
condition, args := h.buildFilterCondition(filters[i], model)
|
||||||
if condition != "" {
|
if condition != "" {
|
||||||
query = query.Where(condition, args...)
|
query = query.Where(condition, args...)
|
||||||
}
|
}
|
||||||
@@ -813,7 +817,7 @@ func (h *Handler) applyFilters(query common.SelectQuery, filters []common.Filter
|
|||||||
|
|
||||||
// applyFilterGroup applies a group of filters that should be OR'd together
|
// applyFilterGroup applies a group of filters that should be OR'd together
|
||||||
// Always wraps them in parentheses and applies as a single WHERE clause
|
// Always wraps them in parentheses and applies as a single WHERE clause
|
||||||
func (h *Handler) applyFilterGroup(query common.SelectQuery, filters []common.FilterOption) common.SelectQuery {
|
func (h *Handler) applyFilterGroup(query common.SelectQuery, filters []common.FilterOption, model interface{}) common.SelectQuery {
|
||||||
if len(filters) == 0 {
|
if len(filters) == 0 {
|
||||||
return query
|
return query
|
||||||
}
|
}
|
||||||
@@ -823,7 +827,7 @@ func (h *Handler) applyFilterGroup(query common.SelectQuery, filters []common.Fi
|
|||||||
var args []interface{}
|
var args []interface{}
|
||||||
|
|
||||||
for _, filter := range filters {
|
for _, filter := range filters {
|
||||||
condition, filterArgs := h.buildFilterCondition(filter)
|
condition, filterArgs := h.buildFilterCondition(filter, model)
|
||||||
if condition != "" {
|
if condition != "" {
|
||||||
conditions = append(conditions, condition)
|
conditions = append(conditions, condition)
|
||||||
args = append(args, filterArgs...)
|
args = append(args, filterArgs...)
|
||||||
@@ -844,8 +848,14 @@ func (h *Handler) applyFilterGroup(query common.SelectQuery, filters []common.Fi
|
|||||||
return query.Where(groupedCondition, args...)
|
return query.Where(groupedCondition, args...)
|
||||||
}
|
}
|
||||||
|
|
||||||
// buildFilterCondition builds a filter condition and returns it with args
|
// buildFilterCondition builds a filter condition and returns it with args.
|
||||||
func (h *Handler) buildFilterCondition(filter common.FilterOption) (conditionString string, conditionArgs []interface{}) {
|
// model, when non-nil, lets JSON sub-field references (data->>'x', data#>>'{a,b}',
|
||||||
|
// or the dotted data.x shorthand for a JSON column) resolve to a safe,
|
||||||
|
// parameterised expression before the ordinary operator handling below.
|
||||||
|
func (h *Handler) buildFilterCondition(filter common.FilterOption, model interface{}) (conditionString string, conditionArgs []interface{}) {
|
||||||
|
if cond, jargs, ok := common.BuildJSONFilterCondition(model, "", filter.Column, filter.Operator, filter.Value); ok {
|
||||||
|
return cond, jargs
|
||||||
|
}
|
||||||
if strings.EqualFold(filter.Operator, "in") {
|
if strings.EqualFold(filter.Operator, "in") {
|
||||||
cond, args := common.BuildInCondition(filter.Column, filter.Value)
|
cond, args := common.BuildInCondition(filter.Column, filter.Value)
|
||||||
return cond, args
|
return cond, args
|
||||||
|
|||||||
@@ -68,6 +68,14 @@ See [`resolvespec-python/todo.md`](./resolvespec-python/todo.md) for detailed Py
|
|||||||
- [ ] Computed column improvements
|
- [ ] Computed column improvements
|
||||||
- [ ] Recursive query support
|
- [ ] Recursive query support
|
||||||
|
|
||||||
|
3. **PostGIS & pgvector (PostgreSQL)**
|
||||||
|
- [x] Custom types: `SqlGeometry`/`SqlGeography` (GeoJSON I/O), `SqlHalfVector`, `SqlSparseVector`, `SqlBitVector`
|
||||||
|
- [x] Spatial filter operators (`st_dwithin`, `st_intersects`, `bbox`, …) in resolvespec + restheadspec
|
||||||
|
- [x] Vector similarity filter operators (`l2_within`, `cosine_within`, `ip_within`)
|
||||||
|
- [x] KNN structured option (`options.vector_search` / `X-Vector-Search-*` headers)
|
||||||
|
- [x] Metadata + OpenAPI report geometry/vector column types
|
||||||
|
- [ ] Integration tests against a PostGIS + pgvector database (needs test DB with extensions)
|
||||||
|
|
||||||
3. **Testing & Quality**
|
3. **Testing & Quality**
|
||||||
- [ ] Increase test coverage to 70%+
|
- [ ] Increase test coverage to 70%+
|
||||||
- [ ] Add integration tests for all ORMs
|
- [ ] Add integration tests for all ORMs
|
||||||
@@ -96,5 +104,5 @@ See [`resolvespec-python/todo.md`](./resolvespec-python/todo.md) for detailed Py
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**Last Updated:** 2026-02-07
|
**Last Updated:** 2026-08-29
|
||||||
**Updated:** Added resolvespec-js client testing and implementation tasks
|
**Updated:** Added PostGIS + pgvector support (types, filter operators, KNN, metadata/OpenAPI)
|
||||||
|
|||||||
Reference in New Issue
Block a user