feat(pgsql): support vector and PostGIS indexes with extensions
* Add handling for pgvector and PostGIS extensions in migration scripts * Implement operator class and storage parameters for vector indexes * Update tests to validate new index behaviors and extension creation
This commit is contained in:
@@ -171,6 +171,7 @@ When `include_audit` is enabled, adds:
|
||||
- Function-based indexes
|
||||
- Concurrent index creation (`CREATE INDEX CONCURRENTLY`) via `Index.Concurrent`
|
||||
- Check constraints with expressions
|
||||
- Extension types and indexes: PostGIS, pgvector, citext, hstore, ltree (see below)
|
||||
|
||||
## Data Types
|
||||
|
||||
@@ -186,6 +187,108 @@ Supports all PostgreSQL data types:
|
||||
- Network: INET, CIDR, MACADDR
|
||||
- Special: ARRAY, HSTORE
|
||||
|
||||
## Extension Types (PostGIS, pgvector)
|
||||
|
||||
Extension column types are preserved verbatim, including their type modifier:
|
||||
|
||||
| Type | Example column type | Extension |
|
||||
|------|---------------------|-----------|
|
||||
| PostGIS | `geometry(Point,4326)`, `geography(Point)`, `box2d`, `raster` | `postgis`, `postgis_raster`, `postgis_topology` |
|
||||
| pgvector | `vector(1536)`, `halfvec(768)`, `sparsevec(1000)` | `vector` |
|
||||
| Other | `citext`, `hstore`, `ltree` | `citext`, `hstore`, `ltree` |
|
||||
|
||||
`CREATE EXTENSION IF NOT EXISTS <ext>;` is emitted automatically for every extension the
|
||||
schema needs. See [Extensions](#extensions).
|
||||
|
||||
### Extension Indexes
|
||||
|
||||
`Index.Type` selects the access method: `gist`, `spgist`, `brin` (PostGIS), `hnsw`, `ivfflat`
|
||||
(pgvector), `vchordrq`, `vchordg` (VectorChord), `bm25` (pg_search).
|
||||
|
||||
Operator class and access-method parameters ride in `Index.Comment`:
|
||||
|
||||
```
|
||||
opclass=vector_l2_ops; with (lists=100)
|
||||
```
|
||||
|
||||
- `opclass=<name>` — used only when compatible with the column type; otherwise ignored.
|
||||
Bare operator class names in the comment (e.g. `gin_trgm_ops`) are also recognized.
|
||||
- `with (k=v, …)` — rendered as `WITH (k = v, …)`. Only well-formed `key = value` pairs are
|
||||
kept, so comment prose never reaches the DDL. Values may be bare (`lists=100`), quoted
|
||||
(`key_field='id'`), or dollar-quoted (`options=$$[build.internal]$$`).
|
||||
|
||||
Defaults when no operator class is requested:
|
||||
|
||||
| Access method | Column type | Emitted operator class |
|
||||
|---------------|-------------|------------------------|
|
||||
| `hnsw`, `ivfflat`, `vchordrq`, `vchordg` | `vector` / `halfvec` / `sparsevec` / `bit` | `vector_cosine_ops` / `halfvec_cosine_ops` / `sparsevec_cosine_ops` / `bit_hamming_ops` |
|
||||
| `gist`, `spgist`, `brin` | `geometry`, `geography` | none (PostGIS default operator class) |
|
||||
| `gin` | text / `jsonb` / array | `gin_trgm_ops` / `jsonb_ops` / `array_ops` |
|
||||
|
||||
pgvector defines no default operator class, so a vector index always names one.
|
||||
|
||||
```sql
|
||||
CREATE INDEX IF NOT EXISTS idx_documents_embedding
|
||||
ON public.documents USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100);
|
||||
CREATE INDEX IF NOT EXISTS idx_documents_location
|
||||
ON public.documents USING gist (location);
|
||||
```
|
||||
|
||||
Migrations only recreate an index when both sides specify a hint and they differ, so a model
|
||||
without hints does not churn against a live database.
|
||||
|
||||
## Extensions
|
||||
|
||||
`CREATE EXTENSION IF NOT EXISTS <ext>;` is emitted per schema, deduplicated and ordered so
|
||||
dependencies come first (`postgis` before `postgis_topology`/`postgis_raster`/`pgrouting`,
|
||||
`vector` before `vchord`). Names needing quoting are quoted: `CREATE EXTENSION IF NOT EXISTS "uuid-ossp";`
|
||||
|
||||
### Detection
|
||||
|
||||
| Source | Example | Extension |
|
||||
|--------|---------|-----------|
|
||||
| Column type | `vector(1536)`, `geometry(Point,4326)`, `citext`, `ltree` | `vector`, `postgis`, `citext`, `ltree` |
|
||||
| Index access method | `hnsw`, `ivfflat` / `vchordrq`, `vchordg` / `bm25` | `vector` / `vchord` / `pg_search` |
|
||||
| Operator class | `gin_trgm_ops`, `gist_ltree_ops` | `pg_trgm`, `ltree` |
|
||||
| GIN/GiST on a scalar type | `USING gin (views)` | `btree_gin` / `btree_gist` |
|
||||
| Function in a default, CHECK, index `WHERE`, or view body | `uuid_generate_v4()`, `crypt()`, `ST_Area()`, `unaccent()`, `json_matches_schema()` | `uuid-ossp`, `pgcrypto`, `postgis`, `unaccent`, `pg_jsonschema` |
|
||||
|
||||
`gen_random_uuid()` is built in since PostgreSQL 13 and does not pull in `pgcrypto`.
|
||||
|
||||
### Declaring extensions explicitly
|
||||
|
||||
Extensions that leave no trace in the schema go in `schema.Metadata["extensions"]`, as a list
|
||||
or a comma-separated string. Dependencies are pulled in automatically; unknown names are kept
|
||||
as given. The PostgreSQL reader populates this from `pg_extension` for the schemas it reads.
|
||||
|
||||
```yaml
|
||||
metadata:
|
||||
extensions: [pg_cron, timescaledb, pg_stat_statements]
|
||||
```
|
||||
|
||||
### Recognized extensions
|
||||
|
||||
| Category | Extensions |
|
||||
|----------|------------|
|
||||
| ai/search | `vector`, `vchord` |
|
||||
| document | `hstore`, `ltree` |
|
||||
| federation | `postgres_fdw` |
|
||||
| geospatial | `postgis`, `postgis_raster`, `postgis_topology`, `pgrouting` |
|
||||
| indexing | `btree_gin`, `btree_gist` |
|
||||
| integration | `http` |
|
||||
| integrity | `amcheck` |
|
||||
| jobs / scheduling | `pg_background`, `pg_cron` |
|
||||
| maintenance | `pg_repack`, `pgstattuple` |
|
||||
| observability | `pg_qualstats`, `pg_stat_statements` |
|
||||
| partitioning | `pg_partman` |
|
||||
| procedural | `plpython3u` |
|
||||
| search | `pg_search`, `pg_textsearch` |
|
||||
| security | `pgcrypto` |
|
||||
| text | `citext`, `fuzzystrmatch`, `pg_trgm`, `unaccent` |
|
||||
| time-series | `timescaledb` |
|
||||
| utility | `uuid-ossp` |
|
||||
| validation | `pg_jsonschema` |
|
||||
|
||||
## Notes
|
||||
|
||||
- Generated SQL is formatted and readable
|
||||
|
||||
Reference in New Issue
Block a user