Files
relspecgo/pkg/writers/pgsql/README.md
T
warkanum ab3c9217df feat(pgsql): support vector and PostGIS indexes with extensions
* Add handling for pgvector and PostGIS extensions in migration scripts
* Implement operator class and storage parameters for vector indexes
* Update tests to validate new index behaviors and extension creation
2026-08-29 20:39:57 +02:00

300 lines
9.0 KiB
Markdown

# PostgreSQL Writer
Generates PostgreSQL DDL (Data Definition Language) SQL scripts from database schema information.
## Overview
The PostgreSQL Writer converts RelSpec's internal database model representation into PostgreSQL-compatible SQL DDL scripts, including CREATE TABLE statements, constraints, indexes, views, and sequences.
## Features
- Generates complete PostgreSQL DDL
- Creates schemas, tables, columns
- Defines constraints (PK, FK, unique, check)
- Creates indexes
- Generates views and sequences
- Supports migration scripts
- Includes audit triggers (optional)
- Handles PostgreSQL-specific data types
## Usage
### Basic Example
```go
package main
import (
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/writers"
"git.warky.dev/wdevs/relspecgo/pkg/writers/pgsql"
)
func main() {
options := &writers.WriterOptions{
OutputPath: "schema.sql",
}
writer := pgsql.NewWriter(options)
err := writer.WriteDatabase(db)
if err != nil {
panic(err)
}
}
```
### CLI Examples
```bash
# Generate PostgreSQL DDL from JSON schema
relspec --input json \
--in-file schema.json \
--output pgsql \
--out-file schema.sql
# Convert GORM models to PostgreSQL DDL
relspec --input gorm \
--in-file models.go \
--output pgsql \
--out-file create_tables.sql
# Export live database schema to SQL
relspec --input pgsql \
--conn "postgres://localhost/source_db" \
--output pgsql \
--out-file backup_schema.sql
```
## Generated SQL Example
```sql
-- Schema: public
CREATE SCHEMA IF NOT EXISTS public;
-- Table: public.users
CREATE TABLE IF NOT EXISTS public.users (
id BIGSERIAL PRIMARY KEY,
username VARCHAR(50) NOT NULL,
email VARCHAR(100) NOT NULL,
bio TEXT,
created_at TIMESTAMP NOT NULL DEFAULT NOW()
);
-- Constraints for public.users
ALTER TABLE public.users
ADD CONSTRAINT uq_users_username UNIQUE (username);
-- Indexes for public.users
CREATE INDEX idx_users_email ON public.users (email);
-- Table: public.posts
CREATE TABLE IF NOT EXISTS public.posts (
id BIGSERIAL PRIMARY KEY,
user_id BIGINT NOT NULL,
title VARCHAR(200) NOT NULL,
content TEXT,
created_at TIMESTAMP DEFAULT NOW()
);
-- Foreign Keys for public.posts
ALTER TABLE public.posts
ADD CONSTRAINT fk_posts_user_id
FOREIGN KEY (user_id)
REFERENCES public.users (id)
ON DELETE CASCADE
ON UPDATE NO ACTION;
-- Indexes for public.posts
CREATE INDEX idx_posts_user_id ON public.posts (user_id);
```
## Writer Options
### Metadata Options
```go
options := &writers.WriterOptions{
OutputPath: "schema.sql",
Metadata: map[string]interface{}{
"include_drop": true, // Include DROP statements
"include_audit": true, // Include audit triggers
"if_not_exists": true, // Use IF NOT EXISTS
"migration_mode": false, // Generate migration script
},
}
```
## Features
### Full DDL Generation
Generates complete database structure:
- CREATE SCHEMA statements
- CREATE TABLE with all columns and types
- PRIMARY KEY constraints
- FOREIGN KEY constraints with actions
- UNIQUE constraints
- CHECK constraints
- CREATE INDEX statements
- CREATE VIEW statements
- CREATE SEQUENCE statements
### Migration Mode
When `migration_mode` is enabled, generates migration scripts with:
- Version tracking
- Up/down migrations
- Transactional DDL
- Rollback support
### Audit Triggers
When `include_audit` is enabled, adds:
- Created/updated timestamp triggers
- Audit logging functionality
- Change tracking
## PostgreSQL-Specific Features
- Serial types (SERIAL, BIGSERIAL)
- Advanced types (UUID, JSONB, ARRAY)
- Schema-qualified names
- Constraint actions (CASCADE, RESTRICT, SET NULL)
- Partial indexes
- Function-based indexes
- Concurrent index creation (`CREATE INDEX CONCURRENTLY`) via `Index.Concurrent`
- Check constraints with expressions
- Extension types and indexes: PostGIS, pgvector, citext, hstore, ltree (see below)
## Data Types
Supports all PostgreSQL data types:
- Integer types: SMALLINT, INTEGER, BIGINT, SERIAL, BIGSERIAL
- Numeric types: NUMERIC, DECIMAL, REAL, DOUBLE PRECISION
- String types: VARCHAR, CHAR, TEXT
- Date/Time: DATE, TIME, TIMESTAMP, TIMESTAMPTZ, INTERVAL
- Boolean: BOOLEAN
- Binary: BYTEA
- JSON: JSON, JSONB
- UUID: UUID
- Network: INET, CIDR, MACADDR
- Special: ARRAY, HSTORE
## Extension Types (PostGIS, pgvector)
Extension column types are preserved verbatim, including their type modifier:
| Type | Example column type | Extension |
|------|---------------------|-----------|
| PostGIS | `geometry(Point,4326)`, `geography(Point)`, `box2d`, `raster` | `postgis`, `postgis_raster`, `postgis_topology` |
| pgvector | `vector(1536)`, `halfvec(768)`, `sparsevec(1000)` | `vector` |
| Other | `citext`, `hstore`, `ltree` | `citext`, `hstore`, `ltree` |
`CREATE EXTENSION IF NOT EXISTS <ext>;` is emitted automatically for every extension the
schema needs. See [Extensions](#extensions).
### Extension Indexes
`Index.Type` selects the access method: `gist`, `spgist`, `brin` (PostGIS), `hnsw`, `ivfflat`
(pgvector), `vchordrq`, `vchordg` (VectorChord), `bm25` (pg_search).
Operator class and access-method parameters ride in `Index.Comment`:
```
opclass=vector_l2_ops; with (lists=100)
```
- `opclass=<name>` — used only when compatible with the column type; otherwise ignored.
Bare operator class names in the comment (e.g. `gin_trgm_ops`) are also recognized.
- `with (k=v, …)` — rendered as `WITH (k = v, …)`. Only well-formed `key = value` pairs are
kept, so comment prose never reaches the DDL. Values may be bare (`lists=100`), quoted
(`key_field='id'`), or dollar-quoted (`options=$$[build.internal]$$`).
Defaults when no operator class is requested:
| Access method | Column type | Emitted operator class |
|---------------|-------------|------------------------|
| `hnsw`, `ivfflat`, `vchordrq`, `vchordg` | `vector` / `halfvec` / `sparsevec` / `bit` | `vector_cosine_ops` / `halfvec_cosine_ops` / `sparsevec_cosine_ops` / `bit_hamming_ops` |
| `gist`, `spgist`, `brin` | `geometry`, `geography` | none (PostGIS default operator class) |
| `gin` | text / `jsonb` / array | `gin_trgm_ops` / `jsonb_ops` / `array_ops` |
pgvector defines no default operator class, so a vector index always names one.
```sql
CREATE INDEX IF NOT EXISTS idx_documents_embedding
ON public.documents USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100);
CREATE INDEX IF NOT EXISTS idx_documents_location
ON public.documents USING gist (location);
```
Migrations only recreate an index when both sides specify a hint and they differ, so a model
without hints does not churn against a live database.
## Extensions
`CREATE EXTENSION IF NOT EXISTS <ext>;` is emitted per schema, deduplicated and ordered so
dependencies come first (`postgis` before `postgis_topology`/`postgis_raster`/`pgrouting`,
`vector` before `vchord`). Names needing quoting are quoted: `CREATE EXTENSION IF NOT EXISTS "uuid-ossp";`
### Detection
| Source | Example | Extension |
|--------|---------|-----------|
| Column type | `vector(1536)`, `geometry(Point,4326)`, `citext`, `ltree` | `vector`, `postgis`, `citext`, `ltree` |
| Index access method | `hnsw`, `ivfflat` / `vchordrq`, `vchordg` / `bm25` | `vector` / `vchord` / `pg_search` |
| Operator class | `gin_trgm_ops`, `gist_ltree_ops` | `pg_trgm`, `ltree` |
| GIN/GiST on a scalar type | `USING gin (views)` | `btree_gin` / `btree_gist` |
| Function in a default, CHECK, index `WHERE`, or view body | `uuid_generate_v4()`, `crypt()`, `ST_Area()`, `unaccent()`, `json_matches_schema()` | `uuid-ossp`, `pgcrypto`, `postgis`, `unaccent`, `pg_jsonschema` |
`gen_random_uuid()` is built in since PostgreSQL 13 and does not pull in `pgcrypto`.
### Declaring extensions explicitly
Extensions that leave no trace in the schema go in `schema.Metadata["extensions"]`, as a list
or a comma-separated string. Dependencies are pulled in automatically; unknown names are kept
as given. The PostgreSQL reader populates this from `pg_extension` for the schemas it reads.
```yaml
metadata:
extensions: [pg_cron, timescaledb, pg_stat_statements]
```
### Recognized extensions
| Category | Extensions |
|----------|------------|
| ai/search | `vector`, `vchord` |
| document | `hstore`, `ltree` |
| federation | `postgres_fdw` |
| geospatial | `postgis`, `postgis_raster`, `postgis_topology`, `pgrouting` |
| indexing | `btree_gin`, `btree_gist` |
| integration | `http` |
| integrity | `amcheck` |
| jobs / scheduling | `pg_background`, `pg_cron` |
| maintenance | `pg_repack`, `pgstattuple` |
| observability | `pg_qualstats`, `pg_stat_statements` |
| partitioning | `pg_partman` |
| procedural | `plpython3u` |
| search | `pg_search`, `pg_textsearch` |
| security | `pgcrypto` |
| text | `citext`, `fuzzystrmatch`, `pg_trgm`, `unaccent` |
| time-series | `timescaledb` |
| utility | `uuid-ossp` |
| validation | `pg_jsonschema` |
## Notes
- Generated SQL is formatted and readable
- Comments are preserved from source schema
- Schema names are fully qualified
- Default values are properly quoted
- Constraint names follow PostgreSQL conventions
- Compatible with PostgreSQL 12+