relspecgo/pkg/writers/pgsql/README.md

# PostgreSQL Writer

Generates PostgreSQL DDL (Data Definition Language) SQL scripts from database schema information.

## Overview

The PostgreSQL Writer converts RelSpec's internal database model representation into PostgreSQL-compatible SQL DDL scripts, including CREATE TABLE statements, constraints, indexes, views, and sequences.

## Features

- Generates complete PostgreSQL DDL
- Creates schemas, tables, columns
- Defines constraints (PK, FK, unique, check)
- Creates indexes
- Generates views and sequences
- Supports migration scripts
- Includes audit triggers (optional)
- Handles PostgreSQL-specific data types

## Identifier and value handling

Generated identifiers are always double-quoted, including reserved words, and
retain the model's case by default. Supply raw model names, without SQL quoting.
Identifiers containing double quotes are escaped by doubling them; NUL is rejected
before database/file output. Catalog checks, comments, audit values, and regclass
names use SQL literal quoting; generated procedural/function bodies choose dollar
delimiters absent from their rendered content. Names in generated line comments
cannot create new SQL lines.

Use CLI `--lowercase`, job option `lowercase_identifiers: true`, or PostgreSQL
`WriterOptions.LowercaseIdentifiers: true` to lowercase model names and structured
references. The transformation copies the model and rejects name collisions. Other
library writers can use `transform.LowercaseIdentifiers` before writing.

SQL expressions are executable input: types, defaults, generated expressions,
checks, index expressions/predicates, dialect directive clauses, audit user
expressions, and `pgsql_using` / `pgsql_preflight` are not sanitized or case-folded.
Adjust identifier references inside these expressions yourself when lowercasing.
Enum values, comments, and other data retain their case. Migration matching respects
identifier case; `"Users"` and `"users"` are distinct objects.

## Usage

### Basic Example

```go
package main

import (
    "git.warky.dev/wdevs/relspecgo/pkg/models"
    "git.warky.dev/wdevs/relspecgo/pkg/writers"
    "git.warky.dev/wdevs/relspecgo/pkg/writers/pgsql"
)

func main() {
    options := &writers.WriterOptions{
        OutputPath: "schema.sql",
    }

    writer := pgsql.NewWriter(options)
    err := writer.WriteDatabase(db)
    if err != nil {
        panic(err)
    }
}
```

### CLI Examples

```bash
# Generate PostgreSQL DDL from JSON schema
relspec --input json \
  --in-file schema.json \
  --output pgsql \
  --out-file schema.sql

# Convert GORM models to PostgreSQL DDL
relspec --input gorm \
  --in-file models.go \
  --output pgsql \
  --out-file create_tables.sql

# Export live database schema to SQL
relspec --input pgsql \
  --conn "postgres://localhost/source_db" \
  --output pgsql \
  --out-file backup_schema.sql
```

## Generated SQL Example

```sql
-- Schema: public

CREATE SCHEMA IF NOT EXISTS public;

-- Table: public.users

CREATE TABLE IF NOT EXISTS public.users (
    id BIGSERIAL PRIMARY KEY,
    username VARCHAR(50) NOT NULL,
    email VARCHAR(100) NOT NULL,
    bio TEXT,
    created_at TIMESTAMP NOT NULL DEFAULT NOW()
);

-- Constraints for public.users

ALTER TABLE public.users
    ADD CONSTRAINT uq_users_username UNIQUE (username);

-- Indexes for public.users

CREATE INDEX idx_users_email ON public.users (email);

-- Table: public.posts

CREATE TABLE IF NOT EXISTS public.posts (
    id BIGSERIAL PRIMARY KEY,
    user_id BIGINT NOT NULL,
    title VARCHAR(200) NOT NULL,
    content TEXT,
    created_at TIMESTAMP DEFAULT NOW()
);

-- Foreign Keys for public.posts

ALTER TABLE public.posts
    ADD CONSTRAINT fk_posts_user_id
    FOREIGN KEY (user_id)
    REFERENCES public.users (id)
    ON DELETE CASCADE
    ON UPDATE NO ACTION;

-- Indexes for public.posts

CREATE INDEX idx_posts_user_id ON public.posts (user_id);
```

## Writer Options

### Metadata Options

```go
options := &writers.WriterOptions{
    OutputPath: "schema.sql",
    Metadata: map[string]interface{}{
        "include_drop":    true,  // Include DROP statements
        "include_audit":   true,  // Include audit triggers
        "if_not_exists":   true,  // Use IF NOT EXISTS
        "migration_mode":  false, // Generate migration script
    },
}
```

## Features

### Full DDL Generation

Generates complete database structure:
- CREATE SCHEMA statements
- CREATE TABLE with all columns and types
- PRIMARY KEY constraints
- FOREIGN KEY constraints with actions
- UNIQUE constraints
- CHECK constraints
- CREATE INDEX statements
- CREATE VIEW statements
- CREATE SEQUENCE statements

### Migration Mode

When `migration_mode` is enabled, generates migration scripts with:
- Version tracking
- Up/down migrations
- Transactional DDL
- Rollback support

### Unique constraints and indexes

- Unique constraint → `ADD CONSTRAINT <name> UNIQUE (cols)`, idempotent (checked by name)
- Migration mode and direct apply both emit them (not only `convert`)
- Same-named model unique index → `UNIQUE USING INDEX` (no duplicate index). Requires: same name, same columns, btree, non-partial, no `INCLUDE`
- Otherwise a plain `ADD CONSTRAINT` (second index is built)
- Naming: unique constraint and unique index both `ukey_<table>_<cols>`; see `NAMING_CONVENTIONS.md`
- Renaming `uidx_…` → `ukey_…` drops the old index and re-adds the constraint on the next diff

### FK deferral

| Model (`Deferrable` / `InitiallyDeferred`) | SQL |
| ------------------------------------------ | --- |
| unset | create: `DEFERRABLE` (migration/apply), none (full DDL constraints); existing FK never diffed |
| `true` / unset or `false` | `DEFERRABLE` |
| `true` / `true`, or unset / `true` | `DEFERRABLE INITIALLY DEFERRED` |
| `false` | `NOT DEFERRABLE` |

- Both fields are `*bool`; `nil` = unspecified
- Specified value differing from the live DB → FK dropped and re-added
- Live reader reads `condeferrable` / `condeferred`
- Initial mode compared only when both sides state it

### Audit Triggers

When `include_audit` is enabled, adds:
- Created/updated timestamp triggers
- Audit logging functionality
- Change tracking

## PostgreSQL-Specific Features

- Serial types (SERIAL, BIGSERIAL)
- Generated columns: `GENERATED ALWAYS AS (expr) STORED` (`Column.Generated` + `GenerationExpression`)
- Identity columns: `GENERATED ALWAYS|BY DEFAULT AS IDENTITY` (`Column.Identity` + `IdentityGeneration`)
- Generated/identity columns never carry a `DEFAULT`; emitted in `CREATE TABLE` and `ADD COLUMN` (full DDL and migration)
- Existing generated/identity columns are not diffed (expression changes are not applied)
- PK sequence = the `nextval()` default's sequence; `setval` moves it past `MAX(pk)`, forward only
- `serial`/`bigserial` columns without a model default keep their `nextval()` default in diffs
- Advanced types (UUID, JSONB, ARRAY)
- Schema-qualified names
- Constraint actions (CASCADE, RESTRICT, SET NULL)
- Partial indexes
- Function-based indexes
- Concurrent index creation (`CREATE INDEX CONCURRENTLY`) via `Index.Concurrent`
- Check constraints with expressions
- Extension types and indexes: PostGIS, pgvector, citext, hstore, ltree (see below)
- DBML dialect directives (`@postgres:` — see below)

### DBML dialect directives

`@postgres:` directives carried on a model object's `Metadata` (typically from a
DBML source file) are translated to SQL:

| Directive | Location | Emitted |
|-----------|----------|---------|
| `@postgres: partition by …` | table | `PARTITION BY …` on `CREATE TABLE` |
| `@postgres: inherits …` | table | `INHERITS (…)` |
| `@postgres: with (…)` | table, index | `WITH (…)` (on an index, overrides the comment-derived `WITH`) |
| `@postgres: tablespace …` | table, index | `TABLESPACE …` |
| `@postgres(col): storage …` | column | `STORAGE …` |
| `@postgres(col): compression …` | column | `COMPRESSION …` |
| `@postgres(col): identity always` / `identity by default` | column | `GENERATED ALWAYS/BY DEFAULT AS IDENTITY` |

Directives for other dialects (`@sqlite:` …) are ignored. With
`WriterOptions.StrictDirectives` (CLI `--strict-directives`) an untranslatable
`@postgres:` key is an error. Full reference: [`docs/DBML_DIRECTIVES.md`](../../../docs/DBML_DIRECTIVES.md).

## Data Types

Supports all PostgreSQL data types:
- Integer types: SMALLINT, INTEGER, BIGINT, SERIAL, BIGSERIAL
- Numeric types: NUMERIC, DECIMAL, REAL, DOUBLE PRECISION
- String types: VARCHAR, CHAR, TEXT
- Date/Time: DATE, TIME, TIMESTAMP, TIMESTAMPTZ, INTERVAL
- Boolean: BOOLEAN
- Binary: BYTEA
- JSON: JSON, JSONB
- UUID: UUID
- Network: INET, CIDR, MACADDR
- Special: ARRAY, HSTORE

## Extension Types (PostGIS, pgvector)

Extension column types are preserved verbatim, including their type modifier:

| Type | Example column type | Extension |
|------|---------------------|-----------|
| PostGIS | `geometry(Point,4326)`, `geography(Point)`, `box2d`, `raster` | `postgis`, `postgis_raster`, `postgis_topology` |
| pgvector | `vector(1536)`, `halfvec(768)`, `sparsevec(1000)` | `vector` |
| Other | `citext`, `hstore`, `ltree` | `citext`, `hstore`, `ltree` |

`CREATE EXTENSION IF NOT EXISTS <ext>;` is emitted automatically for every extension the
schema needs. See [Extensions](#extensions).

### Extension Indexes

`Index.Type` selects the access method: `gist`, `spgist`, `brin` (PostGIS), `hnsw`, `ivfflat`
(pgvector), `vchordrq`, `vchordg` (VectorChord), `bm25` (pg_search).

Operator class and access-method parameters ride in `Index.Comment`:

```
opclass=vector_l2_ops; with (lists=100)
```

- `opclass=<name>` — used only when compatible with the column type; otherwise ignored.
  Bare operator class names in the comment (e.g. `gin_trgm_ops`) are also recognized.
- `with (k=v, …)` — rendered as `WITH (k = v, …)`. Only well-formed `key = value` pairs are
  kept, so comment prose never reaches the DDL. Values may be bare (`lists=100`), quoted
  (`key_field='id'`), or dollar-quoted (`options=$$[build.internal]$$`).

Defaults when no operator class is requested:

| Access method | Column type | Emitted operator class |
|---------------|-------------|------------------------|
| `hnsw`, `ivfflat`, `vchordrq`, `vchordg` | `vector` / `halfvec` / `sparsevec` / `bit` | `vector_cosine_ops` / `halfvec_cosine_ops` / `sparsevec_cosine_ops` / `bit_hamming_ops` |
| `gist`, `spgist`, `brin` | `geometry`, `geography` | none (PostGIS default operator class) |
| `gin` | text / `jsonb` / array | `gin_trgm_ops` / `jsonb_ops` / `array_ops` |

pgvector defines no default operator class, so a vector index always names one.

```sql
CREATE INDEX IF NOT EXISTS idx_documents_embedding
  ON public.documents USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100);
CREATE INDEX IF NOT EXISTS idx_documents_location
  ON public.documents USING gist (location);
```

Migrations only recreate an index when both sides specify a hint and they differ, so a model
without hints does not churn against a live database.

## Extensions

`CREATE EXTENSION IF NOT EXISTS <ext>;` is emitted per schema, deduplicated and ordered so
dependencies come first (`postgis` before `postgis_topology`/`postgis_raster`/`pgrouting`,
`vector` before `vchord`). Names needing quoting are quoted: `CREATE EXTENSION IF NOT EXISTS "uuid-ossp";`

### Detection

| Source | Example | Extension |
|--------|---------|-----------|
| Column type | `vector(1536)`, `geometry(Point,4326)`, `citext`, `ltree` | `vector`, `postgis`, `citext`, `ltree` |
| Index access method | `hnsw`, `ivfflat` / `vchordrq`, `vchordg` / `bm25` | `vector` / `vchord` / `pg_search` |
| Operator class | `gin_trgm_ops`, `gist_ltree_ops` | `pg_trgm`, `ltree` |
| GIN/GiST on a scalar type | `USING gin (views)` | `btree_gin` / `btree_gist` |
| Function in a default, CHECK, index `WHERE`, or view body | `uuid_generate_v4()`, `crypt()`, `ST_Area()`, `unaccent()`, `json_matches_schema()` | `uuid-ossp`, `pgcrypto`, `postgis`, `unaccent`, `pg_jsonschema` |

`gen_random_uuid()` is built in since PostgreSQL 13 and does not pull in `pgcrypto`.

### Declaring extensions explicitly

Extensions that leave no trace in the schema go in `schema.Metadata["extensions"]`, as a list
or a comma-separated string. Dependencies are pulled in automatically; unknown names are kept
as given. The PostgreSQL reader populates this from `pg_extension` for the schemas it reads.

```yaml
metadata:
  extensions: [pg_cron, timescaledb, pg_stat_statements]
```

### Recognized extensions

| Category | Extensions |
|----------|------------|
| ai/search | `vector`, `vchord` |
| document | `hstore`, `ltree` |
| federation | `postgres_fdw` |
| geospatial | `postgis`, `postgis_raster`, `postgis_topology`, `pgrouting` |
| indexing | `btree_gin`, `btree_gist` |
| integration | `http` |
| integrity | `amcheck` |
| jobs / scheduling | `pg_background`, `pg_cron` |
| maintenance | `pg_repack`, `pgstattuple` |
| observability | `pg_qualstats`, `pg_stat_statements` |
| partitioning | `pg_partman` |
| procedural | `plpython3u` |
| search | `pg_search`, `pg_textsearch` |
| security | `pgcrypto` |
| text | `citext`, `fuzzystrmatch`, `pg_trgm`, `unaccent` |
| time-series | `timescaledb` |
| utility | `uuid-ossp` |
| validation | `pg_jsonschema` |

## Notes

- Generated SQL is formatted and readable
- Comments are preserved from source schema
- Schema names are fully qualified
- Default values are properly quoted
- Constraint names follow PostgreSQL conventions
- Compatible with PostgreSQL 12+

## Live execution failures

Live PostgreSQL output stops after the first failed statement by default and returns
`*writers.ExecutionFailure` (also available as `*pgsql.ExecutionFailure`; inspect it
with `errors.As`). `WriterOptions.ContinueOnError` or job
`options.continue_on_error: true` attempts later statements but still returns an error
if any fail, preventing dependent jobs from running. A failed type preflight always
stops live execution, including when continuation is enabled.

PostgreSQL, SQLite, SQL Server, and the PostgreSQL script runner share execution
reports through `WriterOptions.Metadata["execution_report"]`. Optional
`Metadata["report_path"]` writes the report as JSON; write and close failures are
errors. Reports include each operation's status and original number, plus applied,
failed, skipped, and rolled-back counts. Blank and comment entries count as skipped.
Unknown commit/rollback outcomes are explicitly marked `unknown` and are never
reported as confirmed rollback. Connection failures record all planned operations as
skipped. Raw scripts have different reporting limits; see the sqlexec README.

Generated DDL uses independent statements by default: earlier changes can remain
committed after failure. Set `WriterOptions.AtomicExecution = true` to use one
transaction with PostgreSQL, SQLite, or SQL Server generated DDL. For database output
jobs, use:

```yaml
options:
  atomic_execution: true
```

Atomic execution and continuation are mutually exclusive. Rollback covers database
transactional effects; sequence advancement and external effects in SQL expressions
are outside that guarantee. Concurrency locks, migration history, and resumable
migration management remain separate work (audit A09).

## Column type conversions

Type changes propagate errors and never implicitly rename or replace a column.
Before schema changes, full-DDL and diff plans evaluate every existing value through
the conversion expression and target cast. Diff preflights run before dropping
constraints/indexes or creating sequences; database-wide preflights run before
changing any schema. Matching types, missing tables/columns, generated columns, and
identity columns need no conversion check.

Column metadata in JSON/YAML models supports two optional SQL expressions:

```yaml
metadata:
  pgsql_using: "split_part(name, ' ', 1)::integer"
  pgsql_preflight: "name ~ '^[0-9]+ units$'"
```

`pgsql_using` replaces the default `column::target_type` expression in `USING`.
`pgsql_preflight` is a row predicate; false or NULL rejects the conversion. These
fields must contain nonempty strings. Both are trusted executable SQL, like defaults
and check expressions; they are not values or a sandbox. Preflights evaluate the
conversion expression, so use expressions without side effects. New types/extensions
needed by an existing-column conversion must already be available for preflight.

Preflight checks can race concurrent writes and do not prove that locks, defaults,
or every dependency will permit the ALTER. The ALTER still validates the conversion;
use atomic execution when preceding operations also need rollback. Generated SQL
clients must stop on preflight failure; live RelSpec execution enforces this even
with continuation enabled. Rename/replacement/backfill operations still require a
separately reviewed migration.