# RelSpec

[![Release](https://img.shields.io/gitea/v/release/wdevs/relspecgo?gitea_url=https://git.warky.dev&label=release)](https://git.warky.dev/wdevs/relspecgo/releases/latest)
[![CI](https://git.warky.dev/wdevs/relspecgo/actions/workflows/ci.yml/badge.svg)](https://git.warky.dev/wdevs/relspecgo/actions/workflows/ci.yml)
[![Integration Tests](https://git.warky.dev/wdevs/relspecgo/actions/workflows/integration-tests.yml/badge.svg)](https://git.warky.dev/wdevs/relspecgo/actions/workflows/integration-tests.yml)
[![Go Version](https://img.shields.io/badge/go-1.24.0-blue.svg)](https://go.dev/dl/)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)

> Bidirectional database schema conversion, validation, and templating tool.

![RelSpec](./assets/image/relspec1_512.jpg)

## Install

```bash
go install -v git.warky.dev/wdevs/relspecgo/cmd/relspec@latest
```

Windows: download `relspec-setup-windows-amd64.exe` from the
[latest release](https://git.warky.dev/wdevs/relspecgo/releases/latest)
(installs to Program Files and adds `relspec` to `PATH`). See [windows/README.md](windows/README.md).

## Supported Formats

| Direction | Formats |
|-----------|---------|
| **Readers** | `bun` `dbml` `dctx` `drawdb` `drizzle` `gorm` `graphql` `json` `mssql` `mysql` `pgsql` `prisma` `sqldir` `sqlite` `typeorm` `yaml` |
| **Writers** | `bun` `dbml` `dctx` `drawdb` `drizzle` `gorm` `graphql` `json` `mssql` `mysql` `pgsql` `prisma` `sqlexec` `sqlite` `template` `typeorm` `yaml` |

See [docs/FORMAT_EXAMPLES.md](docs/FORMAT_EXAMPLES.md) for usage examples covering every format.

## Commands

### `convert` — Schema conversion

```bash
# PostgreSQL → GORM models
relspec convert --from pgsql --from-conn "postgres://user:pass@localhost/mydb" \
                --to gorm --to-path models/ --package models

# DBML → PostgreSQL DDL
relspec convert --from dbml --from-path schema.dbml --to pgsql --to-path schema.sql

# Lowercase model names and structured references before output
relspec convert --from dbml --from-path schema.dbml --to pgsql --to-path schema.sql --lowercase

# PostgreSQL → SQLite (auto flattens schemas)
relspec convert --from pgsql --from-conn "postgres://..." --to sqlite --to-path schema.sql

# Multiple input files merged
relspec convert --from json --from-list "a.json,b.json" --to yaml --to-path merged.yaml

# Watch mode: regenerate whenever the source file(s) change (Ctrl-C to stop)
relspec convert --from dbml --from-path schema.dbml --to gorm --to-path models/ --package models --watch
```

`--watch` works with `--from-path` and `--from-list` (not live database
connections or `--dry-run`). Source files are polled every `--watch-interval`
(default 500ms), a directory source is watched recursively, and the output path
is ignored so generating into the source tree does not loop. Conversion errors
are printed and watching continues.

#### Identifier casing and PostgreSQL quoting

PostgreSQL output always double-quotes model identifiers, including reserved
words, and preserves their case by default. For example, schema `Sales` and table
`Order` produce `"Sales"."Order"`; with `--lowercase`, they produce
`"sales"."order"`. Quoted names are case-sensitive in PostgreSQL.

`--lowercase` is available on `convert`, `batch`, `merge`, `split`, `templ`, and
`job run`. It lowercases model names and structured references on a copy and
rejects collisions such as `Users` and `users` in the same schema. SQL expressions,
defaults, comments, enum values, and data retain their case. Adjust identifier
references inside your own SQL expressions when opting into lowercasing.

Generated PostgreSQL SQL also escapes identifier quotes and literal values and
chooses dollar-quote delimiters that do not collide with function or procedural
block contents. Raw SQL expressions remain executable input. See the
[PostgreSQL writer documentation](pkg/writers/pgsql/README.md#identifier-and-value-handling)
for library options and details.

#### MySQL fidelity and strict conversion

MySQL output now preserves foreign keys and their actions, checks, indexes,
unsigned types, dimensions, generated columns, and native defaults. Known
unsupported features produce warnings. Add `--strict-conversion` to reject
lossy MySQL output before writing a file or executing DDL:

```bash
relspec convert --from mysql --from-conn "$MYSQL_DSN" \
                --to mysql --to-path schema.sql --strict-conversion
```

Schema jobs can set `options.strict_conversion: true`. This check currently
covers MySQL output; it does not guarantee lossless conversion to every format.
The live suite covers MySQL 8.4 / InnoDB, including JSON/YAML intermediates.
See the [MySQL capability matrix](docs/MYSQL_FIDELITY.md) for unsupported objects,
metadata, SQL-mode requirements, and execution limits. Output creates tables in
existing databases; it is not a data copy or a rerunnable migration.

### `batch` — Convert many inputs in one run

Converts each input independently (one output per input, unlike `--from-list`
which merges). `--input` takes paths or globs; outputs go to `--to-dir`.
Use `--keep-going` to continue past failures (exit is still non-zero) and
`--dry-run` to validate without writing. For named workflows see `relspec job run`.

```bash
relspec batch --from dbml --input "schemas/*.dbml" --to json --to-dir out/
```

PostgreSQL connections opened by relspec set `application_name` by default to
`relspecgo/<version>` (with component suffixes internally, e.g. readers/writers).
If you need a custom value, provide `application_name` explicitly in the connection
string query parameters.

### `merge` — Additive schema merge (never modifies existing items)

```bash
# Merge two JSON schemas
relspec merge --target json --target-path base.json \
              --source json --source-path additions.json \
              --output json --output-path merged.json

# Merge PostgreSQL into JSON, skipping tables
relspec merge --target json --target-path current.json \
              --source pgsql --source-conn "postgres://user:pass@localhost/db" \
              --output json --output-path updated.json \
              --skip-tables "audit_log,temp_tables"
```

Skip flags: `--skip-relations` `--skip-views` `--skip-domains` `--skip-enums` `--skip-sequences`

### `inspect` — Schema validation / linting

```bash
# Validate PostgreSQL database
relspec inspect --from pgsql --from-conn "postgres://user:pass@localhost/mydb"

# Validate DBML with custom rules
relspec inspect --from dbml --from-path schema.dbml --rules .relspec-rules.yaml

# JSON report output
relspec inspect --from json --from-path db.json --output-format json --output report.json

# Filter to specific schema
relspec inspect --from pgsql --from-conn "..." --schema public
```

Rules: naming conventions, PK/FK standards, missing indexes, reserved keywords, circular dependencies.

### `diff` — Schema comparison

```bash
relspec diff --from pgsql --from-conn "postgres://localhost/db1" \
             --to pgsql --to-conn "postgres://localhost/db2"
```

### `templ` — Custom template rendering

```bash
# Render database schema to Markdown docs
relspec templ --from pgsql --from-conn "postgres://user:pass@localhost/db" \
              --template docs.tmpl --output schema-docs.md

# One TypeScript file per table
relspec templ --from dbml --from-path schema.dbml \
              --template ts-model.tmpl --mode table \
              --output ./models/ --filename-pattern "{{.Name | toCamelCase}}.ts"
```

Modes: `database` (default) · `schema` · `table` · `script`

Template functions: string utils (`toCamelCase`, `toSnakeCase`, `pluralize`, …), type converters (`sqlToGo`, `sqlToTypeScript`, …), filters, loop helpers, safe access.

### `job` — Declarative job files

Run named jobs from a `relspec.yml` manifest instead of repeating long command lines.

```bash
# List jobs discovered in ./relspec.yml and ./relspec.<name>.yml (deterministic)
relspec job list

# Validate and print the plan without running anything
relspec job run build-schema --plan

# Run a job (and its declared dependencies)
relspec job run build-schema
```

```yaml
# relspec.yml
version: 1
jobs:
  build-schema:
    command: convert            # closed allow-list: convert | merge | split | scripts-list | scripts-exec | templ | inspect | diff
    description: Merge the DBML sources and emit PostgreSQL DDL
    inputs:
      - path: schema/core.dbml
        format: dbml
      - path: schema/tenant.dbml
        format: dbml
    output:
      format: pgsql
      path: build/schema.sql
      overwrite: true
    options:
      flatten_schema: false
      lowercase_identifiers: true # opt in; defaults to false
    logfile: .relspec/log/build-schema.log
```

The job system is **not** a shell: `command` is a fixed enum, every path is
resolved relative to the job file and may not escape it, and remote database
credentials are referenced by environment-variable name (`conn_env:`) and
redacted from logs. The whole plan — unknown commands/formats, duplicate job
names, missing inputs, path traversal, dependency cycles — is validated before
any job runs. Path-like fields also support `${NAME}` environment-variable
references, which are expanded and safety-checked during pre-flight. See
[docs/JOB_FILES.md](docs/JOB_FILES.md).

Use `options.lowercase_identifiers: true` per job, or
`relspec job run build-schema --lowercase` to enable lowercasing for schema outputs
throughout the selected job plan. For MySQL schema output,
`options.strict_conversion: true` or `--strict-conversion` rejects known lossy
conversions before writing.

### `scripts` — SQL scripts and embedded assets

```bash
relspec scripts list --dir ./migrations
relspec scripts execute --dir ./migrations --conn "$DATABASE_URL"
```

Execution stops on the first error by default. `--continue-on-error` (alias
`--coe`) attempts later scripts and still exits nonzero if any script fails.

Scripts can embed nearby files using a standalone directive comment:

```sql
-- @embed: path=assets/message.txt var=:message mode=text
INSERT INTO messages (body) VALUES (:message);
```

Embed replacement preserves dollar text such as `$1`, `${2}`, and `$HOME` exactly.
Placeholders are replaced only in executable SQL; quoted strings, quoted
identifiers, comments, casts, and dollar-quoted bodies retain their contents.
Inserted content is never scanned for further placeholders. Asset-call named
parameters follow the same SQL lexical boundaries. See
[docs/SCRIPTS_COMMAND.md](docs/SCRIPTS_COMMAND.md) for file containment, limits,
binary embedding, and execution details.

MySQL DDL execution keeps each generated statement whole, preserving multiline
defaults containing semicolons or comment-like text.

### `edit` — Interactive TUI editor

```bash
# Edit DBML schema interactively
relspec edit --from dbml --from-path schema.dbml --to dbml --to-path schema.dbml

# Edit live PostgreSQL database
relspec edit --from pgsql --from-conn "postgres://user:pass@localhost/mydb" \
             --to pgsql --to-conn "postgres://user:pass@localhost/mydb"
```

<p align="center">
<img src="./assets/image/screenshots/main_screen.jpg">
</p>
<p align="center">
<img src="./assets/image/screenshots/table_view.jpg">
</p>
<p align="center">
<img src="./assets/image/screenshots/edit_column.jpg">
</p>

### `update` — Check for a newer release

```bash
relspec update            # check, prompt, then download + run installer (Windows)
relspec update --check    # report only
relspec update --yes      # skip the prompt
```

Non-Windows platforms print the release URL. `dev` and commit-hash builds are never reported as outdated.

## Development

**Prerequisites:** Go 1.24.0+

```bash
make build     # → build/relspec
make test      # race detection + coverage
make lint      # requires golangci-lint
make coverage  # → coverage.html
make install   # → $GOPATH/bin
make installer-windows  # → build/relspec-setup-windows-amd64.exe (needs makensis)
```

## Project Structure

```
cmd/relspec/          CLI commands
pkg/readers/          Input format readers
pkg/writers/          Output format writers
pkg/inspector/        Schema validation
pkg/diff/             Schema comparison
pkg/merge/            Schema merging
pkg/models/           Internal data models
pkg/transform/        Transformation logic
pkg/pgsql/            PostgreSQL utilities
pkg/updatecheck/      Latest-release lookup and version comparison
windows/              NSIS installer script
pkg/sqltypes/         Nullable SQL types for generated/hand-written models (see below)
```

## Nullable Types (`pkg/sqltypes`)

The `bun` and `gorm` writers can generate model structs using
[`pkg/sqltypes`](./pkg/sqltypes/README.md) — nullable types (`SqlString`,
`SqlInt32`, `SqlTimeStamp`, `SqlStringArray`, …) that implement
`database/sql.Scanner`, `driver.Valuer`, and JSON/YAML/XML marshalling in one
type, selected via `--types sqltypes`. See the
[`pkg/sqltypes` README](./pkg/sqltypes/README.md) for the full type
reference, or the [`bun`](./pkg/writers/bun/README.md) /
[`gorm`](./pkg/writers/gorm/README.md) writer docs for the `--types` flag
(`sqltypes`, `stdlib`, or `baselib`). PostgreSQL array columns are the one
exception: the `bun` writer always generates native Go slices (`[]string`,
`[]int32`, …) with an explicit `array` bun tag, regardless of `--types` —
see [`bun`'s `--array-nullable`](./pkg/writers/bun/README.md#nullablearrays)
flag for nullable-array handling. The `SqlXxxArray` wrapper types remain
available in `pkg/sqltypes` and are still used by the `gorm` writer.

#### Custom type mapping

Override the built-in SQL → Go mapping of the `bun` and `gorm` writers with the
repeatable `--type-map sqltype=gotype` flag:

```bash
relspec convert --from pgsql --from-conn "$DSN" --to gorm --to-path models.go \
  --type-map uuid=string --type-map jsonb=json.RawMessage
```

SQL type names are matched case-insensitively on the base type (modifiers such
as `(10,2)` are ignored; aliases like `int4` resolve to `integer`). NOT NULL
columns use the Go type verbatim, nullable columns get a `*` prefix (unless the
type is already a pointer, slice, map or `any`), and arrays become `[]gotype`.
Unmapped types keep their defaults. The flag does not add imports: use types
that need none, or add the import afterwards (e.g. with `goimports`).

## Contributing

1. Register or sign in with GitHub at [git.warky.dev](https://git.warky.dev)
2. Clone the repository: `git clone https://git.warky.dev/wdevs/relspecgo.git`
3. Create a feature branch: `git checkout -b feature/your-feature-name`
4. Commit your changes and push the branch
5. Open a pull request with a description of the new feature or fix

For questions or discussion, join the Discord: [discord.gg/74rcTujp25](https://discord.gg/74rcTujp25) — `warkyhein`

## Links

- [Todo](./TODO.md)
- [AI Use Policy](./AI_USE.md)
- [License](LICENSE) — Apache 2.0 · Copyright 2025 Warky Devs