366 lines
15 KiB
Markdown
366 lines
15 KiB
Markdown
# RelSpec
|
|
|
|
[](https://git.warky.dev/wdevs/relspecgo/releases/latest)
|
|
[](https://git.warky.dev/wdevs/relspecgo/actions/workflows/ci.yml)
|
|
[](https://git.warky.dev/wdevs/relspecgo/actions/workflows/integration-tests.yml)
|
|
[](https://go.dev/dl/)
|
|
[](LICENSE)
|
|
|
|
> Bidirectional database schema conversion, validation, and templating tool.
|
|
|
|
|
|
|
|
## Install
|
|
|
|
```bash
|
|
go install -v git.warky.dev/wdevs/relspecgo/cmd/relspec@latest
|
|
```
|
|
|
|
Windows: download `relspec-setup-windows-amd64.exe` from the
|
|
[latest release](https://git.warky.dev/wdevs/relspecgo/releases/latest)
|
|
(installs to Program Files and adds `relspec` to `PATH`). See [windows/README.md](windows/README.md).
|
|
|
|
## Supported Formats
|
|
|
|
| Direction | Formats |
|
|
|-----------|---------|
|
|
| **Readers** | `bun` `dbml` `dctx` `drawdb` `drizzle` `gorm` `graphql` `json` `mssql` `mysql` `pgsql` `prisma` `sqldir` `sqlite` `typeorm` `yaml` |
|
|
| **Writers** | `bun` `dbml` `dctx` `drawdb` `drizzle` `gorm` `graphql` `json` `mssql` `mysql` `pgsql` `prisma` `sqlexec` `sqlite` `template` `typeorm` `yaml` |
|
|
|
|
See [docs/FORMAT_EXAMPLES.md](docs/FORMAT_EXAMPLES.md) for usage examples covering every format.
|
|
|
|
## Commands
|
|
|
|
### `convert` — Schema conversion
|
|
|
|
```bash
|
|
# PostgreSQL → GORM models
|
|
relspec convert --from pgsql --from-conn "postgres://user:pass@localhost/mydb" \
|
|
--to gorm --to-path models/ --package models
|
|
|
|
# DBML → PostgreSQL DDL
|
|
relspec convert --from dbml --from-path schema.dbml --to pgsql --to-path schema.sql
|
|
|
|
# Lowercase model names and structured references before output
|
|
relspec convert --from dbml --from-path schema.dbml --to pgsql --to-path schema.sql --lowercase
|
|
|
|
# PostgreSQL → SQLite (auto flattens schemas)
|
|
relspec convert --from pgsql --from-conn "postgres://..." --to sqlite --to-path schema.sql
|
|
|
|
# Multiple input files merged
|
|
relspec convert --from json --from-list "a.json,b.json" --to yaml --to-path merged.yaml
|
|
|
|
# Watch mode: regenerate whenever the source file(s) change (Ctrl-C to stop)
|
|
relspec convert --from dbml --from-path schema.dbml --to gorm --to-path models/ --package models --watch
|
|
```
|
|
|
|
`--watch` works with `--from-path` and `--from-list` (not live database
|
|
connections or `--dry-run`). Source files are polled every `--watch-interval`
|
|
(default 500ms), a directory source is watched recursively, and the output path
|
|
is ignored so generating into the source tree does not loop. Conversion errors
|
|
are printed and watching continues.
|
|
|
|
#### Identifier casing and PostgreSQL quoting
|
|
|
|
PostgreSQL output always double-quotes model identifiers, including reserved
|
|
words, and preserves their case by default. For example, schema `Sales` and table
|
|
`Order` produce `"Sales"."Order"`; with `--lowercase`, they produce
|
|
`"sales"."order"`. Quoted names are case-sensitive in PostgreSQL.
|
|
|
|
`--lowercase` is available on `convert`, `batch`, `merge`, `split`, `templ`, and
|
|
`job run`. It lowercases model names and structured references on a copy and
|
|
rejects collisions such as `Users` and `users` in the same schema. SQL expressions,
|
|
defaults, comments, enum values, and data retain their case. Adjust identifier
|
|
references inside your own SQL expressions when opting into lowercasing.
|
|
|
|
Generated PostgreSQL SQL also escapes identifier quotes and literal values and
|
|
chooses dollar-quote delimiters that do not collide with function or procedural
|
|
block contents. Raw SQL expressions remain executable input. See the
|
|
[PostgreSQL writer documentation](pkg/writers/pgsql/README.md#identifier-and-value-handling)
|
|
for library options and details.
|
|
|
|
#### MySQL fidelity and strict conversion
|
|
|
|
MySQL output now preserves foreign keys and their actions, checks, indexes,
|
|
unsigned types, dimensions, generated columns, and native defaults. Known
|
|
unsupported features produce warnings. Add `--strict-conversion` to reject
|
|
lossy MySQL output before writing a file or executing DDL:
|
|
|
|
```bash
|
|
relspec convert --from mysql --from-conn "$MYSQL_DSN" \
|
|
--to mysql --to-path schema.sql --strict-conversion
|
|
```
|
|
|
|
Schema jobs can set `options.strict_conversion: true`. This check currently
|
|
covers MySQL output; it does not guarantee lossless conversion to every format.
|
|
The live suite covers MySQL 8.4 / InnoDB, including JSON/YAML intermediates.
|
|
See the [MySQL capability matrix](docs/MYSQL_FIDELITY.md) for unsupported objects,
|
|
metadata, SQL-mode requirements, and execution limits. Output creates tables in
|
|
existing databases; it is not a data copy or a rerunnable migration.
|
|
|
|
### `batch` — Convert many inputs in one run
|
|
|
|
Converts each input independently (one output per input, unlike `--from-list`
|
|
which merges). `--input` takes paths or globs; outputs go to `--to-dir`.
|
|
Use `--keep-going` to continue past failures (exit is still non-zero) and
|
|
`--dry-run` to validate without writing. For named workflows see `relspec job run`.
|
|
|
|
```bash
|
|
relspec batch --from dbml --input "schemas/*.dbml" --to json --to-dir out/
|
|
```
|
|
|
|
PostgreSQL connections opened by relspec set `application_name` by default to
|
|
`relspecgo/<version>` (with component suffixes internally, e.g. readers/writers).
|
|
If you need a custom value, provide `application_name` explicitly in the connection
|
|
string query parameters.
|
|
|
|
### `merge` — Additive schema merge (never modifies existing items)
|
|
|
|
```bash
|
|
# Merge two JSON schemas
|
|
relspec merge --target json --target-path base.json \
|
|
--source json --source-path additions.json \
|
|
--output json --output-path merged.json
|
|
|
|
# Merge PostgreSQL into JSON, skipping tables
|
|
relspec merge --target json --target-path current.json \
|
|
--source pgsql --source-conn "postgres://user:pass@localhost/db" \
|
|
--output json --output-path updated.json \
|
|
--skip-tables "audit_log,temp_tables"
|
|
```
|
|
|
|
Skip flags: `--skip-relations` `--skip-views` `--skip-domains` `--skip-enums` `--skip-sequences`
|
|
|
|
### `inspect` — Schema validation / linting
|
|
|
|
```bash
|
|
# Validate PostgreSQL database
|
|
relspec inspect --from pgsql --from-conn "postgres://user:pass@localhost/mydb"
|
|
|
|
# Validate DBML with custom rules
|
|
relspec inspect --from dbml --from-path schema.dbml --rules .relspec-rules.yaml
|
|
|
|
# JSON report output
|
|
relspec inspect --from json --from-path db.json --output-format json --output report.json
|
|
|
|
# Filter to specific schema
|
|
relspec inspect --from pgsql --from-conn "..." --schema public
|
|
```
|
|
|
|
Rules: naming conventions, PK/FK standards, missing indexes, reserved keywords, circular dependencies.
|
|
|
|
### `diff` — Schema comparison
|
|
|
|
```bash
|
|
relspec diff --from pgsql --from-conn "postgres://localhost/db1" \
|
|
--to pgsql --to-conn "postgres://localhost/db2"
|
|
```
|
|
|
|
### `templ` — Custom template rendering
|
|
|
|
```bash
|
|
# Render database schema to Markdown docs
|
|
relspec templ --from pgsql --from-conn "postgres://user:pass@localhost/db" \
|
|
--template docs.tmpl --output schema-docs.md
|
|
|
|
# One TypeScript file per table
|
|
relspec templ --from dbml --from-path schema.dbml \
|
|
--template ts-model.tmpl --mode table \
|
|
--output ./models/ --filename-pattern "{{.Name | toCamelCase}}.ts"
|
|
```
|
|
|
|
Modes: `database` (default) · `schema` · `table` · `script`
|
|
|
|
Template functions: string utils (`toCamelCase`, `toSnakeCase`, `pluralize`, …), type converters (`sqlToGo`, `sqlToTypeScript`, …), filters, loop helpers, safe access.
|
|
|
|
### `job` — Declarative job files
|
|
|
|
Run named jobs from a `relspec.yml` manifest instead of repeating long command lines.
|
|
|
|
```bash
|
|
# List jobs discovered in ./relspec.yml and ./relspec.<name>.yml (deterministic)
|
|
relspec job list
|
|
|
|
# Validate and print the plan without running anything
|
|
relspec job run build-schema --plan
|
|
|
|
# Run a job (and its declared dependencies)
|
|
relspec job run build-schema
|
|
```
|
|
|
|
```yaml
|
|
# relspec.yml
|
|
version: 1
|
|
jobs:
|
|
build-schema:
|
|
command: convert # closed allow-list: convert | merge | split | scripts-list | scripts-exec | templ | inspect | diff
|
|
description: Merge the DBML sources and emit PostgreSQL DDL
|
|
inputs:
|
|
- path: schema/core.dbml
|
|
format: dbml
|
|
- path: schema/tenant.dbml
|
|
format: dbml
|
|
output:
|
|
format: pgsql
|
|
path: build/schema.sql
|
|
overwrite: true
|
|
options:
|
|
flatten_schema: false
|
|
lowercase_identifiers: true # opt in; defaults to false
|
|
logfile: .relspec/log/build-schema.log
|
|
```
|
|
|
|
The job system is **not** a shell: `command` is a fixed enum, every path is
|
|
resolved relative to the job file and may not escape it, and remote database
|
|
credentials are referenced by environment-variable name (`conn_env:`) and
|
|
redacted from logs. The whole plan — unknown commands/formats, duplicate job
|
|
names, missing inputs, path traversal, dependency cycles — is validated before
|
|
any job runs. Path-like fields also support `${NAME}` environment-variable
|
|
references, which are expanded and safety-checked during pre-flight. See
|
|
[docs/JOB_FILES.md](docs/JOB_FILES.md).
|
|
|
|
Use `options.lowercase_identifiers: true` per job, or
|
|
`relspec job run build-schema --lowercase` to enable lowercasing for schema outputs
|
|
throughout the selected job plan. For MySQL schema output,
|
|
`options.strict_conversion: true` or `--strict-conversion` rejects known lossy
|
|
conversions before writing.
|
|
|
|
### `scripts` — SQL scripts and embedded assets
|
|
|
|
```bash
|
|
relspec scripts list --dir ./migrations
|
|
relspec scripts execute --dir ./migrations --conn "$DATABASE_URL"
|
|
```
|
|
|
|
Execution stops on the first error by default. `--continue-on-error` (alias
|
|
`--coe`) attempts later scripts and still exits nonzero if any script fails.
|
|
|
|
Scripts can embed nearby files using a standalone directive comment:
|
|
|
|
```sql
|
|
-- @embed: path=assets/message.txt var=:message mode=text
|
|
INSERT INTO messages (body) VALUES (:message);
|
|
```
|
|
|
|
Embed replacement preserves dollar text such as `$1`, `${2}`, and `$HOME` exactly.
|
|
Placeholders are replaced only in executable SQL; quoted strings, quoted
|
|
identifiers, comments, casts, and dollar-quoted bodies retain their contents.
|
|
Inserted content is never scanned for further placeholders. Asset-call named
|
|
parameters follow the same SQL lexical boundaries. See
|
|
[docs/SCRIPTS_COMMAND.md](docs/SCRIPTS_COMMAND.md) for file containment, limits,
|
|
binary embedding, and execution details.
|
|
|
|
MySQL DDL execution keeps each generated statement whole, preserving multiline
|
|
defaults containing semicolons or comment-like text.
|
|
|
|
### `edit` — Interactive TUI editor
|
|
|
|
```bash
|
|
# Edit DBML schema interactively
|
|
relspec edit --from dbml --from-path schema.dbml --to dbml --to-path schema.dbml
|
|
|
|
# Edit live PostgreSQL database
|
|
relspec edit --from pgsql --from-conn "postgres://user:pass@localhost/mydb" \
|
|
--to pgsql --to-conn "postgres://user:pass@localhost/mydb"
|
|
```
|
|
|
|
<p align="center">
|
|
<img src="./assets/image/screenshots/main_screen.jpg">
|
|
</p>
|
|
<p align="center">
|
|
<img src="./assets/image/screenshots/table_view.jpg">
|
|
</p>
|
|
<p align="center">
|
|
<img src="./assets/image/screenshots/edit_column.jpg">
|
|
</p>
|
|
|
|
### `update` — Check for a newer release
|
|
|
|
```bash
|
|
relspec update # check, prompt, then download + run installer (Windows)
|
|
relspec update --check # report only
|
|
relspec update --yes # skip the prompt
|
|
```
|
|
|
|
Non-Windows platforms print the release URL. `dev` and commit-hash builds are never reported as outdated.
|
|
|
|
## Development
|
|
|
|
**Prerequisites:** Go 1.24.0+
|
|
|
|
```bash
|
|
make build # → build/relspec
|
|
make test # race detection + coverage
|
|
make lint # requires golangci-lint
|
|
make coverage # → coverage.html
|
|
make install # → $GOPATH/bin
|
|
make installer-windows # → build/relspec-setup-windows-amd64.exe (needs makensis)
|
|
```
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
cmd/relspec/ CLI commands
|
|
pkg/readers/ Input format readers
|
|
pkg/writers/ Output format writers
|
|
pkg/inspector/ Schema validation
|
|
pkg/diff/ Schema comparison
|
|
pkg/merge/ Schema merging
|
|
pkg/models/ Internal data models
|
|
pkg/transform/ Transformation logic
|
|
pkg/pgsql/ PostgreSQL utilities
|
|
pkg/updatecheck/ Latest-release lookup and version comparison
|
|
windows/ NSIS installer script
|
|
pkg/sqltypes/ Nullable SQL types for generated/hand-written models (see below)
|
|
```
|
|
|
|
## Nullable Types (`pkg/sqltypes`)
|
|
|
|
The `bun` and `gorm` writers can generate model structs using
|
|
[`pkg/sqltypes`](./pkg/sqltypes/README.md) — nullable types (`SqlString`,
|
|
`SqlInt32`, `SqlTimeStamp`, `SqlStringArray`, …) that implement
|
|
`database/sql.Scanner`, `driver.Valuer`, and JSON/YAML/XML marshalling in one
|
|
type, selected via `--types sqltypes`. See the
|
|
[`pkg/sqltypes` README](./pkg/sqltypes/README.md) for the full type
|
|
reference, or the [`bun`](./pkg/writers/bun/README.md) /
|
|
[`gorm`](./pkg/writers/gorm/README.md) writer docs for the `--types` flag
|
|
(`sqltypes`, `stdlib`, or `baselib`). PostgreSQL array columns are the one
|
|
exception: the `bun` writer always generates native Go slices (`[]string`,
|
|
`[]int32`, …) with an explicit `array` bun tag, regardless of `--types` —
|
|
see [`bun`'s `--array-nullable`](./pkg/writers/bun/README.md#nullablearrays)
|
|
flag for nullable-array handling. The `SqlXxxArray` wrapper types remain
|
|
available in `pkg/sqltypes` and are still used by the `gorm` writer.
|
|
|
|
#### Custom type mapping
|
|
|
|
Override the built-in SQL → Go mapping of the `bun` and `gorm` writers with the
|
|
repeatable `--type-map sqltype=gotype` flag:
|
|
|
|
```bash
|
|
relspec convert --from pgsql --from-conn "$DSN" --to gorm --to-path models.go \
|
|
--type-map uuid=string --type-map jsonb=json.RawMessage
|
|
```
|
|
|
|
SQL type names are matched case-insensitively on the base type (modifiers such
|
|
as `(10,2)` are ignored; aliases like `int4` resolve to `integer`). NOT NULL
|
|
columns use the Go type verbatim, nullable columns get a `*` prefix (unless the
|
|
type is already a pointer, slice, map or `any`), and arrays become `[]gotype`.
|
|
Unmapped types keep their defaults. The flag does not add imports: use types
|
|
that need none, or add the import afterwards (e.g. with `goimports`).
|
|
|
|
## Contributing
|
|
|
|
1. Register or sign in with GitHub at [git.warky.dev](https://git.warky.dev)
|
|
2. Clone the repository: `git clone https://git.warky.dev/wdevs/relspecgo.git`
|
|
3. Create a feature branch: `git checkout -b feature/your-feature-name`
|
|
4. Commit your changes and push the branch
|
|
5. Open a pull request with a description of the new feature or fix
|
|
|
|
For questions or discussion, join the Discord: [discord.gg/74rcTujp25](https://discord.gg/74rcTujp25) — `warkyhein`
|
|
|
|
## Links
|
|
|
|
- [Todo](./TODO.md)
|
|
- [AI Use Policy](./AI_USE.md)
|
|
- [License](LICENSE) — Apache 2.0 · Copyright 2025 Warky Devs
|