233 lines
8.6 KiB
Markdown
233 lines
8.6 KiB
Markdown
# RelSpec Job Files
|
|
|
|
Job files let you declare named, repeatable RelSpec workflows in YAML and run
|
|
them with `relspec job run <name>` instead of retyping long command lines.
|
|
|
|
```bash
|
|
relspec job list # deterministic list of discovered jobs
|
|
relspec job run build-schema --plan # validate + print plan, execute nothing
|
|
relspec job run build-schema # run the job (and its dependencies)
|
|
```
|
|
|
|
## Design contract (first release)
|
|
|
|
This is the smallest coherent contract that is safe and useful end to end.
|
|
Anything not listed under "Supported" is intentionally deferred.
|
|
|
|
### Not a shell
|
|
|
|
`command` is a **closed allow-list**. There is no field anywhere that accepts a
|
|
shell string, an executable path, or arbitrary arguments. Adding a new command
|
|
means adding a vetted adapter in the RelSpec source.
|
|
|
|
| command | what it does |
|
|
|----------------|--------------------------------------------------------------------|
|
|
| `convert` | read one or more input schemas, additively merge them, write one output |
|
|
| `merge` | like `convert` but requires ≥2 inputs and exposes `skip_*` merge options |
|
|
| `scripts-list` | deterministically list SQL scripts across one or more directories |
|
|
| `templ` | apply a custom Go text template to one or more input schemas |
|
|
|
|
Deferred (documented, not implemented here): `scripts` execution against a live
|
|
database, `split`, `inspect`, `diff`, job-to-job output wiring,
|
|
log rotation/retention. Live SQL execution already exists as
|
|
`relspec scripts execute`; wiring it into the job runner is a follow-up because
|
|
it needs live database credentials and cannot be covered by offline tests.
|
|
|
|
### Discovery and precedence
|
|
|
|
`relspec job` (no `--file`) scans `--dir` (default `.`) for:
|
|
|
|
1. `relspec.yml` / `relspec.yaml` (the default file), then
|
|
2. `relspec.<name>.yml` / `relspec.<name>.yaml` (extra files),
|
|
|
|
each group sorted lexically. Order is stable across runs. Use `--file <path>`
|
|
(repeatable) to load explicit files and skip discovery.
|
|
|
|
All discovered/selected files are merged into one job namespace. A job name
|
|
defined by **more than one file is a hard error** naming both files. YAML maps
|
|
already forbid duplicate keys within a single file.
|
|
|
|
### Paths
|
|
|
|
* Every path (`inputs[].path`, `output.path`, `script_dirs[]`, `logfile`) is
|
|
**relative to the directory containing the job file that declared the job**,
|
|
not the process working directory.
|
|
* Absolute paths, `~`-relative paths and any path that resolves outside the job
|
|
file directory (`../`, `a/../../b`, …) are **rejected during validation** —
|
|
before anything runs.
|
|
|
|
### Credentials
|
|
|
|
* Database inputs (`format: pgsql` / `mssql`) and database execution outputs
|
|
(`format: pgsql` with `conn_env`) reference an **environment variable name**
|
|
via `conn_env:`. The connection string itself is never stored in the
|
|
manifest.
|
|
* A `conn_env` value that looks like a connection string (contains `:`, `/`,
|
|
`@`, `=`, spaces) is rejected.
|
|
* Missing/empty environment variables are reported during pre-flight, before
|
|
execution.
|
|
* Job logs and `--plan` output show `env:<NAME>`, never the value. Resolved
|
|
secret values and anything matching a connection-string password are
|
|
redacted (`***`) from the logfile and diagnostics.
|
|
|
|
### Validation happens before execution
|
|
|
|
`relspec job list` and `relspec job run` both fully validate the selected set
|
|
first. Nothing is read, written, connected to, or executed if validation fails.
|
|
Checks include:
|
|
|
|
* schema `version` (must be `1`), unknown YAML fields rejected
|
|
* duplicate job names across files
|
|
* unknown / missing `command`
|
|
* per-command input/output shape (`convert`/`merge` need inputs + output;
|
|
`scripts-list` needs `script_dirs` and forbids inputs/output)
|
|
* unknown input/output `format`
|
|
* path traversal / absolute / home-relative paths
|
|
* `depends_on` targets exist
|
|
* dependency cycles (reported as `a -> b -> c -> a`)
|
|
|
|
Then, immediately before running, per-job pre-flight resolves paths and checks:
|
|
|
|
* every input file exists and is a file
|
|
* every `script_dir` exists and is a directory
|
|
* every `conn_env` variable is set
|
|
* `output.path` does not already exist unless `output.overwrite: true`
|
|
|
|
If any pre-flight check fails for **any** job in the plan, **no** job runs.
|
|
|
|
### Execution and exit codes
|
|
|
|
* `relspec job run <name>` runs the job's `depends_on` closure first, in
|
|
topological order (deterministic), then the job. `--no-deps` runs only the
|
|
named job.
|
|
* `--dry-run` (alias `--plan`) prints the resolved plan and exits 0 without
|
|
touching inputs, outputs or databases.
|
|
* A failing job returns the underlying non-zero status (the process exits 1)
|
|
and the error names the job. The logfile records `FAILED: <error>`; a
|
|
successful job records `OK`. No separate success-marker file is written, so a
|
|
failure can never leave a stale "success".
|
|
|
|
## Schema reference
|
|
|
|
```yaml
|
|
version: 1 # required, must be 1
|
|
jobs:
|
|
<job-name>:
|
|
command: convert | merge | scripts-list # required
|
|
description: "free text" # optional, shown by `job list`
|
|
depends_on: [other-job, ...] # optional
|
|
inputs: # convert (≥1) / merge (≥2)
|
|
- path: relative/file.dbml # file inputs
|
|
format: dbml
|
|
- format: pgsql # live-connection inputs
|
|
conn_env: SOURCE_DB_URL # env var NAME
|
|
script_dirs: # scripts-list (≥1)
|
|
- migrations/core
|
|
- migrations/tenant
|
|
template: templates/schema.tmpl # templ (required)
|
|
mode: table # templ: database/schema/script/table
|
|
filename_pattern: "{{.Name}}.go" # templ multi-output modes
|
|
output: # convert / merge (required)
|
|
format: pgsql
|
|
path: build/schema.sql # file output, OR:
|
|
conn_env: TARGET_DB_URL # execute against DB (pgsql only)
|
|
overwrite: false # default false
|
|
options:
|
|
flatten_schema: false
|
|
schema: public
|
|
package: models # for gorm/bun output
|
|
continue_on_error: false # pgsql output
|
|
skip_relations: false # merge only
|
|
skip_enums: false
|
|
skip_views: false
|
|
skip_domains: false
|
|
skip_sequences: false
|
|
logfile: .relspec/log/<job-name>.log # optional; appended to
|
|
```
|
|
|
|
For `templ`, `inputs` use the same file or `pgsql`/`conn_env` source forms as
|
|
schema conversion. `output` is optional (empty means stdout); when present it
|
|
contains only `path` and `overwrite`, because templates do not select a schema
|
|
writer format.
|
|
|
|
### Supported input formats
|
|
|
|
`dbml`, `dctx`, `drawdb`, `graphql`, `json`, `yaml`, `gorm`, `bun`, `drizzle`,
|
|
`prisma`, `typeorm`, `sqlite` (file, via `path`); `pgsql`, `mssql`
|
|
(live, via `conn_env`).
|
|
|
|
### Supported output formats
|
|
|
|
`dbml`, `dctx`, `drawdb`, `graphql`, `json`, `yaml`, `gorm`, `bun`, `drizzle`,
|
|
`prisma`, `typeorm`, `pgsql`, `mssql`, `sqlite` (file, via `path`); `pgsql` also
|
|
supports `conn_env` to execute the generated DDL against a live database.
|
|
|
|
## Examples
|
|
|
|
### Merge many schema files, emit PostgreSQL DDL
|
|
|
|
```yaml
|
|
version: 1
|
|
jobs:
|
|
build-schema:
|
|
command: convert
|
|
inputs:
|
|
- { path: schema/core.dbml, format: dbml }
|
|
- { path: schema/billing.dbml, format: dbml }
|
|
- { path: schema/tenant.dbml, format: dbml }
|
|
output:
|
|
format: pgsql
|
|
path: build/schema.sql
|
|
overwrite: true
|
|
logfile: .relspec/log/build-schema.log
|
|
```
|
|
|
|
### Multiple script directories
|
|
|
|
```yaml
|
|
version: 1
|
|
jobs:
|
|
migration-order:
|
|
command: scripts-list
|
|
script_dirs:
|
|
- migrations/core
|
|
- migrations/tenant
|
|
- migrations/reporting
|
|
logfile: .relspec/log/migration-order.log
|
|
```
|
|
|
|
### Job depending on another job
|
|
|
|
```yaml
|
|
version: 1
|
|
jobs:
|
|
build-schema:
|
|
command: convert
|
|
inputs:
|
|
- { path: schema/core.dbml, format: dbml }
|
|
- { path: schema/tenant.dbml, format: dbml }
|
|
output: { format: json, path: build/schema.json, overwrite: true }
|
|
build-docs:
|
|
command: convert
|
|
depends_on: [build-schema]
|
|
inputs:
|
|
- { path: schema/core.dbml, format: dbml }
|
|
output: { format: yaml, path: build/schema.yaml, overwrite: true }
|
|
```
|
|
|
|
### Reading from a remote database
|
|
|
|
```yaml
|
|
version: 1
|
|
jobs:
|
|
snapshot-prod:
|
|
command: convert
|
|
inputs:
|
|
- format: pgsql
|
|
conn_env: PROD_DB_URL # export PROD_DB_URL=postgres://...
|
|
output:
|
|
format: dbml
|
|
path: snapshots/prod.dbml
|
|
overwrite: true
|
|
```
|