Compare commits

...
25 Commits
Author SHA1 Message Date
warkanum 82e86cdf39 fix(release): run lint and format check before version release
Release / test (push) Successful in 2m7s
Release / release (push) Successful in 1m49s
Release / pkg-aur (push) Successful in 50s
Release / pkg-deb (push) Successful in 1m30s
Release / pkg-rpm (push) Successful in 1m48s
2026-10-02 23:38:28 +02:00
warkanum b7ae00ab52 fix(test): simplify parameter list in identity column builder 2026-10-02 23:35:56 +02:00
warkanum 8f8664357e chore(release): update package version to 1.0.85
Release / test (push) Failing after 2m1s
Release / release (push) Skipped
Release / pkg-aur (push) Skipped
Release / pkg-deb (push) Skipped
Release / pkg-rpm (push) Skipped
2026-10-02 23:32:07 +02:00
warkanum 7bd61a21b5 fix: resolve gofumpt and staticcheck lint errors 2026-10-02 23:31:41 +02:00
warkanum 6f6b9834ca docs(CONTRIBUTING): update setup instructions and add make targets 2026-10-02 23:29:25 +02:00
warkanum 9cc10715e3 feat: generated and identity columns for mssql, sqlite, drizzle, typeorm, bun and gorm readers
- mssql: write computed columns (AS (expr) PERSISTED) and identity; read computed definition and identity
- sqlite: write GENERATED ALWAYS AS (expr) STORED; read generated columns via table_xinfo and parse the expression
- drizzle: generatedAlwaysAs / generated(Always|ByDefault)AsIdentity, read and write
- typeorm: asExpression/generatedType and identity decorators, read and write; decorator scan is now quote-aware
- bun, gorm readers: read generated and identity markers
- readmes updated
2026-10-02 23:04:04 +02:00
warkanum bbab5ce936 feat: mark generated and identity columns across writers and DBML
- bun: scanonly+generated for GENERATED columns; scanonly+identity for ALWAYS identity (PK gets identity only)
- gorm: <-:false+generated / identity, same primary key rule
- dbml: write GENERATED/IDENTITY column notes and read them back into the model
- pgsql: emit generation/identity clauses in migration create-table and add-column, no DEFAULT
- readmes updated
2026-10-02 22:58:00 +02:00
warkanum 275424c605 feat(pgsql): diff against the live database by default for direct output
Direct pgsql output (job output.conn_env, merge --output-conn) now reads the live
schema and executes only the differences. full_ddl: true (job option) or --full-ddl
(merge) restores the full idempotent DDL; file output is unchanged.

- compare PKs by columns, skip constraint-backed indexes, normalize index method,
  FK actions, serial/numeric types, default literals/casts and truncated names
- diff table and column comments instead of re-emitting them
- remove leftover ZZDUMP debug code
2026-10-02 22:52:59 +02:00
warkanum b6c0cd3d1b fix(pgsql-reader): pair composite foreign key columns by position
key_column_usage and constraint_column_usage were joined on constraint name only,
so an N-column foreign key returned N*N column pairs. Read from pg_constraint with
unnest(conkey, confkey) instead.
2026-10-02 22:52:53 +02:00
warkanum d7d1d99ebc fix(pgsql): tie PK sequence to nextval default, setval past data, keep serial defaults
- derive the primary key sequence from the column's nextval() default instead of an unused identity_<table>_<pk> sequence
- setval after table creation (full and diff paths), forward-only, MAX+1 with is_called=false
- do not DROP DEFAULT on serial/bigserial columns without a model default
2026-10-02 22:37:37 +02:00
warkanum 43849324ce chore(release): update package version to 1.0.84
Release / test (push) Successful in 1m25s
Release / release (push) Successful in 2m7s
Release / pkg-aur (push) Successful in 55s
Release / pkg-deb (push) Successful in 1m42s
Release / pkg-rpm (push) Successful in 1m49s
2026-09-24 15:41:46 +02:00
warkanum e1df3b7cee fix(writers): drop version stamps from generated output
Generated files carried the RelSpec version and build date, so every
rebuild produced a diff even when the schema was unchanged.

- bun, gorm, drizzle: remove the GeneratedBy template field and line
- pgsql, mssql, sqlite: emit a constant "-- Generated by RelSpec"
- bun: replace the version header test with a version-free assertion

pkg/buildinfo is unchanged and still backs the CLI version.
2026-09-24 15:40:37 +02:00
warkanum f7b5d5f054 chore(release): update package version to 1.0.83
Release / pkg-deb (push) Successful in 1m47s
Release / test (push) Successful in 59s
Release / release (push) Successful in 2m0s
Release / pkg-aur (push) Successful in 54s
Release / pkg-rpm (push) Successful in 1m56s
2026-09-23 18:52:48 +02:00
warkanum 99fc4b0944 Merge pull request 'Fix/dbml commented refs duplicate indexes' (#31) from fix/dbml-commented-refs-duplicate-indexes into master
Reviewed-on: #31
2026-09-23 16:48:54 +00:00
warkanum 278d488363 chore(lint): use strings.EqualFold in jobs output format check 2026-09-23 18:48:07 +02:00
warkanum 2f69205aa0 fix(dbml): resolve commented cross-file // Ref: lines
Commented refs are collected per file and resolved against the combined
model after all inputs are loaded (directory, --from-list, merge, jobs).
Matched refs become FKs and relationships; duplicates of existing FKs are
skipped; missing targets are skipped with a warning; column type
mismatches warn.

Also keep reused index names within a DBML table instead of overwriting,
give a second FK to the same table a distinct relationship name, and make
the pgsql writer match relationships to FKs by name first.
2026-09-23 18:48:07 +02:00
warkanum b91985c493 feat(inspector): fail on duplicate index names per schema
PostgreSQL index names share the schema-wide relation namespace, so a
reused name makes CREATE INDEX IF NOT EXISTS silently skip the duplicate.
Add duplicate_index_name rule (enforce by default) covering indexes and
PK/unique constraint backing indexes.
2026-09-23 18:48:07 +02:00
warkanum 80a3453233 Merge pull request 'fix(jobs): preserve writer and template output options' (#30) from issue-29-job-manifest-options into master
Reviewed-on: #30
Reviewed-by: Warky <2+warkanum@noreply@warky.dev>
2026-09-21 04:00:50 +00:00
SG Command ade71b75fa fix(jobs): preserve writer and template output options 2026-09-20 23:16:41 +02:00
warkanum 799b7feb53 ci(rpm): update RPM builder to avoid transient mirrors 2026-09-20 21:41:52 +02:00
warkanum eb40201b59 chore(release): update package version to 1.0.82
Release / release (push) Successful in 1m50s
Release / pkg-aur (push) Successful in 1m0s
Release / pkg-deb (push) Successful in 1m17s
Release / test (push) Successful in 53s
Release / pkg-rpm (push) Failing after 16s
2026-09-20 20:37:41 +02:00
warkanum 1461f8f69f chore(release): update version to 1.0.81 and metadata 2026-09-20 20:37:09 +02:00
warkanum ea70e19a46 feat(jobs): expand environment variables in paths 2026-09-20 20:08:20 +02:00
warkanum 52f4642d97 feat(models): add support for generated and identity columns
Release / test (push) Successful in 3m13s
Release / release (push) Successful in 1m52s
Release / pkg-aur (push) Successful in 1m39s
Release / pkg-deb (push) Successful in 2m15s
Release / pkg-rpm (push) Failing after 46s
* extend Column struct with Generated, GenerationExpression, Identity, and IdentityGeneration fields
* update queryColumns to handle new column properties
* modify migration writer to avoid altering generated and identity columns
* add tests to ensure correct handling of generated and identity columns
2026-09-20 18:33:12 +02:00
warkanum a84592374c Merge pull request 'Feat generated headers column order' (#28) from feat-generated-headers-column-order into master
Release / pkg-rpm (push) Successful in 3m58s
Release / release (push) Successful in 1m54s
Release / test (push) Successful in 5m49s
Release / pkg-aur (push) Successful in 7m13s
Release / pkg-deb (push) Successful in 5m8s
Reviewed-on: #28
2026-09-10 20:22:50 +00:00
99 changed files with 4055 additions and 381 deletions
+12 -3
View File
@@ -63,6 +63,8 @@ jobs:
- name: Build release binaries
run: |
VERSION="${{ github.event.inputs.tag || github.ref_name }}"
BUILD_DATE="$(date -u +"%Y-%m-%d %H:%M:%S UTC")"
LDFLAGS="-X 'git.warky.dev/wdevs/relspecgo/pkg/buildinfo.Version=${VERSION}' -X 'git.warky.dev/wdevs/relspecgo/pkg/buildinfo.BuildDate=${BUILD_DATE}'"
for target in "linux/amd64" "linux/arm64" "darwin/amd64" "darwin/arm64" "windows/amd64"; do
GOOS="${target%/*}"
GOARCH="${target#*/}"
@@ -71,7 +73,7 @@ jobs:
NAME="relspec-${GOOS}-${GOARCH}${EXT}"
GOOS="$GOOS" GOARCH="$GOARCH" go build \
-trimpath \
-ldflags "-X git.warky.dev/wdevs/relspecgo/cmd/relspec.version=${VERSION}" \
-ldflags "$LDFLAGS" \
-o "$NAME" ./cmd/relspec
echo "Built $NAME"
done
@@ -234,11 +236,12 @@ jobs:
run: |
VERSION="${{ github.event.inputs.tag || github.ref_name }}"
PKGVER="${VERSION#v}"
BUILD_DATE="$(date -u +"%Y-%m-%d %H:%M:%S UTC")"
for GOARCH in amd64 arm64; do
GOOS=linux GOARCH=$GOARCH go build \
-trimpath \
-ldflags "-X git.warky.dev/wdevs/relspecgo/cmd/relspec.version=${PKGVER}" \
-ldflags "-X 'git.warky.dev/wdevs/relspecgo/pkg/buildinfo.Version=v${PKGVER}' -X 'git.warky.dev/wdevs/relspecgo/pkg/buildinfo.BuildDate=${BUILD_DATE}'" \
-o relspec ./cmd/relspec
PKGDIR="relspec_${PKGVER}_${GOARCH}"
@@ -309,7 +312,13 @@ jobs:
rockylinux:9 \
bash -lc "
set -euo pipefail
dnf install -y rpm-build git &&
# Avoid transient/out-of-sync mirrors while bootstrapping the RPM builder.
sed -i \
-e 's|^mirrorlist=|#mirrorlist=|' \
-e 's|^#baseurl=http://dl.rockylinux.org|baseurl=https://dl.rockylinux.org|' \
/etc/yum.repos.d/rocky*.repo
dnf clean all
dnf -y --setopt=timeout=60 --setopt=retries=5 install rpm-build git
curl -fsSL https://go.dev/dl/go\${GO_VER}.linux-amd64.tar.gz | tar -C /usr/local -xz &&
export PATH=\$PATH:/usr/local/go/bin &&
mkdir -p ~/rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS} &&
+59 -113
View File
@@ -1,145 +1,91 @@
# Contributing to RelSpec
Thank you for your interest in contributing to RelSpec.
## Setup
## Development Setup
- Go 1.25+ (see `go.mod`), Git
- Optional: golangci-lint, Docker/Podman (PostgreSQL integration tests)
### Prerequisites
- Go 1.21 or higher
- Git
- (Optional) golangci-lint for linting
- (Optional) Docker for database testing
### Getting Started
1. Clone the repository:
```bash
git clone https://github.com/wdevs/relspecgo.git
git clone git@git.warky.dev:wdevs/relspecgo.git
cd relspecgo
make deps
make build # outputs build/relspec
```
2. Install dependencies:
```bash
go mod download
```
## Make Targets
3. Run tests:
```bash
go test ./...
```
| Target | Purpose |
|-------------------------|------------------------------------------------------|
| `make test` | Unit tests (race detection, coverage) |
| `make test-integration` | Integration tests (needs `RELSPEC_TEST_PG_CONN`) |
| `make docker-test` | PostgreSQL integration tests via Docker/Podman |
| `make lint` | golangci-lint |
| `make fmt` / `fmt-check`| gofumpt + goimports |
| `make check` | vet, fmt-check, staticcheck, govulncheck |
| `make coverage` | Coverage report |
4. Build the project:
```bash
go build -o relspec ./cmd/relspec
```
Single test: `go test -run TestName ./pkg/readers/dbml`
## Project Structure
## Layout
```
relspecgo/
├── cmd/ # CLI application entry point
├── pkg/
│ ├── readers/ # Input format readers (XML, JSON, DCTX, DB, GORM, Bun)
│ ├── writers/ # Output format writers (GORM, Bun, JSON, YAML)
│ ├── models/ # Internal data models for relations
│ └── transform/ # Transformation and validation logic
├── examples/ # Usage examples and sample files
├── tests/ # Integration tests
└── .claude/ # Claude Code configuration and commands
cmd/relspec/ CLI commands (convert, diff, merge, split, edit, inspect, job, ...)
pkg/models/ Core model: Database > Schema > Table > Column/Constraint/Index/Relationship
pkg/readers/<fmt> One reader per format
pkg/writers/<fmt> One writer per format
pkg/diff, merge, inspector, jobs, transform, ui, pgsql, sqltypes ...
examples/ Sample files
tests/ Integration tests and assets
docs/ Feature docs
```
## Adding New Readers
## Adding a Reader
To add a new input format reader:
1. Create `pkg/readers/<format>/reader.go` with `NewReader(options *readers.ReaderOptions)`.
2. Implement `readers.Reader`: `ReadDatabase`, `ReadSchema`, `ReadTable`.
3. Add `reader_test.go` in the same package.
4. Register the format in the CLI switches (`cmd/relspec/convert.go`, `diff.go`, `edit.go`, ...).
5. Add a `README.md` in the reader directory.
1. Create a new file in `pkg/readers/` (e.g., `myformat_reader.go`)
2. Implement the `Reader` interface:
```go
type Reader interface {
Read(source string) (*models.Schema, error)
}
```
3. Add tests in `pkg/readers/myformat_reader_test.go`
4. Register the reader in the CLI
## Adding a Writer
## Adding New Writers
1. Create `pkg/writers/<format>/writer.go` with `NewWriter(options *writers.WriterOptions)`.
2. Implement `writers.Writer`: `WriteDatabase`, `WriteSchema`, `WriteTable`.
3. Add `writer_test.go` in the same package.
4. Register the format in the CLI switches.
5. Add a `README.md` in the writer directory.
To add a new output format writer:
## Code Rules
1. Create a new file in `pkg/writers/` (e.g., `myformat_writer.go`)
2. Implement the `Writer` interface:
```go
type Writer interface {
Write(schema *models.Schema, destination string) error
}
```
3. Add tests in `pkg/writers/myformat_writer_test.go`
4. Register the writer in the CLI
## Code Style
- Follow standard Go conventions
- Use `gofmt` for formatting
- Run `go vet` to check for issues
- Use meaningful variable and function names
- Add comments for exported functions and types
- Format with gofumpt/goimports (`make fmt`); `make check` must pass.
- Iterate `Table.Columns`, `Constraints`, `Indexes`, `Relationships` in sorted order (maps are unordered; output must be deterministic).
- Every writer stamps `buildinfo.GeneratedComment()` in its file header.
- Comment exported functions and types.
## Testing
- Write unit tests for all new functionality
- Aim for >80% code coverage
- Use table-driven tests where appropriate
- Include both positive and negative test cases
- Tests live in the same package as the code.
- Table-driven tests; cover positive and negative cases.
- Reuse existing test data in `tests/` and `examples/` before adding new data.
- Tests run with `-race`.
### Running Tests
## Commits
```bash
# All tests
go test ./...
- Format: `type(scope): description`
- Types: `feat`, `fix`, `docs`, `test`, `refactor`, `chore`, `ci`
- Keep commits focused. Reference issues where applicable.
# With coverage
go test -cover ./...
## Pull Requests
# Verbose output
go test -v ./...
1. Branch from `master`.
2. Add tests; `make test` and `make check` pass.
3. Update docs/README if behaviour changes.
4. Open a PR with a clear description.
# Specific package
go test ./pkg/readers/...
```
## Security
## Committing Changes
- Write clear, descriptive commit messages
- Follow conventional commits format: `type(scope): description`
- Types: feat, fix, docs, test, refactor, chore
- Example: `feat(readers): add PostgreSQL support`
- Keep commits focused and atomic
- Reference issues in commit messages when applicable
## Pull Request Process
1. Create a feature branch from `master`
2. Make your changes
3. Add tests for new functionality
4. Ensure all tests pass
5. Update documentation if needed
6. Submit a pull request with a clear description
## Claude Code Commands
This project includes Claude Code slash commands for common tasks:
- `/test` - Run all tests
- `/build` - Build the binary
- `/lint` - Run linters
- `/coverage` - Generate coverage report
## Questions or Issues?
- Open an issue for bugs or feature requests
- Start a discussion for questions or ideas
- Check existing issues before creating new ones
Do not report vulnerabilities in public issues. See [SECURITY.md](SECURITY.md).
## License
By contributing to RelSpec, you agree that your contributions will be licensed under the Apache License 2.0.
Contributions are licensed under the Apache License 2.0.
+21 -9
View File
@@ -20,7 +20,9 @@ STATICCHECK = go run honnef.co/go/tools/cmd/staticcheck@latest
GOVULNCHECK = go run golang.org/x/vuln/cmd/govulncheck@latest
# Version information
VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo "dev")
# VERSION file is the single source of truth (bumped by `make release-version`).
# Falls back to git describe for ad-hoc builds where VERSION hasn't been committed yet.
VERSION := $(shell [ -f VERSION ] && echo "v$$(cat VERSION | tr -d '[:space:]')" || git describe --tags --always --dirty 2>/dev/null || echo "dev")
BUILD_DATE := $(shell date -u +"%Y-%m-%d %H:%M:%S UTC")
LDFLAGS := -X 'git.warky.dev/wdevs/relspecgo/pkg/buildinfo.Version=$(VERSION)' -X 'git.warky.dev/wdevs/relspecgo/pkg/buildinfo.BuildDate=$(BUILD_DATE)'
@@ -223,6 +225,15 @@ release: lint fmt-check test build ## Run lint, format check, tests, build, then
echo "Creating new release: $$version"; \
commit_logs=$$(git log "$${latest_tag}..HEAD" --pretty=format:"- %s" --no-merges); \
fi; \
PKGVER=$${version#v}; \
echo "Updating package metadata to $$PKGVER..."; \
echo "$$PKGVER" > VERSION; \
sed -i "s/^pkgver=.*/pkgver=$$PKGVER/" linux/arch/PKGBUILD; \
sed -i "s/^Version:.*/Version: $$PKGVER/" linux/centos/relspec.spec; \
git add VERSION linux/arch/PKGBUILD linux/centos/relspec.spec; \
if ! git diff --cached --quiet; then \
git commit -m "chore(release): update package version to $$PKGVER"; \
fi; \
if [ -z "$$commit_logs" ]; then \
tag_message="Release $$version"; \
else \
@@ -232,17 +243,18 @@ release: lint fmt-check test build ## Run lint, format check, tests, build, then
git push origin "$$version"; \
echo "Tag $$version created and pushed to remote repository."
release-version: ## Auto-increment patch version, update package files, commit, tag, and push
@CURRENT=$$(git describe --tags --abbrev=0 2>/dev/null || echo "v0.0.0"); \
MAJOR=$$(echo $$CURRENT | sed 's/v\([0-9]*\)\.\([0-9]*\)\.\([0-9]*\).*/\1/'); \
MINOR=$$(echo $$CURRENT | sed 's/v\([0-9]*\)\.\([0-9]*\)\.\([0-9]*\).*/\2/'); \
PATCH=$$(echo $$CURRENT | sed 's/v\([0-9]*\)\.\([0-9]*\)\.\([0-9]*\).*/\3/'); \
NEXT="v$$MAJOR.$$MINOR.$$((PATCH + 1))"; \
release-version: lint fmt-check ## Run lint and format check, then auto-increment patch version, update VERSION/package files, commit, tag, and push
@CURRENT=$$([ -f VERSION ] && cat VERSION | tr -d '[:space:]' || (git describe --tags --abbrev=0 2>/dev/null | sed 's/^v//') || echo "0.0.0"); \
MAJOR=$$(echo $$CURRENT | cut -d. -f1); \
MINOR=$$(echo $$CURRENT | cut -d. -f2); \
PATCH=$$(echo $$CURRENT | cut -d. -f3); \
PKGVER="$$MAJOR.$$MINOR.$$((PATCH + 1))"; \
echo "Current: $$CURRENT → Next: $$NEXT"; \
NEXT="v$$PKGVER"; \
echo "Current: v$$CURRENT → Next: $$NEXT"; \
echo "$$PKGVER" > VERSION; \
sed -i "s/^pkgver=.*/pkgver=$$PKGVER/" linux/arch/PKGBUILD; \
sed -i "s/^Version:.*/Version: $$PKGVER/" linux/centos/relspec.spec; \
git add linux/arch/PKGBUILD linux/centos/relspec.spec; \
git add VERSION linux/arch/PKGBUILD linux/centos/relspec.spec; \
git commit -m "chore(release): update package version to $$PKGVER"; \
git tag -a "$$NEXT" -m "Release $$NEXT"; \
git push origin HEAD "$$NEXT"; \
+4 -2
View File
@@ -126,7 +126,7 @@ relspec job run build-schema
version: 1
jobs:
build-schema:
command: convert # closed allow-list: convert | merge | scripts-list
command: convert # closed allow-list: convert | merge | split | scripts-list | scripts-exec | templ | inspect | diff
description: Merge the DBML sources and emit PostgreSQL DDL
inputs:
- path: schema/core.dbml
@@ -147,7 +147,9 @@ resolved relative to the job file and may not escape it, and remote database
credentials are referenced by environment-variable name (`conn_env:`) and
redacted from logs. The whole plan — unknown commands/formats, duplicate job
names, missing inputs, path traversal, dependency cycles — is validated before
any job runs. See [docs/JOB_FILES.md](docs/JOB_FILES.md).
any job runs. Path-like fields also support `${NAME}` environment-variable
references, which are expanded and safety-checked during pre-flight. See
[docs/JOB_FILES.md](docs/JOB_FILES.md).
### `edit` — Interactive TUI editor
+12
View File
@@ -0,0 +1,12 @@
# Security Policy
## Supported versions
Security fixes are released for the latest minor version of RelSpec.
## Reporting a vulnerability
Please do not open a public issue for security problems.
Report privately by email: warkydevs@gmail.com
Reports are reviewed on a best-effort basis. No response time or fix timeline is
guaranteed. Reporters may be credited in the release notes if they wish.
+20 -9
View File
@@ -4,20 +4,25 @@
- [✔️] **Database Inspector**
- [✔️] PostgreSQL driver (reader + writer)
- [ ] MySQL driver
- [ ] MySQL driver (only MariaDB datatype conversion in pkg/mariadb, no reader/writer)
- [✔️] SQLite driver (reader + writer with automatic schema flattening)
- [ ] MSSQL driver
- [✔️] MSSQL driver (reader + writer, generated and identity columns)
- [✔️] Foreign key detection
- [✔️] Index extraction
- [✔️] .sql file generation (PostgreSQL, SQLite)
- [✔️] .dbml: Database Markup Language (DBML) for textual schema representation.
- [✔️] Prisma schema support (PSL format) .prisma
- [ ] Sequelize (Typescript/Javascript) (Use templates, 💲 Someone can do this, not me)
- [✔️] Drizzle ORM support .ts (TypeScript / JavaScript) (Mr. Edd wanted to move from Prisma to Drizzle. If you are bugs, you are welcome to do pull requests or issues)
- [☠️] Entity Framework (.NET) model .edmx (Fuck no, EDMX files were bloated, verbose XML nightmares—hard to merge, error-prone, and a pain in teams. Microsoft wisely ditched them in EF Core for code-first. Classic overkill from old MS era.)
- [✔️] TypeORM support
- [] .hbm.xml / schema.xml: Hibernate/Propel mappings (Java/PHP) (💲 Someone can do this, not me)
- [ ] Django models.py (Python classes), Sequelize migrations (JS) (💲 Someone can do this, not me)
- [] .avsc: Avro schema (JSON format for data serialization) (💲 Someone can do this, not me)
- [ ] .hbm.xml / schema.xml: Hibernate/Propel mappings (Java/PHP) (Use templates, 💲 Someone can do this, not me)
- [ ] Django models.py (Python classes), Sequelize migrations (JS) (Use templates, 💲 Someone can do this, not me)
- [ ] SQLAlchemy, Python (Use templates, 💲 Someone can do this, not me)
- [ ] .avsc: Avro schema (JSON format for data serialization) (Use templates, 💲 Someone can do this, not me)
- [ ] Rails schema in db/schema.rb (Use templates, 💲 Someone can do this, not me)
- [ ] Laravel migration (Use templates, 💲 Someone can do this, not me)
- [ ] Hibernate / JPA (Use templates, 💲 Someone can do this, not me)
- [✔️] GraphQL schema generation
## UI
@@ -39,13 +44,19 @@
## Advanced Features
- [ ] Dry-run mode for validation
- [x] Diff tool for comparing specifications
- [ ] Migration script generation
- [ ] Dry-run mode for validation (only `job run --dry-run`; not on convert/merge/split)
- [✔️] Diff tool for comparing specifications
- [✔️] Migration script generation (PostgreSQL diff, live database by default for direct output)
- [ ] Custom type mapping configuration
- [ ] Batch processing support
- [ ] Batch processing support (partial: job files run named workflows via `relspec job run`)
- [ ] Watch mode for auto-regeneration
## Distribution
- [ ] NSIS Windows installer
- [ ] Check https://git.warky.dev/wdevs/relspecgo/releases for new releases and prompt to update
- [ ] CI pipeline to build and publish the Windows installer
## Future Considerations
- [ ] Web UI for visual editing
+1
View File
@@ -0,0 +1 @@
1.0.85
+13
View File
@@ -229,6 +229,7 @@ func runConvert(cmd *cobra.Command, args []string) error {
if err != nil {
return fmt.Errorf("failed to read source: %w", err)
}
finalizeCommentedRefs(db, stderrWarn)
fmt.Fprintf(os.Stderr, " ✓ Successfully read database '%s'\n", db.Name)
fmt.Fprintf(os.Stderr, " Found: %d schema(s)\n", len(db.Schemas))
@@ -261,6 +262,18 @@ func runConvert(cmd *cobra.Command, args []string) error {
return nil
}
// finalizeCommentedRefs resolves DBML `// Ref:` comments against the fully
// loaded model and warns about refs whose target is not loaded.
func finalizeCommentedRefs(db *models.Database, warn func(string)) {
for _, w := range dbml.ResolveCommentedRefs(db, true) {
warn(w)
}
}
func stderrWarn(msg string) {
fmt.Fprintf(os.Stderr, " ⚠ %s\n", msg)
}
func readDatabaseListForConvert(dbType string, files []string) (*models.Database, error) {
if len(files) == 0 {
return nil, fmt.Errorf("file list is empty")
+57 -19
View File
@@ -60,8 +60,9 @@ inputs, output and options:
logfile: .relspec/log/build-schema.log
Rules and guarantees:
- command is a closed allow-list (convert, merge, scripts-list, templ). Arbitrary
shell strings are never executed.
- command is a closed allow-list (convert, merge, split, scripts-list,
scripts-exec, templ, inspect, diff). Arbitrary shell strings are never
executed.
- Every path is relative to the directory holding the job file and may not
escape it. Absolute and home-relative paths are rejected.
- Remote database credentials are referenced by environment-variable name
@@ -246,22 +247,37 @@ type resolvedInput struct {
func preflightJob(j *jobs.Job, resolvedByName map[string]*resolvedJob) (*resolvedJob, error) {
root := j.Dir()
rj := &resolvedJob{job: j, root: root, logPolicy: j.ResolvedLogPolicy()}
expandPath := func(label, value string) (string, error) {
expanded, err := jobs.ExpandEnv(value)
if err != nil {
return "", fmt.Errorf("%s: %w", label, err)
}
return expanded, nil
}
if j.Logfile != "" {
p, err := jobs.SafeJoin(root, j.Logfile)
logfile, err := expandPath("logfile", j.Logfile)
if err != nil {
return nil, err
}
p, err := jobs.SafeJoin(root, logfile)
if err != nil {
return nil, fmt.Errorf("logfile: %w", err)
}
rj.logPath = p
}
if j.Template != "" {
p, err := jobs.SafeJoin(root, j.Template)
templatePath, err := expandPath("template", j.Template)
if err != nil {
return nil, err
}
p, err := jobs.SafeJoin(root, templatePath)
if err != nil {
return nil, fmt.Errorf("template: %w", err)
}
info, err := os.Stat(p)
if err != nil || info.IsDir() {
return nil, fmt.Errorf("template %q: not found or is a directory", j.Template)
return nil, fmt.Errorf("template %q: not found or is a directory", templatePath)
}
rj.templatePath = p
}
@@ -291,16 +307,20 @@ func preflightJob(j *jobs.Job, resolvedByName map[string]*resolvedJob) (*resolve
ri.connEnv = in.ConnEnv
rj.secrets = append(rj.secrets, v)
} else {
p, err := jobs.SafeJoin(root, in.Path)
inputPath, err := expandPath(fmt.Sprintf("input[%d]", i), in.Path)
if err != nil {
return nil, err
}
p, err := jobs.SafeJoin(root, inputPath)
if err != nil {
return nil, fmt.Errorf("input[%d]: %w", i, err)
}
info, err := os.Stat(p)
if err != nil {
return nil, fmt.Errorf("input[%d]: %s: file not found", i, in.Path)
return nil, fmt.Errorf("input[%d]: %s: file not found", i, inputPath)
}
if info.IsDir() {
return nil, fmt.Errorf("input[%d]: %s: is a directory, not a file", i, in.Path)
return nil, fmt.Errorf("input[%d]: %s: is a directory, not a file", i, inputPath)
}
ri.path = p
}
@@ -308,13 +328,17 @@ func preflightJob(j *jobs.Job, resolvedByName map[string]*resolvedJob) (*resolve
}
for _, d := range j.ScriptDirs {
p, err := jobs.SafeJoin(root, d)
scriptDir, err := expandPath("script_dir", d)
if err != nil {
return nil, err
}
p, err := jobs.SafeJoin(root, scriptDir)
if err != nil {
return nil, fmt.Errorf("script_dir %q: %w", d, err)
}
info, err := os.Stat(p)
if err != nil {
return nil, fmt.Errorf("script_dir %q: not found", d)
return nil, fmt.Errorf("script_dir %q: not found", scriptDir)
}
if !info.IsDir() {
return nil, fmt.Errorf("script_dir %q: not a directory", d)
@@ -332,25 +356,33 @@ func preflightJob(j *jobs.Job, resolvedByName map[string]*resolvedJob) (*resolve
rj.outputConnEnv = j.Output.ConnEnv
rj.secrets = append(rj.secrets, v)
} else {
p, err := jobs.SafeJoin(root, j.Output.Path)
outputPath, err := expandPath("output", j.Output.Path)
if err != nil {
return nil, err
}
p, err := jobs.SafeJoin(root, outputPath)
if err != nil {
return nil, fmt.Errorf("output: %w", err)
}
if _, err := os.Stat(p); err == nil && !j.Output.Overwrite {
return nil, fmt.Errorf("output %s already exists (set output.overwrite: true to replace it)", j.Output.Path)
return nil, fmt.Errorf("output %s already exists (set output.overwrite: true to replace it)", outputPath)
}
rj.outputPath = p
}
}
if j.Rules != "" {
p, err := jobs.SafeJoin(root, j.Rules)
rulesPath, err := expandPath("rules", j.Rules)
if err != nil {
return nil, err
}
p, err := jobs.SafeJoin(root, rulesPath)
if err != nil {
return nil, fmt.Errorf("rules: %w", err)
}
info, err := os.Stat(p)
if err != nil || info.IsDir() {
return nil, fmt.Errorf("rules %q: not found or is a directory", j.Rules)
return nil, fmt.Errorf("rules %q: not found or is a directory", rulesPath)
}
rj.rulesPath = p
}
@@ -358,12 +390,16 @@ func preflightJob(j *jobs.Job, resolvedByName map[string]*resolvedJob) (*resolve
if j.Report != nil {
rj.reportFormat = strings.ToLower(j.Report.Format)
if j.Report.Path != "" {
p, err := jobs.SafeJoin(root, j.Report.Path)
reportPath, err := expandPath("report", j.Report.Path)
if err != nil {
return nil, err
}
p, err := jobs.SafeJoin(root, reportPath)
if err != nil {
return nil, fmt.Errorf("report: %w", err)
}
if _, err := os.Stat(p); err == nil && !j.Report.Overwrite {
return nil, fmt.Errorf("report %s already exists (set report.overwrite: true to replace it)", j.Report.Path)
return nil, fmt.Errorf("report %s already exists (set report.overwrite: true to replace it)", reportPath)
}
rj.reportPath = p
}
@@ -542,6 +578,7 @@ func runMergeJob(rj *resolvedJob, lg *jobLogger) error {
lg.logf("merging: %s", inputLabel(ri))
merge.MergeDatabases(base, db, opts)
}
finalizeCommentedRefs(base, func(w string) { lg.logf("warning: %s", w) })
base.UpdateDate()
return writeJobOutput(rj, base, lg)
}
@@ -778,6 +815,7 @@ func readJobInputs(rj *resolvedJob, lg *jobLogger) (*models.Database, error) {
if base == nil {
return nil, fmt.Errorf("no inputs produced a database")
}
finalizeCommentedRefs(base, func(w string) { lg.logf("warning: %s", w) })
return base, nil
}
@@ -805,8 +843,8 @@ func writeJobOutput(rj *resolvedJob, db *models.Database, lg *jobLogger) error {
return fmt.Errorf("database output is only supported for pgsql (got %q)", rj.job.Output.Format)
}
lg.logf("writing output to database env:%s", rj.outputConnEnv)
writerOpts := newWriterOptions("", o.Package, o.FlattenSchema, "", "", o.ContinueOnError)
writerOpts.Metadata = map[string]interface{}{"connection_string": rj.outputConn}
writerOpts := newWriterOptions("", o.Package, o.FlattenSchema, o.Types, o.ArrayNullable, o.ContinueOnError)
writerOpts.Metadata = map[string]interface{}{"connection_string": rj.outputConn, "full_ddl": o.FullDDL}
return wpgsql.NewWriter(writerOpts).WriteDatabase(db)
}
@@ -816,7 +854,7 @@ func writeJobOutput(rj *resolvedJob, db *models.Database, lg *jobLogger) error {
lg.logf("writing output: %s (%s)", rj.outputPath, format)
write := func(target string) error {
return writeDatabase(db, format, target, o.Package, o.Schema, o.FlattenSchema, "", "", o.ContinueOnError, "")
return writeDatabase(db, format, target, o.Package, o.Schema, o.FlattenSchema, o.Types, o.ArrayNullable, o.ContinueOnError, "")
}
// Single-file formats are written to a temp file and renamed into place so
// a failure never leaves a partial or truncated output. Directory-emitting
+59
View File
@@ -666,6 +666,7 @@ jobs:
format: dbml
template: templates/schema.tmpl
output:
format: text
path: build/schema.txt
overwrite: true
`)
@@ -682,3 +683,61 @@ jobs:
t.Fatalf("templ output missing users table: %s", out)
}
}
func TestJobRun_BunSqlTypesOption(t *testing.T) {
dir := jobFixture(t, `version: 1
jobs:
models:
command: convert
inputs:
- path: schema/core.dbml
format: dbml
output:
format: bun
path: build/models
options:
package: models
types: sqltypes
`)
set := mustLoadSet(t, filepath.Join(dir, "relspec.yml"))
if err := executeJobPlan(set, "models", false, false, &bytes.Buffer{}); err != nil {
t.Fatalf("execute Bun job: %v", err)
}
generated, err := os.ReadFile(filepath.Join(dir, "build", "models"))
if err != nil {
t.Fatalf("read Bun output: %v", err)
}
if !strings.Contains(string(generated), "git.warky.dev/wdevs/relspecgo/pkg/sqltypes") {
t.Fatalf("Bun output did not preserve options.types: %s", generated)
}
}
func TestJobValidation_TemplTextFormat(t *testing.T) {
validDir := jobFixture(t, `version: 1
jobs:
docs:
command: templ
inputs: [{path: schema/core.dbml, format: dbml}]
template: schema.tmpl
output: {format: text, path: build/docs.txt}
`)
valid := mustLoadSet(t, filepath.Join(validDir, "relspec.yml"))
if err := valid.Validate(); err != nil {
t.Fatalf("text output should be valid for templ: %v", err)
}
invalidDir := jobFixture(t, `version: 1
jobs:
docs:
command: templ
inputs: [{path: schema/core.dbml, format: dbml}]
template: schema.tmpl
output: {format: json, path: build/docs.txt}
`)
invalid, err := jobs.Load([]string{filepath.Join(invalidDir, "relspec.yml")})
if err != nil {
t.Fatalf("load invalid manifest: %v", err)
}
if err := invalid.Validate(); err == nil || !strings.Contains(err.Error(), "output.format: text") {
t.Fatalf("expected templ format rejection, got %v", err)
}
}
+4
View File
@@ -59,6 +59,7 @@ var (
mergeSkipTables string // Comma-separated table names to skip
mergeVerbose bool
mergeReportPath string // Path to write merge report
mergeFullDDL bool // Execute full DDL instead of diffing the live pgsql database
mergeFlattenSchema bool
)
@@ -127,6 +128,7 @@ func init() {
mergeCmd.Flags().BoolVar(&mergeSkipSequences, "skip-sequences", false, "Skip sequences during merge")
mergeCmd.Flags().StringVar(&mergeSkipTables, "skip-tables", "", "Comma-separated list of table names to skip during merge")
mergeCmd.Flags().BoolVar(&mergeVerbose, "verbose", false, "Show verbose output")
mergeCmd.Flags().BoolVar(&mergeFullDDL, "full-ddl", false, "pgsql database output: execute the full DDL instead of only the differences from the live database")
mergeCmd.Flags().StringVar(&mergeReportPath, "merge-report", "", "Path to write merge report (JSON format)")
mergeCmd.Flags().BoolVar(&mergeFlattenSchema, "flatten-schema", false, "Flatten schema.table names to schema_table (useful for databases like SQLite that do not support schemas)")
}
@@ -248,6 +250,7 @@ func runMerge(cmd *cobra.Command, args []string) error {
}
result := merge.MergeDatabases(targetDB, sourceDB, opts)
finalizeCommentedRefs(targetDB, stderrWarn)
// Update timestamp
targetDB.UpdateDate()
@@ -442,6 +445,7 @@ func writeDatabaseForMerge(dbType, filePath, connString string, db *models.Datab
if connString != "" {
writerOpts.Metadata = map[string]interface{}{
"connection_string": connString,
"full_ddl": mergeFullDDL,
}
// Add report path if merge report is enabled
if mergeReportPath != "" {
+1
View File
@@ -114,6 +114,7 @@ func runTempl(cmd *cobra.Command, args []string) error {
if err != nil {
return fmt.Errorf("failed to read source: %w", err)
}
finalizeCommentedRefs(db, stderrWarn)
// Print database stats
schemaCount := len(db.Schemas)
+13 -2
View File
@@ -55,6 +55,10 @@ already forbid duplicate keys within a single file.
`script_dirs[]`, `template`, `logfile`) is **relative to the directory
containing the job file that declared the job**, not the process working
directory.
* Those path fields support `${NAME}` environment-variable references. They
are expanded immediately before execution; a missing variable is an error.
The expanded value is still subject to all relative-path and symlink safety
checks.
* Absolute paths, `~`-relative paths and any path that resolves outside the job
file directory (`../`, `a/../../b`, …) are **rejected during validation** —
before anything runs.
@@ -183,12 +187,17 @@ jobs:
format: pgsql
path: build/schema.sql # file output, OR:
conn_env: TARGET_DB_URL # execute against DB (pgsql only)
# default: reads live DB, executes only the diff
# (file output = full DDL); set options.full_ddl to force full DDL
overwrite: false # default false
options:
flatten_schema: false
schema: public
package: models # for gorm/bun output
types: sqltypes # Bun/GORM nullable types: baselib|stdlib|sqltypes
array_nullable: pointer_slice # Bun: slice|pointer_slice
continue_on_error: false # pgsql / scripts-exec output
full_ddl: false # pgsql database output: true = run full DDL; default diffs against live DB
skip_relations: false # merge only
skip_enums: false
skip_views: false
@@ -201,8 +210,10 @@ jobs:
For `templ`, `inputs` use the same file or `pgsql`/`conn_env` source forms as
schema conversion. `output` is optional (empty means stdout); when present it
contains only `path` and `overwrite`, because templates do not select a schema
writer format.
contains `path` and `overwrite`, plus the explicit `format: text` marker. The
marker is templ-only: it describes arbitrary text produced by a Go template,
not a database schema format. It is optional for compatibility with older
manifests and has no effect on template execution.
A `from_job` input takes no `path`, `format` or `conn_env`: it resolves to the
named job's `output.path` and inherits its format, and implies a dependency on
+80
View File
@@ -0,0 +1,80 @@
# RelSpec job examples
Run these commands from this directory:
```bash
cd examples/jobs
# List every job from relspec.yml and the additional relspec.*.yml files.
relspec job list
# Validate a job and print its resolved plan without reading or writing data.
relspec job run build-schema --plan
# Build PostgreSQL DDL from the two DBML inputs.
relspec job run build-schema
# Run a job's dependency chain. This runs build-schema first, then build-json.
relspec job run build-json
# Inspect build-json's output. The from_job input adds the dependency automatically.
relspec job run lint-schema
# Extract only the posts table.
relspec job run posts-only
# List migration scripts in deterministic order without connecting to a database.
relspec job run migration-order
# Render one TypeScript file per table using the example template.
relspec job run generate-typescript
# Compare the two example schemas and write the summary to the logfile.
relspec job run compare-schemas
# Use an environment variable in an output path.
export VST=build/typescript
relspec job run generate-typescript --plan
relspec job run generate-typescript
```
Database jobs use environment variables for connection strings. Set them in
the shell before running the jobs; do not put connection strings in YAML:
```bash
export SOURCE_DB_URL='postgres://user:password@localhost/source_db'
export TARGET_DB_URL='postgres://user:password@localhost/target_db'
# Read the source database and write a local DBML snapshot.
relspec job run snapshot-database
# Preview migration execution without connecting to PostgreSQL.
relspec job run apply-migrations --plan
# Execute the migrations against TARGET_DB_URL.
relspec job run apply-migrations
```
The `apply-migrations` job demonstrates execution from multiple directories:
```yaml
script_dirs:
- migrations/core
- migrations/tenant
output:
format: pgsql
conn_env: TARGET_DB_URL
```
RelSpec combines the SQL files from both folders in deterministic order before
executing them against the configured PostgreSQL database.
To use a different directory or manifest, pass `--dir` or `--file`:
```bash
relspec job list --dir /path/to/project
relspec job run build-schema --file /path/to/project/relspec.yml --plan
```
See [../../docs/JOB_FILES.md](../../docs/JOB_FILES.md) for the complete job
file reference.
+29
View File
@@ -0,0 +1,29 @@
# Database-oriented examples. Set the referenced environment variables before
# running these jobs; connection strings never belong in a job file.
version: 1
jobs:
snapshot-database:
command: convert
description: Read a PostgreSQL database and write a DBML snapshot
inputs:
- format: pgsql
conn_env: SOURCE_DB_URL
output:
format: dbml
path: build/database-snapshot.dbml
overwrite: true
apply-migrations:
command: scripts-exec
description: Execute migrations from core and tenant folders against PostgreSQL
# All SQL files from these directories are discovered in deterministic order.
script_dirs:
- migrations/core
- migrations/tenant
output:
format: pgsql
conn_env: TARGET_DB_URL
options:
continue_on_error: false
logfile: .relspec/log/apply-migrations.log
+32
View File
@@ -0,0 +1,32 @@
# Reporting and templating examples. These jobs are discovered together with
# relspec.yml when running `relspec job list` from this directory.
version: 1
jobs:
generate-typescript:
command: templ
description: Render one TypeScript file per table from the merged schema
inputs:
- path: schema/core.dbml
format: dbml
- path: schema/tenant.dbml
format: dbml
template: templates/schema.tmpl
mode: table
filename_pattern: "{{.Name}}.ts"
output:
format: text
path: "${VST}"
overwrite: true
compare-schemas:
command: diff
description: Compare the two example schemas and print a summary
inputs:
- path: schema/core.dbml
format: dbml
- path: schema/tenant.dbml
format: dbml
report:
format: summary
logfile: .relspec/log/compare-schemas.log
+6
View File
@@ -0,0 +1,6 @@
// Code generated by RelSpec. DO NOT EDIT.
export interface {{.Name}} {
{{- range values .Table.Columns}}
{{.Name}}: {{.Type}};
{{- end}}
}
+2 -2
View File
@@ -1,6 +1,6 @@
# Maintainer: Hein (Warky Devs) <hein@warky.dev>
pkgname=relspec
pkgver=1.0.74
pkgver=1.0.85
pkgrel=1
pkgdesc="RelSpec is a comprehensive database relations management tool that reads, transforms, and writes database table specifications across multiple formats and ORMs."
arch=('x86_64' 'aarch64')
@@ -15,7 +15,7 @@ build() {
export CGO_ENABLED=0
go build \
-trimpath \
-ldflags "-X git.warky.dev/wdevs/relspecgo/cmd/relspec.version=$pkgver" \
-ldflags "-X 'git.warky.dev/wdevs/relspecgo/pkg/buildinfo.Version=v$pkgver' -X 'git.warky.dev/wdevs/relspecgo/pkg/buildinfo.BuildDate=$(date -u +"%Y-%m-%d %H:%M:%S UTC")'" \
-o "$pkgname" ./cmd/relspec
}
+2 -2
View File
@@ -1,5 +1,5 @@
Name: relspec
Version: 1.0.74
Version: 1.0.85
Release: 1%{?dist}
Summary: RelSpec is a comprehensive database relations management tool that reads, transforms, and writes database table specifications across multiple formats and ORMs.
@@ -25,7 +25,7 @@ DBML, GraphQL, and more.
export CGO_ENABLED=0
go build \
-trimpath \
-ldflags "-X git.warky.dev/wdevs/relspecgo/cmd/relspec.version=%{version}" \
-ldflags "-X 'git.warky.dev/wdevs/relspecgo/pkg/buildinfo.Version=v%{version}' -X 'git.warky.dev/wdevs/relspecgo/pkg/buildinfo.BuildDate=$(date -u +"%%Y-%%m-%%d %%H:%%M:%%S UTC")'" \
-o %{name} ./cmd/relspec
%install
+1
View File
@@ -123,6 +123,7 @@ rules:
| `missing_primary_key` | `have_primary_key` | Ensure tables have primary keys |
| `orphaned_foreign_key` | `orphaned_foreign_key` | Detect FKs referencing non-existent tables |
| `circular_dependency` | `circular_dependency` | Detect circular FK dependencies |
| `duplicate_index_name` | `duplicate_index_name` | Index / PK / unique names must be unique per schema (default: `enforce`) |
## Rule Configuration
+1
View File
@@ -161,6 +161,7 @@ func getValidator(functionName string) (validatorFunc, bool) {
"have_primary_key": validateMissingPrimaryKey,
"orphaned_foreign_key": validateOrphanedForeignKey,
"circular_dependency": validateCircularDependency,
"duplicate_index_name": validateDuplicateIndexName,
}
fn, exists := validators[functionName]
+5
View File
@@ -154,6 +154,11 @@ func GetDefaultConfig() *Config {
Function: "circular_dependency",
Message: "Circular foreign key dependency detected",
},
"duplicate_index_name": {
Enabled: "enforce",
Function: "duplicate_index_name",
Message: "Index name is reused within the schema; PostgreSQL skips the duplicate CREATE INDEX IF NOT EXISTS",
},
},
}
}
+1
View File
@@ -37,6 +37,7 @@ func TestGetDefaultConfig(t *testing.T) {
"missing_primary_key",
"orphaned_foreign_key",
"circular_dependency",
"duplicate_index_name",
}
for _, ruleName := range expectedRules {
+61
View File
@@ -643,3 +643,64 @@ func contains(slice []string, value string) bool {
}
return false
}
// validateDuplicateIndexName checks that index names are unique per schema.
// PostgreSQL keeps indexes in the schema-wide relation namespace, so a name
// reused on another table makes CREATE INDEX IF NOT EXISTS silently skip it.
// Primary key and unique constraints create backing indexes and share that
// namespace too. An index and a constraint with the same name on the same
// table describe one object and are not reported.
func validateDuplicateIndexName(db *models.Database, rule Rule, ruleName string) []ValidationResult {
results := []ValidationResult{}
for _, schema := range db.Schemas {
// lowercased name -> "table" entries, one per distinct object
owners := make(map[string][]string)
display := make(map[string]string)
order := []string{}
add := func(name, table string, sameTableMerges bool) {
key := strings.ToLower(name)
if _, seen := owners[key]; !seen {
order = append(order, key)
display[key] = name
}
if sameTableMerges && contains(owners[key], table) {
return
}
owners[key] = append(owners[key], table)
}
for _, table := range schema.Tables {
for _, key := range sortedKeys(table.Indexes) {
if name := table.Indexes[key].Name; name != "" {
add(name, table.Name, false)
}
}
for _, c := range sortConstraints(table.Constraints) {
if c.Name == "" || (c.Type != models.PrimaryKeyConstraint && c.Type != models.UniqueConstraint) {
continue
}
add(c.Name, table.Name, true)
}
}
for _, key := range order {
tables := owners[key]
results = append(results, createResult(
ruleName,
len(tables) == 1,
rule.Message,
formatLocation(schema.Name, display[key], "")+" on "+strings.Join(tables, ", "),
map[string]interface{}{
"schema": schema.Name,
"index": display[key],
"tables": tables,
"occurrences": len(tables),
},
))
}
}
return results
}
+81
View File
@@ -835,3 +835,84 @@ func TestFormatLocation(t *testing.T) {
})
}
}
func TestValidateDuplicateIndexName(t *testing.T) {
db := &models.Database{
Name: "testdb",
Schemas: []*models.Schema{
{
Name: "entity",
Tables: []*models.Table{
{
Name: "actor_phone",
Indexes: map[string]*models.Index{
"idx_actor": {Name: "idx_actor", Columns: []string{"rid_actor"}},
"uk_phone": {Name: "uk_phone", Columns: []string{"phone"}, Unique: true},
},
Constraints: map[string]*models.Constraint{
// Same name as the index on the same table: one object.
"uk_phone": {Name: "uk_phone", Type: models.UniqueConstraint, Columns: []string{"phone"}},
},
},
{
Name: "actor_email",
Indexes: map[string]*models.Index{
"idx_actor": {Name: "idx_actor", Columns: []string{"rid_actor"}},
},
Constraints: map[string]*models.Constraint{
"UK_Phone": {Name: "UK_Phone", Type: models.UniqueConstraint, Columns: []string{"email"}},
},
},
{
Name: "actor_address",
Indexes: map[string]*models.Index{
"idx_actor": {Name: "idx_actor", Columns: []string{"rid_actor"}},
"idx_actor#2": {Name: "idx_actor", Columns: []string{"rid_actor", "kind"}},
"idx_address": {Name: "idx_address", Columns: []string{"line1"}},
},
},
},
},
{
// Same names in another schema do not collide.
Name: "org",
Tables: []*models.Table{
{Name: "api_provider", Indexes: map[string]*models.Index{"idx_actor": {Name: "idx_actor"}}},
},
},
},
}
results := validateDuplicateIndexName(db, Rule{Message: "dup"}, "duplicate_index_name")
got := map[string]bool{}
occ := map[string]int{}
for _, r := range results {
key := r.Context["schema"].(string) + "." + r.Context["index"].(string)
got[key] = r.Passed
occ[key] = r.Context["occurrences"].(int)
}
want := map[string]struct {
passed bool
occ int
}{
"entity.idx_actor": {false, 4},
"entity.uk_phone": {false, 2},
"entity.idx_address": {true, 1},
"org.idx_actor": {true, 1},
}
if len(got) != len(want) {
t.Fatalf("got %d results %v, want %d", len(got), got, len(want))
}
for k, w := range want {
p, ok := got[k]
if !ok {
t.Errorf("missing result for %s", k)
continue
}
if p != w.passed || occ[k] != w.occ {
t.Errorf("%s: passed=%v occurrences=%d, want passed=%v occurrences=%d", k, p, occ[k], w.passed, w.occ)
}
}
}
+42 -9
View File
@@ -19,6 +19,7 @@ import (
"fmt"
"os"
"path/filepath"
"regexp"
"sort"
"strconv"
"strings"
@@ -276,15 +277,22 @@ func parseHumanSize(s string) (int64, error) {
// Options carries the subset of command flags a job file may set.
type Options struct {
FlattenSchema bool `yaml:"flatten_schema"`
Schema string `yaml:"schema"`
Package string `yaml:"package"`
FlattenSchema bool `yaml:"flatten_schema"`
Schema string `yaml:"schema"`
Package string `yaml:"package"`
// Types and ArrayNullable mirror the existing convert/split writer flags.
// They are passed through only to writers that already support them.
Types string `yaml:"types"`
ArrayNullable string `yaml:"array_nullable"`
ContinueOnError bool `yaml:"continue_on_error"`
SkipRelations bool `yaml:"skip_relations"`
SkipEnums bool `yaml:"skip_enums"`
SkipViews bool `yaml:"skip_views"`
SkipDomains bool `yaml:"skip_domains"`
SkipSequences bool `yaml:"skip_sequences"`
// FullDDL makes direct pgsql output execute the full idempotent DDL instead of
// diffing against the live database first.
FullDDL bool `yaml:"full_ddl"`
SkipRelations bool `yaml:"skip_relations"`
SkipEnums bool `yaml:"skip_enums"`
SkipViews bool `yaml:"skip_views"`
SkipDomains bool `yaml:"skip_domains"`
SkipSequences bool `yaml:"skip_sequences"`
}
// Dir returns the directory that a job's relative paths resolve against:
@@ -568,7 +576,9 @@ func (j *Job) validate() []string {
e = append(e, "command \"templ\" does not support database output")
}
if j.Output != nil && j.Output.Format != "" {
e = append(e, "output.format is not valid for command \"templ\"")
if !strings.EqualFold(j.Output.Format, "text") {
e = append(e, "command \"templ\" accepts only output.format: text")
}
}
case CommandSplit:
if len(j.Inputs) < 1 {
@@ -850,6 +860,29 @@ func SafeJoin(root, rel string) (string, error) {
return joined, nil
}
var envReferencePattern = regexp.MustCompile(`\$\{([A-Za-z_][A-Za-z0-9_]*)\}`)
// ExpandEnv expands ${NAME} references using the current process environment.
// It deliberately supports only shell-independent variable references; job
// files are never passed through a shell. An unset variable is an error so a
// typo cannot silently turn into a relative path.
func ExpandEnv(value string) (string, error) {
var missing string
expanded := envReferencePattern.ReplaceAllStringFunc(value, func(reference string) string {
name := reference[2 : len(reference)-1]
resolved, ok := os.LookupEnv(name)
if !ok {
missing = name
return reference
}
return resolved
})
if missing != "" {
return "", fmt.Errorf("environment variable %q referenced by ${%s} is not set", missing, missing)
}
return expanded, nil
}
// deepestExistingAncestor returns p itself if it exists, otherwise the nearest
// existing parent directory (falling back to the filesystem root).
func deepestExistingAncestor(p string) string {
+15
View File
@@ -135,6 +135,21 @@ func TestParseHumanSize(t *testing.T) {
}
}
func TestExpandEnv(t *testing.T) {
t.Setenv("RELSPEC_TEST_ROOT", "build/output")
got, err := ExpandEnv("${RELSPEC_TEST_ROOT}/schema-${RELSPEC_TEST_ROOT}")
if err != nil {
t.Fatal(err)
}
if want := "build/output/schema-build/output"; got != want {
t.Fatalf("ExpandEnv = %q, want %q", got, want)
}
if _, err := ExpandEnv("build/${RELSPEC_TEST_MISSING}"); err == nil {
t.Fatal("expected missing environment variable error")
}
}
func TestLoadRejectsDuplicateJobAcrossFiles(t *testing.T) {
dir := t.TempDir()
a := filepath.Join(dir, "relspec.yml")
+39
View File
@@ -91,6 +91,45 @@ func (r *MergeResult) merge(target, source *models.Database, opts *MergeOptions)
if !opts.SkipDomains {
r.mergeDomains(target, source)
}
mergeDatabaseMetadata(target, source)
}
// mergeDatabaseMetadata adds missing metadata keys and unions []string values,
// so per-file reader state (e.g. pending DBML commented refs) survives a merge.
func mergeDatabaseMetadata(target, source *models.Database) {
if len(source.Metadata) == 0 {
return
}
if target.Metadata == nil {
target.Metadata = make(map[string]any, len(source.Metadata))
}
for key, srcVal := range source.Metadata {
tgtVal, exists := target.Metadata[key]
if !exists {
if list, ok := srcVal.([]string); ok {
srcVal = append([]string(nil), list...)
}
target.Metadata[key] = srcVal
continue
}
tgtList, tgtOK := tgtVal.([]string)
srcList, srcOK := srcVal.([]string)
if !tgtOK || !srcOK {
continue
}
seen := make(map[string]bool, len(tgtList))
for _, v := range tgtList {
seen[v] = true
}
for _, v := range srcList {
if !seen[v] {
tgtList = append(tgtList, v)
seen[v] = true
}
}
target.Metadata[key] = tgtList
}
}
func (r *MergeResult) mergeSchemaContents(target, source *models.Schema, opts *MergeOptions) {
+29
View File
@@ -721,3 +721,32 @@ func TestComplexMerge(t *testing.T) {
t.Error("Expected ukey_users_guid constraint to exist")
}
}
func TestMergeDatabases_Metadata(t *testing.T) {
target := &models.Database{Metadata: map[string]any{
"refs": []string{"a", "b"},
"name": "target",
}}
source := &models.Database{Metadata: map[string]any{
"refs": []string{"b", "c"},
"name": "source",
"extra": []string{"x"},
}}
MergeDatabases(target, source, nil)
if got := target.Metadata["refs"].([]string); strings.Join(got, ",") != "a,b,c" {
t.Errorf("refs = %v, want [a b c]", got)
}
if got := target.Metadata["name"]; got != "target" {
t.Errorf("name = %v, want target (existing scalar keys are kept)", got)
}
extra := target.Metadata["extra"].([]string)
if strings.Join(extra, ",") != "x" {
t.Errorf("extra = %v, want [x]", extra)
}
source.Metadata["extra"].([]string)[0] = "changed"
if extra[0] != "x" {
t.Error("copied slice must not alias the source")
}
}
+21 -17
View File
@@ -227,23 +227,27 @@ func (d *Sequence) SQLName() string {
// Column represents a table column
type Column struct {
Name string `json:"name" yaml:"name" xml:"name"`
Description string `json:"description,omitempty" yaml:"description,omitempty" xml:"description,omitempty"`
Table string `json:"table" yaml:"table" xml:"table"`
Schema string `json:"schema" yaml:"schema" xml:"schema"`
Type string `json:"type" yaml:"type" xml:"type"`
Length int `json:"length,omitempty" yaml:"length,omitempty" xml:"length,omitempty"`
Precision int `json:"precision,omitempty" yaml:"precision,omitempty" xml:"precision,omitempty"`
Scale int `json:"scale,omitempty" yaml:"scale,omitempty" xml:"scale,omitempty"`
NotNull bool `json:"not_null" yaml:"not_null" xml:"not_null"`
Default any `json:"default,omitempty" yaml:"default,omitempty" xml:"default,omitempty"`
AutoIncrement bool `json:"auto_increment" yaml:"auto_increment" xml:"auto_increment"`
IsPrimaryKey bool `json:"is_primary_key" yaml:"is_primary_key" xml:"is_primary_key"`
Comment string `json:"comment,omitempty" yaml:"comment,omitempty" xml:"comment,omitempty"`
Collation string `json:"collation,omitempty" yaml:"collation,omitempty" xml:"collation,omitempty"`
Metadata map[string]any `json:"metadata,omitempty" yaml:"metadata,omitempty" xml:"-"`
Sequence uint `json:"sequence,omitempty" yaml:"sequence,omitempty" xml:"sequence,omitempty"`
GUID string `json:"guid" yaml:"guid" xml:"guid"`
Name string `json:"name" yaml:"name" xml:"name"`
Description string `json:"description,omitempty" yaml:"description,omitempty" xml:"description,omitempty"`
Table string `json:"table" yaml:"table" xml:"table"`
Schema string `json:"schema" yaml:"schema" xml:"schema"`
Type string `json:"type" yaml:"type" xml:"type"`
Length int `json:"length,omitempty" yaml:"length,omitempty" xml:"length,omitempty"`
Precision int `json:"precision,omitempty" yaml:"precision,omitempty" xml:"precision,omitempty"`
Scale int `json:"scale,omitempty" yaml:"scale,omitempty" xml:"scale,omitempty"`
NotNull bool `json:"not_null" yaml:"not_null" xml:"not_null"`
Default any `json:"default,omitempty" yaml:"default,omitempty" xml:"default,omitempty"`
AutoIncrement bool `json:"auto_increment" yaml:"auto_increment" xml:"auto_increment"`
IsPrimaryKey bool `json:"is_primary_key" yaml:"is_primary_key" xml:"is_primary_key"`
Comment string `json:"comment,omitempty" yaml:"comment,omitempty" xml:"comment,omitempty"`
Collation string `json:"collation,omitempty" yaml:"collation,omitempty" xml:"collation,omitempty"`
Metadata map[string]any `json:"metadata,omitempty" yaml:"metadata,omitempty" xml:"-"`
Sequence uint `json:"sequence,omitempty" yaml:"sequence,omitempty" xml:"sequence,omitempty"`
GUID string `json:"guid" yaml:"guid" xml:"guid"`
Generated bool `json:"generated,omitempty" yaml:"generated,omitempty" xml:"generated,omitempty"`
GenerationExpression string `json:"generation_expression,omitempty" yaml:"generation_expression,omitempty" xml:"generation_expression,omitempty"`
Identity bool `json:"identity,omitempty" yaml:"identity,omitempty" xml:"identity,omitempty"`
IdentityGeneration string `json:"identity_generation,omitempty" yaml:"identity_generation,omitempty" xml:"identity_generation,omitempty"` // "ALWAYS" or "BY DEFAULT"
}
// SQLName returns the column name in lowercase for SQL compatibility.
+2
View File
@@ -64,6 +64,8 @@ The reader recognizes the following Bun struct tags:
- `autoincrement` - Auto-increment column
- `default` - Default value
- `unique` - Unique constraint
- `generated` - Generated column marker (`Generated`; expression is not in the tag)
- `identity` - Identity column marker (`Identity`, `ALWAYS`)
- `rel` - Relationship definition
## Example Bun Model
+48
View File
@@ -0,0 +1,48 @@
package bun
import (
"os"
"path/filepath"
"testing"
"git.warky.dev/wdevs/relspecgo/pkg/readers"
)
func TestReader_GeneratedAndIdentityMarkers(t *testing.T) {
src := `package models
import "github.com/uptrace/bun"
type Person struct {
bun.BaseModel ` + "`bun:\"table:people,alias:p\"`" + `
ID int64 ` + "`bun:\"id,pk,type:bigint,autoincrement,identity\"`" + `
Seq int64 ` + "`bun:\"seq,type:bigint,scanonly,identity,notnull\"`" + `
FullName string ` + "`bun:\"full_name,type:text,scanonly,generated,nullzero\"`" + `
First string ` + "`bun:\"first,type:text,nullzero\"`" + `
}
`
path := filepath.Join(t.TempDir(), "person.go")
if err := os.WriteFile(path, []byte(src), 0o644); err != nil {
t.Fatal(err)
}
db, err := NewReader(&readers.ReaderOptions{FilePath: path}).ReadDatabase()
if err != nil {
t.Fatalf("ReadDatabase() error = %v", err)
}
cols := db.Schemas[0].Tables[0].Columns
if !cols["id"].Identity || cols["id"].IdentityGeneration != "ALWAYS" {
t.Errorf("id: identity = %v/%q, want true/ALWAYS", cols["id"].Identity, cols["id"].IdentityGeneration)
}
if !cols["seq"].Identity || cols["seq"].IdentityGeneration != "ALWAYS" {
t.Errorf("seq: identity = %v/%q, want true/ALWAYS", cols["seq"].Identity, cols["seq"].IdentityGeneration)
}
if !cols["full_name"].Generated {
t.Error("full_name should be generated")
}
if cols["first"].Generated || cols["first"].Identity {
t.Error("first must be an ordinary column")
}
}
+7
View File
@@ -659,6 +659,13 @@ func (r *Reader) parseColumn(fieldName string, fieldType ast.Expr, tag string, s
hasExplicitNullableMarker = true
case "autoincrement":
column.AutoIncrement = true
case "generated":
// GENERATED ... STORED marker written by the Bun writer; the expression is not in the tag
column.Generated = true
case "identity":
// GENERATED ALWAYS AS IDENTITY marker written by the Bun writer
column.Identity = true
column.IdentityGeneration = "ALWAYS"
case "default":
// Default value from Bun tag (e.g., default:gen_random_uuid())
column.Default = value
+18
View File
@@ -90,11 +90,28 @@ Ref: posts.user_id > users.id [delete: cascade]
- Default values (`default`)
- Inline references (`ref`)
- Standalone `Ref` blocks
- Commented cross-file refs (`// Ref:` — see below)
- Indexes and composite indexes
- Table notes and column notes
- Enums
- Dialect directives (`@postgres:` / `@sqlite:` — see below)
## Commented cross-file refs
`// Ref:` / `// ref:` lines (ignored by dbdiagram) become FKs + relationships once both ends are loaded.
| Rule | Behaviour |
|---|---|
| When | Single file / directory: end of read. `--from-list`, `merge`, jobs: after all inputs are combined |
| Match | `schema.table.column` on both sides, case-insensitive |
| Operators | `>`, `<`, `-` (parsed like `Ref:`) |
| Duplicate of an FK on the same columns | Skipped silently |
| Target not loaded | Kept pending; skipped with a warning on the final pass |
| Column type mismatch | Warning, FK still created (`serial`≈`integer`, `bigserial`≈`bigint`) |
| Pending state | `Database.Metadata["dbml.commented_refs"]` (`[]string`) |
API: `dbml.ResolveCommentedRefs(db, final bool) []string` returns warnings.
## Dialect directives
Lines of the form `@<namespace>[(<column>)]: <args>` embed database-specific
@@ -140,6 +157,7 @@ grammar and the supported-directive matrix.
## Notes
- Column notes `GENERATED ALWAYS AS (expr) STORED` and `GENERATED ALWAYS|BY DEFAULT AS IDENTITY` set `Generated`/`GenerationExpression` and `Identity`/`IdentityGeneration`; they are not kept as comments
- DBML is designed for database documentation and diagramming
- Schema name defaults to `public`
- Relationship cardinality is preserved
+218
View File
@@ -0,0 +1,218 @@
package dbml
import (
"fmt"
"regexp"
"strings"
"git.warky.dev/wdevs/relspecgo/pkg/models"
)
// CommentedRefsMetadataKey is the Database.Metadata key holding commented
// `// Ref:` lines not yet resolved against the model ([]string).
const CommentedRefsMetadataKey = "dbml.commented_refs"
// commentedRefRegex matches `// Ref: ...` and `// ref: ...`.
var commentedRefRegex = regexp.MustCompile(`^//\s*[Rr]ef\s*:\s*(.+)$`)
// commentedRef returns the ref body of a trimmed `// Ref:` comment line.
func commentedRef(line string) (string, bool) {
m := commentedRefRegex.FindStringSubmatch(line)
if m == nil {
return "", false
}
ref := strings.TrimSpace(m[1])
return ref, ref != ""
}
// PendingCommentedRefs returns the commented refs not yet resolved.
func PendingCommentedRefs(db *models.Database) []string {
if db == nil || db.Metadata == nil {
return nil
}
refs, _ := db.Metadata[CommentedRefsMetadataKey].([]string)
return refs
}
func setPendingCommentedRefs(db *models.Database, refs []string) {
if len(refs) == 0 {
if db.Metadata != nil {
delete(db.Metadata, CommentedRefsMetadataKey)
}
return
}
if db.Metadata == nil {
db.Metadata = make(map[string]any)
}
db.Metadata[CommentedRefsMetadataKey] = refs
}
// addPendingCommentedRef queues a ref, skipping exact repeats.
func addPendingCommentedRef(db *models.Database, ref string) {
refs := PendingCommentedRefs(db)
for _, existing := range refs {
if existing == ref {
return
}
}
setPendingCommentedRefs(db, append(refs, ref))
}
// ResolveCommentedRefs turns pending commented refs into foreign keys and
// relationships when both ends (schema.table.column) exist in db. A ref that
// matches an existing FK on the same columns is dropped as a duplicate.
// Unmatched refs stay pending unless final is set, in which case they are
// dropped with a warning. Returns human-readable warnings.
func ResolveCommentedRefs(db *models.Database, final bool) []string {
refs := PendingCommentedRefs(db)
if len(refs) == 0 {
return nil
}
var warnings []string
var pending []string
parser := &Reader{}
for _, ref := range refs {
fk := parser.parseRef(ref)
if fk == nil || len(fk.Columns) == 0 || len(fk.Columns) != len(fk.ReferencedColumns) {
warnings = append(warnings, fmt.Sprintf("skipping commented ref %q: cannot parse", ref))
continue
}
srcTable, srcCols, srcMissing := lookupColumns(db, fk.Schema, fk.Table, fk.Columns)
dstTable, dstCols, dstMissing := lookupColumns(db, fk.ReferencedSchema, fk.ReferencedTable, fk.ReferencedColumns)
if srcMissing != "" || dstMissing != "" {
if final {
missing := srcMissing
if missing == "" {
missing = dstMissing
}
warnings = append(warnings, fmt.Sprintf("skipping commented ref %q: %s not found", ref, missing))
} else {
pending = append(pending, ref)
}
continue
}
fk.Schema, fk.Table, fk.Columns = srcTable.Schema, srcTable.Name, columnNames(srcCols)
fk.ReferencedSchema, fk.ReferencedTable, fk.ReferencedColumns = dstTable.Schema, dstTable.Name, columnNames(dstCols)
if hasFKOnColumns(srcTable, fk.Columns) {
continue // already declared by an uncommented Ref or inline ref
}
if _, taken := srcTable.Constraints[fk.Name]; taken {
warnings = append(warnings, fmt.Sprintf("skipping commented ref %q: constraint %s already exists", ref, fk.Name))
continue
}
for i := range srcCols {
if !compatibleFKTypes(srcCols[i].Type, dstCols[i].Type) {
warnings = append(warnings, fmt.Sprintf("commented ref %q: type mismatch %s.%s.%s (%s) -> %s.%s.%s (%s)",
ref, srcTable.Schema, srcTable.Name, srcCols[i].Name, srcCols[i].Type,
dstTable.Schema, dstTable.Name, dstCols[i].Name, dstCols[i].Type))
}
}
if srcTable.Constraints == nil {
srcTable.Constraints = make(map[string]*models.Constraint)
}
srcTable.Constraints[fk.Name] = fk
addFKRelationship(srcTable, fk)
}
setPendingCommentedRefs(db, pending)
return warnings
}
// lookupColumns finds a table and its columns, case-insensitively. missing
// names the first object not found, or is empty.
func lookupColumns(db *models.Database, schemaName, tableName string, cols []string) (*models.Table, []*models.Column, string) {
qualified := schemaName + "." + tableName
var table *models.Table
for _, schema := range db.Schemas {
if !strings.EqualFold(schema.Name, schemaName) {
continue
}
for _, t := range schema.Tables {
if strings.EqualFold(t.Name, tableName) {
table = t
break
}
}
}
if table == nil {
return nil, nil, "table " + qualified
}
found := make([]*models.Column, 0, len(cols))
for _, name := range cols {
col := table.Columns[name]
if col == nil {
for _, c := range table.Columns {
if strings.EqualFold(c.Name, name) {
col = c
break
}
}
}
if col == nil {
return nil, nil, "column " + qualified + "." + name
}
found = append(found, col)
}
return table, found, ""
}
func columnNames(cols []*models.Column) []string {
names := make([]string, len(cols))
for i, c := range cols {
names[i] = c.Name
}
return names
}
// hasFKOnColumns reports whether table already has a foreign key over cols.
func hasFKOnColumns(table *models.Table, cols []string) bool {
for _, c := range table.Constraints {
if c.Type != models.ForeignKeyConstraint || len(c.Columns) != len(cols) {
continue
}
same := true
for i := range cols {
if !strings.EqualFold(c.Columns[i], cols[i]) {
same = false
break
}
}
if same {
return true
}
}
return false
}
// fkTypeAliases maps serial and alias spellings to their storage type.
var fkTypeAliases = map[string]string{
"smallserial": "smallint", "serial2": "smallint", "int2": "smallint",
"serial": "integer", "serial4": "integer", "int": "integer", "int4": "integer",
"bigserial": "bigint", "serial8": "bigint", "int8": "bigint",
}
// compatibleFKTypes compares column types ignoring case, length and serial
// vs. integer spelling. Unknown (empty) types are treated as compatible.
func compatibleFKTypes(a, b string) bool {
na, nb := normalizeFKType(a), normalizeFKType(b)
return na == "" || nb == "" || na == nb
}
func normalizeFKType(t string) string {
t = strings.ToLower(strings.TrimSpace(t))
if i := strings.Index(t, "("); i >= 0 {
t = strings.TrimSpace(t[:i])
}
if alias, ok := fkTypeAliases[t]; ok {
return alias
}
return t
}
+260
View File
@@ -0,0 +1,260 @@
package dbml
import (
"os"
"path/filepath"
"strings"
"testing"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/readers"
)
func readDBMLString(t *testing.T, content string) *models.Database {
t.Helper()
path := filepath.Join(t.TempDir(), "in.dbml")
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatalf("write fixture: %v", err)
}
db, err := NewReader(&readers.ReaderOptions{FilePath: path}).ReadDatabase()
if err != nil {
t.Fatalf("ReadDatabase() error = %v", err)
}
return db
}
func findTable(db *models.Database, schema, table string) *models.Table {
for _, s := range db.Schemas {
if s.Name != schema {
continue
}
for _, t := range s.Tables {
if t.Name == table {
return t
}
}
}
return nil
}
func fkOn(table *models.Table, col string) *models.Constraint {
for _, c := range table.Constraints {
if c.Type == models.ForeignKeyConstraint && len(c.Columns) == 1 && c.Columns[0] == col {
return c
}
}
return nil
}
func relFor(table *models.Table, fkName string) *models.Relationship {
for _, r := range table.Relationships {
if r.ForeignKey == fkName {
return r
}
}
return nil
}
func TestCommentedRef(t *testing.T) {
tests := []struct {
line string
want string
ok bool
}{
{`// Ref: a.b.c > d.e.f`, `a.b.c > d.e.f`, true},
{`// ref: a.b.c - d.e.f`, `a.b.c - d.e.f`, true},
{`//Ref:a.b.c > d.e.f`, `a.b.c > d.e.f`, true},
{`// Reference notes`, "", false},
{`// see Ref: a.b.c > d.e.f`, "", false},
{`// Ref:`, "", false},
}
for _, tt := range tests {
got, ok := commentedRef(tt.line)
if ok != tt.ok || got != tt.want {
t.Errorf("commentedRef(%q) = (%q, %v), want (%q, %v)", tt.line, got, ok, tt.want, tt.ok)
}
}
}
// A commented ref whose tables are in the same file resolves on read.
func TestReader_CommentedRefSameFile(t *testing.T) {
db := readDBMLString(t, `Table "org"."department" {
"id_department" bigserial [pk]
}
Table "entity"."employee" {
"id_employee" bigserial [pk]
"rid_department" bigint
}
// Ref: "entity"."employee"."rid_department" > "org"."department"."id_department" [delete: restrict, update: restrict]
`)
emp := findTable(db, "entity", "employee")
fk := fkOn(emp, "rid_department")
if fk == nil {
t.Fatal("expected FK on rid_department")
}
if fk.ReferencedSchema != "org" || fk.ReferencedTable != "department" || fk.ReferencedColumns[0] != "id_department" {
t.Errorf("FK target = %s.%s.%v", fk.ReferencedSchema, fk.ReferencedTable, fk.ReferencedColumns)
}
if fk.OnDelete != "restrict" || fk.OnUpdate != "restrict" {
t.Errorf("FK actions = %q/%q, want restrict/restrict", fk.OnDelete, fk.OnUpdate)
}
if relFor(emp, fk.Name) == nil {
t.Error("expected relationship for FK")
}
if refs := PendingCommentedRefs(db); len(refs) != 0 {
t.Errorf("pending = %v, want none", refs)
}
}
// A cross-file commented ref stays pending after a single-file read.
func TestReader_CommentedRefCrossFilePending(t *testing.T) {
db := readDBMLString(t, `Table "entity"."employee" {
"id_employee" bigserial [pk]
"rid_department" bigint
}
// Ref: "entity"."employee"."rid_department" > "org"."department"."id_department"
`)
if fk := fkOn(findTable(db, "entity", "employee"), "rid_department"); fk != nil {
t.Fatal("FK must not resolve without the target table")
}
if refs := PendingCommentedRefs(db); len(refs) != 1 {
t.Fatalf("pending = %v, want 1", refs)
}
}
// Directory reads resolve commented refs after all files are merged.
func TestReader_CommentedRefDirectory(t *testing.T) {
db, err := NewReader(&readers.ReaderOptions{
FilePath: filepath.Join("..", "..", "..", "tests", "assets", "dbml", "multifile"),
}).ReadDatabase()
if err != nil {
t.Fatalf("ReadDatabase() error = %v", err)
}
fk := fkOn(findTable(db, "public", "posts"), "user_id")
if fk == nil {
t.Fatal("expected FK posts.user_id from 9_refs.dbml commented ref")
}
if fk.ReferencedTable != "users" || fk.OnDelete != "CASCADE" {
t.Errorf("FK = %s ondelete %s, want users ondelete CASCADE", fk.ReferencedTable, fk.OnDelete)
}
}
func crossFileDB(t *testing.T, refs ...string) *models.Database {
t.Helper()
db := readDBMLString(t, `Table "org"."department" {
"id_department" bigserial [pk]
}
Table "entity"."employee" {
"id_employee" bigserial [pk]
"rid_department" bigint
"rid_manager" integer
"rid_team" bigint
}
`)
setPendingCommentedRefs(db, refs)
return db
}
func TestResolveCommentedRefs(t *testing.T) {
t.Run("lowercase ref and one-to-one", func(t *testing.T) {
db := crossFileDB(t,
`"entity"."employee"."rid_department" > "org"."department"."id_department"`,
`entity.employee.rid_team - org.department.id_department`,
)
if w := ResolveCommentedRefs(db, true); len(w) != 0 {
t.Errorf("warnings = %v, want none", w)
}
emp := findTable(db, "entity", "employee")
for _, col := range []string{"rid_department", "rid_team"} {
fk := fkOn(emp, col)
if fk == nil {
t.Fatalf("expected FK on %s", col)
}
if relFor(emp, fk.Name) == nil {
t.Errorf("expected relationship for %s", fk.Name)
}
}
if len(emp.Relationships) != 2 {
t.Errorf("relationships = %d, want 2 (same target must not overwrite)", len(emp.Relationships))
}
})
t.Run("type mismatch warns but resolves", func(t *testing.T) {
db := crossFileDB(t, `entity.employee.rid_manager > org.department.id_department`)
w := ResolveCommentedRefs(db, true)
if len(w) != 1 || !strings.Contains(w[0], "type mismatch") {
t.Errorf("warnings = %v, want one type mismatch", w)
}
if fkOn(findTable(db, "entity", "employee"), "rid_manager") == nil {
t.Error("expected FK on rid_manager")
}
})
t.Run("missing target stays pending until final", func(t *testing.T) {
db := crossFileDB(t, `entity.employee.rid_team > hr.team.id_team`)
if w := ResolveCommentedRefs(db, false); len(w) != 0 {
t.Errorf("non-final warnings = %v, want none", w)
}
if len(PendingCommentedRefs(db)) != 1 {
t.Fatal("ref should stay pending")
}
w := ResolveCommentedRefs(db, true)
if len(w) != 1 || !strings.Contains(w[0], "table hr.team not found") {
t.Errorf("final warnings = %v, want missing table", w)
}
if len(PendingCommentedRefs(db)) != 0 {
t.Error("final pass must clear pending refs")
}
})
t.Run("missing column", func(t *testing.T) {
db := crossFileDB(t, `entity.employee.rid_nope > org.department.id_department`)
w := ResolveCommentedRefs(db, true)
if len(w) != 1 || !strings.Contains(w[0], "column entity.employee.rid_nope not found") {
t.Errorf("warnings = %v, want missing column", w)
}
})
t.Run("deduplicates against uncommented ref", func(t *testing.T) {
db := readDBMLString(t, `Table "org"."department" {
"id_department" bigserial [pk]
}
Table "entity"."employee" {
"id_employee" bigserial [pk]
"rid_department" bigint
}
Ref: "entity"."employee"."rid_department" > "org"."department"."id_department"
// Ref: "entity"."employee"."rid_department" > "org"."department"."id_department"
`)
emp := findTable(db, "entity", "employee")
count := 0
for _, c := range emp.Constraints {
if c.Type == models.ForeignKeyConstraint {
count++
}
}
if count != 1 || len(emp.Relationships) != 1 {
t.Errorf("FKs = %d, relationships = %d, want 1 and 1", count, len(emp.Relationships))
}
})
}
func TestCompatibleFKTypes(t *testing.T) {
tests := []struct {
a, b string
want bool
}{
{"bigint", "bigserial", true},
{"integer", "serial", true},
{"INT4", "integer", true},
{"varchar(10)", "varchar(20)", true},
{"bigint", "serial", false},
{"uuid", "bigint", false},
{"", "bigint", true},
}
for _, tt := range tests {
if got := compatibleFKTypes(tt.a, tt.b); got != tt.want {
t.Errorf("compatibleFKTypes(%q, %q) = %v, want %v", tt.a, tt.b, got, tt.want)
}
}
}
+38
View File
@@ -0,0 +1,38 @@
package dbml
import (
"regexp"
"strings"
"git.warky.dev/wdevs/relspecgo/pkg/models"
)
var (
generatedNoteRegex = regexp.MustCompile(`(?is)^GENERATED\s+ALWAYS\s+AS\s*\((.*)\)\s*STORED$`)
identityNoteRegex = regexp.MustCompile(`(?i)^GENERATED\s+(ALWAYS|BY\s+DEFAULT)\s+AS\s+IDENTITY$`)
)
// applyColumnMarkerNote interprets a column note written by the DBML writer for
// GENERATED ... STORED and GENERATED ... AS IDENTITY columns, setting the matching
// model fields. It returns true when the note was a marker, in which case it is not
// a user comment.
func applyColumnMarkerNote(column *models.Column, rawNote string) bool {
note := strings.TrimSpace(rawNote)
if len(note) >= 2 && (note[0] == '\'' || note[0] == '"') && note[len(note)-1] == note[0] {
note = note[1 : len(note)-1]
}
note = strings.ReplaceAll(strings.ReplaceAll(note, `\'`, `'`), `\\`, `\`)
note = strings.TrimSpace(note)
if m := generatedNoteRegex.FindStringSubmatch(note); m != nil {
column.Generated = true
column.GenerationExpression = strings.TrimSpace(m[1])
return true
}
if m := identityNoteRegex.FindStringSubmatch(note); m != nil {
column.Identity = true
column.IdentityGeneration = strings.Join(strings.Fields(strings.ToUpper(m[1])), " ")
return true
}
return false
}
+72 -15
View File
@@ -48,7 +48,22 @@ func (r *Reader) ReadDatabase() (*models.Database, error) {
return nil, fmt.Errorf("failed to read file: %w", err)
}
return r.parseDBML(string(content))
db, err := r.parseDBML(string(content))
if err != nil {
return nil, err
}
r.resolveCommentedRefs(db)
return db, nil
}
// resolveCommentedRefs resolves the commented refs whose tables are loaded.
// Unmatched refs stay pending for a later pass over a combined model.
func (r *Reader) resolveCommentedRefs(db *models.Database) {
for _, w := range ResolveCommentedRefs(db, false) {
if r.options.Progress != nil {
r.options.Progress("warning: " + w)
}
}
}
// ReadSchema reads and parses DBML input, returning a Schema model
@@ -125,6 +140,7 @@ func (r *Reader) readDirectoryDBML(dirPath string) (*models.Database, error) {
}
}
r.resolveCommentedRefs(db)
return db, nil
}
@@ -440,6 +456,10 @@ func mergeDatabase(baseDB, fileDB *models.Database) {
// Merge domains
baseDB.Domains = append(baseDB.Domains, fileDB.Domains...)
for _, ref := range PendingCommentedRefs(fileDB) {
addPendingCommentedRef(baseDB, ref)
}
// Use first non-empty description
if baseDB.Description == "" && fileDB.Description != "" {
baseDB.Description = fileDB.Description
@@ -493,8 +513,12 @@ func (r *Reader) parseDBML(content string) (*models.Database, error) {
continue
}
// Skip empty lines and comments
// Skip empty lines and comments. A commented `// Ref:` is kept as a
// pending cross-file ref, resolved once every file is loaded.
if line == "" || strings.HasPrefix(line, "//") {
if ref, ok := commentedRef(line); ok {
addPendingCommentedRef(db, ref)
}
continue
}
@@ -581,7 +605,13 @@ func (r *Reader) parseDBML(content string) (*models.Database, error) {
index := r.parseIndex(line, currentTable.Name, currentSchema)
if index != nil {
currentTable.Indexes[index.Name] = index
// Keep a reused name under a unique map key so the duplicate is
// not silently dropped; the inspector reports it.
key := index.Name
for n := 2; currentTable.Indexes[key] != nil; n++ {
key = fmt.Sprintf("%s#%d", index.Name, n)
}
currentTable.Indexes[key] = index
lastIndex = index
}
continue
@@ -659,20 +689,12 @@ func (r *Reader) parseDBML(content string) (*models.Database, error) {
// for DBML refs so diffing equivalent schemas compares the same model.
for _, schema := range schemaMap {
for _, table := range schema.Tables {
for _, constraint := range table.Constraints {
for _, name := range sortedConstraintNames(table.Constraints) {
constraint := table.Constraints[name]
if constraint.Type != models.ForeignKeyConstraint {
continue
}
name := fmt.Sprintf("%s_to_%s", table.Name, constraint.ReferencedTable)
relationship := models.InitRelationship(name, models.OneToMany)
relationship.FromTable = table.Name
relationship.FromSchema = table.Schema
relationship.FromColumns = append([]string(nil), constraint.Columns...)
relationship.ToTable = constraint.ReferencedTable
relationship.ToSchema = constraint.ReferencedSchema
relationship.ToColumns = append([]string(nil), constraint.ReferencedColumns...)
relationship.ForeignKey = constraint.Name
table.Relationships[name] = relationship
addFKRelationship(table, constraint)
}
}
}
@@ -685,6 +707,39 @@ func (r *Reader) parseDBML(content string) (*models.Database, error) {
return db, nil
}
// sortedConstraintNames returns constraint keys in sorted order so derived
// relationship names do not depend on map iteration order.
func sortedConstraintNames(constraints map[string]*models.Constraint) []string {
names := make([]string, 0, len(constraints))
for name := range constraints {
names = append(names, name)
}
sort.Strings(names)
return names
}
// addFKRelationship derives the relationship for a foreign key, matching how
// the PostgreSQL reader models FKs.
func addFKRelationship(table *models.Table, constraint *models.Constraint) {
name := fmt.Sprintf("%s_to_%s", table.Name, constraint.ReferencedTable)
if existing, taken := table.Relationships[name]; taken && existing.ForeignKey != constraint.Name {
// A second FK to the same table must not overwrite the first.
name = fmt.Sprintf("%s_%s", name, strings.Join(constraint.Columns, "_"))
}
relationship := models.InitRelationship(name, models.OneToMany)
relationship.FromTable = table.Name
relationship.FromSchema = table.Schema
relationship.FromColumns = append([]string(nil), constraint.Columns...)
relationship.ToTable = constraint.ReferencedTable
relationship.ToSchema = constraint.ReferencedSchema
relationship.ToColumns = append([]string(nil), constraint.ReferencedColumns...)
relationship.ForeignKey = constraint.Name
if table.Relationships == nil {
table.Relationships = make(map[string]*models.Relationship)
}
table.Relationships[name] = relationship
}
// setTableNote preserves multiple table notes. The first maps to Description
// and the second to Comment, matching the model fields used by code writers.
func setTableNote(table *models.Table, note string) {
@@ -749,7 +804,9 @@ func (r *Reader) parseColumn(line, tableName, schemaName string) (*models.Column
} else if strings.HasPrefix(attr, "note:") {
// Parse column note/comment
note := strings.TrimSpace(strings.TrimPrefix(attr, "note:"))
column.Comment = strings.Trim(note, "'\"")
if !applyColumnMarkerNote(column, note) {
column.Comment = strings.Trim(note, "'\"")
}
} else if strings.HasPrefix(attr, "ref:") {
// Parse inline reference
// DBML semantics depend on context:
+36
View File
@@ -1085,3 +1085,39 @@ func TestReader_MultilineTableNote(t *testing.T) {
t.Errorf("column note = %q, want %q", got, want)
}
}
// A name reused inside one Indexes block must not silently drop the earlier
// index; both are kept so the inspector can report the duplicate.
func TestReader_DuplicateIndexNameInTableKept(t *testing.T) {
dbmlContent := `Table individual_actor {
id bigint [pk]
rid_actor bigint
kind text
Indexes {
(rid_actor) [name: 'idx_actor', unique]
(rid_actor, kind) [name: 'idx_actor']
}
}
`
dir := t.TempDir()
path := filepath.Join(dir, "dup_index.dbml")
if err := os.WriteFile(path, []byte(dbmlContent), 0o644); err != nil {
t.Fatalf("failed to write fixture: %v", err)
}
db, err := NewReader(&readers.ReaderOptions{FilePath: path}).ReadDatabase()
if err != nil {
t.Fatalf("ReadDatabase() error = %v", err)
}
table := db.Schemas[0].Tables[0]
if len(table.Indexes) != 2 {
t.Fatalf("expected 2 indexes, got %d: %v", len(table.Indexes), table.Indexes)
}
for key, idx := range table.Indexes {
if idx.Name != "idx_actor" {
t.Errorf("index %q: Name = %q, want idx_actor", key, idx.Name)
}
}
}
+1
View File
@@ -85,6 +85,7 @@ export const postsRelations = relations(posts, ({ one }) => ({
## Notes
- `.generatedAlwaysAs(sql`expr`)` sets `Generated` + `GenerationExpression`; `.generatedAlwaysAsIdentity()` / `.generatedByDefaultAsIdentity()` set `Identity` + `IdentityGeneration`
- Supports both PostgreSQL and MySQL Drizzle schemas
- Extracts relationship information from `relations` definitions
- Schema defaults to `public` for PostgreSQL
+47 -1
View File
@@ -495,9 +495,24 @@ func (r *Reader) parseColumnModifiers(line string, column *models.Column, table
}
}
// Check for .generatedAlwaysAsIdentity()
// Check for .generatedAlwaysAsIdentity() / .generatedByDefaultAsIdentity()
if strings.Contains(line, ".generatedAlwaysAsIdentity()") {
column.AutoIncrement = true
column.Identity = true
column.IdentityGeneration = "ALWAYS"
}
if strings.Contains(line, ".generatedByDefaultAsIdentity()") {
column.AutoIncrement = true
column.Identity = true
column.IdentityGeneration = "BY DEFAULT"
}
// Check for .generatedAlwaysAs(sql`expr`) (generated column)
if idx := strings.Index(line, ".generatedAlwaysAs("); idx != -1 {
if expr, ok := parseGeneratedAlwaysAs(line[idx+len(".generatedAlwaysAs("):]); ok {
column.Generated = true
column.GenerationExpression = expr
}
}
// Check for .references(() => otherTable.column)
@@ -615,3 +630,34 @@ func (r *Reader) varNameToTableName(varName string) string {
// For now, assume variable name matches table name
return varName
}
// parseGeneratedAlwaysAs extracts the SQL expression from the argument list of
// generatedAlwaysAs(...), which follows the opening parenthesis in rest. The expression is
// the contents of a sql`...` template (backticks inside it are escaped with a backslash)
// or, failing that, a quoted string.
func parseGeneratedAlwaysAs(rest string) (string, bool) {
rest = strings.TrimSpace(rest)
rest = strings.TrimPrefix(rest, "sql")
if rest == "" {
return "", false
}
quote := rest[0]
if quote != '`' && quote != '\'' && quote != '"' {
return "", false
}
var sb strings.Builder
for i := 1; i < len(rest); i++ {
ch := rest[i]
if ch == '\\' && i+1 < len(rest) {
i++
sb.WriteByte(rest[i])
continue
}
if ch == quote {
return strings.TrimSpace(sb.String()), true
}
sb.WriteByte(ch)
}
return "", false
}
+2
View File
@@ -78,6 +78,8 @@ The reader recognizes the following GORM struct tags:
- `not null` - NOT NULL constraint
- `autoIncrement` - Auto-increment column
- `default` - Default value
- `generated` - Generated column marker (`Generated`; expression is not in the tag)
- `identity` - Identity column marker (`Identity`, `ALWAYS`)
- `size` - Column size/length
- `index` - Create index
- `uniqueIndex` - Create unique index
+48
View File
@@ -0,0 +1,48 @@
package gorm
import (
"os"
"path/filepath"
"testing"
"git.warky.dev/wdevs/relspecgo/pkg/readers"
)
func TestReader_GeneratedAndIdentityMarkers(t *testing.T) {
src := `package models
type Person struct {
ID int64 ` + "`gorm:\"column:id;primaryKey;autoIncrement;type:bigint;identity\"`" + `
Seq int64 ` + "`gorm:\"column:seq;type:bigint;<-:false;identity;not null\"`" + `
FullName string ` + "`gorm:\"column:full_name;type:text;<-:false;generated\"`" + `
First string ` + "`gorm:\"column:first;type:text\"`" + `
}
func (Person) TableName() string {
return "people"
}
`
path := filepath.Join(t.TempDir(), "person.go")
if err := os.WriteFile(path, []byte(src), 0o644); err != nil {
t.Fatal(err)
}
db, err := NewReader(&readers.ReaderOptions{FilePath: path}).ReadDatabase()
if err != nil {
t.Fatalf("ReadDatabase() error = %v", err)
}
cols := db.Schemas[0].Tables[0].Columns
if !cols["id"].Identity || cols["id"].IdentityGeneration != "ALWAYS" {
t.Errorf("id: identity = %v/%q, want true/ALWAYS", cols["id"].Identity, cols["id"].IdentityGeneration)
}
if !cols["seq"].Identity {
t.Error("seq should be identity")
}
if !cols["full_name"].Generated {
t.Error("full_name should be generated")
}
if cols["first"].Generated || cols["first"].Identity {
t.Error("first must be an ordinary column")
}
}
+9
View File
@@ -726,6 +726,15 @@ func (r *Reader) parseColumn(fieldName string, fieldType ast.Expr, tag string, s
if _, ok := parts["autoincrement"]; ok {
column.AutoIncrement = true
}
if _, ok := parts["generated"]; ok {
// GENERATED ... STORED marker written by the GORM writer; the expression is not in the tag
column.Generated = true
}
if _, ok := parts["identity"]; ok {
// GENERATED ALWAYS AS IDENTITY marker written by the GORM writer
column.Identity = true
column.IdentityGeneration = "ALWAYS"
}
if def, ok := parts["default"]; ok {
// Default value from GORM tag (e.g., default:gen_random_uuid())
column.Default = def
+5
View File
@@ -25,6 +25,11 @@ sqlserver://user:pass@192.168.1.100:1433/production
sqlserver://localhost/testdb?encrypt=disable
```
## Computed and Identity Columns
- `sys.computed_columns` -> `Generated` + `GenerationExpression` (outer parentheses removed; persisted or not)
- Identity columns -> `Identity` + `IdentityGeneration = ALWAYS`
## Supported Constraints
- Primary Keys
+37 -2
View File
@@ -104,8 +104,12 @@ func (r *Reader) queryColumns(schemaName string) (map[string]map[string]*models.
c.numeric_precision,
c.numeric_scale,
ISNULL(ep.value, '') as description,
COLUMNPROPERTY(OBJECT_ID(QUOTENAME(c.table_schema) + '.' + QUOTENAME(c.table_name)), c.column_name, 'IsIdentity') as is_identity
COLUMNPROPERTY(OBJECT_ID(QUOTENAME(c.table_schema) + '.' + QUOTENAME(c.table_name)), c.column_name, 'IsIdentity') as is_identity,
cc.definition as computed_definition
FROM information_schema.columns c
LEFT JOIN sys.computed_columns cc
ON cc.object_id = OBJECT_ID(QUOTENAME(c.table_schema) + '.' + QUOTENAME(c.table_name))
AND cc.name = c.column_name
LEFT JOIN sys.extended_properties ep
ON ep.major_id = OBJECT_ID(QUOTENAME(c.table_schema) + '.' + QUOTENAME(c.table_name))
AND ep.minor_id = COLUMNPROPERTY(OBJECT_ID(QUOTENAME(c.table_schema) + '.' + QUOTENAME(c.table_name)), c.column_name, 'ColumnId')
@@ -127,8 +131,9 @@ func (r *Reader) queryColumns(schemaName string) (map[string]map[string]*models.
var schema, tableName, columnName, isNullable, dataType, description string
var ordinalPosition int
var columnDefault, charMaxLength, numPrecision, numScale, isIdentity *int
var computedDefinition *string
if err := rows.Scan(&schema, &tableName, &columnName, &ordinalPosition, &columnDefault, &isNullable, &dataType, &charMaxLength, &numPrecision, &numScale, &description, &isIdentity); err != nil {
if err := rows.Scan(&schema, &tableName, &columnName, &ordinalPosition, &columnDefault, &isNullable, &dataType, &charMaxLength, &numPrecision, &numScale, &description, &isIdentity, &computedDefinition); err != nil {
return nil, err
}
@@ -144,6 +149,14 @@ func (r *Reader) queryColumns(schemaName string) (map[string]map[string]*models.
// Check if this is an identity column (auto-increment)
if isIdentity != nil && *isIdentity == 1 {
column.AutoIncrement = true
column.Identity = true
column.IdentityGeneration = "ALWAYS"
}
// Computed columns report their expression wrapped in an extra pair of parentheses
if computedDefinition != nil && strings.TrimSpace(*computedDefinition) != "" {
column.Generated = true
column.GenerationExpression = trimOuterParens(*computedDefinition)
}
if charMaxLength != nil && *charMaxLength > 0 {
@@ -414,3 +427,25 @@ func (r *Reader) queryIndexes(schemaName string) (map[string][]*models.Index, er
return indexes, rows.Err()
}
// trimOuterParens removes one pair of parentheses wrapping the whole expression, as
// SQL Server stores computed column definitions.
func trimOuterParens(expr string) string {
expr = strings.TrimSpace(expr)
if len(expr) < 2 || expr[0] != '(' || expr[len(expr)-1] != ')' {
return expr
}
depth := 0
for i := 0; i < len(expr); i++ {
switch expr[i] {
case '(':
depth++
case ')':
depth--
if depth == 0 && i != len(expr)-1 {
return expr
}
}
}
return strings.TrimSpace(expr[1 : len(expr)-1])
}
+15
View File
@@ -85,3 +85,18 @@ func TestConvertMSSQLToCanonical(t *testing.T) {
})
}
}
func TestTrimOuterParens(t *testing.T) {
tests := map[string]string{
"(([a])+([b]))": "([a])+([b])",
"([a]+[b])": "[a]+[b]",
"([a])+([b])": "([a])+([b])",
"[a]+[b]": "[a]+[b]",
" (upper([a])) ": "upper([a])",
}
for in, want := range tests {
if got := trimOuterParens(in); got != want {
t.Errorf("trimOuterParens(%q) = %q, want %q", in, got, want)
}
}
}
+72 -27
View File
@@ -243,7 +243,11 @@ func (r *Reader) queryColumns(schemaName string) (map[string]map[string]*models.
c.numeric_scale,
c.udt_name,
pg_catalog.format_type(a.atttypid, a.atttypmod) as formatted_data_type,
col_description((c.table_schema||'.'||c.table_name)::regclass, c.ordinal_position) as description
col_description((c.table_schema||'.'||c.table_name)::regclass, c.ordinal_position) as description,
c.is_generated,
c.generation_expression,
c.is_identity,
c.identity_generation
FROM information_schema.columns c
JOIN pg_catalog.pg_namespace n
ON n.nspname = c.table_schema
@@ -268,17 +272,36 @@ func (r *Reader) queryColumns(schemaName string) (map[string]map[string]*models.
columnsMap := make(map[string]map[string]*models.Column)
for rows.Next() {
var schema, tableName, columnName, isNullable, dataType, udtName, formattedDataType string
var schema, tableName, columnName, isNullable, dataType, udtName, formattedDataType, isGenerated, isIdentity string
var ordinalPosition int
var columnDefault, description *string
var columnDefault, description, generationExpression, identityGeneration *string
var charMaxLength, numPrecision, numScale *int
if err := rows.Scan(&schema, &tableName, &columnName, &ordinalPosition, &columnDefault, &isNullable, &dataType, &charMaxLength, &numPrecision, &numScale, &udtName, &formattedDataType, &description); err != nil {
if err := rows.Scan(&schema, &tableName, &columnName, &ordinalPosition, &columnDefault, &isNullable, &dataType, &charMaxLength, &numPrecision, &numScale, &udtName, &formattedDataType, &description, &isGenerated, &generationExpression, &isIdentity, &identityGeneration); err != nil {
return nil, err
}
column := models.InitColumn(columnName, tableName, schema)
// GENERATED ALWAYS ... STORED columns are computed by Postgres and cannot
// have their default altered/dropped like a regular column.
column.Generated = isGenerated == "ALWAYS"
if generationExpression != nil {
column.GenerationExpression = *generationExpression
}
// GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY columns are driven by an
// internal sequence rather than a literal default (unlike serial columns,
// they carry no pg_attrdef row at all), so they need the same DB-side
// handling as generated columns even though the underlying mechanism differs.
column.Identity = isIdentity == "YES"
if identityGeneration != nil {
column.IdentityGeneration = strings.ToUpper(strings.TrimSpace(*identityGeneration))
}
if column.Identity {
column.AutoIncrement = true
}
// Check if this is a serial type (has nextval default)
hasNextval := false
if columnDefault != nil {
@@ -301,6 +324,24 @@ func (r *Reader) queryColumns(schemaName string) (map[string]map[string]*models.
column.Description = *description
}
if column.Generated {
note := "GENERATED ALWAYS AS (" + column.GenerationExpression + ") STORED"
if column.Description != "" {
column.Description = column.Description + " " + note
} else {
column.Description = note
}
}
if column.Identity {
note := "GENERATED " + column.IdentityGeneration + " AS IDENTITY"
if column.Description != "" {
column.Description = column.Description + " " + note
} else {
column.Description = note
}
}
if charMaxLength != nil {
column.Length = *charMaxLength
}
@@ -381,31 +422,35 @@ func (r *Reader) queryPrimaryKeys(schemaName string) (map[string]*models.Constra
// queryForeignKeys retrieves all foreign key constraints for a schema
// Returns map[schema.table][]*Constraint
func (r *Reader) queryForeignKeys(schemaName string) (map[string][]*models.Constraint, error) {
// Columns are paired by position from pg_constraint so composite keys are not
// cross-joined (information_schema.constraint_column_usage has no ordering).
actionCase := func(col string) string {
return `CASE ` + col + ` WHEN 'a' THEN 'NO ACTION' WHEN 'r' THEN 'RESTRICT' WHEN 'c' THEN 'CASCADE' WHEN 'n' THEN 'SET NULL' WHEN 'd' THEN 'SET DEFAULT' END`
}
query := `
SELECT
tc.table_schema,
tc.table_name,
tc.constraint_name,
kcu.table_schema as foreign_table_schema,
kcu.table_name as foreign_table_name,
kcu.column_name as foreign_column,
ccu.table_schema as referenced_table_schema,
ccu.table_name as referenced_table_name,
ccu.column_name as referenced_column,
rc.update_rule,
rc.delete_rule
FROM information_schema.table_constraints tc
JOIN information_schema.key_column_usage kcu
ON tc.constraint_name = kcu.constraint_name
AND tc.table_schema = kcu.table_schema
JOIN information_schema.constraint_column_usage ccu
ON ccu.constraint_name = tc.constraint_name
JOIN information_schema.referential_constraints rc
ON rc.constraint_name = tc.constraint_name
AND rc.constraint_schema = tc.table_schema
WHERE tc.constraint_type = 'FOREIGN KEY'
AND tc.table_schema = $1
ORDER BY tc.table_schema, tc.table_name, tc.constraint_name, kcu.ordinal_position
ns.nspname AS table_schema,
cl.relname AS table_name,
con.conname AS constraint_name,
ns.nspname AS foreign_table_schema,
cl.relname AS foreign_table_name,
att.attname AS foreign_column,
fns.nspname AS referenced_table_schema,
fcl.relname AS referenced_table_name,
fatt.attname AS referenced_column,
` + actionCase("con.confupdtype") + ` AS update_rule,
` + actionCase("con.confdeltype") + ` AS delete_rule
FROM pg_catalog.pg_constraint con
JOIN pg_catalog.pg_class cl ON cl.oid = con.conrelid
JOIN pg_catalog.pg_namespace ns ON ns.oid = cl.relnamespace
JOIN pg_catalog.pg_class fcl ON fcl.oid = con.confrelid
JOIN pg_catalog.pg_namespace fns ON fns.oid = fcl.relnamespace
CROSS JOIN LATERAL unnest(con.conkey, con.confkey) WITH ORDINALITY AS k(attnum, fattnum, ord)
JOIN pg_catalog.pg_attribute att ON att.attrelid = con.conrelid AND att.attnum = k.attnum
JOIN pg_catalog.pg_attribute fatt ON fatt.attrelid = con.confrelid AND fatt.attnum = k.fattnum
WHERE con.contype = 'f'
AND ns.nspname = $1
ORDER BY ns.nspname, cl.relname, con.conname, k.ord
`
rows, err := r.conn.Query(r.ctx, query, schemaName)
+61
View File
@@ -1,10 +1,14 @@
package pgsql
import (
"context"
"os"
"reflect"
"strings"
"testing"
"github.com/jackc/pgx/v5"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/readers"
)
@@ -499,3 +503,60 @@ func TestMapDataType_ExtensionTypesPreserveModifiers(t *testing.T) {
})
}
}
func TestReader_CompositeForeignKeyColumnsArePairedOnce(t *testing.T) {
connStr := getTestConnectionString(t)
ctx := context.Background()
conn, err := pgx.Connect(ctx, connStr)
if err != nil {
t.Fatalf("connect: %v", err)
}
defer conn.Close(ctx)
const schema = "relspec_fk_test"
setup := []string{
"DROP SCHEMA IF EXISTS " + schema + " CASCADE",
"CREATE SCHEMA " + schema,
"CREATE TABLE " + schema + ".parent (a int, b int, PRIMARY KEY (a, b))",
"CREATE TABLE " + schema + ".child (x int, y int, CONSTRAINT fk_child_parent FOREIGN KEY (x, y) REFERENCES " + schema + ".parent (a, b) ON DELETE CASCADE)",
}
for _, stmt := range setup {
if _, err := conn.Exec(ctx, stmt); err != nil {
t.Fatalf("setup %q: %v", stmt, err)
}
}
defer conn.Exec(ctx, "DROP SCHEMA IF EXISTS "+schema+" CASCADE")
reader := NewReader(&readers.ReaderOptions{ConnectionString: connStr})
db, err := reader.ReadDatabase()
if err != nil {
t.Fatalf("ReadDatabase: %v", err)
}
for _, s := range db.Schemas {
if s.Name != schema {
continue
}
for _, tbl := range s.Tables {
if tbl.Name != "child" {
continue
}
fk := tbl.Constraints["fk_child_parent"]
if fk == nil {
t.Fatal("foreign key fk_child_parent not read")
}
if got := strings.Join(fk.Columns, ","); got != "x,y" {
t.Errorf("columns = %q, want x,y", got)
}
if got := strings.Join(fk.ReferencedColumns, ","); got != "a,b" {
t.Errorf("referenced columns = %q, want a,b", got)
}
if fk.OnDelete != "CASCADE" || fk.OnUpdate != "NO ACTION" {
t.Errorf("rules = %s/%s, want CASCADE/NO ACTION", fk.OnDelete, fk.OnUpdate)
}
return
}
}
t.Fatal("test schema/table not found in read result")
}
+1
View File
@@ -40,6 +40,7 @@ options := &readers.ReaderOptions{
- Uses pure Go driver (modernc.org/sqlite) - no CGo required
- Supports both file path and connection string
- Auto-increment detection for INTEGER PRIMARY KEY columns
- Generated columns (virtual and stored) are read via `PRAGMA table_xinfo`; the expression is parsed from the `CREATE TABLE` SQL
- Foreign keys require `PRAGMA foreign_keys = ON` to be set
## Example Schema
+116
View File
@@ -0,0 +1,116 @@
package sqlite
import (
"regexp"
"strings"
)
var generatedAsRegex = regexp.MustCompile(`(?is)\bAS\s*\(`)
// parseGeneratedExpression extracts the expression of a generated column from a CREATE
// TABLE statement, e.g. `full TEXT GENERATED ALWAYS AS (a || b) STORED` yields `a || b`.
// It returns "" when the column or its expression cannot be found.
func parseGeneratedExpression(createSQL, columnName string) string {
open := strings.Index(createSQL, "(")
if open < 0 {
return ""
}
for _, def := range splitTopLevel(createSQL[open+1:]) {
if !strings.EqualFold(firstIdentifier(def), columnName) {
continue
}
loc := generatedAsRegex.FindStringIndex(def)
if loc == nil {
return ""
}
return balancedParens(def[loc[1]-1:])
}
return ""
}
// splitTopLevel splits a table body on commas that are outside parentheses and quotes,
// stopping at the parenthesis that closes the body.
func splitTopLevel(body string) []string {
var parts []string
depth := 0
var quote byte
start := 0
for i := 0; i < len(body); i++ {
ch := body[i]
if quote != 0 {
if ch == quote {
quote = 0
}
continue
}
switch ch {
case '\'', '"', '`':
quote = ch
case '[':
quote = ']'
case '(':
depth++
case ')':
if depth == 0 {
return append(parts, body[start:i])
}
depth--
case ',':
if depth == 0 {
parts = append(parts, body[start:i])
start = i + 1
}
}
}
return append(parts, body[start:])
}
// firstIdentifier returns the first (possibly quoted) identifier of a column definition.
func firstIdentifier(def string) string {
def = strings.TrimSpace(def)
if def == "" {
return ""
}
switch def[0] {
case '"', '\'', '`':
if end := strings.IndexByte(def[1:], def[0]); end >= 0 {
return def[1 : 1+end]
}
case '[':
if end := strings.IndexByte(def, ']'); end >= 0 {
return def[1:end]
}
}
if end := strings.IndexAny(def, " \t\r\n"); end >= 0 {
return def[:end]
}
return def
}
// balancedParens returns the text inside the parenthesis group that starts at s[0].
func balancedParens(s string) string {
depth := 0
var quote byte
for i := 0; i < len(s); i++ {
ch := s[i]
if quote != 0 {
if ch == quote {
quote = 0
}
continue
}
switch ch {
case '\'', '"':
quote = ch
case '(':
depth++
case ')':
depth--
if depth == 0 {
return strings.TrimSpace(s[1:i])
}
}
}
return ""
}
+64
View File
@@ -0,0 +1,64 @@
package sqlite
import (
"database/sql"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"git.warky.dev/wdevs/relspecgo/pkg/readers"
)
func TestParseGeneratedExpression(t *testing.T) {
createSQL := `CREATE TABLE people (
id INTEGER PRIMARY KEY,
"first" TEXT,
last TEXT,
full_name TEXT GENERATED ALWAYS AS (coalesce("first", '') || ' ' || last) STORED,
initials TEXT AS (substr("first", 1, 1) || substr(last, 1, 1)),
plain TEXT NOT NULL
)`
tests := []struct {
column string
want string
}{
{"full_name", `coalesce("first", '') || ' ' || last`},
{"initials", `substr("first", 1, 1) || substr(last, 1, 1)`},
{"plain", ""},
{"missing", ""},
}
for _, tt := range tests {
t.Run(tt.column, func(t *testing.T) {
assert.Equal(t, tt.want, parseGeneratedExpression(createSQL, tt.column))
})
}
}
func TestReader_GeneratedColumns(t *testing.T) {
dbPath := filepath.Join(t.TempDir(), "gen.db")
db, err := sql.Open("sqlite", dbPath)
require.NoError(t, err)
_, err = db.Exec(`CREATE TABLE people (
id INTEGER PRIMARY KEY,
first TEXT,
last TEXT,
full_name TEXT GENERATED ALWAYS AS (first || ' ' || last) STORED,
initials TEXT AS (substr(first, 1, 1) || substr(last, 1, 1))
)`)
require.NoError(t, err)
require.NoError(t, db.Close())
got, err := NewReader(&readers.ReaderOptions{FilePath: dbPath}).ReadDatabase()
require.NoError(t, err)
cols := got.Schemas[0].Tables[0].Columns
require.Contains(t, cols, "full_name", "generated columns must be read")
assert.True(t, cols["full_name"].Generated)
assert.Equal(t, "first || ' ' || last", cols["full_name"].GenerationExpression)
assert.True(t, cols["initials"].Generated)
assert.Equal(t, "substr(first, 1, 1) || substr(last, 1, 1)", cols["initials"].GenerationExpression)
assert.False(t, cols["first"].Generated)
}
+29 -4
View File
@@ -75,7 +75,8 @@ func (r *Reader) queryViews() ([]*models.View, error) {
// queryColumns retrieves all columns for a given table or view
func (r *Reader) queryColumns(tableName string) (map[string]*models.Column, error) {
query := fmt.Sprintf("PRAGMA table_info(%s)", tableName)
// table_xinfo, unlike table_info, also lists generated columns (hidden = 2 virtual, 3 stored)
query := fmt.Sprintf("PRAGMA table_xinfo(%s)", tableName)
rows, err := r.db.QueryContext(r.ctx, query)
if err != nil {
@@ -84,24 +85,38 @@ func (r *Reader) queryColumns(tableName string) (map[string]*models.Column, erro
defer rows.Close()
columns := make(map[string]*models.Column)
var tableSQL string
tableSQLLoaded := false
for rows.Next() {
var cid int
var name, dataType string
var notNull, pk int
var notNull, pk, hidden int
var defaultValue *string
if err := rows.Scan(&cid, &name, &dataType, &notNull, &defaultValue, &pk); err != nil {
if err := rows.Scan(&cid, &name, &dataType, &notNull, &defaultValue, &pk, &hidden); err != nil {
return nil, err
}
// Hidden virtual-table columns (hidden = 1) are not part of the schema
if hidden == 1 {
continue
}
column := models.InitColumn(name, tableName, "main")
column.Type = r.mapDataType(strings.ToUpper(dataType))
column.NotNull = (notNull == 1)
column.IsPrimaryKey = (pk > 0)
column.Sequence = uint(cid + 1)
if defaultValue != nil {
if hidden == 2 || hidden == 3 {
column.Generated = true
if !tableSQLLoaded {
tableSQL = r.tableSQL(tableName)
tableSQLLoaded = true
}
column.GenerationExpression = parseGeneratedExpression(tableSQL, name)
} else if defaultValue != nil {
column.Default = *defaultValue
}
@@ -116,6 +131,16 @@ func (r *Reader) queryColumns(tableName string) (map[string]*models.Column, erro
return columns, rows.Err()
}
// tableSQL returns the CREATE TABLE statement of a table, or "" when it cannot be read.
func (r *Reader) tableSQL(tableName string) string {
var sql string
err := r.db.QueryRowContext(r.ctx, `SELECT sql FROM sqlite_master WHERE type = 'table' AND name = ?`, tableName).Scan(&sql)
if err != nil {
return ""
}
return sql
}
// isAutoIncrement checks if a column is autoincrement
func (r *Reader) isAutoIncrement(tableName, columnName string) bool {
// Check sqlite_sequence table or parse CREATE TABLE statement
+2
View File
@@ -114,6 +114,8 @@ export class Post {
- `@JoinColumn()` - Foreign key column
- `@Index()` - Index definition
- `@Unique()` - Unique constraint
- `asExpression` / `generatedType` - Generated column (`Generated` + `GenerationExpression`)
- `@PrimaryGeneratedColumn('identity')`, `@Generated('identity')`, `generatedIdentity` - Identity column
## Notes
+94 -4
View File
@@ -128,7 +128,6 @@ func (r *Reader) extractEntities(content string) []entityInfo {
scanner := bufio.NewScanner(strings.NewReader(content))
entityRegex := regexp.MustCompile(`^export\s+class\s+(\w+)`)
decoratorRegex := regexp.MustCompile(`^\s*@(\w+)(\([^)]*\))?`)
fieldRegex := regexp.MustCompile(`^\s*(\w+):\s*([^;]+);`)
var currentEntity *entityInfo
@@ -145,8 +144,7 @@ func (r *Reader) extractEntities(content string) []entityInfo {
}
// Check for decorator
if matches := decoratorRegex.FindStringSubmatch(trimmed); matches != nil {
decorator := matches[0]
if decorator, ok := matchDecorator(trimmed); ok {
pendingDecorators = append(pendingDecorators, decorator)
continue
}
@@ -488,7 +486,11 @@ func (r *Reader) parseColumnDecorator(decorator string, column *models.Column, t
column.IsPrimaryKey = true
column.NotNull = true
if strings.Contains(decorator, "'uuid'") {
if strings.Contains(decorator, "'identity'") {
column.AutoIncrement = true
column.Identity = true
column.IdentityGeneration = parseGeneratedIdentity(decorator)
} else if strings.Contains(decorator, "'uuid'") {
column.Type = "uuid"
column.Default = "gen_random_uuid()"
} else if strings.Contains(decorator, "'increment'") || strings.Contains(decorator, "()") {
@@ -497,6 +499,17 @@ func (r *Reader) parseColumnDecorator(decorator string, column *models.Column, t
return
}
// @Generated('identity') on a non-key column; generatedIdentity is read from @Column
if strings.HasPrefix(decorator, "@Generated") {
if strings.Contains(decorator, "'identity'") {
column.Identity = true
if column.IdentityGeneration == "" {
column.IdentityGeneration = "BY DEFAULT"
}
}
return
}
// @Column
if strings.HasPrefix(decorator, "@Column") {
r.parseColumnOptions(decorator, column, table)
@@ -586,6 +599,14 @@ func (r *Reader) parseColumnOptions(decorator string, column *models.Column, tab
}
}
if matches := asExpressionRegex.FindStringSubmatch(content); matches != nil {
column.Generated = true
column.GenerationExpression = unescapeSingleQuoted(matches[1])
}
if strings.Contains(content, "generatedIdentity") {
column.IdentityGeneration = parseGeneratedIdentity(content)
}
if strings.Contains(content, "nullable: true") || strings.Contains(content, "nullable:true") {
column.NotNull = false
}
@@ -834,3 +855,72 @@ func (r *Reader) getPrimaryKeyColumn(table *models.Table) *models.Column {
return pk
}
var (
asExpressionRegex = regexp.MustCompile(`asExpression:\s*'((?:\\.|[^\\'])*)'`)
generatedIdentityRegexp = regexp.MustCompile(`generatedIdentity:\s*['"](ALWAYS|BY DEFAULT)['"]`)
)
// parseGeneratedIdentity returns the identity generation mode named in a decorator
// ("ALWAYS" or "BY DEFAULT"), defaulting to "BY DEFAULT" as TypeORM does.
func parseGeneratedIdentity(decorator string) string {
if matches := generatedIdentityRegexp.FindStringSubmatch(decorator); matches != nil {
return matches[1]
}
return "BY DEFAULT"
}
// unescapeSingleQuoted reverses escaping applied inside a quoted TypeScript string.
func unescapeSingleQuoted(s string) string {
var sb strings.Builder
for i := 0; i < len(s); i++ {
if s[i] == '\\' && i+1 < len(s) {
i++
}
sb.WriteByte(s[i])
}
return sb.String()
}
var decoratorNameRegex = regexp.MustCompile(`^@\w+`)
// matchDecorator returns the decorator at the start of line, including its argument list.
// Parentheses inside quoted strings (e.g. a generated column expression) do not end it.
func matchDecorator(line string) (string, bool) {
name := decoratorNameRegex.FindString(line)
if name == "" {
return "", false
}
rest := line[len(name):]
if !strings.HasPrefix(rest, "(") {
return name, true
}
depth := 0
var quote byte
for i := 0; i < len(rest); i++ {
ch := rest[i]
if quote != 0 {
switch ch {
case '\\':
i++
case quote:
quote = 0
}
continue
}
switch ch {
case '\'', '"', '`':
quote = ch
case '(':
depth++
case ')':
depth--
if depth == 0 {
return name + rest[:i+1], true
}
}
}
// Unterminated argument list: keep the whole line, as a best effort
return line, true
}
+3
View File
@@ -129,6 +129,9 @@ type User struct {
- `default` - Default value
- `rel` - Relationship definition
- `type` - Explicit SQL type
- `scanonly` - Excluded from INSERT/UPDATE: `GENERATED ... STORED` columns, and `GENERATED ALWAYS AS IDENTITY` columns that are not the primary key
- `generated` - Marks a `GENERATED ... STORED` column
- `identity` - Marks a `GENERATED ALWAYS AS IDENTITY` column (a primary key gets only this marker, since Bun drops `scanonly` fields from the PK list)
## Type Mapping
+46
View File
@@ -0,0 +1,46 @@
package bun
import (
"strings"
"testing"
"git.warky.dev/wdevs/relspecgo/pkg/models"
)
func TestBuildBunTag_GeneratedColumn(t *testing.T) {
mapper := NewTypeMapper("", "")
generated := models.InitColumn("full_name", "users", "public")
generated.Type = "text"
generated.Generated = true
generated.GenerationExpression = "first || ' ' || last"
plain := models.InitColumn("first", "users", "public")
plain.Type = "text"
tests := []struct {
name string
column *models.Column
want bool
}{
{"generated column is scanonly and marked", generated, true},
{"ordinary column is untouched", plain, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tag := mapper.BuildBunTag(tt.column, nil)
parts := strings.Split(tag, ",")
has := func(s string) bool {
for _, p := range parts {
if p == s {
return true
}
}
return false
}
if has("scanonly") != tt.want || has("generated") != tt.want {
t.Errorf("tag %q: scanonly/generated present = %v/%v, want %v", tag, has("scanonly"), has("generated"), tt.want)
}
})
}
}
+49
View File
@@ -0,0 +1,49 @@
package bun
import (
"strings"
"testing"
"git.warky.dev/wdevs/relspecgo/pkg/models"
)
func TestBuildBunTag_IdentityColumn(t *testing.T) {
mapper := NewTypeMapper("", "")
build := func(pk, identity bool, generation string) *models.Column {
col := models.InitColumn("id", "users", "public")
col.Type = "bigint"
col.IsPrimaryKey = pk
col.Identity = identity
col.IdentityGeneration = generation
return col
}
tests := []struct {
name string
column *models.Column
wantFlag bool
wantMarker bool
}{
{"always identity non-pk is write-blocked", build(false, true, "ALWAYS"), true, true},
{"always identity pk is only marked", build(true, true, "ALWAYS"), false, true},
{"by default identity is writable", build(false, true, "BY DEFAULT"), false, false},
{"non-identity is untouched", build(false, false, ""), false, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tag := mapper.BuildBunTag(tt.column, nil)
has := func(s string) bool {
for _, p := range strings.Split(tag, ",") {
if p == s {
return true
}
}
return false
}
if has("scanonly") != tt.wantFlag || has("identity") != tt.wantMarker {
t.Errorf("tag %q: scanonly/identity = %v/%v, want %v/%v", tag, has("scanonly"), has("identity"), tt.wantFlag, tt.wantMarker)
}
})
}
}
-3
View File
@@ -6,7 +6,6 @@ import (
"sort"
"strings"
"git.warky.dev/wdevs/relspecgo/pkg/buildinfo"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/writers"
)
@@ -14,7 +13,6 @@ import (
// TemplateData represents the data passed to the template for code generation
type TemplateData struct {
PackageName string
GeneratedBy string
Imports []string
Models []*ModelData
Config *MethodConfig
@@ -167,7 +165,6 @@ func NewTemplateData(packageName string, config *MethodConfig) *TemplateData {
return &TemplateData{
PackageName: packageName,
GeneratedBy: buildinfo.GeneratedComment(),
Imports: make([]string, 0),
Models: make([]*ModelData, 0),
Config: config,
+1 -2
View File
@@ -7,8 +7,7 @@ import (
// modelTemplate defines the template for generating Bun models
const modelTemplate = `// Code generated by relspecgo. DO NOT EDIT.
{{if .GeneratedBy}}// {{.GeneratedBy}}
{{end}}package {{.PackageName}}
package {{.PackageName}}
{{if .Imports -}}
import (
+16
View File
@@ -366,6 +366,22 @@ func (tm *TypeMapper) BuildBunTag(column *models.Column, table *models.Table) st
parts = append(parts, "autoincrement")
}
// GENERATED ALWAYS ... STORED columns are computed by the database and cannot be
// written: scanonly keeps them out of INSERT/UPDATE, generated marks why.
if column.Generated {
parts = append(parts, "scanonly", "generated")
}
// GENERATED ALWAYS AS IDENTITY rejects explicit values. Bun drops scanonly fields
// from the table's PK list, so a primary key only gets the marker; its zero value
// is already sent as DEFAULT via autoincrement.
if column.Identity && strings.EqualFold(column.IdentityGeneration, "ALWAYS") {
if !column.IsPrimaryKey {
parts = append(parts, "scanonly")
}
parts = append(parts, "identity")
}
// Default value
if column.Default != nil {
// Sanitize default value to remove backticks, then quote based on column type
+8 -14
View File
@@ -8,7 +8,6 @@ import (
"strings"
"testing"
"git.warky.dev/wdevs/relspecgo/pkg/buildinfo"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/writers"
)
@@ -121,7 +120,7 @@ func TestWriter_WriteTable_MultilineDescriptionProducesValidGo(t *testing.T) {
}
}
func TestWriter_WriteTable_EmbedsRelspecVersionHeader(t *testing.T) {
func TestWriter_WriteTable_HeaderIsVersionFree(t *testing.T) {
table := models.InitTable("users", "public")
table.Columns["id"] = &models.Column{Name: "id", Type: "bigint", IsPrimaryKey: true, NotNull: true, Sequence: 1}
@@ -141,19 +140,14 @@ func TestWriter_WriteTable_EmbedsRelspecVersionHeader(t *testing.T) {
t.Fatalf("generated code is invalid Go: %v\n%s", err, src)
}
wantHeader := "// " + buildinfo.GeneratedComment()
if !strings.Contains(src, wantHeader) {
t.Errorf("generated code missing RelSpec version header %q\n%s", wantHeader, src)
// Output must be reproducible: no version or build date in the header.
if strings.Contains(src, "(built: ") || strings.Contains(src, "RelSpec ") {
t.Errorf("generated code still carries a version header\n%s", src)
}
// The provenance comment must sit between the "Code generated" marker and the package clause.
genIdx := strings.Index(src, "// Code generated by relspecgo. DO NOT EDIT.")
hdrIdx := strings.Index(src, wantHeader)
pkgIdx := strings.Index(src, "package models")
if genIdx < 0 || hdrIdx < 0 || pkgIdx < 0 || !(genIdx < hdrIdx && hdrIdx < pkgIdx) {
t.Errorf("RelSpec version header is misplaced (gen=%d hdr=%d pkg=%d)\n%s", genIdx, hdrIdx, pkgIdx, src)
}
if !strings.Contains(wantHeader, "RelSpec ") || !strings.Contains(wantHeader, "(built: ") {
t.Errorf("version header not in expected 'RelSpec <version> (built: <date>)' form: %q", wantHeader)
// The "Code generated" marker must sit directly above the package clause.
want := "// Code generated by relspecgo. DO NOT EDIT.\npackage models"
if !strings.Contains(src, want) {
t.Errorf("header not immediately followed by package clause\n%s", src)
}
}
+2
View File
@@ -111,6 +111,8 @@ Table table_name {
- `unique` - Unique constraint
- `default: value` - Default value
- `note: 'text'` - Column note
- `note: 'GENERATED ALWAYS AS (expr) STORED'` - Generated column (`Generated`, `GenerationExpression`)
- `note: 'GENERATED ALWAYS|BY DEFAULT AS IDENTITY'` - Identity column (`Identity`, `IdentityGeneration`)
### Relationships
```dbml
@@ -0,0 +1,77 @@
package dbml
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/readers"
dbmlreader "git.warky.dev/wdevs/relspecgo/pkg/readers/dbml"
"git.warky.dev/wdevs/relspecgo/pkg/writers"
)
func TestGeneratedAndIdentityColumnsRoundTrip(t *testing.T) {
table := models.InitTable("people", "public")
id := models.InitColumn("id", "people", "public")
id.Type = "bigint"
id.IsPrimaryKey = true
id.AutoIncrement = true
id.Identity = true
id.IdentityGeneration = "ALWAYS"
table.Columns["id"] = id
seq := models.InitColumn("seq", "people", "public")
seq.Type = "integer"
seq.Identity = true
seq.IdentityGeneration = "BY DEFAULT"
table.Columns["seq"] = seq
full := models.InitColumn("full_name", "people", "public")
full.Type = "text"
full.Generated = true
full.GenerationExpression = "first || ' ' || last"
table.Columns["full_name"] = full
plain := models.InitColumn("first", "people", "public")
plain.Type = "text"
plain.Comment = "given name"
table.Columns["first"] = plain
schema := models.InitSchema("public")
schema.Tables = append(schema.Tables, table)
db := models.InitDatabase("testdb")
db.Schemas = append(db.Schemas, schema)
out := filepath.Join(t.TempDir(), "people.dbml")
require.NoError(t, NewWriter(&writers.WriterOptions{OutputPath: out}).WriteDatabase(db))
content, err := os.ReadFile(out)
require.NoError(t, err)
assert.Contains(t, string(content), "GENERATED ALWAYS AS (first || \\' \\' || last) STORED")
assert.Contains(t, string(content), "GENERATED ALWAYS AS IDENTITY")
assert.Contains(t, string(content), "GENERATED BY DEFAULT AS IDENTITY")
got, err := dbmlreader.NewReader(&readers.ReaderOptions{FilePath: out}).ReadDatabase()
require.NoError(t, err)
cols := got.Schemas[0].Tables[0].Columns
assert.True(t, cols["id"].Identity)
assert.Equal(t, "ALWAYS", cols["id"].IdentityGeneration)
assert.Empty(t, cols["id"].Comment, "marker note must not become a comment")
assert.True(t, cols["seq"].Identity)
assert.Equal(t, "BY DEFAULT", cols["seq"].IdentityGeneration)
assert.True(t, cols["full_name"].Generated)
assert.Equal(t, "first || ' ' || last", cols["full_name"].GenerationExpression)
assert.Empty(t, cols["full_name"].Comment)
assert.False(t, cols["first"].Generated)
assert.False(t, cols["first"].Identity)
assert.Equal(t, "given name", cols["first"].Comment)
}
+22
View File
@@ -114,6 +114,24 @@ func (w *Writer) schemaToDBML(schema *models.Schema) string {
return sb.String()
}
// columnMarkerNote returns the note marking a column the database computes itself
// (GENERATED ... STORED, or GENERATED ... AS IDENTITY), or "" for an ordinary column.
// The DBML reader turns the note back into the model's Generated/Identity fields.
func columnMarkerNote(column *models.Column) string {
switch {
case column.Generated:
return fmt.Sprintf("GENERATED ALWAYS AS (%s) STORED", column.GenerationExpression)
case column.Identity && column.IdentityGeneration != "":
return fmt.Sprintf("GENERATED %s AS IDENTITY", strings.ToUpper(column.IdentityGeneration))
}
return ""
}
// escapeNote escapes a value for use inside a single-quoted DBML note.
func escapeNote(s string) string {
return strings.ReplaceAll(strings.ReplaceAll(s, `\`, `\\`), `'`, `\'`)
}
// tableToDBML converts a Table to DBML format string
func (w *Writer) tableToDBML(t *models.Table) string {
var sb strings.Builder
@@ -146,6 +164,10 @@ func (w *Writer) tableToDBML(t *models.Table) string {
}
}
if marker := columnMarkerNote(column); marker != "" {
attrs = append(attrs, fmt.Sprintf("note: '%s'", escapeNote(marker)))
}
if len(attrs) > 0 {
fmt.Fprintf(&sb, " [%s]", strings.Join(attrs, ", "))
}
+1
View File
@@ -113,6 +113,7 @@ export const postsRelations = relations(posts, ({ one }) => ({
## Notes
- Generated columns: `.generatedAlwaysAs(sql`expr`)`; identity columns: `.generatedAlwaysAsIdentity()` / `.generatedByDefaultAsIdentity()`
- Table names and column names are preserved as-is
- Relationships are generated as separate relation definitions
- Constraint actions (CASCADE, etc.) are included in references
@@ -0,0 +1,71 @@
package drizzle
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/readers"
drizzlereader "git.warky.dev/wdevs/relspecgo/pkg/readers/drizzle"
"git.warky.dev/wdevs/relspecgo/pkg/writers"
)
func TestGeneratedAndIdentityColumnsRoundTrip(t *testing.T) {
table := models.InitTable("people", "public")
id := models.InitColumn("id", "people", "public")
id.Type = "bigint"
id.IsPrimaryKey = true
id.AutoIncrement = true
id.Identity = true
id.IdentityGeneration = "ALWAYS"
table.Columns["id"] = id
seq := models.InitColumn("seq", "people", "public")
seq.Type = "integer"
seq.AutoIncrement = true
seq.Identity = true
seq.IdentityGeneration = "BY DEFAULT"
table.Columns["seq"] = seq
first := models.InitColumn("first", "people", "public")
first.Type = "text"
table.Columns["first"] = first
full := models.InitColumn("full_name", "people", "public")
full.Type = "text"
full.Generated = true
full.GenerationExpression = "first || ' ' || upper(first)"
table.Columns["full_name"] = full
schema := models.InitSchema("public")
schema.Tables = append(schema.Tables, table)
db := models.InitDatabase("testdb")
db.Schemas = append(db.Schemas, schema)
out := filepath.Join(t.TempDir(), "schema.ts")
require.NoError(t, NewWriter(&writers.WriterOptions{OutputPath: out}).WriteDatabase(db))
content, err := os.ReadFile(out)
require.NoError(t, err)
assert.Contains(t, string(content), "generatedAlwaysAs(sql`first || ' ' || upper(first)`)")
assert.Contains(t, string(content), "generatedAlwaysAsIdentity()")
assert.Contains(t, string(content), "generatedByDefaultAsIdentity()")
got, err := drizzlereader.NewReader(&readers.ReaderOptions{FilePath: out}).ReadDatabase()
require.NoError(t, err)
cols := got.Schemas[0].Tables[0].Columns
assert.True(t, cols["id"].Identity)
assert.Equal(t, "ALWAYS", cols["id"].IdentityGeneration)
assert.True(t, cols["seq"].Identity)
assert.Equal(t, "BY DEFAULT", cols["seq"].IdentityGeneration)
assert.True(t, cols["full_name"].Generated)
assert.Equal(t, "first || ' ' || upper(first)", cols["full_name"].GenerationExpression)
assert.False(t, cols["first"].Generated)
assert.False(t, cols["first"].Identity)
}
+6 -9
View File
@@ -4,16 +4,14 @@ import (
"sort"
"strings"
"git.warky.dev/wdevs/relspecgo/pkg/buildinfo"
"git.warky.dev/wdevs/relspecgo/pkg/models"
)
// TemplateData represents the data passed to the template for code generation
type TemplateData struct {
GeneratedBy string
Imports []string
Enums []*EnumData
Tables []*TableData
Imports []string
Enums []*EnumData
Tables []*TableData
}
// EnumData represents an enum in the schema
@@ -61,10 +59,9 @@ type IndexData struct {
// NewTemplateData creates a new TemplateData
func NewTemplateData() *TemplateData {
return &TemplateData{
GeneratedBy: buildinfo.GeneratedComment(),
Imports: make([]string, 0),
Enums: make([]*EnumData, 0),
Tables: make([]*TableData, 0),
Imports: make([]string, 0),
Enums: make([]*EnumData, 0),
Tables: make([]*TableData, 0),
}
}
+1 -2
View File
@@ -7,8 +7,7 @@ import (
// schemaTemplate defines the template for generating Drizzle schemas
const schemaTemplate = `// Code generated by relspecgo. DO NOT EDIT.
{{if .GeneratedBy}}// {{.GeneratedBy}}
{{end}}{{range .Imports}}{{.}}
{{range .Imports}}{{.}}
{{end}}
{{if .Enums}}
// Enums
+7 -1
View File
@@ -140,7 +140,13 @@ func (tm *TypeMapper) buildColumnModifiers(col *models.Column, table *models.Tab
}
// Default value
if col.AutoIncrement {
if col.Generated && col.GenerationExpression != "" {
// GENERATED ALWAYS AS (expr) STORED
expr := strings.ReplaceAll(col.GenerationExpression, "`", "\\`")
modifiers = append(modifiers, fmt.Sprintf("generatedAlwaysAs(sql`%s`)", expr))
} else if col.Identity && strings.EqualFold(col.IdentityGeneration, "BY DEFAULT") {
modifiers = append(modifiers, "generatedByDefaultAsIdentity()")
} else if col.AutoIncrement || col.Identity {
// For auto-increment, use generatedAlwaysAsIdentity()
modifiers = append(modifiers, "generatedAlwaysAsIdentity()")
} else if col.Default != nil {
+8
View File
@@ -203,6 +203,14 @@ The writer automatically generates relationship fields:
- Relationship field names use 3-letter prefixes
- Includes proper `gorm` tags with `foreignKey` and `references`
## Read-only Columns
| Column | Tags |
|---|---|
| `GENERATED ... STORED` | `<-:false;generated` |
| `GENERATED ALWAYS AS IDENTITY`, not PK | `<-:false;identity` |
| `GENERATED ALWAYS AS IDENTITY`, PK | `identity` only |
## Notes
- Model names are prefixed with "Model" (e.g., `ModelUser`)
+45
View File
@@ -0,0 +1,45 @@
package gorm
import (
"strings"
"testing"
"git.warky.dev/wdevs/relspecgo/pkg/models"
)
func TestBuildGormTag_GeneratedColumn(t *testing.T) {
mapper := NewTypeMapper("")
generated := models.InitColumn("full_name", "users", "public")
generated.Type = "text"
generated.Generated = true
plain := models.InitColumn("first", "users", "public")
plain.Type = "text"
tests := []struct {
name string
column *models.Column
want bool
}{
{"generated column is write-disabled and marked", generated, true},
{"ordinary column is untouched", plain, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tag := mapper.BuildGormTag(tt.column, nil)
parts := strings.Split(tag, ";")
has := func(s string) bool {
for _, p := range parts {
if p == s {
return true
}
}
return false
}
if has("<-:false") != tt.want || has("generated") != tt.want {
t.Errorf("tag %q: <-:false/generated present = %v/%v, want %v", tag, has("<-:false"), has("generated"), tt.want)
}
})
}
}
+49
View File
@@ -0,0 +1,49 @@
package gorm
import (
"strings"
"testing"
"git.warky.dev/wdevs/relspecgo/pkg/models"
)
func TestBuildGormTag_IdentityColumn(t *testing.T) {
mapper := NewTypeMapper("")
build := func(pk, identity bool, generation string) *models.Column {
col := models.InitColumn("id", "users", "public")
col.Type = "bigint"
col.IsPrimaryKey = pk
col.Identity = identity
col.IdentityGeneration = generation
return col
}
tests := []struct {
name string
column *models.Column
wantFlag bool
wantMarker bool
}{
{"always identity non-pk is write-blocked", build(false, true, "ALWAYS"), true, true},
{"always identity pk is only marked", build(true, true, "ALWAYS"), false, true},
{"by default identity is writable", build(false, true, "BY DEFAULT"), false, false},
{"non-identity is untouched", build(false, false, ""), false, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tag := mapper.BuildGormTag(tt.column, nil)
has := func(s string) bool {
for _, p := range strings.Split(tag, ";") {
if p == s {
return true
}
}
return false
}
if has("<-:false") != tt.wantFlag || has("identity") != tt.wantMarker {
t.Errorf("tag %q: <-:false/identity = %v/%v, want %v/%v", tag, has("<-:false"), has("identity"), tt.wantFlag, tt.wantMarker)
}
})
}
}
-3
View File
@@ -4,7 +4,6 @@ import (
"sort"
"strings"
"git.warky.dev/wdevs/relspecgo/pkg/buildinfo"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/writers"
)
@@ -12,7 +11,6 @@ import (
// TemplateData represents the data passed to the template for code generation
type TemplateData struct {
PackageName string
GeneratedBy string
Imports []string
Models []*ModelData
Config *MethodConfig
@@ -81,7 +79,6 @@ func NewTemplateData(packageName string, config *MethodConfig) *TemplateData {
return &TemplateData{
PackageName: packageName,
GeneratedBy: buildinfo.GeneratedComment(),
Imports: make([]string, 0),
Models: make([]*ModelData, 0),
Config: config,
+1 -2
View File
@@ -7,8 +7,7 @@ import (
// modelTemplate defines the template for generating GORM models
const modelTemplate = `// Code generated by relspecgo. DO NOT EDIT.
{{if .GeneratedBy}}// {{.GeneratedBy}}
{{end}}package {{.PackageName}}
package {{.PackageName}}
{{if .Imports -}}
import (
+15
View File
@@ -401,6 +401,21 @@ func (tm *TypeMapper) BuildGormTag(column *models.Column, table *models.Table) s
parts = append(parts, "autoIncrement")
}
// GENERATED ALWAYS ... STORED columns are computed by the database and cannot be
// written: disable write permission so GORM leaves them out of INSERT/UPDATE, and mark why.
if column.Generated {
parts = append(parts, "<-:false", "generated")
}
// GENERATED ALWAYS AS IDENTITY rejects explicit values. A primary key only gets the
// marker so it keeps its normal create/return behavior.
if column.Identity && strings.EqualFold(column.IdentityGeneration, "ALWAYS") {
if !column.IsPrimaryKey {
parts = append(parts, "<-:false")
}
parts = append(parts, "identity")
}
// Not null (skip if primary key, as it's implied)
if column.NotNull && !column.IsPrimaryKey {
parts = append(parts, "not null")
+2 -1
View File
@@ -127,4 +127,5 @@ go test ./pkg/writers/mssql/...
- Views are not currently supported in the writer
- Sequences are not supported (MSSQL uses IDENTITY instead)
- Partitioning and advanced features are not supported
- Generated DDL assumes no triggers or computed columns
- Generated DDL assumes no triggers
- Generated columns are written as computed columns `[col] AS (expr) PERSISTED` (no type, no DEFAULT); identity columns as `IDENTITY(1,1)`
+13 -4
View File
@@ -11,7 +11,6 @@ import (
_ "github.com/microsoft/go-mssqldb" // MSSQL driver
"git.warky.dev/wdevs/relspecgo/pkg/buildinfo"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/mssql"
"git.warky.dev/wdevs/relspecgo/pkg/writers"
@@ -69,7 +68,7 @@ func (w *Writer) WriteDatabase(db *models.Database) error {
// Write header comment
fmt.Fprintf(w.writer, "-- MSSQL Database Schema\n")
fmt.Fprintf(w.writer, "-- Database: %s\n", db.Name)
fmt.Fprintf(w.writer, "-- Generated by %s\n\n", buildinfo.GeneratedComment())
fmt.Fprintf(w.writer, "-- Generated by RelSpec\n\n")
// Process each schema in the database
for _, schema := range db.Schemas {
@@ -188,6 +187,16 @@ func (w *Writer) writeCreateTable(schema *models.Schema, table *models.Table) er
func (w *Writer) generateColumnDefinition(col *models.Column) string {
parts := []string{fmt.Sprintf("[%s]", col.Name)}
// Computed column: "name AS (expr) PERSISTED" has no data type and cannot carry
// IDENTITY or a DEFAULT. PERSISTED mirrors the stored semantics of Column.Generated.
if col.Generated && col.GenerationExpression != "" {
parts = append(parts, fmt.Sprintf("AS (%s) PERSISTED", col.GenerationExpression))
if col.NotNull {
parts = append(parts, "NOT NULL")
}
return strings.Join(parts, " ")
}
// Type with length/precision
baseType := mssql.ConvertCanonicalToMSSQL(col.Type)
typeStr := baseType
@@ -216,7 +225,7 @@ func (w *Writer) generateColumnDefinition(col *models.Column) string {
parts = append(parts, typeStr)
// IDENTITY for auto-increment
if col.AutoIncrement {
if col.AutoIncrement || col.Identity {
parts = append(parts, "IDENTITY(1,1)")
}
@@ -478,7 +487,7 @@ func (w *Writer) executeDatabaseSQL(db *models.Database, connString string) erro
statements := []string{}
statements = append(statements, "-- MSSQL Database Schema")
statements = append(statements, fmt.Sprintf("-- Database: %s", db.Name))
statements = append(statements, "-- Generated by "+buildinfo.GeneratedComment())
statements = append(statements, "-- Generated by RelSpec")
for _, schema := range db.Schemas {
if err := w.generateSchemaStatements(schema, &statements); err != nil {
+16
View File
@@ -204,3 +204,19 @@ func TestWriteComments(t *testing.T) {
assert.Contains(t, output, "User accounts table")
assert.Contains(t, output, "Primary key")
}
func TestGenerateColumnDefinition_ComputedAndIdentity(t *testing.T) {
writer := NewWriter(&writers.WriterOptions{})
computed := &models.Column{
Name: "full_name",
Type: "string",
Generated: true,
GenerationExpression: "[first] + ' ' + [last]",
Default: "ignored",
}
assert.Equal(t, "[full_name] AS ([first] + ' ' + [last]) PERSISTED", writer.generateColumnDefinition(computed))
identity := &models.Column{Name: "id", Type: "int", NotNull: true, Identity: true, IdentityGeneration: "ALWAYS"}
assert.Equal(t, "[id] INT IDENTITY(1,1) NOT NULL", writer.generateColumnDefinition(identity))
}
+6
View File
@@ -164,6 +164,12 @@ When `include_audit` is enabled, adds:
## PostgreSQL-Specific Features
- Serial types (SERIAL, BIGSERIAL)
- Generated columns: `GENERATED ALWAYS AS (expr) STORED` (`Column.Generated` + `GenerationExpression`)
- Identity columns: `GENERATED ALWAYS|BY DEFAULT AS IDENTITY` (`Column.Identity` + `IdentityGeneration`)
- Generated/identity columns never carry a `DEFAULT`; emitted in `CREATE TABLE` and `ADD COLUMN` (full DDL and migration)
- Existing generated/identity columns are not diffed (expression changes are not applied)
- PK sequence = the `nextval()` default's sequence; `setval` moves it past `MAX(pk)`, forward only
- `serial`/`bigserial` columns without a model default keep their `nextval()` default in diffs
- Advanced types (UUID, JSONB, ARRAY)
- Schema-qualified names
- Constraint actions (CASCADE, RESTRICT, SET NULL)
+409
View File
@@ -0,0 +1,409 @@
package pgsql
import (
"strings"
"testing"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/writers"
)
// diffTestDB builds a database with one table per spec: name -> column name -> type.
func diffTestDB(schemaName string, tables map[string]map[string]string) *models.Database {
db := models.InitDatabase("testdb")
schema := models.InitSchema(schemaName)
for tName, cols := range tables {
table := models.InitTable(tName, schemaName)
seq := 1
for cName, cType := range cols {
col := models.InitColumn(cName, tName, schemaName)
col.Type = cType
col.Sequence = uint(seq)
seq++
table.Columns[cName] = col
}
schema.Tables = append(schema.Tables, table)
}
db.Schemas = append(db.Schemas, schema)
return db
}
func diffJoin(stmts []string) string { return strings.Join(stmts, "\n") }
func TestDiffStatements(t *testing.T) {
tests := []struct {
name string
model func() *models.Database
current func() *models.Database
wantContain []string
wantAbsent []string
wantEmpty bool
}{
{
name: "identical schemas produce no statements",
model: func() *models.Database {
return diffTestDB("public", map[string]map[string]string{"users": {"id": "integer", "email": "text"}})
},
current: func() *models.Database {
return diffTestDB("public", map[string]map[string]string{"users": {"id": "integer", "email": "text"}})
},
wantEmpty: true,
},
{
name: "missing table is created",
model: func() *models.Database {
return diffTestDB("public", map[string]map[string]string{"users": {"id": "integer"}, "posts": {"id": "integer"}})
},
current: func() *models.Database {
return diffTestDB("public", map[string]map[string]string{"users": {"id": "integer"}})
},
wantContain: []string{"CREATE TABLE IF NOT EXISTS public.posts"},
wantAbsent: []string{"public.users"},
},
{
name: "missing column is added and existing columns are skipped",
model: func() *models.Database {
return diffTestDB("public", map[string]map[string]string{"users": {"id": "integer", "email": "text"}})
},
current: func() *models.Database {
return diffTestDB("public", map[string]map[string]string{"users": {"id": "integer"}})
},
wantContain: []string{"ADD COLUMN IF NOT EXISTS email text"},
wantAbsent: []string{"COLUMN id"},
},
{
name: "changed column type is altered",
model: func() *models.Database {
return diffTestDB("public", map[string]map[string]string{"users": {"id": "bigint"}})
},
current: func() *models.Database {
return diffTestDB("public", map[string]map[string]string{"users": {"id": "integer"}})
},
wantContain: []string{"ALTER COLUMN id TYPE bigint"},
},
{
name: "missing non-public schema is created before its tables",
model: func() *models.Database {
return diffTestDB("app", map[string]map[string]string{"users": {"id": "integer"}})
},
current: func() *models.Database { return models.InitDatabase("testdb") },
wantContain: []string{
"CREATE SCHEMA IF NOT EXISTS app",
"CREATE TABLE IF NOT EXISTS app.users",
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
w := NewWriter(&writers.WriterOptions{})
stmts, err := w.diffStatements(tt.model(), tt.current())
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
out := diffJoin(stmts)
if tt.wantEmpty && len(stmts) != 0 {
t.Fatalf("expected no statements, got:\n%s", out)
}
for _, want := range tt.wantContain {
if !strings.Contains(out, want) {
t.Errorf("missing %q in:\n%s", want, out)
}
}
for _, absent := range tt.wantAbsent {
if strings.Contains(out, absent) {
t.Errorf("unexpected %q in:\n%s", absent, out)
}
}
})
}
}
func TestDiffStatements_SchemaCreatedBeforeTables(t *testing.T) {
w := NewWriter(&writers.WriterOptions{})
model := diffTestDB("app", map[string]map[string]string{"users": {"id": "integer"}})
stmts, err := w.diffStatements(model, models.InitDatabase("testdb"))
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
schemaIdx, tableIdx := -1, -1
for i, s := range stmts {
if strings.Contains(s, "CREATE SCHEMA") {
schemaIdx = i
}
if strings.Contains(s, "CREATE TABLE") {
tableIdx = i
}
}
if schemaIdx < 0 || tableIdx < 0 || schemaIdx > tableIdx {
t.Fatalf("expected CREATE SCHEMA before CREATE TABLE, got:\n%s", diffJoin(stmts))
}
}
func TestDiffStatements_NewTableCreatesPrimaryKeySequence(t *testing.T) {
model := models.InitDatabase("testdb")
schema := models.InitSchema("public")
table := models.InitTable("users", "public")
id := models.InitColumn("id", "users", "public")
id.Type = "integer"
id.IsPrimaryKey = true
id.Default = "nextval('public.users_id_seq'::regclass)"
table.Columns["id"] = id
schema.Tables = append(schema.Tables, table)
model.Schemas = append(model.Schemas, schema)
w := NewWriter(&writers.WriterOptions{})
stmts, err := w.diffStatements(model, models.InitDatabase("testdb"))
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
out := diffJoin(stmts)
if !strings.Contains(out, "CREATE SEQUENCE IF NOT EXISTS public.users_id_seq") {
t.Fatalf("expected sequence creation, got:\n%s", out)
}
// Existing table: sequence must not be re-emitted.
current := models.InitDatabase("testdb")
cs := models.InitSchema("public")
cs.Tables = append(cs.Tables, table)
current.Schemas = append(current.Schemas, cs)
stmts, err = w.diffStatements(model, current)
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
if strings.Contains(diffJoin(stmts), "CREATE SEQUENCE") {
t.Fatalf("sequence emitted for existing table:\n%s", diffJoin(stmts))
}
}
func TestDiffStatements_NewIndexAndChangedIndexRecreated(t *testing.T) {
build := func(cols ...string) *models.Database {
db := diffTestDB("public", map[string]map[string]string{"users": {"id": "integer", "email": "text", "name": "text"}})
table := db.Schemas[0].Tables[0]
table.Indexes["idx_users_lookup"] = &models.Index{Name: "idx_users_lookup", Columns: cols}
return db
}
w := NewWriter(&writers.WriterOptions{})
// Index missing in current.
noIdx := diffTestDB("public", map[string]map[string]string{"users": {"id": "integer", "email": "text", "name": "text"}})
stmts, err := w.diffStatements(build("email"), noIdx)
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
if !strings.Contains(diffJoin(stmts), "CREATE INDEX IF NOT EXISTS idx_users_lookup") {
t.Fatalf("expected index creation, got:\n%s", diffJoin(stmts))
}
// Index unchanged.
stmts, err = w.diffStatements(build("email"), build("email"))
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
if len(stmts) != 0 {
t.Fatalf("expected no statements for identical index, got:\n%s", diffJoin(stmts))
}
// Index definition changed: dropped and recreated.
stmts, err = w.diffStatements(build("name"), build("email"))
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
out := diffJoin(stmts)
dropIdx := strings.Index(out, "DROP INDEX")
createIdx := strings.Index(out, "CREATE INDEX")
if dropIdx < 0 || createIdx < 0 || dropIdx > createIdx {
t.Fatalf("expected DROP INDEX before CREATE INDEX, got:\n%s", out)
}
}
func TestDiffStatements_FlattenSchemaRejected(t *testing.T) {
w := NewWriter(&writers.WriterOptions{FlattenSchema: true})
if _, err := w.generateLiveDiffStatements(models.InitDatabase("x"), "postgres://unused"); err == nil {
t.Fatal("expected error so the caller falls back to full DDL")
}
}
func TestDiffStatements_LiveDatabaseRepresentationsAreNotDifferences(t *testing.T) {
build := func(live bool) *models.Database {
db := models.InitDatabase("testdb")
schema := models.InitSchema("public")
parent := models.InitTable("parent", "public")
parentID := models.InitColumn("id_parent", "parent", "public")
parentCurrentType := "bigserial"
if live {
parentCurrentType = "bigint"
}
parentID.Type = parentCurrentType
parentID.IsPrimaryKey = true
parentID.NotNull = true
parent.Columns["id_parent"] = parentID
if live {
// A live database reports the PK as a named constraint and the unique
// constraint's backing index as an index too.
parent.Constraints["pk_public_parent"] = &models.Constraint{
Name: "pk_public_parent", Type: models.PrimaryKeyConstraint, Columns: []string{"id_parent"},
}
}
guid := models.InitColumn("guid", "parent", "public")
guid.Type = "uuid"
parent.Columns["guid"] = guid
parent.Constraints["ukey_parent_guid"] = &models.Constraint{
Name: "ukey_parent_guid", Type: models.UniqueConstraint, Columns: []string{"guid"},
}
if live {
parent.Indexes["ukey_parent_guid"] = &models.Index{Name: "ukey_parent_guid", Unique: true, Columns: []string{"guid"}, Type: "btree"}
}
// numeric(10,0) vs numeric(10); quoted default vs unquoted default with cast.
amount := models.InitColumn("amount", "parent", "public")
amount.Type = "numeric(10,0)"
tags := models.InitColumn("tags", "parent", "public")
tags.Type = "jsonb"
tags.Default = "`'[]'`"
if live {
amount.Type = "numeric"
amount.Precision = 10
tags.Default = "[]"
}
parent.Columns["amount"] = amount
parent.Columns["tags"] = tags
parent.Indexes["idx_parent_amount"] = &models.Index{Name: "idx_parent_amount", Columns: []string{"amount"}}
if live {
parent.Indexes["idx_parent_amount"].Type = "btree"
}
child := models.InitTable("child", "public")
cid := models.InitColumn("id_child", "child", "public")
cid.Type = "integer"
child.Columns["id_child"] = cid
rid := models.InitColumn("rid_parent", "child", "public")
rid.Type = "bigint"
child.Columns["rid_parent"] = rid
action := "restrict"
if live {
action = "RESTRICT"
}
child.Constraints["fk_child_rid_parent"] = &models.Constraint{
Name: "fk_child_rid_parent", Type: models.ForeignKeyConstraint, Columns: []string{"rid_parent"},
ReferencedTable: "parent", ReferencedSchema: "public", ReferencedColumns: []string{"id_parent"},
OnDelete: action, OnUpdate: action,
}
parent.Description = "Parent\ntable"
if live {
parent.Description = "Parent\ntable\n"
}
schema.Tables = append(schema.Tables, parent, child)
db.Schemas = append(db.Schemas, schema)
return db
}
w := NewWriter(&writers.WriterOptions{})
stmts, err := w.diffStatements(build(false), build(true))
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
if len(stmts) != 0 {
t.Fatalf("expected live representations to match the model, got:\n%s", diffJoin(stmts))
}
}
func TestDiffStatements_ChangedPrimaryKeyIsRecreated(t *testing.T) {
model := diffTestDB("public", map[string]map[string]string{"users": {"id": "integer", "tenant": "integer"}})
model.Schemas[0].Tables[0].Columns["tenant"].IsPrimaryKey = true
current := diffTestDB("public", map[string]map[string]string{"users": {"id": "integer", "tenant": "integer"}})
current.Schemas[0].Tables[0].Columns["id"].IsPrimaryKey = true
current.Schemas[0].Tables[0].Constraints["pk_public_users"] = &models.Constraint{
Name: "pk_public_users", Type: models.PrimaryKeyConstraint, Columns: []string{"id"},
}
w := NewWriter(&writers.WriterOptions{})
stmts, err := w.diffStatements(model, current)
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
if !strings.Contains(diffJoin(stmts), "DROP CONSTRAINT IF EXISTS pk_public_users") {
t.Fatalf("expected old primary key to be dropped, got:\n%s", diffJoin(stmts))
}
}
func TestDiffStatements_ChangedCommentEmitted(t *testing.T) {
model := diffTestDB("public", map[string]map[string]string{"users": {"id": "integer"}})
model.Schemas[0].Tables[0].Description = "new"
current := diffTestDB("public", map[string]map[string]string{"users": {"id": "integer"}})
current.Schemas[0].Tables[0].Description = "old"
w := NewWriter(&writers.WriterOptions{})
stmts, err := w.diffStatements(model, current)
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
if !strings.Contains(diffJoin(stmts), "COMMENT ON TABLE public.users IS 'new'") {
t.Fatalf("expected changed comment, got:\n%s", diffJoin(stmts))
}
}
func TestDiffStatements_TruncatedConstraintNameMatches(t *testing.T) {
long := "fk_individual_actor_relationship_rid_typelookup_relationship_type"
if len(long) <= maxIdentifierBytes {
t.Fatal("test name must exceed the identifier limit")
}
build := func(name string) *models.Database {
db := diffTestDB("public", map[string]map[string]string{
"parent": {"id": "integer"},
"child": {"rid": "integer"},
})
for _, tbl := range db.Schemas[0].Tables {
if tbl.Name == "child" {
tbl.Constraints[name] = &models.Constraint{
Name: name, Type: models.ForeignKeyConstraint, Columns: []string{"rid"},
ReferencedTable: "parent", ReferencedSchema: "public", ReferencedColumns: []string{"id"},
}
}
}
return db
}
w := NewWriter(&writers.WriterOptions{})
stmts, err := w.diffStatements(build(long), build(pgIdentifier(long)))
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
if len(stmts) != 0 {
t.Fatalf("expected truncated live name to match, got:\n%s", diffJoin(stmts))
}
}
func TestColumnDefaultsEqual_LiveRepresentations(t *testing.T) {
tests := []struct {
name string
model, cur interface{}
want bool
}{
{"cast wrapped in parentheses", "gen_random_uuid()::text", "(gen_random_uuid())::text", true},
{"jsonb re-spaced array", `'["openid","profile","email"]'`, `'["openid", "profile", "email"]'::jsonb`, true},
{"backticked function", "`now()`", "now()", true},
{"quoted literal with cast", "'[]'", "'[]'::jsonb", true},
{"jsonb array already unquoted by the reader", `'["openid","profile"]'`, `["openid", "profile"]`, true},
{"text with spaces and braces", "`'{actor} #{actor_id} - {chartofaccount}'`", "{actor} #{actor_id} - {chartofaccount}", true},
{"cast on plain function stays equal", "gen_random_uuid()::text", "gen_random_uuid()::text", true},
{"different jsonb content", `'["a"]'`, `'["b"]'::jsonb`, false},
{"different function", "gen_random_uuid()::text", "(now())::text", false},
{"different literal", "'active'", "'inactive'::text", false},
{"nil vs value", nil, "now()", false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := columnDefaultsEqual(tt.model, tt.cur); got != tt.want {
t.Errorf("columnDefaultsEqual(%v, %v) = %v, want %v", tt.model, tt.cur, got, tt.want)
}
})
}
}
+101
View File
@@ -0,0 +1,101 @@
package pgsql
import (
"strings"
"testing"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/writers"
)
func generationTestTable(withColumns bool) *models.Table {
table := models.InitTable("people", "public")
first := models.InitColumn("first", "people", "public")
first.Type = "text"
first.Sequence = 1
table.Columns["first"] = first
if !withColumns {
return table
}
full := models.InitColumn("full_name", "people", "public")
full.Type = "text"
full.Sequence = 2
full.Generated = true
full.GenerationExpression = "first || ' ' || last"
full.Default = "ignored"
table.Columns["full_name"] = full
ident := models.InitColumn("seq", "people", "public")
ident.Type = "bigint"
ident.Sequence = 3
ident.NotNull = true
ident.Identity = true
ident.IdentityGeneration = "ALWAYS"
table.Columns["seq"] = ident
return table
}
func generationTestDB(table *models.Table) *models.Database {
db := models.InitDatabase("testdb")
schema := models.InitSchema("public")
schema.Tables = append(schema.Tables, table)
db.Schemas = append(db.Schemas, schema)
return db
}
func TestGeneratedColumns_MigrationCreateAndAddColumn(t *testing.T) {
want := []string{
"GENERATED ALWAYS AS (first || ' ' || last) STORED",
"GENERATED ALWAYS AS IDENTITY",
}
tests := []struct {
name string
current *models.Database
}{
{"new table", models.InitDatabase("testdb")},
{"new columns on existing table", generationTestDB(generationTestTable(false))},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
w := NewWriter(&writers.WriterOptions{})
stmts, err := w.diffStatements(generationTestDB(generationTestTable(true)), tt.current)
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
out := diffJoin(stmts)
for _, s := range want {
if !strings.Contains(out, s) {
t.Errorf("missing %q:\n%s", s, out)
}
}
if strings.Contains(out, "DEFAULT ignored") || strings.Contains(out, "DEFAULT 'ignored'") {
t.Errorf("generated column must not carry a DEFAULT:\n%s", out)
}
})
}
}
func TestGeneratedColumns_FullDDL(t *testing.T) {
w := NewWriter(&writers.WriterOptions{})
stmts, err := w.GenerateDatabaseStatements(generationTestDB(generationTestTable(true)))
if err != nil {
t.Fatalf("GenerateDatabaseStatements failed: %v", err)
}
out := diffJoin(stmts)
for _, s := range []string{
"full_name text GENERATED ALWAYS AS (first || ' ' || last) STORED",
"seq bigint GENERATED ALWAYS AS IDENTITY NOT NULL",
} {
if !strings.Contains(out, s) {
t.Errorf("missing %q:\n%s", s, out)
}
}
if strings.Contains(out, "DEFAULT 'ignored'") {
t.Errorf("generated column must not carry a DEFAULT:\n%s", out)
}
}
+253 -41
View File
@@ -1,13 +1,14 @@
package pgsql
import (
"bytes"
"encoding/json"
"fmt"
"io"
"os"
"sort"
"strings"
"git.warky.dev/wdevs/relspecgo/pkg/buildinfo"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/pgsql"
"git.warky.dev/wdevs/relspecgo/pkg/writers"
@@ -47,10 +48,11 @@ func NewMigrationWriter(options *writers.WriterOptions) (*MigrationWriter, error
}, nil
}
// WriteMigration generates migration scripts using templates
func (w *MigrationWriter) WriteMigration(model, current *models.Database) error {
// GenerateScripts computes the differential migration scripts between current and model,
// sorted by priority and sequence.
func (w *MigrationWriter) GenerateScripts(model, current *models.Database) ([]MigrationScript, error) {
if model == nil {
return fmt.Errorf("model database is required")
return nil, fmt.Errorf("model database is required")
}
if w.options == nil {
w.options = &writers.WriterOptions{}
@@ -59,26 +61,6 @@ func (w *MigrationWriter) WriteMigration(model, current *models.Database) error
current = models.InitDatabase(model.Name)
}
var writer io.Writer
var file *os.File
var err error
// Use existing writer if already set (for testing)
if w.writer != nil {
writer = w.writer
} else if w.options.OutputPath != "" {
file, err = os.Create(w.options.OutputPath)
if err != nil {
return fmt.Errorf("failed to create output file: %w", err)
}
defer file.Close()
writer = file
} else {
writer = os.Stdout
}
w.writer = writer
// Check if audit is configured in metadata
var auditConfig *AuditConfig
if w.options.Metadata != nil {
@@ -94,7 +76,7 @@ func (w *MigrationWriter) WriteMigration(model, current *models.Database) error
if auditConfig != nil && len(auditConfig.EnabledTables) > 0 {
auditTableScript, err := w.generateAuditTablesScript(auditConfig)
if err != nil {
return fmt.Errorf("failed to generate audit tables: %w", err)
return nil, fmt.Errorf("failed to generate audit tables: %w", err)
}
scripts = append(scripts, auditTableScript...)
}
@@ -120,7 +102,7 @@ func (w *MigrationWriter) WriteMigration(model, current *models.Database) error
// Generate schema-level scripts
schemaScripts, err := w.generateSchemaScripts(modelSchema, currentSchema)
if err != nil {
return fmt.Errorf("failed to generate schema scripts: %w", err)
return nil, fmt.Errorf("failed to generate schema scripts: %w", err)
}
scripts = append(scripts, schemaScripts...)
@@ -128,7 +110,7 @@ func (w *MigrationWriter) WriteMigration(model, current *models.Database) error
if auditConfig != nil {
auditScripts, err := w.generateAuditScripts(modelSchema, auditConfig)
if err != nil {
return fmt.Errorf("failed to generate audit scripts: %w", err)
return nil, fmt.Errorf("failed to generate audit scripts: %w", err)
}
scripts = append(scripts, auditScripts...)
}
@@ -142,9 +124,48 @@ func (w *MigrationWriter) WriteMigration(model, current *models.Database) error
return scripts[i].Sequence < scripts[j].Sequence
})
return scripts, nil
}
// WriteMigration generates migration scripts using templates
func (w *MigrationWriter) WriteMigration(model, current *models.Database) error {
if model == nil {
return fmt.Errorf("model database is required")
}
if w.options == nil {
w.options = &writers.WriterOptions{}
}
if current == nil {
current = models.InitDatabase(model.Name)
}
scripts, err := w.GenerateScripts(model, current)
if err != nil {
return err
}
var writer io.Writer
var file *os.File
// Use existing writer if already set (for testing)
if w.writer != nil {
writer = w.writer
} else if w.options.OutputPath != "" {
file, err = os.Create(w.options.OutputPath)
if err != nil {
return fmt.Errorf("failed to create output file: %w", err)
}
defer file.Close()
writer = file
} else {
writer = os.Stdout
}
w.writer = writer
// Write header
fmt.Fprintf(w.writer, "-- PostgreSQL Migration Script\n")
fmt.Fprintf(w.writer, "-- Generated by %s\n", buildinfo.GeneratedComment())
fmt.Fprintf(w.writer, "-- Generated by RelSpec\n")
fmt.Fprintf(w.writer, "-- Source: %s -> %s\n", current.Name, model.Name)
if w.options.ContinueOnError {
fmt.Fprintf(w.writer, "\\set ON_ERROR_STOP off\n")
@@ -242,10 +263,14 @@ func (w *MigrationWriter) generateDropScripts(model, current *models.Schema) ([]
// Check each constraint in current database
for _, currentConstraint := range sortConstraints(currentTable.Constraints) {
constraintName := currentConstraint.Name
modelConstraint, existsInModel := modelTable.Constraints[constraintName]
modelConstraint, existsInModel := lookupConstraint(modelTable.Constraints, constraintName)
shouldDrop := false
if !existsInModel {
if currentConstraint.Type == models.PrimaryKeyConstraint {
// Model primary keys usually live on the columns (not as a named constraint), so
// compare by key columns instead of constraint name.
shouldDrop = !primaryKeyColumnsMatch(modelTable, currentConstraint)
} else if !existsInModel {
shouldDrop = true
} else if !constraintsEqual(modelConstraint, currentConstraint) {
shouldDrop = true
@@ -317,6 +342,12 @@ func (w *MigrationWriter) generateDropScripts(model, current *models.Schema) ([]
indexName := currentIndex.Name
modelIndex, existsInModel := modelTable.Indexes[indexName]
// A live database reports the index backing a unique/primary constraint as an
// index too; it belongs to the constraint and is handled there.
if _, backsConstraint := currentTable.Constraints[indexName]; backsConstraint {
continue
}
shouldDrop := false
if !existsInModel {
shouldDrop = true
@@ -411,6 +442,7 @@ func (w *MigrationWriter) generateAlterTableScripts(schema *models.Schema, model
if !exists {
// Column doesn't exist, add it
_, defaultVal := formatColumnDefaultSQL(modelCol)
generated, identity := columnGenerationClauses(modelCol)
sql, err := w.executor.ExecuteAddColumn(AddColumnData{
SchemaName: schema.Name,
@@ -419,6 +451,8 @@ func (w *MigrationWriter) generateAlterTableScripts(schema *models.Schema, model
ColumnType: effectiveColumnSQLType(modelCol),
Default: defaultVal,
NotNull: modelCol.NotNull,
Generated: generated,
Identity: identity,
})
if err != nil {
return nil, err
@@ -433,6 +467,16 @@ func (w *MigrationWriter) generateAlterTableScripts(schema *models.Schema, model
Body: sql,
}
scripts = append(scripts, script)
} else if currentCol.Generated || currentCol.Identity {
// GENERATED ALWAYS ... STORED columns and GENERATED ... AS IDENTITY
// columns are both driven by Postgres-side machinery (an expression, or
// a backing sequence + attidentity flag) that DBML cannot express, so
// the model side always looks like drift (missing type modifiers, no
// default, NOT NULL mismatches, etc). None of their properties are
// alterable the normal way (e.g. dropping a generated column's default
// requires DROP EXPRESSION, not DROP DEFAULT; identity columns have no
// default to alter at all), so these DB-side-only columns are left
// untouched rather than diffed.
} else if !columnsEqual(modelCol, currentCol) {
// Column exists but properties changed
if !columnTypesEqual(modelCol, currentCol) {
@@ -461,7 +505,7 @@ func (w *MigrationWriter) generateAlterTableScripts(schema *models.Schema, model
}
// Check default value changes
if !columnDefaultsEqual(modelCol.Default, currentCol.Default) {
if !isSerialWithoutDefault(modelCol) && !columnDefaultsEqual(modelCol.Default, currentCol.Default) {
setDefault, defaultVal := formatColumnDefaultSQL(modelCol)
sql, err := w.executor.ExecuteAlterColumnDefaultWithCheck(AlterColumnDefaultWithCheckData{
@@ -738,7 +782,7 @@ func (w *MigrationWriter) generateForeignKeyScripts(model, current *models.Schem
if !shouldCreate {
if currentTable == nil {
shouldCreate = true
} else if currentConstraint, exists := currentTable.Constraints[constraintName]; !exists {
} else if currentConstraint, exists := lookupConstraint(currentTable.Constraints, constraintName); !exists {
shouldCreate = true
} else if !constraintsEqual(constraint, currentConstraint) {
shouldCreate = true
@@ -790,12 +834,20 @@ func (w *MigrationWriter) generateForeignKeyScripts(model, current *models.Schem
// generateCommentScripts generates COMMENT ON scripts using templates
func (w *MigrationWriter) generateCommentScripts(model, current *models.Schema) ([]MigrationScript, error) {
scripts := make([]MigrationScript, 0)
_ = current // TODO: Compare with current schema to only add new/changed comments
currentTables := make(map[string]*models.Table)
if current != nil {
for _, table := range current.Tables {
currentTables[strings.ToLower(table.Name)] = table
}
}
// Process each model table
for _, modelTable := range model.Tables {
// Table comment
if modelTable.Description != "" {
currentTable := currentTables[strings.ToLower(modelTable.Name)]
// Table comment (skipped when the live table already carries the same comment)
if modelTable.Description != "" && (currentTable == nil || strings.TrimSpace(currentTable.Description) != strings.TrimSpace(modelTable.Description)) {
sql, err := w.executor.ExecuteCommentTable(CommentTableData{
SchemaName: model.Name,
TableName: modelTable.Name,
@@ -818,7 +870,7 @@ func (w *MigrationWriter) generateCommentScripts(model, current *models.Schema)
// Column comments
for _, col := range sortColumns(modelTable.Columns) {
if col.Description != "" {
if col.Description != "" && !currentColumnHasDescription(currentTable, col) {
sql, err := w.executor.ExecuteCommentColumn(CommentColumnData{
SchemaName: model.Name,
TableName: modelTable.Name,
@@ -985,18 +1037,80 @@ func normalizeDefaultForCompare(value interface{}) string {
return ""
}
if s, ok := value.(string); ok {
return strings.TrimSpace(stripBackticks(s))
return normalizeDefaultLiteral(strings.TrimSpace(stripBackticks(s)))
}
return fmt.Sprintf("%v", value)
}
// normalizeDefaultLiteral removes a trailing ::type cast and the quotes around a plain string
// literal, so a DBML default ('[]') and the one a live database reports ('[]'::jsonb) compare equal.
func normalizeDefaultLiteral(s string) string {
if strings.HasPrefix(s, "'") {
if end := strings.LastIndex(s, "'"); end > 0 {
rest := strings.TrimSpace(s[end+1:])
if rest == "" || strings.HasPrefix(rest, "::") {
return normalizeJSONLiteral(strings.ReplaceAll(s[1:end], "''", "'"))
}
}
return s
}
if strings.HasPrefix(s, "(") && strings.Contains(s, ")::") {
return normalizeCastExpression(s)
}
return normalizeJSONLiteral(s)
}
// normalizeJSONLiteral compacts JSON text, since jsonb reports defaults re-spaced
// ('["a", "b"]') while the model usually holds them minified.
func normalizeJSONLiteral(s string) string {
if !strings.HasPrefix(s, "[") && !strings.HasPrefix(s, "{") {
return s
}
var buf bytes.Buffer
if err := json.Compact(&buf, []byte(s)); err != nil {
return s
}
return buf.String()
}
// normalizeCastExpression strips whitespace and the redundant parentheses PostgreSQL adds
// around an expression it casts: (gen_random_uuid())::text -> gen_random_uuid()::text.
func normalizeCastExpression(s string) string {
s = strings.Join(strings.Fields(s), "")
for strings.HasPrefix(s, "(") {
end := strings.LastIndex(s, ")::")
if end < 0 {
break
}
inner := s[1:end]
depth := 0
balanced := true
for _, r := range inner {
if r == '(' {
depth++
} else if r == ')' {
depth--
if depth < 0 {
balanced = false
break
}
}
}
if !balanced || depth != 0 {
break
}
s = inner + s[end+1:]
}
return s
}
func columnTypesEqual(col1, col2 *models.Column) bool {
if col1 == nil || col2 == nil {
return false
}
return strings.EqualFold(
pgsql.NormalizeEquivalentSQLType(effectiveColumnSQLType(col1)),
pgsql.NormalizeEquivalentSQLType(effectiveColumnSQLType(col2)),
normalizeZeroScale(pgsql.NormalizeEquivalentSQLType(effectiveAlterColumnSQLType(col1))),
normalizeZeroScale(pgsql.NormalizeEquivalentSQLType(effectiveAlterColumnSQLType(col2))),
)
}
@@ -1032,7 +1146,7 @@ func constraintsEqual(c1, c2 *models.Constraint) bool {
return false
}
}
if c1.OnDelete != c2.OnDelete || c1.OnUpdate != c2.OnUpdate {
if !fkActionsEqual(c1.OnDelete, c2.OnDelete) || !fkActionsEqual(c1.OnUpdate, c2.OnUpdate) {
return false
}
}
@@ -1048,7 +1162,7 @@ func indexesEqual(idx1, idx2 *models.Index) bool {
if idx1.Unique != idx2.Unique {
return false
}
if !strings.EqualFold(idx1.Type, idx2.Type) {
if !strings.EqualFold(normalizeIndexMethod(idx1.Type), normalizeIndexMethod(idx2.Type)) {
return false
}
if len(idx1.Columns) != len(idx2.Columns) {
@@ -1068,6 +1182,104 @@ func indexesEqual(idx1, idx2 *models.Index) bool {
return indexHintsEqual(indexStorageParameters(idx1.Comment), indexStorageParameters(idx2.Comment))
}
// normalizeZeroScale turns numeric(p,0) into numeric(p); PostgreSQL treats them as the same type.
func normalizeZeroScale(sqlType string) string {
if i := strings.Index(sqlType, "("); i >= 0 && strings.HasSuffix(sqlType, ",0)") && !strings.Contains(sqlType[i:], "[") {
return strings.TrimSuffix(sqlType, ",0)") + ")"
}
return sqlType
}
// maxIdentifierBytes is PostgreSQL's identifier length limit; longer names are truncated on creation.
const maxIdentifierBytes = 63
// pgIdentifier returns name as PostgreSQL stores it (truncated to 63 bytes).
func pgIdentifier(name string) string {
if len(name) > maxIdentifierBytes {
return name[:maxIdentifierBytes]
}
return name
}
// lookupConstraint finds a constraint by name, also matching names that differ only because
// PostgreSQL truncated an over-long identifier.
func lookupConstraint(constraints map[string]*models.Constraint, name string) (*models.Constraint, bool) {
if c, ok := constraints[name]; ok {
return c, true
}
want := pgIdentifier(name)
for n, c := range constraints {
if pgIdentifier(n) == want {
return c, true
}
}
return nil, false
}
// currentColumnHasDescription reports whether the live table's column already carries
// the model column's comment.
func currentColumnHasDescription(currentTable *models.Table, col *models.Column) bool {
if currentTable == nil {
return false
}
for _, cc := range currentTable.Columns {
if strings.EqualFold(cc.Name, col.Name) {
return strings.TrimSpace(cc.Description) == strings.TrimSpace(col.Description)
}
}
return false
}
// primaryKeyColumnsMatch reports whether the model table's primary key (an explicit PK
// constraint or columns flagged IsPrimaryKey) has the same columns, in order, as current.
func primaryKeyColumnsMatch(modelTable *models.Table, current *models.Constraint) bool {
var modelCols []string
for _, c := range sortConstraints(modelTable.Constraints) {
if c.Type == models.PrimaryKeyConstraint {
modelCols = c.Columns
break
}
}
if modelCols == nil {
for _, col := range getSortedColumns(modelTable.Columns) {
if col.IsPrimaryKey {
modelCols = append(modelCols, col.Name)
}
}
}
if len(modelCols) != len(current.Columns) {
return false
}
for i, col := range modelCols {
if !strings.EqualFold(col, current.Columns[i]) {
return false
}
}
return true
}
// normalizeIndexMethod maps an unspecified index method to PostgreSQL's default (btree),
// which is what a live database reports for it.
func normalizeIndexMethod(method string) string {
if strings.TrimSpace(method) == "" {
return "btree"
}
return method
}
// fkActionsEqual compares referential actions case-insensitively, treating an unspecified
// action as PostgreSQL's default (NO ACTION).
func fkActionsEqual(a, b string) bool {
norm := func(s string) string {
s = strings.ToUpper(strings.TrimSpace(s))
if s == "" {
return "NO ACTION"
}
return s
}
return norm(a) == norm(b)
}
// indexHintsEqual compares two optional index hints, treating an unspecified hint as a match.
func indexHintsEqual(hint1, hint2 string) bool {
if hint1 == "" || hint2 == "" {
@@ -219,6 +219,91 @@ func TestWriteMigration_AltersColumnNullabilityWhenNotNullDiffers(t *testing.T)
}
}
func TestWriteMigration_DoesNotDropDefaultOnGeneratedColumn(t *testing.T) {
current := models.InitDatabase("testdb")
currentSchema := models.InitSchema("public")
currentTable := models.InitTable("info_city", "public")
currentGeom := models.InitColumn("geom", "info_city", "public")
currentGeom.Type = "text"
currentGeom.Generated = true
currentGeom.GenerationExpression = "st_makepoint(lon, lat)"
currentTable.Columns["geom"] = currentGeom
currentSchema.Tables = append(currentSchema.Tables, currentTable)
current.Schemas = append(current.Schemas, currentSchema)
model := models.InitDatabase("testdb")
modelSchema := models.InitSchema("public")
modelTable := models.InitTable("info_city", "public")
modelGeom := models.InitColumn("geom", "info_city", "public")
modelGeom.Type = "text"
// DBML cannot express GENERATED ALWAYS columns, so the model declares
// this as a plain column with no default — it must not be treated as a
// default-value drift against the DB-side generated column.
modelTable.Columns["geom"] = modelGeom
modelSchema.Tables = append(modelSchema.Tables, modelTable)
model.Schemas = append(model.Schemas, modelSchema)
var buf bytes.Buffer
writer, err := NewMigrationWriter(&writers.WriterOptions{})
if err != nil {
t.Fatalf("Failed to create writer: %v", err)
}
writer.writer = &buf
if err := writer.WriteMigration(model, current); err != nil {
t.Fatalf("WriteMigration failed: %v", err)
}
output := buf.String()
if strings.Contains(output, "DROP DEFAULT") {
t.Fatalf("did not expect DROP DEFAULT against a generated column, got:\n%s", output)
}
}
func TestWriteMigration_DoesNotAlterIdentityColumn(t *testing.T) {
current := models.InitDatabase("testdb")
currentSchema := models.InitSchema("public")
currentTable := models.InitTable("users", "public")
currentID := models.InitColumn("id", "users", "public")
currentID.Type = "bigint"
currentID.NotNull = true
currentID.Identity = true
currentID.IdentityGeneration = "ALWAYS"
currentTable.Columns["id"] = currentID
currentSchema.Tables = append(currentSchema.Tables, currentTable)
current.Schemas = append(current.Schemas, currentSchema)
model := models.InitDatabase("testdb")
modelSchema := models.InitSchema("public")
modelTable := models.InitTable("users", "public")
modelID := models.InitColumn("id", "users", "public")
modelID.Type = "bigint"
// DBML cannot express GENERATED ... AS IDENTITY, so the model declares this
// as a plain nullable column — it must not be diffed as NOT NULL drift
// against the DB-side identity column (Postgres rejects DROP NOT NULL on
// an identity column).
modelID.NotNull = false
modelTable.Columns["id"] = modelID
modelSchema.Tables = append(modelSchema.Tables, modelTable)
model.Schemas = append(model.Schemas, modelSchema)
var buf bytes.Buffer
writer, err := NewMigrationWriter(&writers.WriterOptions{})
if err != nil {
t.Fatalf("Failed to create writer: %v", err)
}
writer.writer = &buf
if err := writer.WriteMigration(model, current); err != nil {
t.Fatalf("WriteMigration failed: %v", err)
}
output := buf.String()
if strings.Contains(output, "DROP NOT NULL") {
t.Fatalf("did not expect DROP NOT NULL against an identity column, got:\n%s", output)
}
}
func TestWriteMigration_UsesStorageTypeForSerialAlterStatements(t *testing.T) {
current := models.InitDatabase("testdb")
currentSchema := models.InitSchema("public")
+163
View File
@@ -0,0 +1,163 @@
package pgsql
import (
"bytes"
"strings"
"testing"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/writers"
)
func serialTestTable(colType string, def interface{}) *models.Table {
table := models.InitTable("login_event", "identity")
id := models.InitColumn("id_login_event", "login_event", "identity")
id.Type = colType
id.IsPrimaryKey = true
id.Default = def
table.Columns["id_login_event"] = id
return table
}
func serialTestDB(table *models.Table) *models.Database {
db := models.InitDatabase("testdb")
schema := models.InitSchema("identity")
schema.Tables = append(schema.Tables, table)
db.Schemas = append(db.Schemas, schema)
return db
}
func TestSequences_FollowPrimaryKeyDefault(t *testing.T) {
tests := []struct {
name string
table *models.Table
wantContain []string
wantAbsent []string
}{
{
name: "nextval default uses its own sequence",
table: serialTestTable("bigint", "nextval('identity.login_event_id_login_event_seq'::regclass)"),
wantContain: []string{
"login_event_id_login_event_seq",
"setval(",
},
wantAbsent: []string{"identity_login_event_id_login_event"},
},
{
name: "no default creates no sequence",
table: serialTestTable("bigint", nil),
wantAbsent: []string{"CREATE SEQUENCE", "setval("},
},
{
name: "bigserial without default creates no extra sequence",
table: serialTestTable("bigserial", nil),
wantAbsent: []string{"CREATE SEQUENCE", "setval(", "identity_login_event_id_login_event"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var buf bytes.Buffer
w := NewWriter(&writers.WriterOptions{})
w.writer = &buf
if err := w.WriteDatabase(serialTestDB(tt.table)); err != nil {
t.Fatalf("WriteDatabase failed: %v", err)
}
full := buf.String()
stmts, err := w.GenerateDatabaseStatements(serialTestDB(tt.table))
if err != nil {
t.Fatalf("GenerateDatabaseStatements failed: %v", err)
}
for label, out := range map[string]string{"WriteDatabase": full, "GenerateDatabaseStatements": diffJoin(stmts)} {
for _, s := range tt.wantContain {
if !strings.Contains(out, s) {
t.Errorf("%s: missing %q\n%s", label, s, out)
}
}
for _, s := range tt.wantAbsent {
if strings.Contains(out, s) {
t.Errorf("%s: unexpected %q\n%s", label, s, out)
}
}
}
})
}
}
func TestSetSequenceValue_OnlyMovesForwardPastData(t *testing.T) {
w := NewWriter(&writers.WriterOptions{})
table := serialTestTable("bigint", "nextval('identity.login_event_id_login_event_seq'::regclass)")
stmt, err := w.primaryKeySetvalStatement(serialTestDB(table).Schemas[0], table)
if err != nil {
t.Fatal(err)
}
for _, want := range []string{"MAX(", "is_called", "m_cnt > m_next", ", m_cnt, false)"} {
if !strings.Contains(stmt, want) {
t.Errorf("setval statement missing %q:\n%s", want, stmt)
}
}
}
func TestDiffStatements_ExistingTableNewSequenceIsSetPastData(t *testing.T) {
def := "nextval('identity.login_event_id_login_event_seq'::regclass)"
model := serialTestDB(serialTestTable("bigint", def))
current := serialTestDB(serialTestTable("bigint", nil))
w := NewWriter(&writers.WriterOptions{})
stmts, err := w.diffStatements(model, current)
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
out := diffJoin(stmts)
seqIdx := strings.Index(out, "CREATE SEQUENCE IF NOT EXISTS identity.login_event_id_login_event_seq")
setvalIdx := strings.Index(out, "setval(")
if seqIdx < 0 || setvalIdx < 0 || seqIdx > setvalIdx {
t.Fatalf("expected sequence creation before setval, got:\n%s", out)
}
// Sequence already present in the database: nothing to create or move.
current.Schemas[0].Sequences = append(current.Schemas[0].Sequences,
models.InitSequence("login_event_id_login_event_seq", "identity"))
stmts, err = w.diffStatements(model, current)
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
if strings.Contains(diffJoin(stmts), "setval(") || strings.Contains(diffJoin(stmts), "CREATE SEQUENCE") {
t.Fatalf("existing sequence must be left alone:\n%s", diffJoin(stmts))
}
}
func TestSerialWithoutDefault_DoesNotDropExistingDefault(t *testing.T) {
def := "nextval('identity.login_event_id_login_event_seq'::regclass)"
model := serialTestDB(serialTestTable("bigserial", nil))
current := serialTestDB(serialTestTable("bigserial", def))
w := NewWriter(&writers.WriterOptions{})
stmts, err := w.diffStatements(model, current)
if err != nil {
t.Fatalf("diffStatements failed: %v", err)
}
if strings.Contains(strings.ToUpper(diffJoin(stmts)), "DROP DEFAULT") {
t.Fatalf("serial default must not be dropped:\n%s", diffJoin(stmts))
}
alter, err := w.GenerateAlterColumnDefaultStatements(model.Schemas[0])
if err != nil {
t.Fatal(err)
}
if strings.Contains(strings.ToUpper(diffJoin(alter)), "DROP DEFAULT") {
t.Fatalf("full-DDL path must not drop serial default:\n%s", diffJoin(alter))
}
// A plain bigint with no model default still has its default dropped.
model = serialTestDB(serialTestTable("bigint", nil))
current = serialTestDB(serialTestTable("bigint", def))
stmts, err = w.diffStatements(model, current)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(strings.ToUpper(diffJoin(stmts)), "DROP DEFAULT") {
t.Fatalf("non-serial default removal should still be emitted:\n%s", diffJoin(stmts))
}
}
+23 -1
View File
@@ -77,6 +77,10 @@ type ColumnData struct {
Type string
Default string
NotNull bool
// Generated is the expression of a GENERATED ALWAYS AS (...) STORED column.
Generated string
// Identity is "ALWAYS" or "BY DEFAULT" for a GENERATED ... AS IDENTITY column.
Identity string
}
// AddColumnData contains data for add column template
@@ -87,6 +91,10 @@ type AddColumnData struct {
ColumnType string
Default string
NotNull bool
// Generated is the expression of a GENERATED ALWAYS AS (...) STORED column.
Generated string
// Identity is "ALWAYS" or "BY DEFAULT" for a GENERATED ... AS IDENTITY column.
Identity string
}
// AlterColumnTypeWithCheckData contains data for the guarded alter column
@@ -520,6 +528,19 @@ func (te *TemplateExecutor) ExecuteCreatePrimaryKeyWithAutoGenCheck(data CreateP
// Helper functions to build template data from models
// columnGenerationClauses returns the generation expression of a GENERATED ... STORED
// column and the generation mode of an identity column ("" for each that does not apply).
// Such columns cannot carry a DEFAULT.
func columnGenerationClauses(col *models.Column) (generated, identity string) {
if col.Generated && col.GenerationExpression != "" {
return col.GenerationExpression, ""
}
if col.Identity && col.IdentityGeneration != "" {
return "", strings.ToUpper(col.IdentityGeneration)
}
return "", ""
}
// BuildCreateTableData builds CreateTableData from a models.Table
func BuildCreateTableData(schemaName string, table *models.Table) CreateTableData {
columns := make([]ColumnData, 0, len(table.Columns))
@@ -532,7 +553,8 @@ func BuildCreateTableData(schemaName string, table *models.Table) CreateTableDat
Type: col.Type,
NotNull: col.NotNull,
}
if col.Default != nil {
colData.Generated, colData.Identity = columnGenerationClauses(col)
if col.Default != nil && colData.Generated == "" && colData.Identity == "" {
if value, ok := col.Default.(string); ok {
colData.Default = writers.QuoteDefaultValue(stripBackticks(value), col.Type)
} else {
+3 -1
View File
@@ -1,4 +1,6 @@
ALTER TABLE {{qual_table .SchemaName .TableName}}
ADD COLUMN IF NOT EXISTS {{quote_ident .ColumnName}} {{.ColumnType}}
{{- if .Default}} DEFAULT {{.Default}}{{end}}
{{- if .Generated}} GENERATED ALWAYS AS ({{.Generated}}) STORED{{end}}
{{- if .Identity}} GENERATED {{.Identity}} AS IDENTITY{{end}}
{{- if and .Default (not .Generated) (not .Identity)}} DEFAULT {{.Default}}{{end}}
{{- if .NotNull}} NOT NULL{{end}};
@@ -2,7 +2,9 @@ CREATE TABLE IF NOT EXISTS {{qual_table .SchemaName .TableName}} (
{{- range $i, $col := .Columns}}
{{- if $i}},{{end}}
{{quote_ident $col.Name}} {{$col.Type}}
{{- if $col.Default}} DEFAULT {{$col.Default}}{{end}}
{{- if $col.Generated}} GENERATED ALWAYS AS ({{$col.Generated}}) STORED{{end}}
{{- if $col.Identity}} GENERATED {{$col.Identity}} AS IDENTITY{{end}}
{{- if and $col.Default (not $col.Generated) (not $col.Identity)}} DEFAULT {{$col.Default}}{{end}}
{{- if $col.NotNull}} NOT NULL{{end}}
{{- end}}
);
+3 -1
View File
@@ -3,7 +3,9 @@
{{/* Column definition fragment */}}
{{- define "column_definition" -}}
{{.Name}} {{.Type}}
{{- if .Default}} DEFAULT {{.Default}}{{end}}
{{- if .Generated}} GENERATED ALWAYS AS ({{.Generated}}) STORED{{end}}
{{- if .Identity}} GENERATED {{.Identity}} AS IDENTITY{{end}}
{{- if and .Default (not .Generated) (not .Identity)}} DEFAULT {{.Default}}{{end}}
{{- if .NotNull}} NOT NULL{{end}}
{{- end -}}
@@ -1,6 +1,7 @@
DO $$
DECLARE
m_cnt bigint;
m_next bigint;
BEGIN
IF EXISTS (
SELECT 1 FROM pg_class c
@@ -12,8 +13,15 @@ BEGIN
SELECT COALESCE(MAX({{quote_ident .ColumnName}}), 0) + 1
FROM {{qual_table .SchemaName .TableName}}
INTO m_cnt;
PERFORM setval('{{qual_table_raw .SchemaName .SequenceName}}'::regclass, m_cnt);
SELECT CASE WHEN is_called THEN last_value + 1 ELSE last_value END
FROM {{qual_table .SchemaName .SequenceName}}
INTO m_next;
-- Only move the sequence forward; never rewind one that is already past the data.
IF m_cnt > m_next THEN
PERFORM setval('{{qual_table_raw .SchemaName .SequenceName}}'::regclass, m_cnt, false);
END IF;
END IF;
END;
$$;
+268 -47
View File
@@ -12,9 +12,10 @@ import (
"sync"
"time"
"git.warky.dev/wdevs/relspecgo/pkg/buildinfo"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/pgsql"
"git.warky.dev/wdevs/relspecgo/pkg/readers"
rpgsql "git.warky.dev/wdevs/relspecgo/pkg/readers/pgsql"
"git.warky.dev/wdevs/relspecgo/pkg/writers"
)
@@ -102,7 +103,7 @@ func (w *Writer) WriteDatabase(db *models.Database) error {
// Write header comment
fmt.Fprintf(w.writer, "-- PostgreSQL Database Schema\n")
fmt.Fprintf(w.writer, "-- Database: %s\n", db.Name)
fmt.Fprintf(w.writer, "-- Generated by %s\n", buildinfo.GeneratedComment())
fmt.Fprintf(w.writer, "-- Generated by RelSpec\n")
if w.options.ContinueOnError {
fmt.Fprintf(w.writer, "\\set ON_ERROR_STOP off\n")
}
@@ -126,7 +127,7 @@ func (w *Writer) GenerateDatabaseStatements(db *models.Database) ([]string, erro
// Add header comment
statements = append(statements, "-- PostgreSQL Database Schema")
statements = append(statements, fmt.Sprintf("-- Database: %s", db.Name))
statements = append(statements, "-- Generated by "+buildinfo.GeneratedComment())
statements = append(statements, "-- Generated by RelSpec")
// Process each schema in the database
for _, schema := range db.Schemas {
@@ -140,6 +141,50 @@ func (w *Writer) GenerateDatabaseStatements(db *models.Database) ([]string, erro
return statements, nil
}
// primaryKeySequenceName returns the name of the sequence behind a table's integer
// primary key nextval() default, or "" when the key has no such default.
func primaryKeySequenceName(table *models.Table) string {
pk := table.GetPrimaryKey()
if pk == nil || !isIntegerType(pk.Type) || pk.Default == nil {
return ""
}
defaultStr, ok := pk.Default.(string)
if !ok || !strings.Contains(strings.ToLower(defaultStr), "nextval") {
return ""
}
return extractSequenceName(defaultStr)
}
// primaryKeySequenceStatement returns the CREATE SEQUENCE statement backing a table's
// integer primary key nextval() default, or "" when the table has no such sequence.
func (w *Writer) primaryKeySequenceStatement(schema *models.Schema, table *models.Table) string {
seqName := primaryKeySequenceName(table)
if seqName == "" {
return ""
}
return fmt.Sprintf("CREATE SEQUENCE IF NOT EXISTS %s\n INCREMENT 1\n MINVALUE 1\n MAXVALUE 9223372036854775807\n START 1\n CACHE 1",
w.qualTable(schema.SQLName(), seqName))
}
// primaryKeySetvalStatement returns the statement that moves a table's primary key
// sequence past the existing rows, or "" when the table has no such sequence.
func (w *Writer) primaryKeySetvalStatement(schema *models.Schema, table *models.Table) (string, error) {
seqName := primaryKeySequenceName(table)
if seqName == "" {
return "", nil
}
pk := table.GetPrimaryKey()
return w.executor.ExecuteSetSequenceValue(SetSequenceValueData{
SchemaName: schema.Name,
TableName: table.Name,
SequenceName: seqName,
ColumnName: pk.Name,
})
}
// GenerateSchemaStatements generates SQL statements as a list for a single schema
func (w *Writer) GenerateSchemaStatements(schema *models.Schema) ([]string, error) {
statements := []string{}
@@ -160,24 +205,9 @@ func (w *Writer) GenerateSchemaStatements(schema *models.Schema) ([]string, erro
// Phase 2: Create sequences
for _, table := range schema.Tables {
pk := table.GetPrimaryKey()
if pk == nil || !isIntegerType(pk.Type) || pk.Default == "" {
continue
if stmt := w.primaryKeySequenceStatement(schema, table); stmt != "" {
statements = append(statements, stmt)
}
defaultStr, ok := pk.Default.(string)
if !ok || !strings.Contains(strings.ToLower(defaultStr), "nextval") {
continue
}
seqName := extractSequenceName(defaultStr)
if seqName == "" {
continue
}
stmt := fmt.Sprintf("CREATE SEQUENCE IF NOT EXISTS %s\n INCREMENT 1\n MINVALUE 1\n MAXVALUE 9223372036854775807\n START 1\n CACHE 1",
w.qualTable(schema.SQLName(), seqName))
statements = append(statements, stmt)
}
// Phase 3: Create tables
@@ -392,6 +422,17 @@ func (w *Writer) GenerateSchemaStatements(schema *models.Schema) ([]string, erro
}
}
// Phase 6.5: Move primary key sequences past existing rows
for _, table := range schema.Tables {
stmt, err := w.primaryKeySetvalStatement(schema, table)
if err != nil {
return nil, fmt.Errorf("failed to generate set sequence value for %s.%s: %w", schema.Name, table.Name, err)
}
if stmt != "" {
statements = append(statements, stmt)
}
}
// Phase 7: Comments
for _, table := range schema.Tables {
if table.Comment != "" {
@@ -462,6 +503,12 @@ func (w *Writer) GenerateAlterColumnTypeStatements(schema *models.Schema) ([]str
for _, table := range schema.Tables {
columns := getSortedColumns(table.Columns)
for _, col := range columns {
if col.Generated || col.Identity {
// Type is derived from the generation expression (generated columns) or
// tied to the backing sequence (identity columns); leave both alone here
// rather than risk an ALTER COLUMN TYPE Postgres won't accept cleanly.
continue
}
targetType := effectiveAlterColumnSQLType(col)
stmt, err := w.executor.ExecuteAlterColumnTypeWithCheck(AlterColumnTypeWithCheckData{
SchemaName: schema.Name,
@@ -492,6 +539,19 @@ func (w *Writer) GenerateAlterColumnDefaultStatements(schema *models.Schema) ([]
for _, table := range schema.Tables {
columns := getSortedColumns(table.Columns)
for _, col := range columns {
if col.Generated || col.Identity {
// Generated columns: Postgres stores the generation expression in
// pg_attrdef just like a real default, so the guard template would see
// it as a stray default and emit DROP DEFAULT, which Postgres rejects.
// Identity columns never carry a DEFAULT at all (they're driven by a
// backing sequence), so there is nothing for this generator to manage.
continue
}
if isSerialWithoutDefault(col) {
// serial/bigserial columns get their nextval() default from the
// type itself; a model with no explicit default must not drop it.
continue
}
setDefault, defaultVal := formatColumnDefaultSQL(col)
stmt, err := w.executor.ExecuteAlterColumnDefaultWithCheck(AlterColumnDefaultWithCheckData{
SchemaName: schema.Name,
@@ -510,6 +570,19 @@ func (w *Writer) GenerateAlterColumnDefaultStatements(schema *models.Schema) ([]
return statements, nil
}
// isSerialWithoutDefault reports whether col is a serial-family column with no
// explicit default, i.e. one whose nextval() default is implied by its type.
func isSerialWithoutDefault(col *models.Column) bool {
if col == nil || col.Default != nil {
return false
}
switch strings.ToLower(strings.TrimSpace(col.Type)) {
case "serial", "bigserial", "smallserial", "serial4", "serial8", "serial2":
return true
}
return false
}
// formatColumnDefaultSQL renders a column's model-level default into the
// SQL literal/expression used by ALTER COLUMN ... SET DEFAULT, shared by
// the full-schema writer and the diff-based migration writer.
@@ -556,7 +629,7 @@ func (w *Writer) GenerateAddColumnsForDatabase(db *models.Database) ([]string, e
statements = append(statements, "-- Add missing columns to existing tables")
statements = append(statements, fmt.Sprintf("-- Database: %s", db.Name))
statements = append(statements, "-- Generated by "+buildinfo.GeneratedComment())
statements = append(statements, "-- Generated by RelSpec")
for _, schema := range db.Schemas {
schemaStatements, err := w.GenerateAddColumnStatements(schema)
@@ -605,13 +678,25 @@ func (w *Writer) generateColumnDefinition(col *models.Column) string {
parts = append(parts, effectiveColumnSQLType(col))
// GENERATED ALWAYS ... STORED columns are computed from an expression and
// cannot also carry a DEFAULT, so this takes the place of the DEFAULT clause.
if col.Generated && col.GenerationExpression != "" {
parts = append(parts, fmt.Sprintf("GENERATED ALWAYS AS (%s) STORED", col.GenerationExpression))
}
// GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY columns are driven by an
// internal sequence, not a literal DEFAULT, and cannot carry one either.
if col.Identity && col.IdentityGeneration != "" {
parts = append(parts, fmt.Sprintf("GENERATED %s AS IDENTITY", col.IdentityGeneration))
}
// NOT NULL
if col.NotNull {
parts = append(parts, "NOT NULL")
}
// DEFAULT
if col.Default != nil {
if !col.Generated && !col.Identity && col.Default != nil {
switch v := col.Default.(type) {
case string:
parts = append(parts, fmt.Sprintf("DEFAULT %s", writers.QuoteDefaultValue(stripBackticks(v), col.Type)))
@@ -847,18 +932,12 @@ func (w *Writer) writeSequences(schema *models.Schema) error {
fmt.Fprintf(w.writer, "-- Sequences for schema: %s\n", schema.Name)
for _, table := range schema.Tables {
pk := table.GetPrimaryKey()
if pk == nil {
// Only create the sequence the primary key's nextval() default uses
seqName := primaryKeySequenceName(table)
if seqName == "" {
continue
}
// Only create sequences for integer-type PKs with identity
if !isIntegerType(pk.Type) {
continue
}
seqName := fmt.Sprintf("identity_%s_%s", table.SQLName(), pk.SQLName())
data := CreateSequenceData{
SchemaName: schema.Name,
SequenceName: seqName,
@@ -1233,13 +1312,18 @@ func (w *Writer) writeForeignKeys(schema *models.Schema) error {
fkName = fmt.Sprintf("fk_%s_%s", table.SQLName(), rel.ToTable)
}
// Find the foreign key constraint that matches this relationship
// Find the foreign key constraint that matches this relationship.
// Prefer the exact name: with several FKs to the same table, a
// referenced-table match alone picks an arbitrary one.
var fkConstraint *models.Constraint
for _, constraint := range table.Constraints {
if constraint.Type == models.ForeignKeyConstraint &&
(constraint.Name == fkName || constraint.ReferencedTable == rel.ToTable) {
fkConstraint = constraint
break
if c, ok := table.Constraints[fkName]; ok && c.Type == models.ForeignKeyConstraint {
fkConstraint = c
} else {
for _, constraint := range sortConstraints(table.Constraints) {
if constraint.Type == models.ForeignKeyConstraint && constraint.ReferencedTable == rel.ToTable {
fkConstraint = constraint
break
}
}
}
@@ -1393,12 +1477,11 @@ func (w *Writer) writeSetSequenceValues(schema *models.Schema) error {
fmt.Fprintf(w.writer, "-- Set sequence values for schema: %s\n", schema.Name)
for _, table := range schema.Tables {
pk := table.GetPrimaryKey()
if pk == nil || !isIntegerType(pk.Type) {
seqName := primaryKeySequenceName(table)
if seqName == "" {
continue
}
seqName := fmt.Sprintf("identity_%s_%s", table.SQLName(), pk.SQLName())
pk := table.GetPrimaryKey()
// Use template executor to generate set sequence value statement
data := SetSequenceValueData{
@@ -1963,7 +2046,9 @@ func extractSequenceName(defaultExpr string) string {
return fullName
}
// executeDatabaseSQL executes SQL statements directly on a PostgreSQL database
// executeDatabaseSQL applies db to a PostgreSQL database. By default it reads the live schema
// and executes only the differences; Metadata["full_ddl"]=true (or a failed/unsupported live
// read) executes the full idempotent DDL instead.
func (w *Writer) executeDatabaseSQL(db *models.Database, connString string) error {
// Initialize execution report
w.executionReport = &ExecutionReport{
@@ -1972,13 +2057,145 @@ func (w *Writer) executeDatabaseSQL(db *models.Database, connString string) erro
Errors: make([]ExecutionError, 0),
}
// Generating a large schema can take time before any statement is executed.
fmt.Fprintln(os.Stderr, " → Generating PostgreSQL statements...")
statements, err := w.GenerateDatabaseStatements(db)
if err != nil {
return fmt.Errorf("failed to generate SQL statements: %w", err)
var statements []string
if fullDDL, _ := w.options.Metadata["full_ddl"].(bool); !fullDDL {
diffStatements, err := w.generateLiveDiffStatements(db, connString)
if err != nil {
fmt.Fprintf(os.Stderr, "⚠ Warning: live diff unavailable (%v); falling back to full DDL\n", err)
} else {
statements = diffStatements
if len(statements) == 0 {
w.executionReport.EndTime = getCurrentTimestamp()
fmt.Fprintln(os.Stderr, "✓ Database is already up to date; nothing to execute")
return w.finishReport()
}
}
}
if statements == nil {
// Generating a large schema can take time before any statement is executed.
fmt.Fprintln(os.Stderr, " → Generating PostgreSQL statements...")
var err error
statements, err = w.GenerateDatabaseStatements(db)
if err != nil {
return fmt.Errorf("failed to generate SQL statements: %w", err)
}
}
return w.executeStatements(statements, connString)
}
// generateLiveDiffStatements reads the live database and returns only the statements needed
// to bring it in line with db. An error means the diff could not be computed.
func (w *Writer) generateLiveDiffStatements(db *models.Database, connString string) ([]string, error) {
if w.options.FlattenSchema {
return nil, fmt.Errorf("flatten_schema output cannot be compared with the live schemas")
}
fmt.Fprintln(os.Stderr, " → Reading live database to compute differences...")
current, err := rpgsql.NewReader(&readers.ReaderOptions{ConnectionString: connString}).ReadDatabase()
if err != nil {
return nil, fmt.Errorf("failed to read live database: %w", err)
}
return w.diffStatements(db, current)
}
// diffStatements returns the ordered statements that migrate current to model.
func (w *Writer) diffStatements(model, current *models.Database) ([]string, error) {
currentSchemas := make(map[string]*models.Schema)
for _, cs := range current.Schemas {
if cs != nil {
currentSchemas[strings.ToLower(cs.Name)] = cs
}
}
statements := []string{}
// Primary key sequences that must be moved past existing rows once the
// migration scripts (which set the column defaults) have run.
var setvalStatements []string
for _, schema := range model.Schemas {
if schema == nil {
continue
}
if err := w.checkDirectives(schema); err != nil {
return nil, err
}
cs := currentSchemas[strings.ToLower(schema.Name)]
if cs == nil && schema.Name != "public" {
statements = append(statements,
fmt.Sprintf("-- Schema: %s", schema.Name),
fmt.Sprintf("CREATE SCHEMA IF NOT EXISTS %s", schema.SQLName()))
}
existing := make(map[string]*models.Table)
knownSequences := make(map[string]bool)
if cs != nil {
for _, t := range cs.Tables {
existing[strings.ToLower(t.Name)] = t
}
for _, seq := range cs.Sequences {
knownSequences[strings.ToLower(seq.Name)] = true
}
}
for _, table := range schema.Tables {
currentTable, ok := existing[strings.ToLower(table.Name)]
if !ok {
// New table: no rows yet, so the sequence only needs creating.
if stmt := w.primaryKeySequenceStatement(schema, table); stmt != "" {
statements = append(statements, stmt)
}
continue
}
// Existing table whose primary key is being pointed at a sequence the
// database does not have yet: create it and move it past the data,
// otherwise it restarts at 1 and collides with existing rows.
seqName := primaryKeySequenceName(table)
if seqName == "" || knownSequences[strings.ToLower(seqName)] {
continue
}
if cpk := currentTable.GetPrimaryKey(); cpk != nil && columnDefaultsEqual(table.GetPrimaryKey().Default, cpk.Default) {
continue
}
statements = append(statements, w.primaryKeySequenceStatement(schema, table))
setval, err := w.primaryKeySetvalStatement(schema, table)
if err != nil {
return nil, fmt.Errorf("failed to generate set sequence value for %s.%s: %w", schema.Name, table.Name, err)
}
setvalStatements = append(setvalStatements, setval)
}
}
mw, err := NewMigrationWriter(w.options)
if err != nil {
return nil, err
}
scripts, err := mw.GenerateScripts(model, current)
if err != nil {
return nil, fmt.Errorf("failed to generate migration scripts: %w", err)
}
lastSchema := ""
for _, script := range scripts {
body := strings.TrimSpace(script.Body)
if body == "" {
continue
}
if script.Schema != "" && script.Schema != lastSchema {
statements = append(statements, fmt.Sprintf("-- Schema: %s", script.Schema))
lastSchema = script.Schema
}
statements = append(statements, body)
}
statements = append(statements, setvalStatements...)
return statements, nil
}
// executeStatements runs statements one by one against the database and writes the report.
func (w *Writer) executeStatements(statements []string, connString string) error {
w.executionReport.TotalStatements = len(statements)
// Connect to database
@@ -2078,7 +2295,11 @@ func (w *Writer) executeDatabaseSQL(db *models.Database, connString string) erro
}
w.executionReport.EndTime = getCurrentTimestamp()
return w.finishReport()
}
// finishReport writes the optional report file and prints the execution summary.
func (w *Writer) finishReport() error {
// Write report if path is specified
if reportPath, ok := w.options.Metadata["report_path"].(string); ok && reportPath != "" {
if err := w.writeReport(reportPath); err != nil {
+88
View File
@@ -1510,3 +1510,91 @@ func TestIndexStorageParameters(t *testing.T) {
})
}
}
func TestGenerateColumnDefinition_GeneratedColumnEmitsGeneratedClauseNotDefault(t *testing.T) {
w := &Writer{}
col := models.InitColumn("geom", "info_city", "public")
col.Type = "text"
col.NotNull = true
col.Generated = true
col.GenerationExpression = "st_makepoint(lon, lat)"
// A generated column's introspected "default" is actually its generation
// expression surfaced via pg_attrdef; it must never be rendered as DEFAULT.
col.Default = "st_makepoint(lon, lat)"
got := w.generateColumnDefinition(col)
want := "geom text GENERATED ALWAYS AS (st_makepoint(lon, lat)) STORED NOT NULL"
if got != want {
t.Fatalf("generateColumnDefinition() = %q, want %q", got, want)
}
}
func TestGenerateColumnDefinition_IdentityColumnEmitsIdentityClauseNotDefault(t *testing.T) {
w := &Writer{}
col := models.InitColumn("id", "users", "public")
col.Type = "bigint"
col.NotNull = true
col.Identity = true
col.IdentityGeneration = "ALWAYS"
// Identity columns never carry a literal default; this must never surface.
col.Default = "some-stray-value"
got := w.generateColumnDefinition(col)
want := "id bigint GENERATED ALWAYS AS IDENTITY NOT NULL"
if got != want {
t.Fatalf("generateColumnDefinition() = %q, want %q", got, want)
}
}
// Two FKs to the same table: each relationship must emit its own FK columns,
// not whichever FK map iteration returns first.
func TestWriteDatabase_MultipleForeignKeysToSameTable(t *testing.T) {
db := models.InitDatabase("testdb")
schema := models.InitSchema("public")
dept := models.InitTable("department", "public")
id := models.InitColumn("id_department", "department", "public")
id.Type = "bigint"
id.IsPrimaryKey = true
dept.Columns["id_department"] = id
emp := models.InitTable("employee", "public")
for _, name := range []string{"rid_department", "rid_manager"} {
col := models.InitColumn(name, "employee", "public")
col.Type = "bigint"
emp.Columns[name] = col
fkName := "fk_employee_" + name
fk := models.InitConstraint(fkName, models.ForeignKeyConstraint)
fk.Schema, fk.Table, fk.Columns = "public", "employee", []string{name}
fk.ReferencedSchema, fk.ReferencedTable, fk.ReferencedColumns = "public", "department", []string{"id_department"}
emp.Constraints[fkName] = fk
rel := models.InitRelationship("employee_to_department_"+name, models.OneToMany)
rel.FromTable, rel.ToTable, rel.ToSchema, rel.ForeignKey = "employee", "department", "public", fkName
emp.Relationships[rel.Name] = rel
}
schema.Tables = append(schema.Tables, dept, emp)
db.Schemas = append(db.Schemas, schema)
for i := 0; i < 20; i++ {
var buf bytes.Buffer
writer := NewWriter(&writers.WriterOptions{})
writer.writer = &buf
if err := writer.WriteDatabase(db); err != nil {
t.Fatalf("WriteDatabase failed: %v", err)
}
output := buf.String()
for _, name := range []string{"rid_department", "rid_manager"} {
want := "ADD CONSTRAINT fk_employee_" + name + "\n FOREIGN KEY (" + name + ")"
if !strings.Contains(output, want) {
t.Fatalf("run %d: missing %q in output:\n%s", i, want, output)
}
}
}
}
+1
View File
@@ -229,6 +229,7 @@ func main() {
## Notes
- Generated columns are written as `col TYPE GENERATED ALWAYS AS (expr) STORED` (no DEFAULT)
- Schema flattening is **always enabled** for SQLite output (cannot be disabled); the default schema (`public`/`main`) produces bare table names, other schemas are prefixed
- Constraint and index names are prefixed with the flattened table name to avoid collisions
- Generated SQL is compatible with SQLite 3.x
@@ -1,7 +1,7 @@
CREATE TABLE {{quote_ident (qualified_table_name .Schema .Name)}} (
{{- $hasAutoIncrement := false}}
{{- range $i, $col := .Columns}}{{if $i}},{{end}}
{{quote_ident $col.Name}} {{map_type $col.Type}}{{column_collate $col}}{{if is_autoincrement $col}}{{$hasAutoIncrement = true}} PRIMARY KEY AUTOINCREMENT{{else}}{{if $col.NotNull}} NOT NULL{{end}}{{if ne (format_default $col) ""}} DEFAULT {{format_default $col}}{{end}}{{end}}
{{quote_ident $col.Name}} {{map_type $col.Type}}{{column_collate $col}}{{if and $col.Generated $col.GenerationExpression}} GENERATED ALWAYS AS ({{$col.GenerationExpression}}) STORED{{if $col.NotNull}} NOT NULL{{end}}{{else if is_autoincrement $col}}{{$hasAutoIncrement = true}} PRIMARY KEY AUTOINCREMENT{{else}}{{if $col.NotNull}} NOT NULL{{end}}{{if ne (format_default $col) ""}} DEFAULT {{format_default $col}}{{end}}{{end}}
{{- end}}
{{- if and .PrimaryKey (not $hasAutoIncrement)}}{{if gt (len .Columns) 0}},{{end}}
PRIMARY KEY ({{range $i, $colName := .PrimaryKey.Columns}}{{if $i}}, {{end}}{{quote_ident $colName}}{{end}})
+1 -2
View File
@@ -10,7 +10,6 @@ import (
_ "modernc.org/sqlite" // SQLite driver
"git.warky.dev/wdevs/relspecgo/pkg/buildinfo"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/writers"
)
@@ -73,7 +72,7 @@ func (w *Writer) writeContent(db *models.Database) error {
// Write header comment
fmt.Fprintf(w.writer, "-- SQLite Database Schema\n")
fmt.Fprintf(w.writer, "-- Database: %s\n", db.Name)
fmt.Fprintf(w.writer, "-- Generated by %s\n", buildinfo.GeneratedComment())
fmt.Fprintf(w.writer, "-- Generated by RelSpec\n")
fmt.Fprintf(w.writer, "-- Note: SQLite has no schema concept; non-default schema names are flattened into table name prefixes (e.g., auth.sessions -> auth_sessions)\n\n")
// Enable foreign keys
+36
View File
@@ -421,3 +421,39 @@ func TestQuoteIdentifier(t *testing.T) {
}
}
}
func TestWriteDatabase_GeneratedColumn(t *testing.T) {
table := models.InitTable("people", "public")
first := models.InitColumn("first", "people", "public")
first.Type = "text"
first.Sequence = 1
table.Columns["first"] = first
full := models.InitColumn("full_name", "people", "public")
full.Type = "text"
full.Sequence = 2
full.NotNull = true
full.Generated = true
full.GenerationExpression = "first || ' ' || first"
full.Default = "ignored"
table.Columns["full_name"] = full
schema := models.InitSchema("public")
schema.Tables = append(schema.Tables, table)
db := models.InitDatabase("testdb")
db.Schemas = append(db.Schemas, schema)
var buf bytes.Buffer
writer := NewWriter(&writers.WriterOptions{})
writer.writer = &buf
if err := writer.WriteDatabase(db); err != nil {
t.Fatalf("WriteDatabase failed: %v", err)
}
want := `"full_name" TEXT GENERATED ALWAYS AS (first || ' ' || first) STORED NOT NULL`
if !strings.Contains(buf.String(), want) {
t.Errorf("missing %q in:\n%s", want, buf.String())
}
if strings.Contains(buf.String(), "DEFAULT") && strings.Contains(buf.String(), "ignored") {
t.Errorf("generated column must not carry a DEFAULT:\n%s", buf.String())
}
}
+6
View File
@@ -146,6 +146,12 @@ export class Post {
})
```
Generated and identity columns:
- `asExpression: '...'`, `generatedType: 'STORED'` - generated column (no default)
- `@PrimaryGeneratedColumn('identity', { generatedIdentity: 'ALWAYS' })` - identity primary key
- `@Generated('identity')` + `generatedIdentity: '...'` - identity column that is not the primary key
## Type Mapping
| SQL Type | TypeScript Type | TypeORM Type |
@@ -0,0 +1,76 @@
package typeorm
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"git.warky.dev/wdevs/relspecgo/pkg/models"
"git.warky.dev/wdevs/relspecgo/pkg/readers"
typeormreader "git.warky.dev/wdevs/relspecgo/pkg/readers/typeorm"
"git.warky.dev/wdevs/relspecgo/pkg/writers"
)
func TestGeneratedAndIdentityColumnsRoundTrip(t *testing.T) {
table := models.InitTable("people", "public")
id := models.InitColumn("id", "people", "public")
id.Type = "bigint"
id.IsPrimaryKey = true
id.NotNull = true
id.AutoIncrement = true
id.Identity = true
id.IdentityGeneration = "ALWAYS"
table.Columns["id"] = id
seq := models.InitColumn("seq", "people", "public")
seq.Type = "integer"
seq.NotNull = true
seq.Identity = true
seq.IdentityGeneration = "BY DEFAULT"
table.Columns["seq"] = seq
first := models.InitColumn("first", "people", "public")
first.Type = "text"
first.NotNull = true
table.Columns["first"] = first
full := models.InitColumn("full_name", "people", "public")
full.Type = "text"
full.NotNull = true
full.Generated = true
full.GenerationExpression = "first || ' ' || upper(first)"
table.Columns["full_name"] = full
schema := models.InitSchema("public")
schema.Tables = append(schema.Tables, table)
db := models.InitDatabase("testdb")
db.Schemas = append(db.Schemas, schema)
out := filepath.Join(t.TempDir(), "entities.ts")
require.NoError(t, NewWriter(&writers.WriterOptions{OutputPath: out}).WriteDatabase(db))
content, err := os.ReadFile(out)
require.NoError(t, err)
assert.Contains(t, string(content), `asExpression: 'first || \' \' || upper(first)'`)
assert.Contains(t, string(content), "generatedType: 'STORED'")
assert.Contains(t, string(content), "@PrimaryGeneratedColumn('identity', { generatedIdentity: 'ALWAYS' })")
assert.Contains(t, string(content), "@Generated('identity')")
assert.Contains(t, string(content), "Generated } from 'typeorm'")
got, err := typeormreader.NewReader(&readers.ReaderOptions{FilePath: out}).ReadDatabase()
require.NoError(t, err)
cols := got.Schemas[0].Tables[0].Columns
assert.True(t, cols["id"].Identity)
assert.Equal(t, "ALWAYS", cols["id"].IdentityGeneration)
assert.True(t, cols["seq"].Identity)
assert.Equal(t, "BY DEFAULT", cols["seq"].IdentityGeneration)
assert.True(t, cols["full_name"].Generated)
assert.Equal(t, "first || ' ' || upper(first)", cols["full_name"].GenerationExpression)
assert.False(t, cols["first"].Generated)
assert.False(t, cols["first"].Identity)
}
+35 -3
View File
@@ -97,6 +97,7 @@ func (w *Writer) generateImports(db *models.Database) string {
needsCreateDate := false
needsUpdateDate := false
needsViewEntity := false
needsGenerated := false
for _, schema := range db.Schemas {
// Check for views
@@ -107,6 +108,9 @@ func (w *Writer) generateImports(db *models.Database) string {
for _, table := range schema.Tables {
// Check for timestamp columns
for _, col := range table.Columns {
if isNonKeyIdentity(col) {
needsGenerated = true
}
if col.Default == "now()" {
needsCreateDate = true
}
@@ -161,6 +165,9 @@ func (w *Writer) generateImports(db *models.Database) string {
if needsViewEntity {
imports = append(imports, "ViewEntity")
}
if needsGenerated {
imports = append(imports, "Generated")
}
return fmt.Sprintf("import { %s } from 'typeorm';\n", strings.Join(imports, ", "))
}
@@ -299,14 +306,16 @@ func (w *Writer) columnToField(col *models.Column, table *models.Table) string {
// Generate decorator
if col.IsPrimaryKey {
if col.AutoIncrement {
if col.Identity && col.IdentityGeneration != "" {
fmt.Fprintf(&sb, " @PrimaryGeneratedColumn('identity', { generatedIdentity: '%s' })\n", strings.ToUpper(col.IdentityGeneration))
} else if col.AutoIncrement {
sb.WriteString(" @PrimaryGeneratedColumn('increment')\n")
} else if col.Type == "uuid" || strings.Contains(fmt.Sprint(col.Default), "uuid") {
sb.WriteString(" @PrimaryGeneratedColumn('uuid')\n")
} else {
sb.WriteString(" @PrimaryGeneratedColumn()\n")
}
} else if col.Default == "now()" {
} else if col.Default == "now()" && !col.Generated {
sb.WriteString(" @CreateDateColumn()\n")
} else if strings.Contains(col.Comment, "auto-update") {
sb.WriteString(" @UpdateDateColumn()\n")
@@ -318,6 +327,9 @@ func (w *Writer) columnToField(col *models.Column, table *models.Table) string {
} else {
sb.WriteString(" @Column()\n")
}
if isNonKeyIdentity(col) {
sb.WriteString(" @Generated('identity')\n")
}
}
// Generate field declaration
@@ -351,8 +363,18 @@ func (w *Writer) buildColumnOptions(col *models.Column, table *models.Table) str
options = append(options, "unique: true")
}
// Generated column (GENERATED ALWAYS AS (expr) STORED)
if col.Generated && col.GenerationExpression != "" {
options = append(options, fmt.Sprintf("asExpression: '%s'", escapeSingleQuoted(col.GenerationExpression)), "generatedType: 'STORED'")
}
// Identity column that is not the primary key
if isNonKeyIdentity(col) {
options = append(options, fmt.Sprintf("generatedIdentity: '%s'", strings.ToUpper(col.IdentityGeneration)))
}
// Default
if col.Default != nil && col.Default != "now()" {
if !col.Generated && !col.Identity && col.Default != nil && col.Default != "now()" {
defaultStr := fmt.Sprint(col.Default)
if defaultStr != "" {
options = append(options, fmt.Sprintf("default: '%s'", defaultStr))
@@ -362,6 +384,16 @@ func (w *Writer) buildColumnOptions(col *models.Column, table *models.Table) str
return strings.Join(options, ", ")
}
// isNonKeyIdentity reports whether col is a GENERATED ... AS IDENTITY column outside the primary key.
func isNonKeyIdentity(col *models.Column) bool {
return col.Identity && col.IdentityGeneration != "" && !col.IsPrimaryKey
}
// escapeSingleQuoted escapes a value for use inside a single-quoted TypeScript string.
func escapeSingleQuoted(s string) string {
return strings.ReplaceAll(strings.ReplaceAll(s, `\`, `\\`), `'`, `\'`)
}
// needsExplicitType checks if a SQL type needs explicit type declaration
func (w *Writer) needsExplicitType(sqlType string) bool {
// Types that don't map cleanly to TypeScript types need explicit declaration