feat: initial plan
This commit is contained in:
Hein
@@ -0,0 +1,124 @@
|
||||
## Project Overview
|
||||
|
||||
**PgTidy** is a PostgreSQL-focused linter and formatter written in Go. It enforces a
|
||||
consistent SQL style, detects PostgreSQL-specific issues, and formats migrations, schemas,
|
||||
functions, procedures, triggers, and related database code. The primary day-one use case is
|
||||
formatting PL/pgSQL stored procedures to a defined house style.
|
||||
|
||||
It ships as a CLI, an LSP server, a VSCode extension, and (later) a DataGrip integration.
|
||||
|
||||
# Agent Rules
|
||||
|
||||
Keep your answers short. Question everything, never assume or guess. Ask the user if you are
|
||||
unsure about anything.
|
||||
|
||||
# Architecture
|
||||
|
||||
PgTidy uses a **hybrid** strategy:
|
||||
|
||||
- **Formatting** is driven by a custom **lossless lexer + CST** (concrete syntax tree) that
|
||||
preserves every comment and whitespace span, so formatting can round-trip safely.
|
||||
- **Linting** (later milestones) is driven by the real PostgreSQL grammar via
|
||||
`go-pgquery` (libpg_query compiled to WASM with wazero — no cgo), which yields an accurate
|
||||
AST for deep semantic rules.
|
||||
|
||||
A single Go core backs every frontend (CLI, LSP, editors) and shares one `diagnostics` type.
|
||||
|
||||
```
|
||||
cmd/pgtidy/ — CLI entry; subcommands: fmt, lint (v2), lsp (v3), version
|
||||
pkg/lexer/ — lossless lexer: tokens + trivia (comments/whitespace) attached
|
||||
pkg/cst/ — concrete syntax tree (round-trippable node model)
|
||||
pkg/parser/ — recursive-descent parser → CST (DML + DDL + PL/pgSQL)
|
||||
pkg/format/ — Doc-IR printer (Wadler/Prettier-style) + style application
|
||||
pkg/config/ — .pgtidy.yaml discovery/merge: style + rule config
|
||||
pkg/diagnostics/ — shared diagnostic type (CLI + LSP)
|
||||
pkg/pgast/ — go-pgquery wrapper: SQL → real PG AST (lint, v2)
|
||||
pkg/lint/ — rule engine + rule packs (v2)
|
||||
pkg/lsp/ — LSP server (v3)
|
||||
editors/vscode/ — VSCode extension (v3)
|
||||
editors/datagrip/ — LSP4IJ integration (v4)
|
||||
testdata/corpus/ — real-world .pgsql procedures used as the safety/idempotence harness
|
||||
```
|
||||
|
||||
## Why this architecture
|
||||
|
||||
- **No cgo** (WASM-embedded libpg_query) keeps cross-compilation trivial and lets us bundle a
|
||||
single static binary per platform inside the VSCode extension.
|
||||
- **Formatting needs a lossless CST** — libpg_query drops comments and whitespace, so it cannot
|
||||
be the sole basis for a formatter. We build our own lexer/CST so the formatter never loses a
|
||||
comment.
|
||||
- **Deep lint needs an accurate AST** — go-pgquery gives the real PostgreSQL parse tree.
|
||||
- **One core, many frontends** — CLI, LSP, VSCode, and DataGrip all reuse the same engine and
|
||||
the same `diagnostics` type.
|
||||
|
||||
## Core invariants (the formatter must never violate these)
|
||||
|
||||
1. **Lossless lex**: `emit(lex(src)) == src` byte-for-byte. The lexer keeps all trivia.
|
||||
2. **Semantic equivalence**: formatting only changes trivia/layout. Verify by re-lexing the
|
||||
output and comparing the non-trivia token stream against the input.
|
||||
3. **Idempotence**: `fmt(fmt(x)) == fmt(x)`.
|
||||
4. **Graceful degradation**: any span the parser cannot handle is passed through verbatim
|
||||
rather than corrupted.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
go build ./cmd/pgtidy # Build the binary
|
||||
go test ./... # Run all tests (incl. corpus safety harness)
|
||||
go vet ./... # Static analysis
|
||||
go fmt ./... # Format Go code
|
||||
```
|
||||
|
||||
## House style (formatter defaults; all configurable via .pgtidy.yaml)
|
||||
|
||||
- Keywords UPPERCASE; data types lowercase; identifiers lowercase snake_case.
|
||||
- Indent: 2 spaces per level.
|
||||
- Leading-comma style, one item per line, for SELECT column lists and function params.
|
||||
- Function headers: params one-per-line in parens; LANGUAGE / SECURITY / volatility each on
|
||||
their own line; `AS` then `$$` on its own line; body; closing `$$;` on its own line.
|
||||
- PL/pgSQL: `DECLARE` alone, vars 2-space indented; `--Block--` comment markers preserved;
|
||||
`BEGIN`/`END` at body level; IF/ELSIF/ELSE/END IF, loops, CASE indent their bodies.
|
||||
- Spacing: spaces around binary operators (`=`, `<>`, `||`, …) and `:=`; no space around
|
||||
`::`, `->`, `->>`, array `[...]`, or before a call's `(`.
|
||||
- Dollar-quote tags preserved verbatim (`$$`, `$S$`, `$Z$`, …).
|
||||
|
||||
## Milestones
|
||||
|
||||
### V1 — Formatter + CLI (current priority)
|
||||
1. **Lexer** (`pkg/lexer`): full PG token coverage incl. dollar-quoted strings, `--` and
|
||||
`/* */` comments, operators. Comments + whitespace as leading/trailing trivia on tokens.
|
||||
Acceptance: `emit(lex(src)) == src` byte-for-byte across the corpus.
|
||||
2. **CST + parser** (`pkg/cst`, `pkg/parser`): recursive descent for DML (SELECT/INSERT/
|
||||
UPDATE/DELETE/CTE), DDL (CREATE FUNCTION/PROCEDURE/TABLE/INDEX/TRIGGER, ALTER, DO).
|
||||
3. **PL/pgSQL body parser**: DECLARE/BEGIN/END, IF/CASE/LOOP, assignments, nested SQL.
|
||||
4. **Printer** (`pkg/format`): Doc-IR (group/indent/line/softline) driven by `style` config.
|
||||
Graceful degradation — unparsed spans pass through verbatim.
|
||||
5. **CLI** (`cmd/pgtidy fmt`): `--check`, `--write`/`-w`, stdin→stdout, `--diff`; config
|
||||
discovery walking up to `.pgtidy.yaml`; CI-friendly exit codes.
|
||||
6. **Config** (`pkg/config`): load/merge style config; defaults = house style above.
|
||||
|
||||
### V2 — Linter
|
||||
- `pkg/pgast` go-pgquery (WASM) wrapper; `pkg/lint` rule engine (rule ID, severity, config).
|
||||
- Rule packs: style/consistency, migration safety (ACCESS EXCLUSIVE locks, unsafe ALTER/ADD
|
||||
COLUMN, non-CONCURRENTLY index builds, blocking constraints), naming conventions,
|
||||
correctness/anti-patterns (SELECT *, missing WHERE on UPDATE/DELETE). Emit `diagnostics`.
|
||||
- `pgtidy lint` subcommand; `--fix` for autofixable rules.
|
||||
|
||||
### V3 — LSP + VSCode
|
||||
- `pkg/lsp`: `textDocument/formatting` + range formatting, `publishDiagnostics`, `codeAction`
|
||||
quick-fixes — all reusing the core.
|
||||
- `editors/vscode`: TS extension using `vscode-languageclient`, launches bundled `pgtidy lsp`.
|
||||
Per-platform VSIX (`win32/linux/darwin × x64/arm64`) built in CI matrix.
|
||||
|
||||
### V4 — DataGrip
|
||||
- `editors/datagrip`: integrate via LSP4IJ (free, works across JetBrains editions incl.
|
||||
DataGrip). No core changes expected.
|
||||
|
||||
## Verification
|
||||
- **Formatter:** `go test ./...` runs golden-file tests + corpus harness asserting idempotence
|
||||
and token-stream equality before/after (no semantic change).
|
||||
- **CLI:** `pgtidy fmt --check` returns non-zero on unformatted input, zero when clean.
|
||||
- **Lint (v2):** fixture SQL with known violations → assert expected diagnostics; `--fix`
|
||||
round-trips.
|
||||
- **LSP/VSCode (v3):** load a `.sql` file in a dev-host VSCode, confirm format-on-save and
|
||||
live diagnostics via the bundled binary.
|
||||
@@ -0,0 +1,35 @@
|
||||
# AI Usage Declaration
|
||||
|
||||
This Go project utilizes AI tools for the following purposes:
|
||||
|
||||
- Generating and improving documentation
|
||||
- Writing and enhancing tests
|
||||
- Refactoring and optimizing existing code
|
||||
|
||||
AI is **not** used for core design or architecture decisions.
|
||||
All design decisions are deferred to human discussion.
|
||||
AI is employed only for enhancements to human-written code.
|
||||
|
||||
We are aware of significant AI hallucinations; all AI-generated content is to be reviewed and verified by humans.
|
||||
|
||||
|
||||
.-""""""-.
|
||||
.' '.
|
||||
/ O O \
|
||||
: ` :
|
||||
| |
|
||||
: .------. :
|
||||
\ ' ' /
|
||||
'. .'
|
||||
'-......-'
|
||||
MEGAMIND AI
|
||||
[============]
|
||||
|
||||
___________
|
||||
/___________\
|
||||
/_____________\
|
||||
| ASSIMILATE |
|
||||
| RESISTANCE |
|
||||
| IS FUTILE |
|
||||
\_____________/
|
||||
\___________/
|
||||
@@ -0,0 +1,4 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
You must now read the ./AGENTS.md
|
||||
@@ -0,0 +1,29 @@
|
||||
APP := pgtidy
|
||||
CMD := ./cmd/pgtidy
|
||||
DIST := dist
|
||||
|
||||
.PHONY: build test lint vet fmt clean
|
||||
|
||||
## build: compile binary for the current platform
|
||||
build:
|
||||
go build -trimpath -ldflags "-s -w -X main.version=$$(git describe --tags --abbrev=0 2>/dev/null || echo dev)" -o $(DIST)/$(APP) $(CMD)
|
||||
|
||||
## test: run tests (incl. corpus safety harness)
|
||||
test:
|
||||
go test ./...
|
||||
|
||||
## vet: static analysis
|
||||
vet:
|
||||
go vet ./...
|
||||
|
||||
## fmt: format Go code
|
||||
fmt:
|
||||
go fmt ./...
|
||||
|
||||
## lint: vet + format check
|
||||
lint: vet
|
||||
test -z "$$(gofmt -l .)" || (echo "gofmt needed:"; gofmt -l .; exit 1)
|
||||
|
||||
## clean: remove build artifacts
|
||||
clean:
|
||||
rm -rf $(DIST)
|
||||
@@ -0,0 +1,100 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
|
||||
"github.com/hein/pgtidy/pkg/config"
|
||||
"github.com/hein/pgtidy/pkg/format"
|
||||
"github.com/hein/pgtidy/pkg/parser"
|
||||
)
|
||||
|
||||
// cmdFmt implements `pgtidy fmt`. It follows the gofmt model: with no flags it
|
||||
// prints the formatted result to stdout; -w rewrites in place; -l lists files
|
||||
// that differ; --check exits non-zero if any input is unformatted.
|
||||
func cmdFmt(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
|
||||
var (
|
||||
write bool
|
||||
list bool
|
||||
check bool
|
||||
files []string
|
||||
)
|
||||
for _, a := range args {
|
||||
switch a {
|
||||
case "-w", "--write":
|
||||
write = true
|
||||
case "-l", "--list":
|
||||
list = true
|
||||
case "--check":
|
||||
check = true
|
||||
case "-h", "--help":
|
||||
usage(stdout)
|
||||
return 0
|
||||
default:
|
||||
if len(a) > 1 && a[0] == '-' {
|
||||
fmt.Fprintf(stderr, "pgtidy fmt: unknown flag %q\n", a)
|
||||
return 2
|
||||
}
|
||||
files = append(files, a)
|
||||
}
|
||||
}
|
||||
|
||||
// TODO: discover and parse .pgtidy.yaml; for now use the house-style default.
|
||||
st := config.Default()
|
||||
|
||||
// stdin → stdout when no files are given.
|
||||
if len(files) == 0 {
|
||||
src, err := io.ReadAll(stdin)
|
||||
if err != nil {
|
||||
fmt.Fprintf(stderr, "pgtidy: reading stdin: %v\n", err)
|
||||
return 2
|
||||
}
|
||||
out := format.File(parser.Parse(string(src)), st)
|
||||
if check {
|
||||
if out != string(src) {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
io.WriteString(stdout, out)
|
||||
return 0
|
||||
}
|
||||
|
||||
exit := 0
|
||||
anyDiff := false
|
||||
for _, path := range files {
|
||||
src, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
fmt.Fprintf(stderr, "pgtidy: %v\n", err)
|
||||
exit = 2
|
||||
continue
|
||||
}
|
||||
out := format.File(parser.Parse(string(src)), st)
|
||||
changed := out != string(src)
|
||||
if changed {
|
||||
anyDiff = true
|
||||
}
|
||||
switch {
|
||||
case write:
|
||||
if changed {
|
||||
if err := os.WriteFile(path, []byte(out), 0o644); err != nil {
|
||||
fmt.Fprintf(stderr, "pgtidy: writing %s: %v\n", path, err)
|
||||
exit = 2
|
||||
}
|
||||
}
|
||||
case list:
|
||||
if changed {
|
||||
fmt.Fprintln(stdout, path)
|
||||
}
|
||||
case check:
|
||||
// handled after loop via anyDiff
|
||||
default:
|
||||
io.WriteString(stdout, out)
|
||||
}
|
||||
}
|
||||
if check && anyDiff && exit == 0 {
|
||||
return 1
|
||||
}
|
||||
return exit
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestFmtStdin(t *testing.T) {
|
||||
in := strings.NewReader("create function f() returns int language sql as $$ select 1 $$;\n")
|
||||
var out, errb bytes.Buffer
|
||||
rc := run([]string{"fmt"}, in, &out, &errb)
|
||||
if rc != 0 {
|
||||
t.Fatalf("rc=%d stderr=%s", rc, errb.String())
|
||||
}
|
||||
if !strings.HasPrefix(out.String(), "CREATE FUNCTION f(") {
|
||||
t.Errorf("unexpected output:\n%s", out.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestFmtCheckStdin(t *testing.T) {
|
||||
unformatted := "create function f() returns int language sql as $$ select 1 $$;"
|
||||
|
||||
// Unformatted input → rc 1.
|
||||
var out, errb bytes.Buffer
|
||||
if rc := run([]string{"fmt", "--check"}, strings.NewReader(unformatted), &out, &errb); rc != 1 {
|
||||
t.Errorf("unformatted input: rc=%d, want 1", rc)
|
||||
}
|
||||
|
||||
// Its own formatted output → rc 0 (idempotent + check agree).
|
||||
out.Reset()
|
||||
errb.Reset()
|
||||
run([]string{"fmt"}, strings.NewReader(unformatted), &out, &errb)
|
||||
formatted := out.String()
|
||||
var out2, errb2 bytes.Buffer
|
||||
if rc := run([]string{"fmt", "--check"}, strings.NewReader(formatted), &out2, &errb2); rc != 0 {
|
||||
t.Errorf("formatted input: rc=%d, want 0\noutput was:\n%s", rc, formatted)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFmtWriteInPlace(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "f.pgsql")
|
||||
orig := "create function f() returns int language sql as $$ select 1 $$;\n"
|
||||
if err := os.WriteFile(path, []byte(orig), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var out, errb bytes.Buffer
|
||||
if rc := run([]string{"fmt", "-w", path}, nil, &out, &errb); rc != 0 {
|
||||
t.Fatalf("rc=%d stderr=%s", rc, errb.String())
|
||||
}
|
||||
got, _ := os.ReadFile(path)
|
||||
if string(got) == orig {
|
||||
t.Error("file was not rewritten")
|
||||
}
|
||||
// Second write is a no-op (idempotent).
|
||||
if rc := run([]string{"fmt", "--check", path}, nil, &out, &errb); rc != 0 {
|
||||
t.Errorf("after -w, --check rc=%d, want 0", rc)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUnknownCommand(t *testing.T) {
|
||||
var out, errb bytes.Buffer
|
||||
if rc := run([]string{"frobnicate"}, nil, &out, &errb); rc != 2 {
|
||||
t.Errorf("rc=%d, want 2", rc)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
// Command pgtidy is the PgTidy CLI: a PostgreSQL formatter (and, later, linter)
|
||||
// and LSP server. Today it provides the `fmt` subcommand.
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
)
|
||||
|
||||
// version is overridden at build time via -ldflags "-X main.version=...".
|
||||
var version = "dev"
|
||||
|
||||
func main() {
|
||||
os.Exit(run(os.Args[1:], os.Stdin, os.Stdout, os.Stderr))
|
||||
}
|
||||
|
||||
func run(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
|
||||
if len(args) == 0 {
|
||||
usage(stderr)
|
||||
return 2
|
||||
}
|
||||
switch args[0] {
|
||||
case "fmt", "format":
|
||||
return cmdFmt(args[1:], stdin, stdout, stderr)
|
||||
case "version", "--version", "-v":
|
||||
fmt.Fprintf(stdout, "pgtidy %s\n", version)
|
||||
return 0
|
||||
case "help", "-h", "--help":
|
||||
usage(stdout)
|
||||
return 0
|
||||
default:
|
||||
fmt.Fprintf(stderr, "pgtidy: unknown command %q\n", args[0])
|
||||
usage(stderr)
|
||||
return 2
|
||||
}
|
||||
}
|
||||
|
||||
func usage(w io.Writer) {
|
||||
fmt.Fprint(w, `pgtidy — PostgreSQL formatter and linter
|
||||
|
||||
Usage:
|
||||
pgtidy fmt [flags] [files...] Format SQL/PL-pgSQL (stdin if no files)
|
||||
pgtidy version Print version
|
||||
pgtidy help Show this help
|
||||
|
||||
fmt flags:
|
||||
-w, --write Rewrite files in place
|
||||
-l, --list List files whose formatting differs (no writes)
|
||||
--check Exit non-zero if any input is not already formatted (CI)
|
||||
`)
|
||||
}
|
||||
Executable
+55
@@ -0,0 +1,55 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Detect the latest version tag
|
||||
latest_tag=$(git tag --sort=-v:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | head -1)
|
||||
|
||||
if [[ -z "$latest_tag" ]]; then
|
||||
suggested_version="v1.0.0"
|
||||
else
|
||||
# Increment the patch version
|
||||
IFS='.' read -r major minor patch <<< "${latest_tag#v}"
|
||||
suggested_version="v${major}.${minor}.$((patch + 1))"
|
||||
fi
|
||||
|
||||
echo "Latest tag: ${latest_tag:-none}"
|
||||
echo "Suggested next version: $suggested_version"
|
||||
|
||||
read -p "Do you want to make a release version? (y/n): " make_release
|
||||
|
||||
if [[ $make_release =~ ^[Yy]$ ]]; then
|
||||
read -p "Enter the version number [$suggested_version]: " version
|
||||
version="${version:-$suggested_version}"
|
||||
|
||||
# Validate the version number
|
||||
if ! [[ $version =~ ^(v)?([0-9]+\.){0,2}[0-9]+(\.[0-9]+)?(\.beta)?$ ]]; then
|
||||
echo "Invalid version number format. Please use a valid format like 'v1.0.0'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Prepend 'v' to the version if it doesn't start with it
|
||||
if ! [[ $version =~ ^v ]]; then
|
||||
version="v$version"
|
||||
fi
|
||||
|
||||
# Build tag message with recent commits since last tag
|
||||
if [[ -n "$latest_tag" ]]; then
|
||||
commit_log=$(git log "${latest_tag}..HEAD" --oneline --no-decorate)
|
||||
else
|
||||
commit_log=$(git log --oneline --no-decorate -20)
|
||||
fi
|
||||
|
||||
tag_message="Released $version
|
||||
|
||||
Changes since ${latest_tag:-beginning}:
|
||||
$commit_log"
|
||||
|
||||
# Create an annotated tag
|
||||
git tag -a "$version" -m "$tag_message"
|
||||
|
||||
# Push the tag to the remote repository
|
||||
git push origin "$version"
|
||||
|
||||
echo "Tag $version created and pushed to the remote repository."
|
||||
else
|
||||
echo "No release version created."
|
||||
fi
|
||||
@@ -0,0 +1,55 @@
|
||||
// Package config defines PgTidy's formatter (and, later, linter) configuration.
|
||||
//
|
||||
// Defaults encode the project house style reverse-engineered from the corpus.
|
||||
// A future change will load/merge these from a discovered .pgtidy.yaml file.
|
||||
package config
|
||||
|
||||
// Case controls keyword/identifier casing.
|
||||
type Case string
|
||||
|
||||
const (
|
||||
CaseUpper Case = "upper"
|
||||
CaseLower Case = "lower"
|
||||
// CasePreserve leaves the token text unchanged.
|
||||
CasePreserve Case = "preserve"
|
||||
)
|
||||
|
||||
// CommaStyle controls where separators sit in multi-line lists.
|
||||
type CommaStyle string
|
||||
|
||||
const (
|
||||
// CommaLeading puts the comma at the start of the continuation line
|
||||
// (",col"), the house style.
|
||||
CommaLeading CommaStyle = "leading"
|
||||
// CommaTrailing puts the comma at the end of the preceding line ("col,").
|
||||
CommaTrailing CommaStyle = "trailing"
|
||||
)
|
||||
|
||||
// Style is the formatter configuration.
|
||||
type Style struct {
|
||||
// Indent is one indentation level (default two spaces).
|
||||
Indent string
|
||||
// Newline is the line terminator emitted by the formatter.
|
||||
Newline string
|
||||
// KeywordCase controls SQL keyword casing (types excluded — see TypeCase).
|
||||
KeywordCase Case
|
||||
// IdentCase controls unquoted identifier casing (quoted identifiers are
|
||||
// never touched).
|
||||
IdentCase Case
|
||||
// TypeCase controls built-in type-name casing.
|
||||
TypeCase Case
|
||||
// Commas controls list separator placement.
|
||||
Commas CommaStyle
|
||||
}
|
||||
|
||||
// Default returns the house-style configuration.
|
||||
func Default() Style {
|
||||
return Style{
|
||||
Indent: " ",
|
||||
Newline: "\n",
|
||||
KeywordCase: CaseUpper,
|
||||
IdentCase: CaseLower,
|
||||
TypeCase: CaseLower,
|
||||
Commas: CommaLeading,
|
||||
}
|
||||
}
|
||||
+173
@@ -0,0 +1,173 @@
|
||||
// Package cst defines PgTidy's concrete syntax tree.
|
||||
//
|
||||
// The CST is lossless: every lexer token (significant tokens and the trivia —
|
||||
// whitespace/comments — attached to them) is reachable, so File.Source()
|
||||
// reproduces the original input byte-for-byte. The parser builds structured
|
||||
// nodes only where it is confident; everything else is captured verbatim in a
|
||||
// Raw node. This makes "graceful degradation" a property of the data model
|
||||
// rather than something the printer must remember to do.
|
||||
package cst
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"github.com/hein/pgtidy/pkg/lexer"
|
||||
)
|
||||
|
||||
// Trivia is a run of whitespace/comment tokens preceding a significant token.
|
||||
type Trivia []lexer.Token
|
||||
|
||||
// Tok is a significant (non-trivia) token together with the trivia that
|
||||
// immediately precedes it in the source.
|
||||
type Tok struct {
|
||||
Lead Trivia
|
||||
Tok lexer.Token
|
||||
}
|
||||
|
||||
// Text returns the significant token's text.
|
||||
func (t Tok) Text() string { return t.Tok.Text }
|
||||
|
||||
// Is reports whether the token is an unquoted word equal (case-insensitively)
|
||||
// to kw.
|
||||
func (t Tok) Is(kw string) bool {
|
||||
return t.Tok.Kind == lexer.Ident && strings.EqualFold(t.Tok.Text, kw)
|
||||
}
|
||||
|
||||
// Comments returns just the comment tokens from the leading trivia.
|
||||
func (t Tok) Comments() []lexer.Token {
|
||||
var out []lexer.Token
|
||||
for _, tr := range t.Lead {
|
||||
if tr.Kind == lexer.LineComment || tr.Kind == lexer.BlockComment {
|
||||
out = append(out, tr)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Attach converts a lossless lexer stream into significant tokens, each
|
||||
// carrying its leading trivia, plus any trailing trivia before EOF.
|
||||
func Attach(toks []lexer.Token) (sig []Tok, trailing Trivia) {
|
||||
var lead Trivia
|
||||
for _, t := range toks {
|
||||
if t.Kind == lexer.EOF {
|
||||
trailing = lead
|
||||
break
|
||||
}
|
||||
if t.IsTrivia() {
|
||||
lead = append(lead, t)
|
||||
continue
|
||||
}
|
||||
sig = append(sig, Tok{Lead: lead, Tok: t})
|
||||
lead = nil
|
||||
}
|
||||
return sig, trailing
|
||||
}
|
||||
|
||||
// Node is any element of a File.
|
||||
type Node interface {
|
||||
appendTokens(*[]Tok)
|
||||
}
|
||||
|
||||
// Tokens returns all significant tokens of a node in source order.
|
||||
func Tokens(n Node) []Tok {
|
||||
var t []Tok
|
||||
n.appendTokens(&t)
|
||||
return t
|
||||
}
|
||||
|
||||
// File is the whole parsed input.
|
||||
type File struct {
|
||||
Items []Node
|
||||
Trailing Trivia // trivia after the last significant token
|
||||
}
|
||||
|
||||
// Source reconstructs the original input from the tree. With a correct parse
|
||||
// this is byte-for-byte identical to the lexer input.
|
||||
func (f *File) Source() string {
|
||||
var toks []Tok
|
||||
for _, it := range f.Items {
|
||||
it.appendTokens(&toks)
|
||||
}
|
||||
var b strings.Builder
|
||||
for _, t := range toks {
|
||||
for _, tr := range t.Lead {
|
||||
b.WriteString(tr.Text)
|
||||
}
|
||||
b.WriteString(t.Tok.Text)
|
||||
}
|
||||
for _, tr := range f.Trailing {
|
||||
b.WriteString(tr.Text)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// Raw is an unstructured statement: a verbatim run of significant tokens. This
|
||||
// is the graceful-degradation fallback for anything the parser does not (yet)
|
||||
// structure.
|
||||
type Raw struct {
|
||||
Toks []Tok
|
||||
}
|
||||
|
||||
func (r *Raw) appendTokens(out *[]Tok) { *out = append(*out, r.Toks...) }
|
||||
|
||||
// Param is one entry in a function/procedure parameter list, with the comma
|
||||
// that separates it from the next entry (nil on the last parameter).
|
||||
type Param struct {
|
||||
Toks []Tok
|
||||
Sep *Tok // trailing comma, nil if last
|
||||
}
|
||||
|
||||
// CreateFunction is a parsed CREATE [OR REPLACE] FUNCTION|PROCEDURE statement.
|
||||
//
|
||||
// Field order matches source order; appendTokens emits them contiguously so the
|
||||
// node round-trips exactly.
|
||||
type CreateFunction struct {
|
||||
Head []Tok // CREATE [OR REPLACE] FUNCTION|PROCEDURE
|
||||
Name []Tok // name (possibly schema-qualified)
|
||||
LParen Tok //
|
||||
Params []Param // parameter list (may be empty)
|
||||
RParen Tok //
|
||||
Options [][]Tok // option clauses before the body (RETURNS/LANGUAGE/VOLATILE/...)
|
||||
As *Tok // the AS keyword, if present
|
||||
Body *Tok // the body token (dollar-quoted string or string), if present
|
||||
Tail [][]Tok // option clauses after the body (rare)
|
||||
Semi *Tok // terminating semicolon, if present
|
||||
}
|
||||
|
||||
// IsProcedure reports whether this is a PROCEDURE (vs FUNCTION).
|
||||
func (c *CreateFunction) IsProcedure() bool {
|
||||
for _, t := range c.Head {
|
||||
if t.Is("procedure") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (c *CreateFunction) appendTokens(out *[]Tok) {
|
||||
*out = append(*out, c.Head...)
|
||||
*out = append(*out, c.Name...)
|
||||
*out = append(*out, c.LParen)
|
||||
for _, p := range c.Params {
|
||||
*out = append(*out, p.Toks...)
|
||||
if p.Sep != nil {
|
||||
*out = append(*out, *p.Sep)
|
||||
}
|
||||
}
|
||||
*out = append(*out, c.RParen)
|
||||
for _, cl := range c.Options {
|
||||
*out = append(*out, cl...)
|
||||
}
|
||||
if c.As != nil {
|
||||
*out = append(*out, *c.As)
|
||||
}
|
||||
if c.Body != nil {
|
||||
*out = append(*out, *c.Body)
|
||||
}
|
||||
for _, cl := range c.Tail {
|
||||
*out = append(*out, cl...)
|
||||
}
|
||||
if c.Semi != nil {
|
||||
*out = append(*out, *c.Semi)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,310 @@
|
||||
// Package format renders a cst.File back to source text in the configured
|
||||
// house style.
|
||||
//
|
||||
// Scope (V1): CREATE FUNCTION/PROCEDURE headers are laid out to house style
|
||||
// (one parameter per line with leading commas, each option clause on its own
|
||||
// line, AS/$$ on their own lines). The PL/pgSQL body and any statement the
|
||||
// parser left as cst.Raw are emitted verbatim — this upholds the safety
|
||||
// invariants while body formatting is built out (task #4).
|
||||
//
|
||||
// Guarantees: formatting changes only trivia/layout (never literal/identifier
|
||||
// semantics), and is idempotent.
|
||||
package format
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"github.com/hein/pgtidy/pkg/config"
|
||||
"github.com/hein/pgtidy/pkg/cst"
|
||||
"github.com/hein/pgtidy/pkg/lexer"
|
||||
)
|
||||
|
||||
// File formats a parsed file with the given style.
|
||||
func File(f *cst.File, st config.Style) string {
|
||||
p := &printer{st: st}
|
||||
for i, item := range f.Items {
|
||||
toks := cst.Tokens(item)
|
||||
if len(toks) == 0 {
|
||||
continue
|
||||
}
|
||||
lead := toks[0].Lead
|
||||
if i > 0 {
|
||||
p.nl()
|
||||
if hasBlankLine(lead) {
|
||||
p.nl()
|
||||
}
|
||||
}
|
||||
p.leadingComments(lead)
|
||||
p.writeItem(item)
|
||||
}
|
||||
p.trailingComments(f.Trailing)
|
||||
return ensureTrailingNewline(p.b.String(), st.Newline)
|
||||
}
|
||||
|
||||
type printer struct {
|
||||
st config.Style
|
||||
b strings.Builder
|
||||
}
|
||||
|
||||
func (p *printer) nl() { p.b.WriteString(p.st.Newline) }
|
||||
|
||||
func (p *printer) writeItem(n cst.Node) {
|
||||
switch v := n.(type) {
|
||||
case *cst.CreateFunction:
|
||||
p.writeCreateFunction(v)
|
||||
default:
|
||||
p.b.WriteString(verbatimSpan(cst.Tokens(n)))
|
||||
}
|
||||
}
|
||||
|
||||
func (p *printer) writeCreateFunction(cf *cst.CreateFunction) {
|
||||
// Safety: if the header carries comments we cannot confidently relocate,
|
||||
// emit the whole statement verbatim rather than risk dropping them.
|
||||
if headerHasComments(cf) {
|
||||
p.b.WriteString(verbatimSpan(cst.Tokens(cf)))
|
||||
return
|
||||
}
|
||||
|
||||
head := p.inline(cf.Head)
|
||||
name := p.inline(cf.Name)
|
||||
p.b.WriteString(head)
|
||||
if name != "" {
|
||||
p.b.WriteString(" ")
|
||||
p.b.WriteString(name)
|
||||
}
|
||||
p.b.WriteString("(")
|
||||
|
||||
first := p.st.Indent + " " // align item text one column past the comma
|
||||
cont := p.st.Indent
|
||||
for i, param := range cf.Params {
|
||||
p.nl()
|
||||
text := p.inline(param.Toks)
|
||||
if i == 0 || p.st.Commas != config.CommaLeading {
|
||||
p.b.WriteString(first)
|
||||
p.b.WriteString(text)
|
||||
if p.st.Commas == config.CommaTrailing && i < len(cf.Params)-1 {
|
||||
p.b.WriteString(",")
|
||||
}
|
||||
} else {
|
||||
p.b.WriteString(cont)
|
||||
p.b.WriteString(",")
|
||||
p.b.WriteString(text)
|
||||
}
|
||||
}
|
||||
p.nl()
|
||||
p.b.WriteString(")")
|
||||
|
||||
for _, clause := range cf.Options {
|
||||
p.nl()
|
||||
p.b.WriteString(p.inline(clause))
|
||||
}
|
||||
if cf.As != nil {
|
||||
p.nl()
|
||||
p.b.WriteString(p.inline([]cst.Tok{{Tok: cf.As.Tok}}))
|
||||
}
|
||||
if cf.Body != nil {
|
||||
p.nl()
|
||||
p.b.WriteString(cf.Body.Tok.Text) // body emitted verbatim (formatted later)
|
||||
}
|
||||
for _, clause := range cf.Tail {
|
||||
p.nl()
|
||||
p.b.WriteString(p.inline(clause))
|
||||
}
|
||||
if cf.Semi != nil {
|
||||
p.b.WriteString(";")
|
||||
}
|
||||
}
|
||||
|
||||
// inline renders a run of tokens on one line, applying spacing and casing.
|
||||
// If the run contains comment trivia it is emitted verbatim to avoid losing
|
||||
// or misplacing the comments.
|
||||
func (p *printer) inline(toks []cst.Tok) string {
|
||||
if len(toks) == 0 {
|
||||
return ""
|
||||
}
|
||||
// A comment on the first token's lead is the span's leading/separation
|
||||
// comment, handled by the caller and never emitted here, so it does not
|
||||
// force verbatim. Internal comments (on later tokens) do.
|
||||
if anyComment(toks[1:]) {
|
||||
return verbatimSpan(toks)
|
||||
}
|
||||
var b strings.Builder
|
||||
for i, t := range toks {
|
||||
if i > 0 && needSpace(toks[i-1].Tok, t.Tok) {
|
||||
b.WriteByte(' ')
|
||||
}
|
||||
b.WriteString(caseText(t.Tok, p.st))
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func (p *printer) leadingComments(lead cst.Trivia) {
|
||||
for _, tr := range lead {
|
||||
if tr.Kind == lexer.LineComment || tr.Kind == lexer.BlockComment {
|
||||
p.b.WriteString(strings.TrimRight(tr.Text, " \t"))
|
||||
p.nl()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (p *printer) trailingComments(lead cst.Trivia) {
|
||||
cs := commentsOf(lead)
|
||||
for _, c := range cs {
|
||||
p.nl()
|
||||
p.b.WriteString(strings.TrimRight(c.Text, " \t"))
|
||||
}
|
||||
}
|
||||
|
||||
// --- spacing & casing ---
|
||||
|
||||
// tightOps are operators printed without surrounding spaces.
|
||||
var tightOps = map[string]bool{"::": true, ":": true, "->": true, "->>": true}
|
||||
|
||||
func needSpace(a, b lexer.Token) bool {
|
||||
// No space after.
|
||||
switch a.Kind {
|
||||
case lexer.LParen, lexer.LBracket, lexer.Dot:
|
||||
return false
|
||||
case lexer.Operator:
|
||||
if tightOps[a.Text] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
// No space before.
|
||||
switch b.Kind {
|
||||
case lexer.RParen, lexer.RBracket, lexer.Comma, lexer.Semicolon, lexer.Dot:
|
||||
return false
|
||||
case lexer.LParen:
|
||||
switch a.Kind {
|
||||
case lexer.Ident, lexer.QuotedIdent, lexer.RParen, lexer.RBracket, lexer.Param:
|
||||
return false // function call / type modifier
|
||||
}
|
||||
case lexer.Operator:
|
||||
if tightOps[b.Text] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func caseText(t lexer.Token, st config.Style) string {
|
||||
if t.Kind != lexer.Ident {
|
||||
return t.Text // only unquoted words are re-cased
|
||||
}
|
||||
low := lowerASCII(t.Text)
|
||||
switch {
|
||||
case isTypeName(low):
|
||||
return applyCase(t.Text, st.TypeCase)
|
||||
case isKeyword(low):
|
||||
return applyCase(t.Text, st.KeywordCase)
|
||||
default:
|
||||
return applyCase(t.Text, st.IdentCase)
|
||||
}
|
||||
}
|
||||
|
||||
func applyCase(s string, c config.Case) string {
|
||||
switch c {
|
||||
case config.CaseUpper:
|
||||
return strings.ToUpper(s)
|
||||
case config.CaseLower:
|
||||
return strings.ToLower(s)
|
||||
default:
|
||||
return s
|
||||
}
|
||||
}
|
||||
|
||||
// --- helpers ---
|
||||
|
||||
// headerHasComments reports whether the function header carries comment trivia
|
||||
// the formatter cannot confidently relocate. The first token's leading trivia
|
||||
// is excluded: that is the statement's leading comment, which File() emits
|
||||
// separately. The body token's own text is excluded too (it is emitted
|
||||
// verbatim), but a comment in front of the body is caught.
|
||||
func headerHasComments(cf *cst.CreateFunction) bool {
|
||||
all := cst.Tokens(cf)
|
||||
for i, t := range all {
|
||||
if i == 0 {
|
||||
continue
|
||||
}
|
||||
if t.Tok.Kind == lexer.Semicolon {
|
||||
continue
|
||||
}
|
||||
if hasComment(t) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func anyComment(toks []cst.Tok) bool {
|
||||
for _, t := range toks {
|
||||
if hasComment(t) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func hasComment(t cst.Tok) bool {
|
||||
for _, tr := range t.Lead {
|
||||
if tr.Kind == lexer.LineComment || tr.Kind == lexer.BlockComment {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func commentsOf(lead cst.Trivia) []lexer.Token {
|
||||
var out []lexer.Token
|
||||
for _, tr := range lead {
|
||||
if tr.Kind == lexer.LineComment || tr.Kind == lexer.BlockComment {
|
||||
out = append(out, tr)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// verbatimSpan emits tokens exactly as in source, excluding the leading trivia
|
||||
// of the first token (separation is controlled by the caller).
|
||||
func verbatimSpan(toks []cst.Tok) string {
|
||||
var b strings.Builder
|
||||
for i, t := range toks {
|
||||
if i > 0 {
|
||||
for _, tr := range t.Lead {
|
||||
b.WriteString(tr.Text)
|
||||
}
|
||||
}
|
||||
b.WriteString(t.Tok.Text)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// hasBlankLine reports whether leading whitespace trivia contains a blank line
|
||||
// (two or more newlines), indicating the author wanted statements separated.
|
||||
func hasBlankLine(lead cst.Trivia) bool {
|
||||
n := 0
|
||||
for _, tr := range lead {
|
||||
if tr.Kind == lexer.Whitespace {
|
||||
n += strings.Count(tr.Text, "\n")
|
||||
}
|
||||
}
|
||||
return n >= 2
|
||||
}
|
||||
|
||||
func ensureTrailingNewline(s, nl string) string {
|
||||
s = strings.TrimRight(s, "\n")
|
||||
if s == "" {
|
||||
return s
|
||||
}
|
||||
return s + nl
|
||||
}
|
||||
|
||||
func lowerASCII(s string) string {
|
||||
b := []byte(s)
|
||||
for i, c := range b {
|
||||
if c >= 'A' && c <= 'Z' {
|
||||
b[i] = c + 32
|
||||
}
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
package format
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/hein/pgtidy/pkg/config"
|
||||
"github.com/hein/pgtidy/pkg/lexer"
|
||||
"github.com/hein/pgtidy/pkg/parser"
|
||||
)
|
||||
|
||||
func format(src string) string {
|
||||
return File(parser.Parse(src), config.Default())
|
||||
}
|
||||
|
||||
func TestFormatHeaderGolden(t *testing.T) {
|
||||
src := "--select * from dropall('resolvespec_login');\n" +
|
||||
"create or replace function resolvespec_login(\n" +
|
||||
"INOUT p_data jsonb, OUT p_success boolean, OUT p_error text)\n" +
|
||||
"language plpgsql volatile security definer\n" +
|
||||
"as $$\nbegin end;\n$$;\n"
|
||||
|
||||
want := "--select * from dropall('resolvespec_login');\n" +
|
||||
"CREATE OR REPLACE FUNCTION resolvespec_login(\n" +
|
||||
" INOUT p_data jsonb\n" +
|
||||
" ,OUT p_success boolean\n" +
|
||||
" ,OUT p_error text\n" +
|
||||
")\n" +
|
||||
"LANGUAGE plpgsql\n" +
|
||||
"VOLATILE\n" +
|
||||
"SECURITY DEFINER\n" +
|
||||
"AS\n" +
|
||||
"$$\nbegin end;\n$$;\n"
|
||||
|
||||
got := format(src)
|
||||
if got != want {
|
||||
t.Errorf("header format mismatch\n--- got ---\n%s\n--- want ---\n%s", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIdempotentSmall(t *testing.T) {
|
||||
src := "create function f(a int,b text) returns void language sql as $$ select 1 $$;"
|
||||
once := format(src)
|
||||
twice := format(once)
|
||||
if once != twice {
|
||||
t.Errorf("not idempotent\n--- once ---\n%s\n--- twice ---\n%s", once, twice)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCorpusIdempotentAndSafe(t *testing.T) {
|
||||
dir := filepath.Join("..", "..", "testdata", "corpus")
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
t.Skipf("no corpus: %v", err)
|
||||
}
|
||||
var seen int
|
||||
for _, e := range entries {
|
||||
if e.IsDir() || !strings.HasSuffix(e.Name(), ".pgsql") {
|
||||
continue
|
||||
}
|
||||
seen++
|
||||
data, err := os.ReadFile(filepath.Join(dir, e.Name()))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
src := string(data)
|
||||
once := format(src)
|
||||
twice := format(once)
|
||||
if once != twice {
|
||||
t.Errorf("%s: not idempotent", e.Name())
|
||||
}
|
||||
if !semanticallyEqual(src, once) {
|
||||
t.Errorf("%s: formatting changed semantics", e.Name())
|
||||
}
|
||||
}
|
||||
if seen == 0 {
|
||||
t.Skip("no corpus files")
|
||||
}
|
||||
t.Logf("formatted %d corpus files (idempotent + semantically equal)", seen)
|
||||
}
|
||||
|
||||
// semanticallyEqual compares the non-trivia token streams of two sources,
|
||||
// treating unquoted identifiers/keywords case-insensitively and everything
|
||||
// else (strings, numbers, dollar bodies, operators, punctuation) exactly. This
|
||||
// validates that formatting changed only layout/casing, never meaning.
|
||||
func semanticallyEqual(a, b string) bool {
|
||||
ta := significant(a)
|
||||
tb := significant(b)
|
||||
if len(ta) != len(tb) {
|
||||
return false
|
||||
}
|
||||
for i := range ta {
|
||||
if ta[i].Kind != tb[i].Kind {
|
||||
return false
|
||||
}
|
||||
if ta[i].Kind == lexer.Ident {
|
||||
if !strings.EqualFold(ta[i].Text, tb[i].Text) {
|
||||
return false
|
||||
}
|
||||
} else if ta[i].Text != tb[i].Text {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func significant(src string) []lexer.Token {
|
||||
var out []lexer.Token
|
||||
for _, t := range lexer.Lex(src) {
|
||||
if t.Kind == lexer.EOF || t.IsTrivia() {
|
||||
continue
|
||||
}
|
||||
out = append(out, t)
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
package format
|
||||
|
||||
// keywords are SQL / PL/pgSQL words the formatter may re-case via KeywordCase.
|
||||
// Type-name words are deliberately excluded (see typeNames) so the house style
|
||||
// can keep types lowercase while keywords are uppercased.
|
||||
var keywords = words(`
|
||||
add after all alter analyze and any array as asc atomic begin between by
|
||||
called cascade case cast check close coalesce collate column commit
|
||||
concurrently constraint create cross cube current_date current_time
|
||||
current_timestamp current_user cursor declare default deferrable definer delete desc
|
||||
distinct do drop each else elsif end except exception execute exists external
|
||||
fetch filter first for foreach foreign from full function get grant group
|
||||
grouping having if ilike immutable in index inner inout insert intersect into
|
||||
into invoker is join key language last leakproof left like limit localtime
|
||||
localtimestamp loop materialized natural new next no not nothing notify null
|
||||
nulls of off offset old on only open or order out outer over overriding
|
||||
parallel partition perform precision primary procedure raise
|
||||
references refresh rename replace reset restrict return returning returns
|
||||
revoke right rollback row rows safe schema secdef security select sequence set
|
||||
setof some stable stacked strict table temp temporary then to transaction
|
||||
trigger truncate union unique unsafe update using vacuum values variadic view
|
||||
volatile when where while window with within
|
||||
`)
|
||||
|
||||
// typeNames are built-in/common type words kept lowercase by the house style.
|
||||
var typeNames = words(`
|
||||
bigint bigserial bit bool boolean box bytea char character cidr circle citext
|
||||
date daterange decimal double float float4 float8 hstore inet int int2 int4
|
||||
int8 integer interval json jsonb line lseg macaddr macaddr8 money numeric oid
|
||||
path pg_lsn point polygon real serial serial2 serial4 serial8 smallint
|
||||
smallserial text time timestamp timestamptz timetz tsquery tsrange tstzrange
|
||||
tsvector uuid varbit varchar xml
|
||||
`)
|
||||
|
||||
func words(s string) map[string]bool {
|
||||
m := make(map[string]bool)
|
||||
w := ""
|
||||
for _, r := range s {
|
||||
if r == ' ' || r == '\n' || r == '\t' || r == '\r' {
|
||||
if w != "" {
|
||||
m[w] = true
|
||||
w = ""
|
||||
}
|
||||
continue
|
||||
}
|
||||
w += string(r)
|
||||
}
|
||||
if w != "" {
|
||||
m[w] = true
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func isKeyword(lower string) bool { return keywords[lower] }
|
||||
func isTypeName(lower string) bool { return typeNames[lower] }
|
||||
@@ -0,0 +1,370 @@
|
||||
package lexer
|
||||
|
||||
import "strings"
|
||||
|
||||
// opChars are the characters PostgreSQL allows in operator names.
|
||||
const opChars = "+-*/<>=~!@#%^&|`?"
|
||||
|
||||
// opSpecial is the subset whose presence lets an operator end in + or -.
|
||||
const opSpecial = "~!@#%^&|`?"
|
||||
|
||||
// Lex tokenizes src into a lossless token stream. The concatenation of every
|
||||
// returned token's Text equals src. The final token is always EOF (empty Text).
|
||||
func Lex(src string) []Token {
|
||||
l := &lexer{src: src, line: 1, col: 1}
|
||||
return l.run()
|
||||
}
|
||||
|
||||
type lexer struct {
|
||||
src string
|
||||
pos int
|
||||
line int
|
||||
col int
|
||||
out []Token
|
||||
|
||||
// position of the token currently being scanned, captured each iteration
|
||||
tokOff, tokLine, tokCol int
|
||||
}
|
||||
|
||||
func (l *lexer) run() []Token {
|
||||
n := len(l.src)
|
||||
for l.pos < n {
|
||||
l.tokOff, l.tokLine, l.tokCol = l.pos, l.line, l.col
|
||||
start := l.pos
|
||||
c := l.src[l.pos]
|
||||
switch {
|
||||
case isSpace(c):
|
||||
l.scanWhile(isSpace)
|
||||
l.emit(Whitespace, start)
|
||||
case c == '-' && l.peek(1) == '-':
|
||||
l.scanLineComment()
|
||||
l.emit(LineComment, start)
|
||||
case c == '/' && l.peek(1) == '*':
|
||||
l.scanBlockComment()
|
||||
l.emit(BlockComment, start)
|
||||
case c == '\'':
|
||||
l.scanString('\'', false)
|
||||
l.emit(String, start)
|
||||
case c == '"':
|
||||
l.scanString('"', false)
|
||||
l.emit(QuotedIdent, start)
|
||||
case c == '$':
|
||||
l.scanDollar(start)
|
||||
case c == '(':
|
||||
l.advance(1)
|
||||
l.emit(LParen, start)
|
||||
case c == ')':
|
||||
l.advance(1)
|
||||
l.emit(RParen, start)
|
||||
case c == '[':
|
||||
l.advance(1)
|
||||
l.emit(LBracket, start)
|
||||
case c == ']':
|
||||
l.advance(1)
|
||||
l.emit(RBracket, start)
|
||||
case c == ',':
|
||||
l.advance(1)
|
||||
l.emit(Comma, start)
|
||||
case c == ';':
|
||||
l.advance(1)
|
||||
l.emit(Semicolon, start)
|
||||
case c == ':':
|
||||
// :: (cast), := (assign), or bare : (slice).
|
||||
if l.peek(1) == ':' || l.peek(1) == '=' {
|
||||
l.advance(2)
|
||||
} else {
|
||||
l.advance(1)
|
||||
}
|
||||
l.emit(Operator, start)
|
||||
case c == '.':
|
||||
if isDigit(l.peek(1)) {
|
||||
l.scanNumber()
|
||||
l.emit(Number, start)
|
||||
} else {
|
||||
l.advance(1)
|
||||
l.emit(Dot, start)
|
||||
}
|
||||
case isDigit(c):
|
||||
l.scanNumber()
|
||||
l.emit(Number, start)
|
||||
case isIdentStart(c):
|
||||
l.scanWord(start)
|
||||
case strings.IndexByte(opChars, c) >= 0:
|
||||
l.scanOperator()
|
||||
l.emit(Operator, start)
|
||||
default:
|
||||
l.advance(1)
|
||||
l.emit(Unknown, start)
|
||||
}
|
||||
}
|
||||
l.out = append(l.out, Token{Kind: EOF, Off: l.pos, Line: l.line, Col: l.col})
|
||||
return l.out
|
||||
}
|
||||
|
||||
// scanWord handles identifiers and the typed-string prefixes E' B' X' U&' / U&".
|
||||
func (l *lexer) scanWord(start int) {
|
||||
c := l.src[l.pos]
|
||||
switch c {
|
||||
case 'E', 'e':
|
||||
if l.peek(1) == '\'' {
|
||||
l.advance(1)
|
||||
l.scanString('\'', true)
|
||||
l.emit(EscapeString, start)
|
||||
return
|
||||
}
|
||||
case 'B', 'b':
|
||||
if l.peek(1) == '\'' {
|
||||
l.advance(1)
|
||||
l.scanString('\'', false)
|
||||
l.emit(BitString, start)
|
||||
return
|
||||
}
|
||||
case 'X', 'x':
|
||||
if l.peek(1) == '\'' {
|
||||
l.advance(1)
|
||||
l.scanString('\'', false)
|
||||
l.emit(HexString, start)
|
||||
return
|
||||
}
|
||||
case 'U', 'u':
|
||||
if l.peek(1) == '&' && (l.peek(2) == '\'' || l.peek(2) == '"') {
|
||||
q := l.peek(2)
|
||||
l.advance(2)
|
||||
l.scanString(q, false)
|
||||
l.emit(UnicodeString, start)
|
||||
return
|
||||
}
|
||||
}
|
||||
l.scanWhile(isIdentCont)
|
||||
l.emit(Ident, start)
|
||||
}
|
||||
|
||||
// scanDollar handles dollar-quoted strings ($tag$...$tag$), positional
|
||||
// parameters ($1), and a lone $.
|
||||
func (l *lexer) scanDollar(start int) {
|
||||
if tag, ok := l.dollarTag(); ok {
|
||||
// Opening delimiter is "$tag$"; find the matching close.
|
||||
open := "$" + tag + "$"
|
||||
l.advance(len(open))
|
||||
if idx := strings.Index(l.src[l.pos:], open); idx >= 0 {
|
||||
l.advance(idx + len(open))
|
||||
} else {
|
||||
l.advance(len(l.src) - l.pos) // unterminated: consume to EOF
|
||||
}
|
||||
l.emit(DollarString, start)
|
||||
return
|
||||
}
|
||||
if isDigit(l.peek(1)) {
|
||||
l.advance(1)
|
||||
l.scanWhile(isDigit)
|
||||
l.emit(Param, start)
|
||||
return
|
||||
}
|
||||
l.advance(1)
|
||||
l.emit(Unknown, start)
|
||||
}
|
||||
|
||||
// dollarTag checks whether the current $ begins a dollar-quote opening
|
||||
// delimiter and, if so, returns the (possibly empty) tag.
|
||||
func (l *lexer) dollarTag() (string, bool) {
|
||||
// src[pos] == '$'
|
||||
j := l.pos + 1
|
||||
n := len(l.src)
|
||||
if j < n && l.src[j] == '$' {
|
||||
return "", true // "$$"
|
||||
}
|
||||
k := j
|
||||
if k < n && isIdentStart(l.src[k]) {
|
||||
k++
|
||||
// Tag chars follow identifier rules but exclude '$' (which closes the tag).
|
||||
for k < n && isIdentCont(l.src[k]) && l.src[k] != '$' {
|
||||
k++
|
||||
}
|
||||
if k < n && l.src[k] == '$' {
|
||||
return l.src[j:k], true
|
||||
}
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
func (l *lexer) scanLineComment() {
|
||||
// Consume until newline (exclusive); the newline is whitespace.
|
||||
for l.pos < len(l.src) && l.src[l.pos] != '\n' {
|
||||
l.advance(1)
|
||||
}
|
||||
}
|
||||
|
||||
func (l *lexer) scanBlockComment() {
|
||||
l.advance(2) // /*
|
||||
depth := 1
|
||||
for l.pos < len(l.src) && depth > 0 {
|
||||
if l.src[l.pos] == '/' && l.peek(1) == '*' {
|
||||
l.advance(2)
|
||||
depth++
|
||||
} else if l.src[l.pos] == '*' && l.peek(1) == '/' {
|
||||
l.advance(2)
|
||||
depth--
|
||||
} else {
|
||||
l.advance(1)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// scanString consumes a quoted run beginning at the current quote character.
|
||||
// A doubled quote (”) is an embedded quote; with backslash=true a backslash
|
||||
// escapes the next byte (escape-string syntax).
|
||||
func (l *lexer) scanString(quote byte, backslash bool) {
|
||||
l.advance(1) // opening quote
|
||||
for l.pos < len(l.src) {
|
||||
c := l.src[l.pos]
|
||||
if backslash && c == '\\' && l.pos+1 < len(l.src) {
|
||||
l.advance(2)
|
||||
continue
|
||||
}
|
||||
if c == quote {
|
||||
if l.peek(1) == quote {
|
||||
l.advance(2) // doubled quote
|
||||
continue
|
||||
}
|
||||
l.advance(1) // closing quote
|
||||
return
|
||||
}
|
||||
l.advance(1)
|
||||
}
|
||||
}
|
||||
|
||||
func (l *lexer) scanNumber() {
|
||||
// Non-decimal integer literals: 0x.., 0o.., 0b..
|
||||
if l.src[l.pos] == '0' {
|
||||
switch l.peek(1) {
|
||||
case 'x', 'X', 'o', 'O', 'b', 'B':
|
||||
l.advance(2)
|
||||
l.scanWhile(isHexOrSep)
|
||||
return
|
||||
}
|
||||
}
|
||||
l.scanWhile(isDigitOrSep)
|
||||
if l.pos < len(l.src) && l.src[l.pos] == '.' {
|
||||
l.advance(1)
|
||||
l.scanWhile(isDigitOrSep)
|
||||
}
|
||||
if c := l.cur(); c == 'e' || c == 'E' {
|
||||
if p := l.peek(1); isDigit(p) || ((p == '+' || p == '-') && isDigit(l.peek(2))) {
|
||||
l.advance(1)
|
||||
if l.cur() == '+' || l.cur() == '-' {
|
||||
l.advance(1)
|
||||
}
|
||||
l.scanWhile(isDigit)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// scanOperator consumes a run of operator characters, applying PostgreSQL's
|
||||
// rule that a multi-character operator may not end in + or - unless it also
|
||||
// contains one of ~ ! @ # % ^ & | ` ?.
|
||||
func (l *lexer) scanOperator() {
|
||||
start := l.pos
|
||||
hasSpecial := false
|
||||
for l.pos < len(l.src) {
|
||||
c := l.src[l.pos]
|
||||
if strings.IndexByte(opChars, c) < 0 {
|
||||
break
|
||||
}
|
||||
if c == '-' && l.peek(1) == '-' {
|
||||
break // comment start
|
||||
}
|
||||
if c == '/' && l.peek(1) == '*' {
|
||||
break // comment start
|
||||
}
|
||||
if strings.IndexByte(opSpecial, c) >= 0 {
|
||||
hasSpecial = true
|
||||
}
|
||||
l.advance(1)
|
||||
}
|
||||
if !hasSpecial {
|
||||
for l.pos-1 > start {
|
||||
last := l.src[l.pos-1]
|
||||
if last != '+' && last != '-' {
|
||||
break
|
||||
}
|
||||
l.backup()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --- low-level helpers ---
|
||||
|
||||
func (l *lexer) cur() byte {
|
||||
if l.pos < len(l.src) {
|
||||
return l.src[l.pos]
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func (l *lexer) peek(k int) byte {
|
||||
if l.pos+k < len(l.src) {
|
||||
return l.src[l.pos+k]
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func (l *lexer) scanWhile(pred func(byte) bool) {
|
||||
for l.pos < len(l.src) && pred(l.src[l.pos]) {
|
||||
l.advance(1)
|
||||
}
|
||||
}
|
||||
|
||||
// advance moves forward k bytes, maintaining line/col.
|
||||
func (l *lexer) advance(k int) {
|
||||
for i := 0; i < k && l.pos < len(l.src); i++ {
|
||||
if l.src[l.pos] == '\n' {
|
||||
l.line++
|
||||
l.col = 1
|
||||
} else {
|
||||
l.col++
|
||||
}
|
||||
l.pos++
|
||||
}
|
||||
}
|
||||
|
||||
// backup moves back one byte. Only used within scanOperator, which never
|
||||
// spans a newline, so column bookkeeping is safe.
|
||||
func (l *lexer) backup() {
|
||||
l.pos--
|
||||
l.col--
|
||||
}
|
||||
|
||||
// emit appends a token covering src[start:pos], using the start position
|
||||
// captured at the top of the scan loop.
|
||||
func (l *lexer) emit(kind Kind, start int) {
|
||||
l.out = append(l.out, Token{
|
||||
Kind: kind,
|
||||
Text: l.src[start:l.pos],
|
||||
Off: l.tokOff,
|
||||
Line: l.tokLine,
|
||||
Col: l.tokCol,
|
||||
})
|
||||
}
|
||||
|
||||
func isSpace(c byte) bool {
|
||||
switch c {
|
||||
case ' ', '\t', '\n', '\r', '\v', '\f':
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isDigit(c byte) bool { return c >= '0' && c <= '9' }
|
||||
func isDigitOrSep(c byte) bool { return isDigit(c) || c == '_' }
|
||||
|
||||
func isHexOrSep(c byte) bool {
|
||||
return isDigit(c) || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F') || c == '_'
|
||||
}
|
||||
|
||||
func isIdentStart(c byte) bool {
|
||||
return c == '_' || (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || c >= 0x80
|
||||
}
|
||||
|
||||
func isIdentCont(c byte) bool {
|
||||
return isIdentStart(c) || isDigit(c) || c == '$'
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
package lexer
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// emit reconstructs the source from a token stream.
|
||||
func emit(toks []Token) string {
|
||||
var b strings.Builder
|
||||
for _, t := range toks {
|
||||
b.WriteString(t.Text)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func TestRoundTripSmall(t *testing.T) {
|
||||
cases := []string{
|
||||
"",
|
||||
"SELECT 1;",
|
||||
"select * from t where a = b;",
|
||||
"-- a comment\nSELECT 1",
|
||||
"/* block /* nested */ still */ SELECT 1",
|
||||
"SELECT 'it''s', E'a\\nb', $$dollar$$, $tag$x$tag$, $1;",
|
||||
"a->>'b'::text",
|
||||
"x := y + 1;",
|
||||
"a=-b",
|
||||
"SELECT 1.5, .5, 1e10, 0xFF, 1_000;",
|
||||
"arr[1:2]",
|
||||
"\"Quoted Ident\".col",
|
||||
"U&'d\\0061t'",
|
||||
}
|
||||
for _, src := range cases {
|
||||
got := emit(Lex(src))
|
||||
if got != src {
|
||||
t.Errorf("round-trip mismatch\n in: %q\nout: %q", src, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestKinds(t *testing.T) {
|
||||
toks := nonEOF(Lex("a->>'b'::text"))
|
||||
want := []Kind{Ident, Operator, String, Operator, Ident}
|
||||
if len(toks) != len(want) {
|
||||
t.Fatalf("got %d tokens, want %d: %v", len(toks), len(want), toks)
|
||||
}
|
||||
for i, k := range want {
|
||||
if toks[i].Kind != k {
|
||||
t.Errorf("token %d: got %s, want %s (text %q)", i, toks[i].Kind, k, toks[i].Text)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestOperatorTrailingRule(t *testing.T) {
|
||||
// "=-" must split into "=" and "-" (no special char, can't end in -).
|
||||
toks := nonEOF(Lex("a=-b"))
|
||||
if len(toks) != 4 || toks[1].Text != "=" || toks[2].Text != "-" {
|
||||
t.Fatalf("a=-b mis-lexed: %v", toks)
|
||||
}
|
||||
// "@-" keeps trailing - because @ is special.
|
||||
toks = nonEOF(Lex("a@-b"))
|
||||
if toks[1].Text != "@-" {
|
||||
t.Fatalf("@- should stay one operator, got %q", toks[1].Text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDollarQuote(t *testing.T) {
|
||||
toks := nonEOF(Lex("$func$ body $$ inner $func$"))
|
||||
if len(toks) != 1 || toks[0].Kind != DollarString {
|
||||
t.Fatalf("dollar quote not single token: %v", toks)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLineColumns(t *testing.T) {
|
||||
toks := nonEOF(Lex("ab\n cd"))
|
||||
// ab(1,1) ws cd(2,3)
|
||||
if toks[0].Line != 1 || toks[0].Col != 1 {
|
||||
t.Errorf("ab at %d:%d, want 1:1", toks[0].Line, toks[0].Col)
|
||||
}
|
||||
cd := toks[len(toks)-1]
|
||||
if cd.Text != "cd" || cd.Line != 2 || cd.Col != 3 {
|
||||
t.Errorf("cd at %d:%d, want 2:3", cd.Line, cd.Col)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCorpusRoundTrip is the core lossless invariant: lexing then re-emitting
|
||||
// every real-world .pgsql fixture must reproduce it byte-for-byte.
|
||||
func TestCorpusRoundTrip(t *testing.T) {
|
||||
dir := filepath.Join("..", "..", "testdata", "corpus")
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
t.Skipf("no corpus dir: %v", err)
|
||||
}
|
||||
var seen int
|
||||
for _, e := range entries {
|
||||
if e.IsDir() || !strings.HasSuffix(e.Name(), ".pgsql") {
|
||||
continue
|
||||
}
|
||||
seen++
|
||||
path := filepath.Join(dir, e.Name())
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("read %s: %v", path, err)
|
||||
}
|
||||
src := string(data)
|
||||
if got := emit(Lex(src)); got != src {
|
||||
t.Errorf("%s: round-trip mismatch (len in=%d out=%d)", e.Name(), len(src), len(got))
|
||||
reportFirstDiff(t, e.Name(), src, got)
|
||||
}
|
||||
}
|
||||
if seen == 0 {
|
||||
t.Skip("corpus dir has no .pgsql files")
|
||||
}
|
||||
t.Logf("round-tripped %d corpus files", seen)
|
||||
}
|
||||
|
||||
func reportFirstDiff(t *testing.T, name, a, b string) {
|
||||
t.Helper()
|
||||
n := len(a)
|
||||
if len(b) < n {
|
||||
n = len(b)
|
||||
}
|
||||
for i := 0; i < n; i++ {
|
||||
if a[i] != b[i] {
|
||||
lo := i - 20
|
||||
if lo < 0 {
|
||||
lo = 0
|
||||
}
|
||||
t.Logf("%s: first diff at byte %d\n in: %q\n out: %q", name, i, a[lo:min(i+20, len(a))], b[lo:min(i+20, len(b))])
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func nonEOF(toks []Token) []Token {
|
||||
var out []Token
|
||||
for _, t := range toks {
|
||||
if t.Kind == EOF {
|
||||
continue
|
||||
}
|
||||
out = append(out, t)
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,102 @@
|
||||
// Package lexer implements a lossless lexer for PostgreSQL SQL and PL/pgSQL.
|
||||
//
|
||||
// "Lossless" means every byte of the input is represented by exactly one token,
|
||||
// including whitespace and comments. Concatenating the Text of all tokens in
|
||||
// order reproduces the original source byte-for-byte:
|
||||
//
|
||||
// emit(Lex(src)) == src
|
||||
//
|
||||
// This property is the foundation of the formatter: comments and whitespace are
|
||||
// first-class tokens (trivia) so the parser/printer can preserve them.
|
||||
package lexer
|
||||
|
||||
// Kind classifies a token.
|
||||
type Kind int
|
||||
|
||||
const (
|
||||
EOF Kind = iota
|
||||
|
||||
// Trivia — insignificant to the grammar but preserved losslessly.
|
||||
Whitespace
|
||||
LineComment // -- ... (up to, not including, the newline)
|
||||
BlockComment // /* ... */ (nestable)
|
||||
|
||||
// Words.
|
||||
Ident // unquoted identifier or keyword (keyword-ness resolved later)
|
||||
QuotedIdent // "..."
|
||||
|
||||
// Literals.
|
||||
String // '...'
|
||||
EscapeString // E'...'
|
||||
BitString // B'...'
|
||||
HexString // X'...'
|
||||
UnicodeString // U&'...' or U&"..."
|
||||
DollarString // $tag$...$tag$
|
||||
Number // 123, 1.5, .5, 1e10, 0xff, 1_000
|
||||
Param // $1, $2
|
||||
|
||||
// Operators (incl. ::, :=, :, ->, ->>, and op-char runs).
|
||||
Operator
|
||||
|
||||
// Structural punctuation.
|
||||
LParen
|
||||
RParen
|
||||
LBracket
|
||||
RBracket
|
||||
Comma
|
||||
Semicolon
|
||||
Dot
|
||||
|
||||
Unknown // a byte that fits no other category
|
||||
)
|
||||
|
||||
var kindNames = map[Kind]string{
|
||||
EOF: "EOF",
|
||||
Whitespace: "Whitespace",
|
||||
LineComment: "LineComment",
|
||||
BlockComment: "BlockComment",
|
||||
Ident: "Ident",
|
||||
QuotedIdent: "QuotedIdent",
|
||||
String: "String",
|
||||
EscapeString: "EscapeString",
|
||||
BitString: "BitString",
|
||||
HexString: "HexString",
|
||||
UnicodeString: "UnicodeString",
|
||||
DollarString: "DollarString",
|
||||
Number: "Number",
|
||||
Param: "Param",
|
||||
Operator: "Operator",
|
||||
LParen: "LParen",
|
||||
RParen: "RParen",
|
||||
LBracket: "LBracket",
|
||||
RBracket: "RBracket",
|
||||
Comma: "Comma",
|
||||
Semicolon: "Semicolon",
|
||||
Dot: "Dot",
|
||||
Unknown: "Unknown",
|
||||
}
|
||||
|
||||
func (k Kind) String() string {
|
||||
if s, ok := kindNames[k]; ok {
|
||||
return s
|
||||
}
|
||||
return "Kind(?)"
|
||||
}
|
||||
|
||||
// Token is a single lexical unit covering Text == src[Off:Off+len(Text)].
|
||||
type Token struct {
|
||||
Kind Kind
|
||||
Text string
|
||||
Off int // byte offset of the first byte
|
||||
Line int // 1-based line of the first byte
|
||||
Col int // 1-based byte column of the first byte
|
||||
}
|
||||
|
||||
// IsTrivia reports whether the token is whitespace or a comment.
|
||||
func (t Token) IsTrivia() bool {
|
||||
switch t.Kind {
|
||||
case Whitespace, LineComment, BlockComment:
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,263 @@
|
||||
// Package parser builds a cst.File from PostgreSQL source.
|
||||
//
|
||||
// The parser is intentionally partial: it splits input into statements and
|
||||
// structures the constructs the formatter currently understands (today:
|
||||
// CREATE FUNCTION/PROCEDURE). Anything else is preserved verbatim as a cst.Raw
|
||||
// node. This guarantees the parser never loses or corrupts input — see the
|
||||
// round-trip test.
|
||||
package parser
|
||||
|
||||
import (
|
||||
"github.com/hein/pgtidy/pkg/cst"
|
||||
"github.com/hein/pgtidy/pkg/lexer"
|
||||
)
|
||||
|
||||
// Parse lexes and parses src into a lossless cst.File.
|
||||
func Parse(src string) *cst.File {
|
||||
sig, trailing := cst.Attach(lexer.Lex(src))
|
||||
f := &cst.File{Trailing: trailing}
|
||||
for _, stmt := range splitStatements(sig) {
|
||||
f.Items = append(f.Items, parseStatement(stmt))
|
||||
}
|
||||
return f
|
||||
}
|
||||
|
||||
// splitStatements breaks the significant-token stream into statements at
|
||||
// top-level (paren-depth 0) semicolons. The terminating semicolon is included
|
||||
// in the statement it ends.
|
||||
func splitStatements(sig []cst.Tok) [][]cst.Tok {
|
||||
var stmts [][]cst.Tok
|
||||
var cur []cst.Tok
|
||||
depth := 0
|
||||
for _, t := range sig {
|
||||
cur = append(cur, t)
|
||||
switch t.Tok.Kind {
|
||||
case lexer.LParen:
|
||||
depth++
|
||||
case lexer.RParen:
|
||||
if depth > 0 {
|
||||
depth--
|
||||
}
|
||||
case lexer.Semicolon:
|
||||
if depth == 0 {
|
||||
stmts = append(stmts, cur)
|
||||
cur = nil
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(cur) > 0 {
|
||||
stmts = append(stmts, cur)
|
||||
}
|
||||
return stmts
|
||||
}
|
||||
|
||||
func parseStatement(stmt []cst.Tok) cst.Node {
|
||||
if isCreateFunction(stmt) {
|
||||
if cf, ok := parseCreateFunction(stmt); ok {
|
||||
return cf
|
||||
}
|
||||
}
|
||||
return &cst.Raw{Toks: stmt}
|
||||
}
|
||||
|
||||
// isCreateFunction reports whether stmt begins with CREATE ... FUNCTION|PROCEDURE.
|
||||
func isCreateFunction(stmt []cst.Tok) bool {
|
||||
if len(stmt) == 0 || !stmt[0].Is("create") {
|
||||
return false
|
||||
}
|
||||
for i := 1; i < len(stmt) && i < 4; i++ {
|
||||
if stmt[i].Is("function") || stmt[i].Is("procedure") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// parseCreateFunction structures a CREATE FUNCTION/PROCEDURE statement. It
|
||||
// returns ok=false (so the caller falls back to Raw) if the shape is not what
|
||||
// it expects.
|
||||
func parseCreateFunction(stmt []cst.Tok) (*cst.CreateFunction, bool) {
|
||||
cf := &cst.CreateFunction{}
|
||||
i := 0
|
||||
|
||||
// Head: CREATE [OR REPLACE] (FUNCTION|PROCEDURE)
|
||||
if i >= len(stmt) || !stmt[i].Is("create") {
|
||||
return nil, false
|
||||
}
|
||||
i++
|
||||
if i+1 < len(stmt) && stmt[i].Is("or") && stmt[i+1].Is("replace") {
|
||||
i += 2
|
||||
}
|
||||
if i >= len(stmt) || !(stmt[i].Is("function") || stmt[i].Is("procedure")) {
|
||||
return nil, false
|
||||
}
|
||||
i++
|
||||
cf.Head = stmt[:i]
|
||||
|
||||
// Name: everything up to the opening '(' of the parameter list.
|
||||
nameStart := i
|
||||
for i < len(stmt) && stmt[i].Tok.Kind != lexer.LParen {
|
||||
i++
|
||||
}
|
||||
if i >= len(stmt) {
|
||||
return nil, false // no parameter list
|
||||
}
|
||||
cf.Name = stmt[nameStart:i]
|
||||
cf.LParen = stmt[i]
|
||||
i++
|
||||
|
||||
// Parameters: up to the matching ')'.
|
||||
params, rparen, ok := parseParamList(stmt, i)
|
||||
if !ok {
|
||||
return nil, false
|
||||
}
|
||||
cf.Params = params
|
||||
cf.RParen = stmt[rparen]
|
||||
i = rparen + 1
|
||||
|
||||
// Trailing semicolon (handle now so it isn't swept into options/tail).
|
||||
end := len(stmt)
|
||||
if end > 0 && stmt[end-1].Tok.Kind == lexer.Semicolon {
|
||||
cf.Semi = &stmt[end-1]
|
||||
end--
|
||||
}
|
||||
|
||||
// Options up to AS; then the body; then any tail options.
|
||||
asIdx := indexOfKeyword(stmt, i, end, "as")
|
||||
if asIdx < 0 {
|
||||
// No AS clause (e.g. RETURN / BEGIN ATOMIC bodies). Keep options
|
||||
// structured but leave the body unstructured.
|
||||
cf.Options = splitClauses(stmt[i:end])
|
||||
return cf, true
|
||||
}
|
||||
cf.Options = splitClauses(stmt[i:asIdx])
|
||||
cf.As = &stmt[asIdx]
|
||||
bi := asIdx + 1
|
||||
if bi < end && isBodyToken(stmt[bi]) {
|
||||
cf.Body = &stmt[bi]
|
||||
bi++
|
||||
}
|
||||
if bi < end {
|
||||
cf.Tail = splitClauses(stmt[bi:end])
|
||||
}
|
||||
return cf, true
|
||||
}
|
||||
|
||||
// parseParamList parses comma-separated parameters starting at index `start`
|
||||
// (the token after '('). It returns the parameters, the index of the matching
|
||||
// ')', and ok.
|
||||
func parseParamList(stmt []cst.Tok, start int) ([]cst.Param, int, bool) {
|
||||
var params []cst.Param
|
||||
depth := 0
|
||||
itemStart := start
|
||||
i := start
|
||||
flush := func(endExclusive int, sep *cst.Tok) {
|
||||
toks := stmt[itemStart:endExclusive]
|
||||
if len(toks) == 0 && sep == nil {
|
||||
return // empty parameter list ()
|
||||
}
|
||||
params = append(params, cst.Param{Toks: toks, Sep: sep})
|
||||
}
|
||||
for ; i < len(stmt); i++ {
|
||||
switch stmt[i].Tok.Kind {
|
||||
case lexer.LParen, lexer.LBracket:
|
||||
depth++
|
||||
case lexer.RParen:
|
||||
if depth == 0 {
|
||||
// Closing the parameter list.
|
||||
flush(i, nil)
|
||||
return params, i, true
|
||||
}
|
||||
depth--
|
||||
case lexer.RBracket:
|
||||
if depth > 0 {
|
||||
depth--
|
||||
}
|
||||
case lexer.Comma:
|
||||
if depth == 0 {
|
||||
sep := &stmt[i]
|
||||
flush(i, sep)
|
||||
itemStart = i + 1
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil, 0, false // unbalanced
|
||||
}
|
||||
|
||||
// splitClauses groups a run of option tokens into clauses, each beginning at a
|
||||
// recognized clause keyword (at paren depth 0). Tokens before the first
|
||||
// recognized keyword, if any, are attached to a leading clause so nothing is
|
||||
// dropped.
|
||||
func splitClauses(toks []cst.Tok) [][]cst.Tok {
|
||||
if len(toks) == 0 {
|
||||
return nil
|
||||
}
|
||||
var clauses [][]cst.Tok
|
||||
depth := 0
|
||||
start := 0
|
||||
for i, t := range toks {
|
||||
switch t.Tok.Kind {
|
||||
case lexer.LParen, lexer.LBracket:
|
||||
depth++
|
||||
case lexer.RParen, lexer.RBracket:
|
||||
if depth > 0 {
|
||||
depth--
|
||||
}
|
||||
}
|
||||
if i > start && depth == 0 && isClauseStarter(t) {
|
||||
clauses = append(clauses, toks[start:i])
|
||||
start = i
|
||||
}
|
||||
}
|
||||
clauses = append(clauses, toks[start:])
|
||||
return clauses
|
||||
}
|
||||
|
||||
var clauseStarters = map[string]bool{
|
||||
"returns": true, "language": true, "transform": true, "window": true,
|
||||
"immutable": true, "stable": true, "volatile": true, "leakproof": true,
|
||||
"not": true, "called": true, "strict": true, "external": true,
|
||||
"security": true, "parallel": true, "cost": true, "rows": true,
|
||||
"support": true, "set": true, "as": true,
|
||||
}
|
||||
|
||||
func isClauseStarter(t cst.Tok) bool {
|
||||
if t.Tok.Kind != lexer.Ident {
|
||||
return false
|
||||
}
|
||||
return clauseStarters[lowerASCII(t.Tok.Text)]
|
||||
}
|
||||
|
||||
func isBodyToken(t cst.Tok) bool {
|
||||
return t.Tok.Kind == lexer.DollarString || t.Tok.Kind == lexer.String
|
||||
}
|
||||
|
||||
// indexOfKeyword returns the index in [from,to) of the first top-level (paren
|
||||
// depth 0) token equal to kw, or -1.
|
||||
func indexOfKeyword(stmt []cst.Tok, from, to int, kw string) int {
|
||||
depth := 0
|
||||
for i := from; i < to; i++ {
|
||||
switch stmt[i].Tok.Kind {
|
||||
case lexer.LParen, lexer.LBracket:
|
||||
depth++
|
||||
case lexer.RParen, lexer.RBracket:
|
||||
if depth > 0 {
|
||||
depth--
|
||||
}
|
||||
}
|
||||
if depth == 0 && stmt[i].Is(kw) {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
func lowerASCII(s string) string {
|
||||
b := []byte(s)
|
||||
for i, c := range b {
|
||||
if c >= 'A' && c <= 'Z' {
|
||||
b[i] = c + 32
|
||||
}
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
package parser
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/hein/pgtidy/pkg/cst"
|
||||
)
|
||||
|
||||
func TestParseRoundTripSmall(t *testing.T) {
|
||||
cases := []string{
|
||||
"",
|
||||
"SELECT 1;",
|
||||
"-- lead\nSELECT 1; SELECT 2;",
|
||||
"CREATE FUNCTION f() RETURNS int LANGUAGE sql AS $$ SELECT 1 $$;",
|
||||
"create or replace function s.g(a int, b text default 'x') returns void language plpgsql as $$ begin end; $$;\n",
|
||||
"INSERT INTO t (a,b) VALUES (1,2); -- trailing\n",
|
||||
}
|
||||
for _, src := range cases {
|
||||
if got := Parse(src).Source(); got != src {
|
||||
t.Errorf("round-trip mismatch\n in: %q\nout: %q", src, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseCreateFunctionShape(t *testing.T) {
|
||||
src := "CREATE OR REPLACE FUNCTION resolvespec_login(\n" +
|
||||
" INOUT p_data jsonb\n" +
|
||||
" ,OUT p_success boolean\n" +
|
||||
" ,OUT p_error text\n" +
|
||||
")\nLANGUAGE plpgsql VOLATILE\nSECURITY DEFINER\nAS\n$$\nbegin end;\n$$;\n"
|
||||
|
||||
f := Parse(src)
|
||||
if len(f.Items) != 1 {
|
||||
t.Fatalf("want 1 item, got %d", len(f.Items))
|
||||
}
|
||||
cf, ok := f.Items[0].(*cst.CreateFunction)
|
||||
if !ok {
|
||||
t.Fatalf("want *CreateFunction, got %T", f.Items[0])
|
||||
}
|
||||
if cf.IsProcedure() {
|
||||
t.Error("should be FUNCTION not PROCEDURE")
|
||||
}
|
||||
if len(cf.Params) != 3 {
|
||||
t.Fatalf("want 3 params, got %d", len(cf.Params))
|
||||
}
|
||||
// Leading-comma style: in source the comma precedes the next param, but the
|
||||
// parser associates each comma as the separator after the preceding param.
|
||||
if cf.Params[0].Sep == nil || cf.Params[1].Sep == nil || cf.Params[2].Sep != nil {
|
||||
t.Errorf("param separators wrong: %v %v %v",
|
||||
cf.Params[0].Sep != nil, cf.Params[1].Sep != nil, cf.Params[2].Sep != nil)
|
||||
}
|
||||
if cf.Body == nil || !strings.Contains(cf.Body.Text(), "begin") {
|
||||
t.Errorf("body not captured: %+v", cf.Body)
|
||||
}
|
||||
if cf.As == nil {
|
||||
t.Error("AS not captured")
|
||||
}
|
||||
// Options should include a LANGUAGE clause, a VOLATILE clause and a SECURITY clause.
|
||||
var langs, vols, secs int
|
||||
for _, cl := range cf.Options {
|
||||
if len(cl) == 0 {
|
||||
continue
|
||||
}
|
||||
switch {
|
||||
case cl[0].Is("language"):
|
||||
langs++
|
||||
case cl[0].Is("volatile"):
|
||||
vols++
|
||||
case cl[0].Is("security"):
|
||||
secs++
|
||||
}
|
||||
}
|
||||
if langs != 1 || vols != 1 || secs != 1 {
|
||||
t.Errorf("clause split wrong: language=%d volatile=%d security=%d", langs, vols, secs)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseFallbackRaw(t *testing.T) {
|
||||
f := Parse("SELECT a, b FROM t WHERE c = 1;")
|
||||
if len(f.Items) != 1 {
|
||||
t.Fatalf("want 1 item, got %d", len(f.Items))
|
||||
}
|
||||
if _, ok := f.Items[0].(*cst.Raw); !ok {
|
||||
t.Fatalf("want *Raw, got %T", f.Items[0])
|
||||
}
|
||||
}
|
||||
|
||||
// TestCorpusRoundTrip proves the parser is lossless on real-world input: the
|
||||
// reconstructed source must equal the original byte-for-byte.
|
||||
func TestCorpusRoundTrip(t *testing.T) {
|
||||
dir := filepath.Join("..", "..", "testdata", "corpus")
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
t.Skipf("no corpus dir: %v", err)
|
||||
}
|
||||
var seen, fns int
|
||||
for _, e := range entries {
|
||||
if e.IsDir() || !strings.HasSuffix(e.Name(), ".pgsql") {
|
||||
continue
|
||||
}
|
||||
seen++
|
||||
data, err := os.ReadFile(filepath.Join(dir, e.Name()))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
src := string(data)
|
||||
f := Parse(src)
|
||||
if got := f.Source(); got != src {
|
||||
t.Errorf("%s: round-trip mismatch (len in=%d out=%d)", e.Name(), len(src), len(got))
|
||||
}
|
||||
for _, it := range f.Items {
|
||||
if _, ok := it.(*cst.CreateFunction); ok {
|
||||
fns++
|
||||
}
|
||||
}
|
||||
}
|
||||
if seen == 0 {
|
||||
t.Skip("no corpus files")
|
||||
}
|
||||
t.Logf("round-tripped %d corpus files; structured %d CREATE FUNCTION/PROCEDURE", seen, fns)
|
||||
}
|
||||
@@ -0,0 +1,144 @@
|
||||
# PgTidy — PostgreSQL Linter & Formatter
|
||||
|
||||
## Context
|
||||
|
||||
We are starting a greenfield project, **PgTidy**: a PostgreSQL-focused linter and
|
||||
formatter that enforces consistent SQL style, detects PostgreSQL-specific issues, and
|
||||
formats migrations, schemas, functions, procedures, triggers, and related DB code.
|
||||
The primary day-one use case is **formatting PL/pgSQL stored procedures** to match an
|
||||
existing house style (sample corpus at `testdata/corpus`).
|
||||
|
||||
The core is written in **Go**. We ship a CLI, then an LSP server that powers a **VSCode
|
||||
extension** (priority) and later a **DataGrip/JetBrains** integration (lower priority).
|
||||
|
||||
### Decisions locked in (from planning Q&A)
|
||||
|
||||
- **Parsing = hybrid.** Real PostgreSQL grammar via **go-pgquery** (libpg_query compiled
|
||||
to WASM with `wazero`, **no cgo**) powers deep semantic lint rules. A **custom lossless
|
||||
lexer + CST** powers the comment-preserving formatter. (libpg_query drops comments &
|
||||
whitespace, so it cannot be the sole basis for a formatter — confirmed.)
|
||||
- **V1 = Formatter + CLI first.** Lint → LSP/VSCode → DataGrip follow.
|
||||
- **Lint scope (later milestones):** style/consistency, migration safety (Squawk-style),
|
||||
naming conventions, correctness/anti-patterns — all four.
|
||||
- **Editors:** one Go LSP core → VSCode now (bundled per-platform binary), DataGrip via
|
||||
the free **LSP4IJ** plugin later.
|
||||
|
||||
## Why this architecture
|
||||
|
||||
- **No cgo** (WASM-embedded libpg_query) keeps cross-compilation trivial and lets us bundle
|
||||
a single static binary per platform inside the VSCode extension. sqlc migrated to this exact
|
||||
approach to escape cgo pain.
|
||||
- **Formatting needs a lossless CST** (round-trippable, comments + whitespace preserved). This
|
||||
is the dominant pattern in mature tools (Roslyn, rust-analyzer/rowan, Biome, SQLFluff). We
|
||||
build our own lexer/CST so the formatter never loses a comment.
|
||||
- **Deep lint needs an accurate AST** — go-pgquery gives the real PostgreSQL parse tree.
|
||||
- **One core, many frontends** — CLI, LSP, VSCode and DataGrip all reuse the same engine and
|
||||
the same `diagnostics` type.
|
||||
|
||||
## Proposed layout
|
||||
|
||||
```
|
||||
pgtidy/
|
||||
cmd/pgtidy/ — CLI entry; subcommands: fmt, lint (v2), lsp (v3), version
|
||||
pkg/lexer/ — lossless lexer: tokens + trivia (comments/whitespace) attached
|
||||
pkg/cst/ — concrete syntax tree (round-trippable node model)
|
||||
pkg/parser/ — recursive-descent parser → CST (DML + DDL + PL/pgSQL)
|
||||
pkg/format/ — Doc-IR printer (Wadler/Prettier-style) + style application
|
||||
pkg/pgast/ — go-pgquery wrapper: SQL → real PG AST (for lint, v2)
|
||||
pkg/lint/ — rule engine + rule packs (v2)
|
||||
pkg/config/ — .pgtidy.yaml discovery/merge: style + rule config
|
||||
pkg/diagnostics/ — shared diagnostic type (CLI + LSP)
|
||||
pkg/lsp/ — LSP server (v3)
|
||||
editors/vscode/ — VSCode extension (TS), bundles pgtidy binary (v3)
|
||||
editors/datagrip/ — LSP4IJ integration (v4)
|
||||
testdata/ — golden formatter fixtures + lint fixtures + corpus
|
||||
```
|
||||
|
||||
## House style (formatter defaults — reverse-engineered from the corpus)
|
||||
|
||||
These become the default `style` config; all are configurable. The corpus had human
|
||||
inconsistencies (e.g. a DECLARE var at column 0, mixed `=`/`:=`); the formatter **normalizes**
|
||||
to the intended style below.
|
||||
|
||||
- Keywords **UPPERCASE**; data types **lowercase**; identifiers lowercase snake_case.
|
||||
- Indent: **2 spaces** per level.
|
||||
- **Leading-comma** style, one item per line, for SELECT column lists and function params.
|
||||
- Function headers: params one-per-line in parens; `LANGUAGE` / `SECURITY` / volatility each
|
||||
on own line; `AS` then `$$` on its own line; body; closing `$$;` on its own line.
|
||||
- PL/pgSQL: `DECLARE` alone, vars 2-space indented; `--Block--` comment markers preserved;
|
||||
`BEGIN`/`END` at body level; `IF/THEN/ELSIF/ELSE/END IF`, loops, `CASE` indent their bodies.
|
||||
- Spacing: spaces around binary operators (`=`,`<>`,`||`,…) and `:=`; **no** space around
|
||||
`::`, `->`, `->>`, array `[...]`, or before a call's `(`.
|
||||
- Dollar-quote tags preserved verbatim (`$$`, `$S$`, `$Z$`, …).
|
||||
|
||||
## Milestones
|
||||
|
||||
### V1 — Formatter + CLI (priority)
|
||||
|
||||
1. **Lexer** (`pkg/lexer`): full PG token coverage incl. dollar-quoted strings, `--` and
|
||||
`/* */` comments, operators. Comments + whitespace captured as **leading/trailing trivia**
|
||||
on tokens. Acceptance: `emit(lex(src)) == src` byte-for-byte across the corpus.
|
||||
2. **CST + parser** (`pkg/cst`, `pkg/parser`): recursive descent for DML (SELECT/INSERT/
|
||||
UPDATE/DELETE/CTE), DDL (CREATE FUNCTION/PROCEDURE/TABLE/INDEX/TRIGGER, ALTER, DO).
|
||||
3. **PL/pgSQL body parser**: DECLARE/BEGIN/END, IF/CASE/LOOP, assignments, nested SQL — the
|
||||
crux for stored-procedure formatting.
|
||||
4. **Printer** (`pkg/format`): Doc-IR (group/indent/line/softline) driven by `style` config.
|
||||
**Graceful degradation** — any span the parser can't handle passes through verbatim rather
|
||||
than being corrupted.
|
||||
5. **CLI** (`cmd/pgtidy fmt`): `--check`, `--write`/`-w`, stdin→stdout, `--diff`; config
|
||||
discovery walking up to `.pgtidy.yaml`; CI-friendly exit codes.
|
||||
6. **Config** (`pkg/config`): load/merge style config; defaults = house style above.
|
||||
|
||||
**Safety guarantees (tested):** semantic equivalence (re-lex output, compare non-trivia token
|
||||
stream to input), and idempotence (`fmt(fmt(x)) == fmt(x)`). The corpus is the
|
||||
primary safety/idempotence harness; golden-file tests for targeted cases.
|
||||
|
||||
### V2 — Linter
|
||||
|
||||
- `pkg/pgast` go-pgquery (WASM) wrapper; `pkg/lint` rule engine (rule ID, severity, config).
|
||||
- Rule packs: **style/consistency**, **migration safety** (ACCESS EXCLUSIVE locks, unsafe
|
||||
`ALTER`/`ADD COLUMN`, non-`CONCURRENTLY` index builds, blocking constraints), **naming**
|
||||
(configurable table/column/index/constraint patterns), **correctness/anti-patterns**
|
||||
(`SELECT *`, missing `WHERE` on UPDATE/DELETE, deprecated syntax). Emit `diagnostics`.
|
||||
- `pgtidy lint` subcommand; `--fix` for autofixable rules.
|
||||
|
||||
### V3 — LSP + VSCode
|
||||
|
||||
- `pkg/lsp`: `textDocument/formatting` + range formatting, `publishDiagnostics`, `codeAction`
|
||||
quick-fixes — all reusing the core.
|
||||
- `editors/vscode`: TS extension using `vscode-languageclient`, launches bundled `pgtidy lsp`.
|
||||
Build **per-platform VSIX** (`win32/linux/darwin × x64/arm64`) in a CI matrix (rust-analyzer
|
||||
model), with a target-less fallback.
|
||||
|
||||
### V4 — DataGrip
|
||||
|
||||
- `editors/datagrip`: integrate via **LSP4IJ** (free, works across JetBrains editions incl.
|
||||
DataGrip). No core changes expected.
|
||||
|
||||
## Build / repo hygiene
|
||||
|
||||
- Replace boilerplate `AGENTS.md`/`CLAUDE.md` with PgTidy content; rewrite `Makefile`
|
||||
(`APP := pgtidy`, `CMD := ./cmd/pgtidy`; keep build/test/lint/release-version targets).
|
||||
- Add **goreleaser** for the multi-platform binary matrix (clean, since no cgo).
|
||||
- `go.mod`: deps = `github.com/wasilibs/go-pgquery` (v2), `wazero`, a YAML lib, an LSP lib
|
||||
(e.g. `go.lsp.dev/protocol`) in v3.
|
||||
|
||||
## Verification
|
||||
|
||||
- **Formatter:** `go test ./...` runs golden-file tests + the corpus harness asserting
|
||||
(a) idempotence and (b) token-stream equality before/after (no semantic change).
|
||||
Manual: `pgtidy fmt --diff` against several procedures; confirm output matches
|
||||
the house style and comments/dollar-quote tags survive.
|
||||
- **CLI:** `pgtidy fmt --check` returns non-zero on unformatted input, zero when clean.
|
||||
- **Lint (v2):** fixture SQL with known violations → assert expected diagnostics; `--fix`
|
||||
round-trips.
|
||||
- **LSP/VSCode (v3):** load a `.sql` file in a dev-host VSCode, confirm format-on-save and
|
||||
live diagnostics via the bundled binary.
|
||||
|
||||
## Open risks
|
||||
|
||||
- A lossless PL/pgSQL recursive-descent parser is the largest single effort; the
|
||||
pass-through-on-unparsed fallback bounds the risk and lets us ship incrementally by construct.
|
||||
- go-pgquery tracks PG17 (PG18 not yet) — fine for lint; irrelevant to the formatter path.
|
||||
- Leading-comma + one-per-line is unusual vs. most formatters; it's a first-class style option,
|
||||
not an afterthought.
|
||||
+169
@@ -0,0 +1,169 @@
|
||||
--select * from dropall('resolvespec_login');
|
||||
CREATE OR REPLACE FUNCTION resolvespec_login(
|
||||
INOUT p_data jsonb
|
||||
,OUT p_success boolean
|
||||
,OUT p_error text
|
||||
)
|
||||
LANGUAGE plpgsql VOLATILE
|
||||
SECURITY DEFINER
|
||||
AS
|
||||
$$
|
||||
DECLARE
|
||||
--Error Handling--
|
||||
m_funcname text = 'resolvespec_login';
|
||||
m_errmsg text;
|
||||
m_errcontext text;
|
||||
m_errdetail text;
|
||||
m_errhint text;
|
||||
m_errstate text;
|
||||
m_retval integer;
|
||||
--Error Handling--
|
||||
m_rid_user integer;
|
||||
m_rid_hub integer;
|
||||
m_pass_hashed citext[];
|
||||
m_session jsonb;
|
||||
m_allow_hash_auth boolean;
|
||||
BEGIN
|
||||
m_allow_hash_auth = _try_integer( p_data->'claims'->>'rid_user',0) > 0;
|
||||
create extension if not exists pgcrypto;
|
||||
|
||||
perform log_event(m_funcname,format('API Login username: %s hh=%s claims: %s',p_data->>'username',m_allow_hash_auth,p_data->'claims'), bt_enum('eventlog','local notice'));
|
||||
|
||||
select h.rid_hub
|
||||
from public.user h
|
||||
where h.usercode = p_data->>'username'
|
||||
into m_rid_hub;
|
||||
|
||||
if m_rid_hub is null
|
||||
and exists (select 1 from information_schema.tables t where t.table_schema = 'public' and t.table_name = 'users')
|
||||
then
|
||||
select u.rid_hub, u.rid_user
|
||||
from public.users u
|
||||
where u.rid_user = _try_integer( p_data->'claims'->>'rid_user',0)
|
||||
into m_rid_hub,m_rid_user;
|
||||
|
||||
end if;
|
||||
|
||||
if m_rid_hub is null
|
||||
then
|
||||
raise exception 'Invalid username / password';
|
||||
end if;
|
||||
|
||||
m_pass_hashed = array[encode(digest(format('%s:%s',p_data->>'username',p_data->>'password'), 'sha512'), 'hex')
|
||||
,encode(digest(format('%s:%s',p_data->>'username',p_data->>'password'), 'md5'), 'hex')
|
||||
,encode(digest(format('%s',p_data->>'password'), 'md5'), 'hex')
|
||||
]::citext[];
|
||||
|
||||
if m_allow_hash_auth
|
||||
then
|
||||
m_pass_hashed := m_pass_hashed || array[
|
||||
p_data->>'password'
|
||||
]::citext[];
|
||||
end if;
|
||||
|
||||
--select $A${"meta": null, "claims": {"rid_user": 30000024}, "password": "c4ca4238a0b923820dcc509a6f75849b", "username": "SUPPORT"}$A$::jsonb->>'password'
|
||||
--c4ca4238a0b923820dcc509a6f75849b
|
||||
--select * from v_eventlog
|
||||
|
||||
if exists (
|
||||
select 1
|
||||
from information_schema.tables t
|
||||
where t.table_schema = 'public'
|
||||
and t.table_name = 'users'
|
||||
)
|
||||
then
|
||||
if not exists (
|
||||
select 1
|
||||
from public.user h
|
||||
left outer join public.users usr on usr.rid_hub = h.rid_hub
|
||||
where h.rid_hub = m_rid_hub
|
||||
and (
|
||||
h.password = any (m_pass_hashed)
|
||||
and nv(h.password) <> ''
|
||||
or usr.password = any(m_pass_hashed)
|
||||
and nv(usr.password) <> ''
|
||||
)
|
||||
)
|
||||
then
|
||||
raise exception 'Password incorrect';
|
||||
end if;
|
||||
|
||||
elsif not exists (
|
||||
select 1
|
||||
from public.user h
|
||||
where h.rid_hub = m_rid_hub
|
||||
and h.password = any(m_pass_hashed)
|
||||
and nv(h.password) <> ''
|
||||
)
|
||||
then
|
||||
raise exception 'Password incorrect';
|
||||
end if;
|
||||
|
||||
if _try_bool(p_data->'jsonvalue'->>'issecurity',false)
|
||||
and not exists (
|
||||
select h.rid_hub
|
||||
from public.user h
|
||||
inner join public.user_all_parents(m_rid_hub) p on p.parent_rid_hub = h.rid_hub
|
||||
where h.hubname ilike '%Access Control%'
|
||||
)
|
||||
then
|
||||
raise exception 'Cannot login with security mode. User must be in Access Control group';
|
||||
end if;
|
||||
|
||||
with newsession as (
|
||||
insert into core._loginsession (createtm, modifytm, rid_user, usertable, sessionid, token, useragent, location,
|
||||
ipaddress, expiretm, jsonvalue)
|
||||
select now(), now(), m_rid_hub, 'hub',newid(), newid(),p_data->>'user-agent',p_data->>'fromurl'
|
||||
,p_data->>'host', (now() + '31 days'::interval), p_data->'jsonvalue'
|
||||
returning *
|
||||
)
|
||||
select to_jsonb(newsession)
|
||||
from newsession
|
||||
into m_session;
|
||||
|
||||
if _try_integer(m_session->>'rid_user',0) > 0
|
||||
then
|
||||
update public.user u
|
||||
set jsonvalue = _jsonb_object_cat(u.jsonvalue, jsonb_build_object('lastlogin',to_char(now(),'YYYY-MM-DD HH24:mi:SS')))
|
||||
where u.rid_hub = m_rid_hub;
|
||||
end if;
|
||||
|
||||
|
||||
|
||||
|
||||
select jsonb_build_object('token',m_session->>'token'
|
||||
,'session',m_session->>'session'
|
||||
, 'user', _jsonb_object_cat(jsonb_build_object(
|
||||
'user_id', h.rid_hub
|
||||
,'username', h.usercode
|
||||
,'email',null
|
||||
,'user_level', 0
|
||||
,'roles', jsonb_build_array()
|
||||
,'session_id', m_session->>'sessionid'
|
||||
,'token', m_session->>'token'
|
||||
,'session_rid', _try_integer(m_session->>'id')
|
||||
), (
|
||||
select jsonb_build_object('program_user_table', r.tablename,'program_user_id', _try_integer(r.key,0))
|
||||
from public.user_tableinfo(m_rid_hub) r
|
||||
)
|
||||
)
|
||||
,'expires_in', 86400
|
||||
)
|
||||
from public.user h
|
||||
where h.rid_hub = m_rid_hub
|
||||
into p_data;
|
||||
|
||||
p_success = true;
|
||||
EXCEPTION
|
||||
WHEN others THEN
|
||||
GET STACKED DIAGNOSTICS
|
||||
m_errmsg = MESSAGE_TEXT
|
||||
,m_errcontext = PG_EXCEPTION_CONTEXT
|
||||
,m_errdetail = PG_EXCEPTION_DETAIL
|
||||
,m_errhint = PG_EXCEPTION_HINT
|
||||
,m_errstate = RETURNED_SQLSTATE;
|
||||
|
||||
p_error := get_err_msg(m_funcname, m_errmsg, m_errcontext, m_errdetail, m_errhint, m_errstate);
|
||||
p_success := false;
|
||||
END;
|
||||
$$;
|
||||
+790
@@ -0,0 +1,790 @@
|
||||
--select * from dropall('migrations_read','meta');
|
||||
CREATE OR REPLACE FUNCTION meta.migration_read(
|
||||
p_id_migration_model integer default null
|
||||
,OUT p_retval integer
|
||||
,OUT p_errmsg text
|
||||
,OUT p_info jsonb
|
||||
)
|
||||
LANGUAGE plpgsql VOLATILE
|
||||
SECURITY DEFINER
|
||||
AS
|
||||
$$
|
||||
DECLARE
|
||||
m_funcname text = 'migrations_read';
|
||||
m_errmsg text;
|
||||
m_errcontext text;
|
||||
m_errdetail text;
|
||||
m_errhint text;
|
||||
m_errstate text;
|
||||
m_retval integer;
|
||||
m_id_migration_model integer;
|
||||
m_payload text;
|
||||
m_tm timestamp;
|
||||
|
||||
m_xml xml;
|
||||
m_json json;
|
||||
j_err jsonb;
|
||||
BEGIN
|
||||
m_tm = clock_timestamp();
|
||||
|
||||
select f.id_migration_model
|
||||
,convert_from(case when f_iscompressed(f.modelfile) = 'gzip' then pl_gzip_bytes(0,f.modelfile) else f.modelfile end, 'utf8')
|
||||
from meta.migration_model f
|
||||
where f.id_migration_model = p_id_migration_model
|
||||
or p_id_migration_model is null
|
||||
order by f.version desc , f.id_migration_model desc
|
||||
into m_id_migration_model, m_payload;
|
||||
|
||||
if m_payload ilike '%<%>%'
|
||||
then
|
||||
m_xml = m_payload::xml;
|
||||
raise notice 'XML File set';
|
||||
elseif m_payload ilike '%{%}%'
|
||||
then
|
||||
raise notice 'JSON File set';
|
||||
m_json = m_payload::json;
|
||||
end if;
|
||||
|
||||
if m_xml is not null
|
||||
then
|
||||
|
||||
perform pg_notify('upgrade.events', json_build_object('type','upgrade','status',1,'objecttype',m_funcname)::text);
|
||||
|
||||
p_info = jsonb_build_object('format','xml');
|
||||
|
||||
delete from meta.migration_table
|
||||
where ismodel;
|
||||
|
||||
delete from meta.migration_index
|
||||
where ismodel;
|
||||
|
||||
delete from meta.migration_relation_col d
|
||||
where d.rid_migration_relation in (
|
||||
select r.id_migration_relation
|
||||
from meta.migration_relation r
|
||||
where r.ismodel
|
||||
);
|
||||
|
||||
delete from meta.migration_relation
|
||||
where ismodel;
|
||||
|
||||
raise notice 'inserting meta.migration_table @ %', (clock_timestamp() - m_tm)::interval;
|
||||
m_tm = clock_timestamp();
|
||||
|
||||
drop table if exists tmp_meta_migration_table;
|
||||
create temp table tmp_meta_migration_table as
|
||||
select
|
||||
lower((xpath('/table/tableident/text()', node.x))[1]::text)::citext as tableident
|
||||
, lower((xpath('/table/prefix/text()', node.x))[1]::text)::citext as tableprefix
|
||||
, lower((xpath('/table/tablename/text()', node.x))[1]::text)::citext as tablename
|
||||
, lower(coalesce((xpath('/table/schema/text()', node.x))[1]::text,(xpath('/table/schemaname/text()', node.x))[1]::text)) as schemaname
|
||||
, lower((xpath('/table/version/text()', node.x))[1]::text)::citext as version
|
||||
, coalesce((xpath('/table/@seq', node.x))[1]::text,'0')::integer + 1 as schemapriority
|
||||
, node.x as xml
|
||||
from unnest(xpath('/root/tables/table', m_xml)) node(x)
|
||||
;
|
||||
|
||||
insert into meta.migration_table(rid_migration_model ,guid,prefix, tablename, schemaname, version,ismodel, isdb,schemapriority)
|
||||
select distinct on (r.tableident,r.tablename,r.tableprefix,r.schemaname)
|
||||
m_id_migration_model
|
||||
,r.tableident
|
||||
,r.tableprefix
|
||||
,r.tablename
|
||||
,r.schemaname
|
||||
,r.version
|
||||
,true
|
||||
,false
|
||||
,r.schemapriority
|
||||
from tmp_meta_migration_table r
|
||||
;
|
||||
|
||||
|
||||
raise notice 'inserting meta.migration_column @ %', (clock_timestamp() - m_tm)::interval;
|
||||
m_tm = clock_timestamp();
|
||||
|
||||
insert into meta.migration_column(rid_migration_table, columnname, guid, columntype,indextype,seqseed, columnlen, precision, defaultval)
|
||||
select distinct on (r.id_migration_table,r.columnname,r.columnident)
|
||||
r.id_migration_table,r.columnname,r.columnident,r.columntype,r.indextype,r.seq_seed,r.columnlen,r.precision
|
||||
,r.defaultval
|
||||
from (
|
||||
select
|
||||
mt.id_migration_table
|
||||
, lower((xpath('/column/columnname/text()', col.x))[1]::text) as columnname
|
||||
, (xpath('/column/columnident/text()', col.x))[1]::text as columnident
|
||||
, (xpath('/column/columntype/text()', col.x))[1]::text as columntype
|
||||
, (xpath('/column/indextype/text()', col.x))[1]::text as indextype
|
||||
, coalesce((xpath('/column/seed/text()', col.x))[1]::text,'0')::bigint as seq_seed
|
||||
, coalesce((xpath('/column/columnlen/text()', col.x))[1]::text, '0')::integer as columnlen
|
||||
, coalesce((xpath('/column/precision/text()', col.x))[1]::text, '')::text as precision
|
||||
, coalesce((xpath('/column/defaultval/text()', col.x))[1]::text, '')::text as defaultval
|
||||
from tmp_meta_migration_table tbl
|
||||
cross join unnest(xpath('/table/columns/column', tbl.xml)) col(x)
|
||||
inner join meta.migration_table mt on mt.ismodel
|
||||
and mt.rid_migration_model = m_id_migration_model
|
||||
-- and lower(mt.guid) = lower((xpath('/table/tableident/text()', tbl.x))[1]::text)
|
||||
and lower(mt.tablename) = lower(tbl.tablename)
|
||||
and lower(mt.schemaname) =lower(tbl.schemaname)
|
||||
) r
|
||||
;
|
||||
|
||||
|
||||
raise notice 'inserting meta.migration_index @ %', (clock_timestamp() - m_tm)::interval;
|
||||
m_tm = clock_timestamp();
|
||||
|
||||
insert into meta.migration_index(rid_migration_model,rid_migration_table, indexname, indextype, ispk,isduplicate, ispartial,partialstr, isunique, sequence, guid,ismodel)
|
||||
select distinct on (r.id_migration_table,r.indexname)
|
||||
m_id_migration_model
|
||||
,r.id_migration_table
|
||||
,r.indexname
|
||||
|
||||
,null
|
||||
,r.indexprimary in ('1','true')
|
||||
,r.indexduplicate in ('1','true')
|
||||
,length(r.indexpartial) > 3
|
||||
,r.indexpartial
|
||||
,(not coalesce(r.indexduplicate,'') in ('1','true') and (r.indexunique in ('1','true') or r.indexprimary in ('1','true')))
|
||||
,row_number() over (order by r.indexname)
|
||||
,r.indexname
|
||||
,true
|
||||
from (
|
||||
select
|
||||
mt.id_migration_table
|
||||
, mt.schemaname
|
||||
, mt.tablename
|
||||
, lower((xpath('/index/indexname/text()', idx.x))[1]::text) as indexname
|
||||
, coalesce(lower((xpath('/index/indexprimary/text()', idx.x))[1]::text),'') as indexprimary
|
||||
, coalesce(lower((xpath('/index/indexduplicate/text()', idx.x))[1]::text),'') as indexduplicate
|
||||
, coalesce(lower((xpath('/index/indexpartial/text()', idx.x))[1]::text),'') as indexpartial
|
||||
, coalesce(lower((xpath('/index/indexunique/text()', idx.x))[1]::text),'') as indexunique
|
||||
from tmp_meta_migration_table tbl
|
||||
cross join unnest(xpath('/table/indexes/index', tbl.xml)) idx(x)
|
||||
inner join meta.migration_table mt on mt.ismodel
|
||||
and mt.rid_migration_model = m_id_migration_model
|
||||
and lower(mt.tablename) = lower(tbl.tablename)
|
||||
and lower(mt.schemaname) =lower(tbl.schemaname)
|
||||
) r
|
||||
;
|
||||
|
||||
|
||||
update meta.migration_index u
|
||||
set isunique = true
|
||||
where u.indexname ilike 'uk_%'
|
||||
and not u.ispk
|
||||
and not u.ispartial
|
||||
and not u.isduplicate
|
||||
;
|
||||
|
||||
-- update meta.migration_index u
|
||||
-- set ispartial = true
|
||||
-- where u.indexname ilike 'k_%'
|
||||
-- and not u.isunique
|
||||
-- and not u.ispk
|
||||
-- ;
|
||||
|
||||
insert into meta.migration_index_col(rid_migration_index,rid_migration_column_parent,sequence)
|
||||
select distinct on (r.id_migration_index,r.id_migration_column)
|
||||
r.id_migration_index
|
||||
,r.id_migration_column
|
||||
,r.seq
|
||||
from (
|
||||
select
|
||||
midx.id_migration_index
|
||||
,mc.id_migration_column
|
||||
,coalesce((xpath('/indexcolumn/@seq', idxcol.x))[1]::text,'0')::integer as seq
|
||||
from tmp_meta_migration_table tbl
|
||||
cross join unnest(xpath('/table/indexes/index', tbl.xml)) idx(x)
|
||||
inner join meta.migration_table mt on mt.ismodel
|
||||
and mt.rid_migration_model = m_id_migration_model
|
||||
and lower(mt.tablename) = lower(tbl.tablename)
|
||||
and lower(mt.schemaname) =lower(tbl.schemaname)
|
||||
inner join meta.migration_index midx on midx.rid_migration_table = mt.id_migration_table
|
||||
and midx.indexname = lower((xpath('/index/indexname/text()', idx.x))[1]::text)
|
||||
|
||||
cross join unnest(xpath('/index/indexcolumns/indexcolumn', idx.x)) idxcol(x)
|
||||
inner join meta.migration_column mc on mt.id_migration_table = mc.rid_migration_table
|
||||
and lower(mc.columnname) = lower((xpath('/indexcolumn/text()', idxcol.x))[1]::text)
|
||||
) r
|
||||
;
|
||||
|
||||
raise notice 'inserting meta.migration_relation temp table @ %', (clock_timestamp() - m_tm)::interval;
|
||||
m_tm = clock_timestamp();
|
||||
|
||||
drop table if exists tmp_meta_gigration_relation;
|
||||
create temp table tmp_meta_gigration_relation as
|
||||
select
|
||||
tbl.tablename
|
||||
,tbl.schemaname
|
||||
, lower((xpath('/relation/childtable/text()', rel.x))[1]::text) as childtable
|
||||
, lower((xpath('/relation/relationname/text()', rel.x))[1]::text) as relationname
|
||||
, lower((xpath('/relation/relationguid/text()', rel.x))[1]::text) as relationguid
|
||||
, lower((xpath('/relation/deleteconstraint/text()', rel.x))[1]::text) as deleteconstraint
|
||||
, lower((xpath('/relation/updateconstraint/text()', rel.x))[1]::text) as updateconstraint
|
||||
,rel.x as relation
|
||||
from tmp_meta_migration_table tbl
|
||||
cross join unnest(xpath('/table/relations/relation', tbl.xml)) rel(x)
|
||||
;
|
||||
|
||||
raise notice 'inserting meta.migration_relation insert @ %', (clock_timestamp() - m_tm)::interval;
|
||||
m_tm = clock_timestamp();
|
||||
|
||||
insert
|
||||
into meta.migration_relation( rid_migration_model
|
||||
, rid_migration_table_parent
|
||||
, rid_migration_table_child
|
||||
, relationname
|
||||
, guid
|
||||
, updateconstraint
|
||||
, deleteconstraint
|
||||
, sequence
|
||||
, ismodel)
|
||||
select m_id_migration_model
|
||||
,mt.id_migration_table
|
||||
,ct.id_migration_table as rid_child_table
|
||||
,src.relationname
|
||||
,src.relationguid
|
||||
,src.updateconstraint
|
||||
,src.deleteconstraint
|
||||
,row_number() over (order by src.relationname)
|
||||
,true
|
||||
from tmp_meta_gigration_relation as src
|
||||
inner join meta.migration_table mt on mt.ismodel
|
||||
and mt.rid_migration_model = m_id_migration_model
|
||||
and lower(mt.tablename) = lower(src.tablename)
|
||||
and lower(mt.schemaname) = lower(src.schemaname)
|
||||
inner join meta.migration_table ct on ct.ismodel
|
||||
and ct.rid_migration_model = m_id_migration_model
|
||||
and lower(ct.schemaname) = lower(mt.schemaname)
|
||||
and lower(ct.tablename) = lower(src.childtable)
|
||||
;
|
||||
|
||||
raise notice 'inserting meta.migration_relation_col @ %', (clock_timestamp() - m_tm)::interval;
|
||||
m_tm = clock_timestamp();
|
||||
|
||||
insert into meta.migration_relation_col(rid_migration_relation, rid_migration_column_parent, rid_migration_column_child, sequence)
|
||||
select mrel.id_migration_relation
|
||||
, parcol.id_migration_column
|
||||
, cldcol.id_migration_column
|
||||
,coalesce((xpath('/keyfields/@seq', key.x))[1]::text,'0')::integer as seq
|
||||
from tmp_meta_gigration_relation src
|
||||
inner join meta.migration_table mt on mt.ismodel
|
||||
and mt.rid_migration_model = m_id_migration_model
|
||||
and lower(mt.tablename) = lower(src.tablename)
|
||||
and lower(mt.schemaname) = lower(src.schemaname)
|
||||
inner join meta.migration_relation mrel on mrel.rid_migration_table_parent = mt.id_migration_table
|
||||
and lower(mrel.relationname) = lower(src.relationname)
|
||||
cross join unnest(xpath('/relation/keyfields', src.relation)) key(x)
|
||||
inner join meta.migration_column parcol on mrel.rid_migration_table_parent = parcol.rid_migration_table
|
||||
and lower(parcol.columnname) = lower((xpath('/keyfields/parentcolumn/text()', key.x))[1]::text)
|
||||
inner join meta.migration_column cldcol on mrel.rid_migration_table_child = cldcol.rid_migration_table
|
||||
and lower(cldcol.columnname) = lower((xpath('/keyfields/childcolumn/text()', key.x))[1]::text)
|
||||
;
|
||||
|
||||
raise notice 'inserting meta.migration_object @ %', (clock_timestamp() - m_tm)::interval;
|
||||
m_tm = clock_timestamp();
|
||||
|
||||
delete from meta.migration_object
|
||||
where ismodel;
|
||||
|
||||
insert into meta.migration_object(rid_migration_model,objecttype,objectname, schema, version, checksum, sequence, priority, guid, body, ismodel, isdb)
|
||||
select m_id_migration_model
|
||||
,format('script:%s',r.scripttype)
|
||||
,format('%s [%s]',r.scriptname,r.id_scriptcode)
|
||||
,r.dbschema,r.version
|
||||
, case when coalesce(r.scriptchecksum) <> '' then r.scriptchecksum else encode(sha256(convert_to(r.scriptcode,'utf8')),'hex') end
|
||||
, r.sequence, r.priority, r.id_scriptcode::text,r.scriptcode,true,false
|
||||
from (
|
||||
select (xpath('/script/priority/text()', node.x))[1]::text::integer as priority
|
||||
, (xpath('/script/sequence/text()', node.x))[1]::text::integer as sequence
|
||||
, lower((xpath('/script/scriptname/text()', node.x))[1]::text)::citext as scriptname
|
||||
, lower((xpath('/script/scripttype/text()', node.x))[1]::text)::citext as scripttype
|
||||
, lower((xpath('/script/dbschema/text()', node.x))[1]::text)::citext as dbschema
|
||||
, lower((xpath('/script/programname/text()', node.x))[1]::text)::citext as programname
|
||||
, lower((xpath('/script/version/text()', node.x))[1]::text)::citext as version
|
||||
, lower((xpath('/script/scriptchecksum/text()', node.x))[1]::text)::citext as scriptchecksum
|
||||
, xml_extract_value('/script/code', node.x)::text as scriptcode
|
||||
,'dct' as source
|
||||
,(xpath('/script/id_scriptcode/text()', node.x))[1]::text::integer as id_scriptcode
|
||||
from unnest(xpath('/root/scripts/script', m_xml )) node(x)
|
||||
) r
|
||||
;
|
||||
|
||||
elsif m_json is not null
|
||||
then
|
||||
p_info = jsonb_build_object('format','json');
|
||||
raise exception 'Not yet supported';
|
||||
else
|
||||
p_info = jsonb_build_object('format','unknown');
|
||||
raise exception 'Unsupported input file, Content: %', substr(m_payload, 1,20);
|
||||
end if;
|
||||
|
||||
|
||||
|
||||
|
||||
insert into meta.migration_column(rid_migration_table, columnname, columntype, guid)
|
||||
select distinct on (t.id_migration_table)
|
||||
t.id_migration_table, 'updatecnt', 'integer',format('updatecnt_%s', t.id_migration_table)
|
||||
from meta.migration_table t
|
||||
left outer join meta.migration_column c on c.rid_migration_table = t.id_migration_table
|
||||
and c.columnname = 'updatecnt'
|
||||
where t.ismodel
|
||||
and t.tablename not in (
|
||||
select uut.tablename
|
||||
from meta.f_upgrade_table() uut
|
||||
)
|
||||
and c.id_migration_column is null
|
||||
;
|
||||
|
||||
raise notice 'updates section @ %', (clock_timestamp() - m_tm)::interval;
|
||||
m_tm = clock_timestamp();
|
||||
|
||||
|
||||
--Set primary key field from indexes
|
||||
update meta.migration_column u
|
||||
set ispk = true
|
||||
from meta.migration_column c
|
||||
inner join meta.migration_table t on t.id_migration_table = c.rid_migration_table
|
||||
and t.ismodel
|
||||
inner join meta.migration_index idx on idx.rid_migration_table = t.id_migration_table
|
||||
inner join meta.migration_index_col idxcol on idxcol.rid_migration_index = idx.id_migration_index
|
||||
and idxcol.rid_migration_column_parent = c.id_migration_column
|
||||
where idx.ispk
|
||||
and u.id_migration_column = c.id_migration_column
|
||||
;
|
||||
|
||||
|
||||
|
||||
--Set length for strings
|
||||
update meta.migration_column
|
||||
set columnlen = reverse(substr(reverse(columntype),2,strpos(reverse(columntype),'(')-2))::integer
|
||||
- (case when columntype ilike '%cstring%' then 1 else 0 end)
|
||||
where rid_migration_table in (select t.id_migration_table from meta.migration_table t where t.ismodel)
|
||||
and columntype ilike '%string%'
|
||||
;
|
||||
|
||||
---Set the length and precision values
|
||||
update meta.migration_column u
|
||||
set columnlen = case when coalesce(u.columnlen,0) = 0 and (r.precision > 0 or r.scale > 0)
|
||||
then coalesce(r.precision,0) + coalesce(r.scale,0)
|
||||
else u.columnlen
|
||||
end
|
||||
,precision = format('%s,%s',r.precision,r.scale)
|
||||
from (
|
||||
select id_migration_column
|
||||
,substring(columntype FROM '\((\d+),(\d+)\)')::integer AS precision
|
||||
, substring(columntype FROM '\(\d+,(\d+)\)')::integer AS scale
|
||||
from meta.migration_column
|
||||
where
|
||||
rid_migration_table in (
|
||||
select t.id_migration_table from meta.migration_table t where t.ismodel
|
||||
)
|
||||
and columntype ilike '%(%)'
|
||||
) r
|
||||
where u.id_migration_column = r.id_migration_column
|
||||
;
|
||||
|
||||
--Set default values for prefixes
|
||||
update meta.migration_column u
|
||||
set defaultval = (
|
||||
select quote_literal(upper(t.prefix))
|
||||
from meta.migration_table t
|
||||
where t.id_migration_table = u.rid_migration_table
|
||||
and t.ismodel
|
||||
and nv(t.prefix) <> ''
|
||||
)
|
||||
where u.rid_migration_table in (select t.id_migration_table from meta.migration_table t where t.ismodel)
|
||||
and u.columnname = 'prefix'
|
||||
and coalesce(u.defaultval,'') = ''
|
||||
;
|
||||
|
||||
--Set GUID field types. e.g. text, uuid
|
||||
with option as (
|
||||
select name, value
|
||||
from meta.migration_option
|
||||
where name = 'guidtype'
|
||||
)
|
||||
update meta.migration_column u
|
||||
set columntype = option.value
|
||||
,defaultval = case when nv(u.defaultval) = '' then 'newid()' else u.defaultval end
|
||||
from option
|
||||
where rid_migration_table in (
|
||||
select t.id_migration_table
|
||||
from meta.migration_table t
|
||||
where t.ismodel
|
||||
and not exists (
|
||||
select 1
|
||||
from meta.migration_option o
|
||||
cross join regexp_split_to_table(o.value, ',') rex(v)
|
||||
where o.name = 'textguid'
|
||||
and rex.v::citext = t.tablename
|
||||
and t.schemaname = 'public'
|
||||
)
|
||||
)
|
||||
and u.columnname in ('guid','uuid')
|
||||
and u.columntype is distinct from option.value
|
||||
;
|
||||
|
||||
|
||||
--Limit length constraints
|
||||
with option as (
|
||||
select name, value::numeric as numvalue
|
||||
from meta.migration_option
|
||||
where name = 'max_constraint'
|
||||
and value ~ '^-?[0-9]+(\.[0-9]+)?$'
|
||||
)
|
||||
update meta.migration_column u
|
||||
set columnlen = 0
|
||||
from option
|
||||
where rid_migration_table in (select t.id_migration_table from meta.migration_table t where t.ismodel)
|
||||
and u.columnlen >= option.numvalue
|
||||
;
|
||||
|
||||
--Force names if json type options
|
||||
with option as (
|
||||
select name, value
|
||||
from meta.migration_option
|
||||
where name = 'settype_names_jsonb'
|
||||
and value is not null
|
||||
)
|
||||
update meta.migration_column u
|
||||
set columntype = 'jsonb'
|
||||
from option
|
||||
where rid_migration_table in (select t.id_migration_table from meta.migration_table t where t.ismodel)
|
||||
and lower(u.columnname) in (
|
||||
SELECT lower(t.v) from regexp_split_to_table(option.value, ',') t(v)
|
||||
)
|
||||
;
|
||||
|
||||
--Convert the program types to postgres types
|
||||
update meta.migration_column u
|
||||
set columntype = meta.f_datatype_map(u.columntype)
|
||||
where rid_migration_table in (select t.id_migration_table from meta.migration_table t where t.ismodel)
|
||||
and meta.f_datatype_map(u.columntype) not ilike '%unknown%'
|
||||
;
|
||||
|
||||
update meta.migration_column u
|
||||
set columntype = case when u.columntype in ('date','text','citext') and (u.columnname ilike '%datetime%' or u.columnname ilike '%timestamp%')
|
||||
then 'timestamp'
|
||||
else u.columntype
|
||||
end
|
||||
where rid_migration_table in (select t.id_migration_table from meta.migration_table t where t.ismodel)
|
||||
and meta.f_datatype_map(u.columntype) not ilike '%unknown%'
|
||||
;
|
||||
|
||||
|
||||
--Larges objects has no lengths
|
||||
update meta.migration_column u
|
||||
set columnlen = 0
|
||||
where rid_migration_table in (select t.id_migration_table from meta.migration_table t where t.ismodel)
|
||||
and lower(u.columntype) in ('blob,0','jsonb','json','blob','bytea')
|
||||
;
|
||||
|
||||
j_err = null;
|
||||
select jsonb_agg(
|
||||
jsonb_build_object('tablename',t.tablename,'schemaname',t.schemaname,'word', kw.word)
|
||||
)
|
||||
from meta.migration_table t
|
||||
inner join pg_get_keywords() kw on lower(kw.word) = lower(t.tablename)
|
||||
and lower(kw.catdesc::text) = 'reserved'
|
||||
where t.ismodel
|
||||
into j_err
|
||||
;
|
||||
|
||||
if jsonb_typeof(j_err) = 'array'
|
||||
then
|
||||
p_info = p_info || jsonb_build_object('table_reserved_words',j_err) ;
|
||||
end if;
|
||||
|
||||
j_err = null;
|
||||
select jsonb_agg(
|
||||
jsonb_build_object('columnname',u.columnname,'tablename',t.tablename,'schemaname',t.schemaname,'word', kw.word)
|
||||
)
|
||||
from meta.migration_column u
|
||||
inner join meta.migration_table t on t.id_migration_table = u.rid_migration_table
|
||||
inner join pg_get_keywords() kw on lower(kw.word) = lower(u.columnname)
|
||||
and lower(kw.catdesc::text) = 'reserved'
|
||||
where u.rid_migration_table in (select t.id_migration_table from meta.migration_table t where t.ismodel)
|
||||
into j_err
|
||||
;
|
||||
|
||||
if jsonb_typeof(j_err) = 'array'
|
||||
then
|
||||
p_info = p_info || jsonb_build_object('column_reserved_words',j_err) ;
|
||||
end if;
|
||||
|
||||
----Set the default value to the identity if the pk type is identity
|
||||
update meta.migration_column u
|
||||
set defaultval = r.def
|
||||
from (
|
||||
select
|
||||
c.id_migration_column
|
||||
,format($S$nextval('%s.identity_%s_%s'::regclass)$S$,t.schemaname,t.tablename,c.columnname) as def
|
||||
from meta.migration_table t
|
||||
inner join meta.migration_column c on c.rid_migration_table = t.id_migration_table
|
||||
where
|
||||
t.ismodel
|
||||
and c.ispk
|
||||
and nv(c.defaultval) = ''
|
||||
and c.indextype = 'identity'
|
||||
) r
|
||||
where u.id_migration_column = r.id_migration_column;
|
||||
|
||||
update meta.migration_table u
|
||||
set ismodel = false
|
||||
where format('%s_%s',u.schemaname, u.tablename) in (
|
||||
select format('%s_%s',split_part(o.name,':',2)
|
||||
,t.name
|
||||
)
|
||||
from meta.migration_option o
|
||||
cross join regexp_split_to_table(o.value,',') t(name)
|
||||
where o.name ilike 'exclude:%'
|
||||
and t.name <> ''
|
||||
)
|
||||
and u.ismodel
|
||||
;
|
||||
|
||||
raise notice 'duplicates section @ %', (clock_timestamp() - m_tm)::interval;
|
||||
m_tm = clock_timestamp();
|
||||
|
||||
|
||||
--Fallback fix missing primary keys
|
||||
|
||||
update meta.migration_column u
|
||||
set ispk = true
|
||||
from meta.migration_table t
|
||||
where t.id_migration_table = u.rid_migration_table
|
||||
and t.ismodel
|
||||
and not exists (
|
||||
select 1
|
||||
from meta.migration_column c2
|
||||
where c2.rid_migration_table = t.id_migration_table
|
||||
and c2.ispk
|
||||
)
|
||||
and u.indextype = 'identity'
|
||||
and u.columntype ilike '%int%'
|
||||
and u.columnname ilike '%id_%'
|
||||
;
|
||||
|
||||
|
||||
--Move duplicate tables that exists in the settings to their default location. Move the keys as well.
|
||||
with move as (
|
||||
select
|
||||
t1.schemaname
|
||||
, t1.tablename
|
||||
, t1.id_migration_table as id_migration_table_dest
|
||||
, t2.schemaname
|
||||
, t2.tablename
|
||||
, t2.id_migration_table as id_migration_table_src
|
||||
,format('%s.%s',t1.schemaname, t1.tablename) as named
|
||||
from meta.migration_table t1
|
||||
inner join meta.migration_table t2 on t2.ismodel
|
||||
and t2.tablename = t1.tablename
|
||||
and t2.schemaname is distinct from t1.schemaname
|
||||
where
|
||||
t1.ismodel
|
||||
--and not t1.isdb
|
||||
and (
|
||||
format('%s.%s',t1.schemaname, t1.tablename) in (
|
||||
select format('%s.%s',o.value,lower(split_part(o.name, ':', 2)))
|
||||
from meta.migration_option o
|
||||
where o.name ilike 'default_schema:%'
|
||||
)
|
||||
)
|
||||
), cols as (
|
||||
update meta.migration_column u
|
||||
set rid_migration_table = m.id_migration_table_dest
|
||||
from move m
|
||||
inner join meta.migration_column c1 on c1.rid_migration_table = m.id_migration_table_src
|
||||
and c1.columnname not in (
|
||||
select c2.columnname
|
||||
from meta.migration_column c2
|
||||
where c2.rid_migration_table = m.id_migration_table_dest
|
||||
)
|
||||
where u.id_migration_column = c1.id_migration_column
|
||||
returning *
|
||||
), relations1 as (
|
||||
update meta.migration_relation u
|
||||
set rid_migration_table_parent = m.id_migration_table_dest
|
||||
,relationname = format('%s_merged',u.relationname)
|
||||
from move m
|
||||
where u.rid_migration_table_parent = m.id_migration_table_src
|
||||
returning *
|
||||
), relations2 as (
|
||||
update meta.migration_relation u
|
||||
set rid_migration_table_child = m.id_migration_table_dest
|
||||
,relationname = format('%s_merged',u.relationname)
|
||||
from move m
|
||||
where u.rid_migration_table_child = m.id_migration_table_src
|
||||
returning *
|
||||
), relationscols as (
|
||||
update meta.migration_relation_col u
|
||||
set rid_migration_column_child = mc2.id_migration_column
|
||||
from move m
|
||||
inner join meta.migration_column mc on mc.rid_migration_table = m.id_migration_table_src
|
||||
inner join meta.migration_column mc2 on mc2.rid_migration_table = m.id_migration_table_dest
|
||||
and mc2.columnname = mc.columnname
|
||||
inner join meta.migration_relation_col rc on rc.rid_migration_column_child = mc.id_migration_column
|
||||
where u.id_migration_relationcol = rc.id_migration_relationcol
|
||||
), relationscols2 as (
|
||||
update meta.migration_relation_col u
|
||||
set rid_migration_column_parent = mc2.id_migration_column
|
||||
from move m
|
||||
inner join meta.migration_column mc on mc.rid_migration_table = m.id_migration_table_src
|
||||
inner join meta.migration_column mc2 on mc2.rid_migration_table = m.id_migration_table_dest
|
||||
and mc2.columnname = mc.columnname
|
||||
inner join meta.migration_relation_col rc on rc.rid_migration_column_parent = mc.id_migration_column
|
||||
where u.id_migration_relationcol = rc.id_migration_relationcol
|
||||
), idx as (
|
||||
update meta.migration_index u
|
||||
set rid_migration_table = m.id_migration_table_dest
|
||||
from move m
|
||||
where u.rid_migration_table = m.id_migration_table_src
|
||||
returning *
|
||||
), idxcols as (
|
||||
update meta.migration_index_col u
|
||||
set rid_migration_column_parent = col.id_migration_column
|
||||
from move m
|
||||
inner join meta.migration_column col on col.rid_migration_table = m.id_migration_table_src
|
||||
inner join meta.migration_column col2 on col2.rid_migration_table = m.id_migration_table_dest
|
||||
and col2.columnname = col.columnname
|
||||
where u.rid_migration_column_parent = col.id_migration_column
|
||||
)
|
||||
update meta.migration_table u
|
||||
set ismodel = false
|
||||
from move mv
|
||||
where u.id_migration_table = mv.id_migration_table_src;
|
||||
|
||||
|
||||
update meta.migration_index u
|
||||
set indexname = format('%s_%s_%s',u.indexname,tbl.schemaname,tbl.tablename)
|
||||
from meta.migration_table tbl
|
||||
where u.rid_migration_table = tbl.id_migration_table
|
||||
and u.indexname not ilike '%' || tbl.schemaname ||'%' || tbl.tablename || '%s'
|
||||
;
|
||||
|
||||
|
||||
|
||||
|
||||
--disable and report duplicates. Take the high priority schema
|
||||
|
||||
with dup as (
|
||||
select
|
||||
t.tablename as tablename_src
|
||||
, t2.tablename as tablename_dest
|
||||
, t.schemaname as schemaname_src
|
||||
, t2.schemaname as schemaname_dest
|
||||
, t.id_migration_table as id_migration_table_dest
|
||||
, t2.id_migration_table as id_migration_table_src
|
||||
from meta.migration_table t
|
||||
inner join meta.migration_table t2 on t2.ismodel
|
||||
and t.tablename = t2.tablename
|
||||
and t.schemaname is distinct from t2.schemaname
|
||||
and t.schemapriority < t2.schemapriority
|
||||
where t.ismodel
|
||||
and exists (
|
||||
select *
|
||||
from meta.migration_option o
|
||||
where o.name ilike 'default_schema:%'
|
||||
and (split_part(o.name, ':', 2) = t.tablename
|
||||
or split_part(o.name, ':', 2) = t2.tablename
|
||||
)
|
||||
)
|
||||
), upd as (
|
||||
update meta.migration_table u
|
||||
set ismodel = false
|
||||
from dup
|
||||
where u.id_migration_table = dup.id_migration_table_src
|
||||
returning *
|
||||
)
|
||||
select jsonb_agg(to_jsonb(dup))
|
||||
from dup
|
||||
into m_json;
|
||||
|
||||
|
||||
|
||||
insert into meta.table_prefix(schemaname,tablename, prefix)
|
||||
select distinct t.schemaname,t.tablename,t.prefix
|
||||
from meta.migration_table t
|
||||
where t.ismodel
|
||||
and coalesce(t.prefix,'') <> ''
|
||||
and not exists (
|
||||
select 1 from meta.table_prefix p where p.schemaname = t.schemaname and p.tablename = t.tablename
|
||||
);
|
||||
|
||||
update meta.table_prefix u
|
||||
set prefix = t.prefix
|
||||
from meta.migration_table t
|
||||
where t.ismodel
|
||||
and u.tablename = t.tablename
|
||||
and u.schemaname = t.schemaname
|
||||
and u.prefix is distinct from t.prefix
|
||||
;
|
||||
|
||||
update meta.migration_relation u
|
||||
set ismodel = false
|
||||
from (
|
||||
select max(mr.rid_migration_table_child) as rid_migration_table_child
|
||||
, max(mr.rid_migration_table_parent) as rid_migration_table_parent
|
||||
, count(1) as cnt
|
||||
, array_agg(mr.id_migration_relation) as a_id_migration_relation
|
||||
, min(mr.id_migration_relation) as id_migration_relation_keep
|
||||
, array_agg(mr.relationname)
|
||||
,string_agg(c1.columnname,'_')
|
||||
from meta.migration_relation mr
|
||||
inner join meta.migration_relation_col rc on rc.rid_migration_relation = mr.id_migration_relation
|
||||
inner join meta.migration_table t1 on t1.id_migration_table = mr.rid_migration_table_parent
|
||||
inner join meta.migration_table t2 on t2.id_migration_table = mr.rid_migration_table_child
|
||||
inner join meta.migration_column c1 on c1.id_migration_column = rc.rid_migration_column_parent
|
||||
inner join meta.migration_column c2 on c2.id_migration_column = rc.rid_migration_column_child
|
||||
where mr.ismodel
|
||||
|
||||
group by t1.schemaname, t1.tablename,t2.schemaname,t2.tablename,c1.columnname,c2.columnname
|
||||
having count(1) > 1
|
||||
) r
|
||||
where u.id_migration_relation = any(a_id_migration_relation)
|
||||
and u.id_migration_relation <> r.id_migration_relation_keep
|
||||
;
|
||||
|
||||
/*
|
||||
update meta.migration_relation u
|
||||
set ismodel = false
|
||||
,isdb = false
|
||||
from (
|
||||
select mr.relationname
|
||||
, mr.rid_migration_table_child
|
||||
, mr.rid_migration_table_parent
|
||||
, count(1) as cnt
|
||||
, array_agg(mr.id_migration_relation) as a_id_migration_relation
|
||||
, min(mr.id_migration_relation) as id_migration_relation
|
||||
from meta.migration_relation mr
|
||||
where mr.ismodel
|
||||
group by mr.relationname, mr.rid_migration_table_child, mr.rid_migration_table_parent
|
||||
) r
|
||||
where u.id_migration_relation = any(r.a_id_migration_relation)
|
||||
and u.id_migration_relation is distinct from r.id_migration_relation
|
||||
;
|
||||
*/
|
||||
|
||||
|
||||
raise notice 'done @ %', (clock_timestamp() - m_tm)::interval;
|
||||
m_tm = clock_timestamp();
|
||||
|
||||
|
||||
|
||||
p_info = coalesce(jsonb_concat(p_info, m_json::jsonb),p_info);
|
||||
perform pg_notify('upgrade.events', json_build_object('type','upgrade','status',2,'objecttype',m_funcname)::text);
|
||||
|
||||
EXCEPTION
|
||||
WHEN others THEN
|
||||
GET STACKED DIAGNOSTICS
|
||||
m_errmsg = MESSAGE_TEXT
|
||||
,m_errcontext = PG_EXCEPTION_CONTEXT
|
||||
,m_errdetail = PG_EXCEPTION_DETAIL
|
||||
,m_errhint = PG_EXCEPTION_HINT
|
||||
,m_errstate = RETURNED_SQLSTATE;
|
||||
|
||||
p_retval = 1;
|
||||
p_errmsg = format('%s Context %s State: %s', m_errmsg, m_errcontext, m_errstate);
|
||||
raise warning '% % hint:% state:% context:%',m_errmsg,m_errdetail,m_errhint,m_errstate,m_errcontext;
|
||||
perform pg_notify('upgrade.events', json_build_object('type','upgrade','status',3,'error',m_errmsg,'objecttype',m_funcname)::text);
|
||||
END;
|
||||
$$;
|
||||
@@ -0,0 +1,100 @@
|
||||
# PgTidy — TODO / Progress
|
||||
|
||||
Tracks what is done and what remains. See `plan.md` for the full design and rationale.
|
||||
|
||||
Legend: ✅ done · 🚧 in progress · ⬜ not started
|
||||
|
||||
---
|
||||
|
||||
## V1 — Formatter + CLI (current milestone)
|
||||
|
||||
### ✅ Scaffold module + repo hygiene
|
||||
- `go.mod` (`module github.com/hein/pgtidy`, go 1.26).
|
||||
- Replaced WkMailSync boilerplate: `AGENTS.md` (PgTidy architecture + invariants),
|
||||
`CLAUDE.md`, `Makefile` (`build`/`test`/`vet`/`fmt`/`lint`/`clean`).
|
||||
- Directory layout created (`cmd/`, `pkg/...`, `testdata/`).
|
||||
- Copied 4 real-world procedures into `testdata/corpus/*.pgsql` (safety harness).
|
||||
|
||||
### ✅ Lossless lexer — `pkg/lexer`
|
||||
- `token.go`: `Kind` enum (trivia, words, literals, operators, punctuation) + `Token`.
|
||||
- `lexer.go`: full PG token coverage — dollar-quoted strings (`$tag$`, nested-tag aware),
|
||||
`--` / nestable `/* */` comments, standard/escape/bit/hex/unicode strings,
|
||||
numbers (decimal, exponent, leading dot, `0x/0o/0b`, `_` separators), positional params
|
||||
(`$1`), operator runs with PostgreSQL's trailing `+/-` rule, `::` / `:=` / `:`, punctuation.
|
||||
Tracks line/col per token.
|
||||
- `lexer_test.go`: unit tests (kinds, operator trailing rule, dollar quotes, line/col) +
|
||||
**corpus round-trip** asserting `emit(Lex(src)) == src` byte-for-byte.
|
||||
- **Status:** all tests pass; round-trips all 4 corpus files. Invariant #1 satisfied.
|
||||
|
||||
### ✅ CST node model + parser — `pkg/cst`, `pkg/parser`
|
||||
- `pkg/cst`: lossless node model — `Tok` (significant token + leading trivia), `Attach`,
|
||||
`File.Source()` (byte-exact reconstruction), `Raw` (verbatim fallback), `CreateFunction`
|
||||
(Head/Name/Params/Options/As/Body/Tail/Semi), `Param` (with separator comma).
|
||||
- `pkg/parser`: statement splitting at top-level `;`; structures CREATE FUNCTION/PROCEDURE
|
||||
(head, qualified name, comma-split params, option clauses split by keyword, AS + body);
|
||||
everything else → `Raw`. Graceful degradation is a property of the data model.
|
||||
- `parser_test.go`: small round-trips, CreateFunction shape assertions, Raw fallback, and
|
||||
**corpus round-trip** — reconstructs all 4 files byte-for-byte; structures all 4 functions.
|
||||
- **Status:** all tests pass.
|
||||
- _Still TODO (later): DML/other-DDL structuring (currently Raw) for full formatting._
|
||||
|
||||
### ⬜ PL/pgSQL body parser ← NEXT (the remaining V1 piece)
|
||||
- Parse `$$ ... $$` bodies: DECLARE/BEGIN/END, IF/ELSIF/ELSE/END IF, LOOP/FOR/WHILE,
|
||||
CASE, assignments (`:=` / `=`), RAISE, nested SQL statements, EXCEPTION blocks.
|
||||
- The crux for stored-procedure formatting (primary use case). Currently the body is
|
||||
emitted verbatim; this milestone formats inside it. Reuse `inline`/spacing/casing from
|
||||
`pkg/format` and keep verbatim fallback for unparsable constructs.
|
||||
|
||||
### ✅ Printer + style config — `pkg/format`, `pkg/config`
|
||||
- `pkg/config`: `Style` struct + `Default()` = house style (UPPERCASE keywords, lowercase
|
||||
types, 2-space indent, leading commas, spacing rules).
|
||||
- `pkg/format`: formats CREATE FUNCTION/PROCEDURE **headers** to house style (params
|
||||
one-per-line leading-comma, option clauses each on own line, AS/`$$` own lines); body and
|
||||
`Raw` statements emitted verbatim. Spacing engine (`needSpace`, tight ops `:: : -> ->>`)
|
||||
+ casing (`keywords`/`typeNames` sets). Comment-safety: verbatim fallback if a header
|
||||
carries comments it cannot relocate.
|
||||
- Tests: golden header, idempotence, corpus idempotence + **semantic equivalence**.
|
||||
- _Note: not a full Wadler Doc-IR yet — fixed-layout printer. Doc-IR for width-based
|
||||
expression wrapping can come when DML structuring lands._
|
||||
|
||||
### ✅ CLI `fmt` + safety harness — `cmd/pgtidy`
|
||||
- `pgtidy fmt` (gofmt model): default stdin→stdout; `-w`/`--write`, `-l`/`--list`,
|
||||
`--check` (CI exit codes); `version`/`help`. _`.pgtidy.yaml` discovery + `-d` diff: TODO._
|
||||
- `fmt_test.go`: stdin, --check (un/formatted), -w idempotence, unknown-command.
|
||||
- Safety invariants #2 (semantic equivalence), #3 (idempotence), #4 (graceful degradation)
|
||||
are tested in `pkg/format` over the corpus.
|
||||
|
||||
---
|
||||
|
||||
## V2 — Linter (later)
|
||||
- `pkg/pgast`: `go-pgquery` (WASM, no cgo) wrapper → real PG AST.
|
||||
- `pkg/lint`: rule engine + packs — style/consistency, **migration safety** (locks, unsafe
|
||||
ALTER/ADD COLUMN, non-CONCURRENTLY index, blocking constraints), naming, correctness.
|
||||
- `pkg/diagnostics`: shared diagnostic type (CLI + LSP).
|
||||
- `pgtidy lint` subcommand; `--fix` for autofixable rules.
|
||||
|
||||
## V3 — LSP + VSCode (later)
|
||||
- `pkg/lsp`: formatting + range formatting, publishDiagnostics, codeAction quick-fixes.
|
||||
- `editors/vscode`: TS extension (`vscode-languageclient`) launching bundled `pgtidy lsp`;
|
||||
per-platform VSIX matrix in CI (rust-analyzer model) + target-less fallback.
|
||||
|
||||
## V4 — DataGrip (later)
|
||||
- `editors/datagrip`: integrate via free **LSP4IJ** plugin.
|
||||
|
||||
---
|
||||
|
||||
## Build / release (cross-cutting)
|
||||
- ⬜ Add goreleaser for the multi-platform binary matrix (clean: no cgo).
|
||||
- `make_release.sh` retained from boilerplate (generic version tagging).
|
||||
|
||||
## Core invariants (must always hold — tested)
|
||||
1. ✅ Lossless lex: `emit(Lex(src)) == src` (corpus round-trip).
|
||||
2. ✅ Semantic equivalence: formatting changes only trivia/layout (corpus token-stream check).
|
||||
3. ✅ Idempotence: `fmt(fmt(x)) == fmt(x)` (corpus + CLI tests).
|
||||
4. ✅ Graceful degradation: unparsable spans pass through verbatim (Raw nodes + verbatim body).
|
||||
|
||||
## Open risks
|
||||
- Lossless PL/pgSQL recursive-descent parser is the largest effort; pass-through fallback
|
||||
bounds risk and allows shipping construct-by-construct.
|
||||
- `go-pgquery` tracks PG17 (not PG18) — fine for lint; irrelevant to formatter path.
|
||||
- Leading-comma + one-per-line is a first-class style option, not an afterthought.
|
||||
Reference in New Issue
Block a user