PgTidy/todo.md

PgTidy — TODO / Progress

Tracks what is done and what remains. See plan.md for the full design and rationale.

Legend: ✅ done · 🚧 in progress · ⬜ not started

---

V1 — Formatter + CLI (current milestone)

✅ Scaffold module + repo hygiene

• go.mod (module github.com/hein/pgtidy, go 1.26).
• Replaced WkMailSync boilerplate: AGENTS.md (PgTidy architecture + invariants), CLAUDE.md, Makefile (build/test/vet/fmt/lint/clean).
• Directory layout created (cmd/, pkg/..., testdata/).
• Copied 4 real-world procedures into testdata/corpus/*.pgsql (safety harness).

✅ Lossless lexer — pkg/lexer

• token.go: Kind enum (trivia, words, literals, operators, punctuation) + Token.
• lexer.go: full PG token coverage — dollar-quoted strings ($tag$, nested-tag aware), -- / nestable /* */ comments, standard/escape/bit/hex/unicode strings, numbers (decimal, exponent, leading dot, 0x/0o/0b, _ separators), positional params ($1), operator runs with PostgreSQL's trailing +/- rule, :: / := / :, punctuation. Tracks line/col per token.
• lexer_test.go: unit tests (kinds, operator trailing rule, dollar quotes, line/col) + corpus round-trip asserting emit(Lex(src)) == src byte-for-byte.
• Status: all tests pass; round-trips all 4 corpus files. Invariant #1 satisfied.

✅ CST node model + parser — pkg/cst, pkg/parser

• pkg/cst: lossless node model — Tok (significant token + leading trivia), Attach, File.Source() (byte-exact reconstruction), Raw (verbatim fallback), CreateFunction (Head/Name/Params/Options/As/Body/Tail/Semi), Param (with separator comma).
• pkg/parser: statement splitting at top-level ;; structures CREATE FUNCTION/PROCEDURE (head, qualified name, comma-split params, option clauses split by keyword, AS + body); everything else → Raw. Graceful degradation is a property of the data model.
• parser_test.go: small round-trips, CreateFunction shape assertions, Raw fallback, and corpus round-trip — reconstructs all 4 files byte-for-byte; structures all 4 functions.
• Status: all tests pass.
• Still TODO (later): DML/other-DDL structuring (currently Raw) for full formatting.

🚧 PL/pgSQL body parser ← NEXT (the remaining V1 piece)

✅ DECLARE section — pkg/format/body.go

• formatBody splits the dollar-quote tag, calls formatBodyInner.
• formatBodyInner locates DECLARE and BEGIN at depth 0, formats the declare block, then emits BEGIN onwards verbatim.
• formatDeclareVars: each variable declaration collapsed to one line ( name type [= expr];), --Block-- comment markers preserved on their own lines, mid-declaration block comments trigger verbatim fallback.
• needSpace fixed for LBracket — no space before [ after ident/closing bracket (fixes citext[], array subscripts).
• semanticallyEqual in tests updated to recurse into dollar-quoted body tokens so whitespace normalization inside the body does not falsely fail the semantic check.
• Added testdata/corpus/test_a_broken.pgsql — a CRLF corpus file with intentionally broken layout (split-line variables + split-line body statements) used as a formatting target; testdata/corpus/test_a.pgsql is the golden output.
• Status: all tests pass; DECLARE section formats correctly.

⬜ Body statement formatter — pkg/format/body.go (next step)

Two-phase formatter for the BEGIN … END block:

1. Block-depth tracker — recognise BEGIN/END, IF/THEN/ELSIF/ELSE/END IF, LOOP/END LOOP, FOR/END LOOP, CASE/END CASE, EXCEPTION/WHEN to maintain current indent depth (depth × 2 spaces).

2. Split-line join — within each statement (tokens up to ; at depth 0), a physical line whose first non-whitespace token is at column 0 is a broken continuation; join it to the preceding line with a single space. Exceptions: SQL clause keywords (FROM, WHERE, INTO, HAVING, GROUP, ORDER, RETURNING) stay on their own line.

3. Base-indent normalisation — after joining, each logical line is re-emitted at depth × 2 spaces, stripping its original leading whitespace and replacing it with the computed indent. Internal relative indentation of multi-line expressions is preserved.

4. Blank-line preservation — blank lines between statements in the original are kept (they carry author intent about logical grouping).

5. Verbatim fallback — any construct the formatter cannot classify cleanly passes through verbatim, upholding invariant #4.

Acceptance: pgtidy fmt testdata/corpus/test_a_broken.pgsql produces output matching testdata/corpus/test_a.pgsql (new golden-file test).

✅ Printer + style config — pkg/format, pkg/config

• pkg/config: Style struct + Default() = house style (UPPERCASE keywords, lowercase types, 2-space indent, leading commas, spacing rules).
• pkg/format: formats CREATE FUNCTION/PROCEDURE headers to house style (params one-per-line leading-comma, option clauses each on own line, AS/$$ own lines); DECLARE section formatted (see body parser entry); Raw statements emitted verbatim. Spacing engine (needSpace, tight ops :: : -> ->>, array []) + casing (keywords/typeNames sets). Comment-safety: verbatim fallback if a header carries comments it cannot relocate.
• pkg/format/body.go: DECLARE section formatter (see body parser entry).
• Tests: golden header, idempotence, corpus idempotence + semantic equivalence (updated to recurse into dollar-quoted body tokens).
• Note: not a full Wadler Doc-IR yet — fixed-layout printer. Doc-IR for width-based expression wrapping can come when DML structuring lands.

✅ CLI fmt + safety harness — cmd/pgtidy

• pgtidy fmt (gofmt model): default stdin→stdout; -w/--write, -l/--list, --check (CI exit codes); version/help. .pgtidy.yaml discovery + -d diff: TODO.
• fmt_test.go: stdin, --check (un/formatted), -w idempotence, unknown-command.
• Safety invariants #2 (semantic equivalence), #3 (idempotence), #4 (graceful degradation) are tested in pkg/format over the corpus.

---

V2 — Linter (later)

• pkg/pgast: go-pgquery (WASM, no cgo) wrapper → real PG AST.
• pkg/lint: rule engine + packs — style/consistency, migration safety (locks, unsafe ALTER/ADD COLUMN, non-CONCURRENTLY index, blocking constraints), naming, correctness.
• pkg/diagnostics: shared diagnostic type (CLI + LSP).
• pgtidy lint subcommand; --fix for autofixable rules.

V3 — LSP + VSCode (later)

• pkg/lsp: formatting + range formatting, publishDiagnostics, codeAction quick-fixes.
• editors/vscode: TS extension (vscode-languageclient) launching bundled pgtidy lsp; per-platform VSIX matrix in CI (rust-analyzer model) + target-less fallback.

V4 — DataGrip (later)

• editors/datagrip: integrate via free LSP4IJ plugin.

---

Build / release (cross-cutting)

• ⬜ Add goreleaser for the multi-platform binary matrix (clean: no cgo).
• make_release.sh retained from boilerplate (generic version tagging).

Core invariants (must always hold — tested)

1. ✅ Lossless lex: emit(Lex(src)) == src (corpus round-trip).
2. ✅ Semantic equivalence: formatting changes only trivia/layout (corpus token-stream check).
3. ✅ Idempotence: fmt(fmt(x)) == fmt(x) (corpus + CLI tests).
4. ✅ Graceful degradation: unparsable spans pass through verbatim (Raw nodes + verbatim body).

Open risks

• Lossless PL/pgSQL recursive-descent parser is the largest effort; pass-through fallback bounds risk and allows shipping construct-by-construct.
• go-pgquery tracks PG17 (not PG18) — fine for lint; irrelevant to formatter path.
• Leading-comma + one-per-line is a first-class style option, not an afterthought.