PgTidy/todo.md

# PgTidy — TODO / Progress

Tracks what is done and what remains. See `plan.md` for the full design and rationale.

Legend: ✅ done · 🚧 in progress · ⬜ not started

---

## V1 — Formatter + CLI (current milestone)

### ✅ Scaffold module + repo hygiene
- `go.mod` (`module github.com/hein/pgtidy`, go 1.26).
- Replaced WkMailSync boilerplate: `AGENTS.md` (PgTidy architecture + invariants),
  `CLAUDE.md`, `Makefile` (`build`/`test`/`vet`/`fmt`/`lint`/`clean`).
- Directory layout created (`cmd/`, `pkg/...`, `testdata/`).
- Copied 4 real-world procedures into `testdata/corpus/*.pgsql` (safety harness).

### ✅ Lossless lexer — `pkg/lexer`
- `token.go`: `Kind` enum (trivia, words, literals, operators, punctuation) + `Token`.
- `lexer.go`: full PG token coverage — dollar-quoted strings (`$tag$`, nested-tag aware),
  `--` / nestable `/* */` comments, standard/escape/bit/hex/unicode strings,
  numbers (decimal, exponent, leading dot, `0x/0o/0b`, `_` separators), positional params
  (`$1`), operator runs with PostgreSQL's trailing `+/-` rule, `::` / `:=` / `:`, punctuation.
  Tracks line/col per token.
- `lexer_test.go`: unit tests (kinds, operator trailing rule, dollar quotes, line/col) +
  **corpus round-trip** asserting `emit(Lex(src)) == src` byte-for-byte.
- **Status:** all tests pass; round-trips all 4 corpus files. Invariant #1 satisfied.

### ✅ CST node model + parser — `pkg/cst`, `pkg/parser`
- `pkg/cst`: lossless node model — `Tok` (significant token + leading trivia), `Attach`,
  `File.Source()` (byte-exact reconstruction), `Raw` (verbatim fallback), `CreateFunction`
  (Head/Name/Params/Options/As/Body/Tail/Semi), `Param` (with separator comma).
- `pkg/parser`: statement splitting at top-level `;`; structures CREATE FUNCTION/PROCEDURE
  (head, qualified name, comma-split params, option clauses split by keyword, AS + body);
  everything else → `Raw`. Graceful degradation is a property of the data model.
- `parser_test.go`: small round-trips, CreateFunction shape assertions, Raw fallback, and
  **corpus round-trip** — reconstructs all 4 files byte-for-byte; structures all 4 functions.
- **Status:** all tests pass.
- _Still TODO (later): DML/other-DDL structuring (currently Raw) for full formatting._

### 🚧 PL/pgSQL body parser  ← NEXT (the remaining V1 piece)

#### ✅ DECLARE section — `pkg/format/body.go`
- `formatBody` splits the dollar-quote tag, calls `formatBodyInner`.
- `formatBodyInner` locates `DECLARE` and `BEGIN` at depth 0, formats the declare block,
  then emits `BEGIN` onwards verbatim.
- `formatDeclareVars`: each variable declaration collapsed to one line
  (`  name type [= expr];`), `--Block--` comment markers preserved on their own lines,
  mid-declaration block comments trigger verbatim fallback.
- `needSpace` fixed for `LBracket` — no space before `[` after ident/closing bracket
  (fixes `citext[]`, array subscripts).
- `semanticallyEqual` in tests updated to recurse into dollar-quoted body tokens so
  whitespace normalization inside the body does not falsely fail the semantic check.
- Added `testdata/corpus/test_a_broken.pgsql` — a CRLF corpus file with intentionally
  broken layout (split-line variables + split-line body statements) used as a formatting
  target; `testdata/corpus/test_a.pgsql` is the golden output.
- **Status:** all tests pass; DECLARE section formats correctly.

#### ⬜ Body statement formatter — `pkg/format/body.go` (next step)
Two-phase formatter for the `BEGIN … END` block:

1. **Block-depth tracker** — recognise `BEGIN`/`END`, `IF`/`THEN`/`ELSIF`/`ELSE`/`END IF`,
   `LOOP`/`END LOOP`, `FOR`/`END LOOP`, `CASE`/`END CASE`, `EXCEPTION`/`WHEN` to maintain
   current indent depth (`depth × 2` spaces).

2. **Split-line join** — within each statement (tokens up to `;` at depth 0), a physical
   line whose first non-whitespace token is at column 0 is a broken continuation; join it to
   the preceding line with a single space. Exceptions: SQL clause keywords (`FROM`, `WHERE`,
   `INTO`, `HAVING`, `GROUP`, `ORDER`, `RETURNING`) stay on their own line.

3. **Base-indent normalisation** — after joining, each logical line is re-emitted at
   `depth × 2` spaces, stripping its original leading whitespace and replacing it with the
   computed indent. Internal relative indentation of multi-line expressions is preserved.

4. **Blank-line preservation** — blank lines between statements in the original are kept
   (they carry author intent about logical grouping).

5. **Verbatim fallback** — any construct the formatter cannot classify cleanly passes
   through verbatim, upholding invariant #4.

Acceptance: `pgtidy fmt testdata/corpus/test_a_broken.pgsql` produces output matching
`testdata/corpus/test_a.pgsql` (new golden-file test).

### ✅ Printer + style config — `pkg/format`, `pkg/config`
- `pkg/config`: `Style` struct + `Default()` = house style (UPPERCASE keywords, lowercase
  types, 2-space indent, leading commas, spacing rules).
- `pkg/format`: formats CREATE FUNCTION/PROCEDURE **headers** to house style (params
  one-per-line leading-comma, option clauses each on own line, AS/`$$` own lines); DECLARE
  section formatted (see body parser entry); `Raw` statements emitted verbatim.
  Spacing engine (`needSpace`, tight ops `:: : -> ->>`, array `[]`) + casing
  (`keywords`/`typeNames` sets). Comment-safety: verbatim fallback if a header carries
  comments it cannot relocate.
- `pkg/format/body.go`: DECLARE section formatter (see body parser entry).
- Tests: golden header, idempotence, corpus idempotence + **semantic equivalence**
  (updated to recurse into dollar-quoted body tokens).
- _Note: not a full Wadler Doc-IR yet — fixed-layout printer. Doc-IR for width-based
  expression wrapping can come when DML structuring lands._

### ✅ CLI `fmt` + safety harness — `cmd/pgtidy`
- `pgtidy fmt` (gofmt model): default stdin→stdout; `-w`/`--write`, `-l`/`--list`,
  `--check` (CI exit codes); `version`/`help`. _`.pgtidy.yaml` discovery + `-d` diff: TODO._
- `fmt_test.go`: stdin, --check (un/formatted), -w idempotence, unknown-command.
- Safety invariants #2 (semantic equivalence), #3 (idempotence), #4 (graceful degradation)
  are tested in `pkg/format` over the corpus.

---

## V2 — Linter (later)
- `pkg/pgast`: `go-pgquery` (WASM, no cgo) wrapper → real PG AST.
- `pkg/lint`: rule engine + packs — style/consistency, **migration safety** (locks, unsafe
  ALTER/ADD COLUMN, non-CONCURRENTLY index, blocking constraints), naming, correctness.
- `pkg/diagnostics`: shared diagnostic type (CLI + LSP).
- `pgtidy lint` subcommand; `--fix` for autofixable rules.

## V3 — LSP + VSCode (later)
- `pkg/lsp`: formatting + range formatting, publishDiagnostics, codeAction quick-fixes.
- `editors/vscode`: TS extension (`vscode-languageclient`) launching bundled `pgtidy lsp`;
  per-platform VSIX matrix in CI (rust-analyzer model) + target-less fallback.

## V4 — DataGrip (later)
- `editors/datagrip`: integrate via free **LSP4IJ** plugin.

---

## Build / release (cross-cutting)
- ⬜ Add goreleaser for the multi-platform binary matrix (clean: no cgo).
- `make_release.sh` retained from boilerplate (generic version tagging).

## Core invariants (must always hold — tested)
1. ✅ Lossless lex: `emit(Lex(src)) == src` (corpus round-trip).
2. ✅ Semantic equivalence: formatting changes only trivia/layout (corpus token-stream check).
3. ✅ Idempotence: `fmt(fmt(x)) == fmt(x)` (corpus + CLI tests).
4. ✅ Graceful degradation: unparsable spans pass through verbatim (Raw nodes + verbatim body).

## Open risks
- Lossless PL/pgSQL recursive-descent parser is the largest effort; pass-through fallback
  bounds risk and allows shipping construct-by-construct.
- `go-pgquery` tracks PG17 (not PG18) — fine for lint; irrelevant to formatter path.
- Leading-comma + one-per-line is a first-class style option, not an afterthought.