relspecgo/pkg/inspector/PLAN.md

Inspector Feature Implementation Plan

Overview

Add a model inspection feature that validates database schemas against configurable rules. The inspector will read any supported format, apply validation rules from a YAML config, and output a report in markdown or JSON format.

Architecture

Core Components

1. CLI Command (cmd/relspec/inspect.go)

   • New subcommand: relspec inspect
   • Flags:
     • --from (required): Input format (dbml, pgsql, json, etc.)
     • --from-path: File path for file-based formats
     • --from-conn: Connection string for database formats
     • --rules (optional): Path to rules YAML file (default: .relspec-rules.yaml)
     • --output-format: Report format (markdown, json) (default: markdown)
     • --output: Output file path (default: stdout)
     • --schema: Schema name filter (optional)

2. Inspector Package (pkg/inspector/)

   • inspector.go: Main inspector logic
   • rules.go: Rule definitions and configuration
   • validators.go: Individual validation rule implementations
   • report.go: Report generation (markdown, JSON)
   • config.go: YAML config loading and parsing

Data Flow

Input Format → Reader → Database Model → Inspector → Validation Results → Report Formatter → Output

Rules Configuration Structure

YAML Schema (rules.yaml)

version: "1.0"
rules:
  # Primary Key Rules
  primary_key_naming:
    enabled: enforce|warn|off
    pattern: "^id_"  # regex pattern
    message: "Primary key columns must start with 'id_'"

  primary_key_datatype:
    enabled: enforce|warn|off
    allowed_types: ["bigserial", "bigint", "int", "serial", "integer"]
    message: "Primary keys must use approved integer types"

  primary_key_auto_increment:
    enabled: enforce|warn|off
    require_auto_increment: true|false
    message: "Primary keys without auto-increment detected"

  # Foreign Key Rules
  foreign_key_column_naming:
    enabled: enforce|warn|off
    pattern: "^rid_"
    message: "Foreign key columns must start with 'rid_'"

  foreign_key_constraint_naming:
    enabled: enforce|warn|off
    pattern: "^fk_"
    message: "Foreign key constraint names must start with 'fk_'"

  foreign_key_index:
    enabled: enforce|warn|off
    require_index: true
    message: "Foreign keys should have indexes"

  # Naming Convention Rules
  table_naming_case:
    enabled: enforce|warn|off
    case: "lowercase"  # lowercase, uppercase, snake_case, camelCase
    pattern: "^[a-z][a-z0-9_]*$"
    message: "Table names must be lowercase with underscores"

  column_naming_case:
    enabled: enforce|warn|off
    case: "lowercase"
    pattern: "^[a-z][a-z0-9_]*$"
    message: "Column names must be lowercase with underscores"

  # Length Rules
  table_name_length:
    enabled: enforce|warn|off
    max_length: 64
    message: "Table name exceeds maximum length"

  column_name_length:
    enabled: enforce|warn|off
    max_length: 64
    message: "Column name exceeds maximum length"

  # Reserved Keywords
  reserved_keywords:
    enabled: enforce|warn|off
    check_tables: true
    check_columns: true
    message: "Using reserved SQL keywords"

  # Schema Integrity Rules
  missing_primary_key:
    enabled: enforce|warn|off
    message: "Table missing primary key"

  orphaned_foreign_key:
    enabled: enforce|warn|off
    message: "Foreign key references non-existent table"

  circular_dependency:
    enabled: enforce|warn|off
    message: "Circular foreign key dependency detected"

Rule Levels

• enforce: Violations are errors (exit code 1)
• warn: Violations are warnings (exit code 0)
• off: Rule disabled

Implementation Details

1. Inspector Core (pkg/inspector/inspector.go)

type Inspector struct {
    config *Config
    db     *models.Database
}

type ValidationResult struct {
    RuleName    string
    Level       string // "error" or "warning"
    Message     string
    Location    string // e.g., "schema.table.column"
    Context     map[string]interface{}
    Passed      bool
}

type InspectorReport struct {
    Summary     ReportSummary
    Violations  []ValidationResult
    GeneratedAt time.Time
    Database    string
    SourceFormat string
}

type ReportSummary struct {
    TotalRules      int
    RulesChecked    int
    RulesSkipped    int
    ErrorCount      int
    WarningCount    int
    PassedCount     int
}

func NewInspector(db *models.Database, config *Config) *Inspector
func (i *Inspector) Inspect() (*InspectorReport, error)
func (i *Inspector) validateDatabase() []ValidationResult
func (i *Inspector) validateSchema(schema *models.Schema) []ValidationResult
func (i *Inspector) validateTable(table *models.Table) []ValidationResult

2. Rule Definitions (pkg/inspector/rules.go)

type Config struct {
    Version string
    Rules   map[string]Rule
}

type Rule struct {
    Enabled       string // "enforce", "warn", "off"
    Message       string
    Pattern       string
    AllowedTypes  []string
    MaxLength     int
    Case          string
    RequireIndex  bool
    CheckTables   bool
    CheckColumns  bool
    // ... rule-specific fields
}

type RuleValidator interface {
    Name() string
    Validate(db *models.Database, rule Rule) []ValidationResult
}

func LoadConfig(path string) (*Config, error)
func GetDefaultConfig() *Config

Configuration Loading Behavior:

• If --rules flag is provided but file not found: Use default configuration (don't error)
• If file exists but is invalid YAML: Return error
• Default configuration has sensible rules enabled at "warn" level
• Users can override by creating their own .relspec-rules.yaml file

3. Validators (pkg/inspector/validators.go)

Each validator implements rule logic:

// Primary Key Validators
func validatePrimaryKeyNaming(db *models.Database, rule Rule) []ValidationResult
func validatePrimaryKeyDatatype(db *models.Database, rule Rule) []ValidationResult
func validatePrimaryKeyAutoIncrement(db *models.Database, rule Rule) []ValidationResult

// Foreign Key Validators
func validateForeignKeyColumnNaming(db *models.Database, rule Rule) []ValidationResult
func validateForeignKeyConstraintNaming(db *models.Database, rule Rule) []ValidationResult
func validateForeignKeyIndex(db *models.Database, rule Rule) []ValidationResult

// Naming Convention Validators
func validateTableNamingCase(db *models.Database, rule Rule) []ValidationResult
func validateColumnNamingCase(db *models.Database, rule Rule) []ValidationResult

// Length Validators
func validateTableNameLength(db *models.Database, rule Rule) []ValidationResult
func validateColumnNameLength(db *models.Database, rule Rule) []ValidationResult

// Reserved Keywords Validator
func validateReservedKeywords(db *models.Database, rule Rule) []ValidationResult

// Integrity Validators
func validateMissingPrimaryKey(db *models.Database, rule Rule) []ValidationResult
func validateOrphanedForeignKey(db *models.Database, rule Rule) []ValidationResult
func validateCircularDependency(db *models.Database, rule Rule) []ValidationResult

// Registry of all validators
var validators = map[string]RuleValidator{
    "primary_key_naming": primaryKeyNamingValidator{},
    // ...
}

4. Report Formatting (pkg/inspector/report.go)

type ReportFormatter interface {
    Format(report *InspectorReport) (string, error)
}

type MarkdownFormatter struct {
    UseColors bool  // ANSI colors for terminal output
}
type JSONFormatter struct{}

func (f *MarkdownFormatter) Format(report *InspectorReport) (string, error)
func (f *JSONFormatter) Format(report *InspectorReport) (string, error)

// Helper to detect if output is a TTY (terminal)
func isTerminal(w io.Writer) bool

Output Behavior:

• Markdown format will use ANSI color codes when outputting to a terminal (TTY)
• When piped or redirected to a file, plain markdown without colors
• Colors: Red for errors, Yellow for warnings, Green for passed checks

Markdown Format Example:

# RelSpec Inspector Report

**Database:** my_database
**Source Format:** pgsql
**Generated:** 2025-12-31 10:30:45

## Summary
- Rules Checked: 12
- Errors: 3
- Warnings: 5
- Passed: 4

## Violations

### Errors (3)

#### primary_key_naming
**Table:** users, **Column:** user_id
Primary key columns must start with 'id_'

#### table_name_length
**Table:** user_authentication_sessions_with_metadata
Table name exceeds maximum length (64 characters)

### Warnings (5)

#### foreign_key_index
**Table:** orders, **Column:** customer_id
Foreign keys should have indexes

...

JSON Format Example:

{
  "summary": {
    "total_rules": 12,
    "rules_checked": 12,
    "error_count": 3,
    "warning_count": 5,
    "passed_count": 4
  },
  "violations": [
    {
      "rule_name": "primary_key_naming",
      "level": "error",
      "message": "Primary key columns must start with 'id_'",
      "location": "public.users.user_id",
      "context": {
        "schema": "public",
        "table": "users",
        "column": "user_id",
        "current_name": "user_id",
        "expected_pattern": "^id_"
      },
      "passed": false
    }
  ],
  "generated_at": "2025-12-31T10:30:45Z",
  "database": "my_database",
  "source_format": "pgsql"
}

5. CLI Command (cmd/relspec/inspect.go)

var inspectCmd = &cobra.Command{
    Use:   "inspect",
    Short: "Inspect and validate database schemas against rules",
    Long:  `Read database schemas from various formats and validate against configurable rules.`,
    RunE:  runInspect,
}

func init() {
    inspectCmd.Flags().String("from", "", "Input format (dbml, pgsql, json, etc.)")
    inspectCmd.Flags().String("from-path", "", "Input file path")
    inspectCmd.Flags().String("from-conn", "", "Database connection string")
    inspectCmd.Flags().String("rules", ".relspec-rules.yaml", "Rules configuration file")
    inspectCmd.Flags().String("output-format", "markdown", "Output format (markdown, json)")
    inspectCmd.Flags().String("output", "", "Output file (default: stdout)")
    inspectCmd.Flags().String("schema", "", "Filter by schema name")
    inspectCmd.MarkFlagRequired("from")
}

func runInspect(cmd *cobra.Command, args []string) error {
    // 1. Parse flags
    // 2. Create reader (reuse pattern from convert.go)
    // 3. Read database
    // 4. Load rules config (use defaults if file not found)
    // 5. Create inspector
    // 6. Run inspection
    // 7. Detect if output is terminal (for color support)
    // 8. Format report (with/without ANSI colors)
    // 9. Write output
    // 10. Exit with appropriate code (0 if no errors, 1 if errors)
}

Implementation Phases

Phase 1: Core Infrastructure

1. Create pkg/inspector/ package structure
2. Implement Config and YAML loading
3. Implement Inspector core with basic validation framework
4. Create CLI command skeleton

Phase 2: Basic Validators

1. Implement naming convention validators
   • Primary key naming
   • Foreign key column naming
   • Foreign key constraint naming
   • Table/column case validation
2. Implement length validators
3. Implement reserved keywords validator (leverage pkg/pgsql/keywords.go)

Phase 3: Advanced Validators

1. Implement datatype validators
2. Implement integrity validators (missing PK, orphaned FK, circular deps)
3. Implement foreign key index validator

Phase 4: Reporting

1. Implement InspectorReport structure
2. Implement markdown formatter
3. Implement JSON formatter
4. Add summary statistics

Phase 5: CLI Integration

1. Wire up CLI command with flags
2. Integrate reader factory (from convert.go pattern)
3. Add output file handling
4. Add exit code logic
5. Add progress reporting

Phase 6: Testing & Documentation

1. Unit tests for validators
2. Integration tests with sample schemas
3. Test with all reader formats
4. Update README with inspector documentation
5. Create example rules configuration file

Files to Create

1. pkg/inspector/inspector.go - Core inspector logic
2. pkg/inspector/rules.go - Rule definitions and config loading
3. pkg/inspector/validators.go - Validation implementations
4. pkg/inspector/report.go - Report formatting
5. pkg/inspector/config.go - Config utilities
6. cmd/relspec/inspect.go - CLI command
7. .relspec-rules.yaml.example - Example configuration
8. pkg/inspector/inspector_test.go - Tests

Files to Modify

1. cmd/relspec/root.go - Register inspect command
2. README.md - Add inspector documentation (if requested)

Example Usage

# Inspect a PostgreSQL database with default rules
relspec inspect --from pgsql --from-conn "postgresql://localhost/mydb"

# Inspect a DBML file with custom rules
relspec inspect --from dbml --from-path schema.dbml --rules my-rules.yaml

# Output JSON report to file
relspec inspect --from json --from-path db.json --output-format json --output report.json

# Inspect specific schema only
relspec inspect --from pgsql --from-conn "..." --schema public

# Use custom rules location
relspec inspect --from dbml --from-path schema.dbml --rules /path/to/rules.yaml

Exit Codes

• 0: Success (no errors, only warnings or all passed)
• 1: Validation errors found (rules with level="enforce" failed)
• 2: Runtime error (invalid config, reader error, etc.)

Dependencies

• Existing: pkg/models, pkg/readers, pkg/pgsql/keywords.go
• New: gopkg.in/yaml.v3 for YAML parsing (may already be in go.mod)

Design Decisions

Confirmed Choices (from user)

1. Example config file: Create .relspec-rules.yaml.example in repository root with documented examples
2. Missing rules file: Use sensible built-in defaults (don't error), all rules at "warn" level by default
3. Terminal output: ANSI colors (red/yellow/green) when outputting to terminal, plain markdown when piped/redirected
4. Foreign key naming: Separate configurable rules for both FK column names and FK constraint names

Architecture Rationale

1. Why YAML for config?: Human-readable, supports comments, standard for config files
2. Why three levels (enforce/warn/off)?: Flexibility for gradual adoption, different contexts
3. Why markdown + JSON?: Markdown for human review, JSON for tooling integration
4. Why pkg/inspector?: Follows existing package structure, separates concerns
5. Reuse readers: Leverage existing reader infrastructure, supports all formats automatically
6. Exit codes: Follow standard conventions (0=success, 1=validation fail, 2=error)

Future Enhancements (Not in Scope)

• Auto-fix mode (automatically rename columns, etc.)
• Custom rule plugins
• HTML report format
• Rule templates for different databases
• CI/CD integration examples
• Performance metrics in report