relspecgo/pkg/inspector/PLAN.md

# Inspector Feature Implementation Plan

## Overview
Add a model inspection feature that validates database schemas against configurable rules. The inspector will read any supported format, apply validation rules from a YAML config, and output a report in markdown or JSON format.

## Architecture

### Core Components

1. **CLI Command** (`cmd/relspec/inspect.go`)
   - New subcommand: `relspec inspect`
   - Flags:
     - `--from` (required): Input format (dbml, pgsql, json, etc.)
     - `--from-path`: File path for file-based formats
     - `--from-conn`: Connection string for database formats
     - `--rules` (optional): Path to rules YAML file (default: `.relspec-rules.yaml`)
     - `--output-format`: Report format (markdown, json) (default: markdown)
     - `--output`: Output file path (default: stdout)
     - `--schema`: Schema name filter (optional)

2. **Inspector Package** (`pkg/inspector/`)
   - `inspector.go`: Main inspector logic
   - `rules.go`: Rule definitions and configuration
   - `validators.go`: Individual validation rule implementations
   - `report.go`: Report generation (markdown, JSON)
   - `config.go`: YAML config loading and parsing

### Data Flow
```
Input Format → Reader → Database Model → Inspector → Validation Results → Report Formatter → Output
```

## Rules Configuration Structure

### YAML Schema (`rules.yaml`)
```yaml
version: "1.0"
rules:
  # Primary Key Rules
  primary_key_naming:
    enabled: enforce|warn|off
    pattern: "^id_"  # regex pattern
    message: "Primary key columns must start with 'id_'"

  primary_key_datatype:
    enabled: enforce|warn|off
    allowed_types: ["bigserial", "bigint", "int", "serial", "integer"]
    message: "Primary keys must use approved integer types"

  primary_key_auto_increment:
    enabled: enforce|warn|off
    require_auto_increment: true|false
    message: "Primary keys without auto-increment detected"

  # Foreign Key Rules
  foreign_key_column_naming:
    enabled: enforce|warn|off
    pattern: "^rid_"
    message: "Foreign key columns must start with 'rid_'"

  foreign_key_constraint_naming:
    enabled: enforce|warn|off
    pattern: "^fk_"
    message: "Foreign key constraint names must start with 'fk_'"

  foreign_key_index:
    enabled: enforce|warn|off
    require_index: true
    message: "Foreign keys should have indexes"

  # Naming Convention Rules
  table_naming_case:
    enabled: enforce|warn|off
    case: "lowercase"  # lowercase, uppercase, snake_case, camelCase
    pattern: "^[a-z][a-z0-9_]*$"
    message: "Table names must be lowercase with underscores"

  column_naming_case:
    enabled: enforce|warn|off
    case: "lowercase"
    pattern: "^[a-z][a-z0-9_]*$"
    message: "Column names must be lowercase with underscores"

  # Length Rules
  table_name_length:
    enabled: enforce|warn|off
    max_length: 64
    message: "Table name exceeds maximum length"

  column_name_length:
    enabled: enforce|warn|off
    max_length: 64
    message: "Column name exceeds maximum length"

  # Reserved Keywords
  reserved_keywords:
    enabled: enforce|warn|off
    check_tables: true
    check_columns: true
    message: "Using reserved SQL keywords"

  # Schema Integrity Rules
  missing_primary_key:
    enabled: enforce|warn|off
    message: "Table missing primary key"

  orphaned_foreign_key:
    enabled: enforce|warn|off
    message: "Foreign key references non-existent table"

  circular_dependency:
    enabled: enforce|warn|off
    message: "Circular foreign key dependency detected"
```

### Rule Levels
- **enforce**: Violations are errors (exit code 1)
- **warn**: Violations are warnings (exit code 0)
- **off**: Rule disabled

## Implementation Details

### 1. Inspector Core (`pkg/inspector/inspector.go`)

```go
type Inspector struct {
    config *Config
    db     *models.Database
}

type ValidationResult struct {
    RuleName    string
    Level       string // "error" or "warning"
    Message     string
    Location    string // e.g., "schema.table.column"
    Context     map[string]interface{}
    Passed      bool
}

type InspectorReport struct {
    Summary     ReportSummary
    Violations  []ValidationResult
    GeneratedAt time.Time
    Database    string
    SourceFormat string
}

type ReportSummary struct {
    TotalRules      int
    RulesChecked    int
    RulesSkipped    int
    ErrorCount      int
    WarningCount    int
    PassedCount     int
}

func NewInspector(db *models.Database, config *Config) *Inspector
func (i *Inspector) Inspect() (*InspectorReport, error)
func (i *Inspector) validateDatabase() []ValidationResult
func (i *Inspector) validateSchema(schema *models.Schema) []ValidationResult
func (i *Inspector) validateTable(table *models.Table) []ValidationResult
```

### 2. Rule Definitions (`pkg/inspector/rules.go`)

```go
type Config struct {
    Version string
    Rules   map[string]Rule
}

type Rule struct {
    Enabled       string // "enforce", "warn", "off"
    Message       string
    Pattern       string
    AllowedTypes  []string
    MaxLength     int
    Case          string
    RequireIndex  bool
    CheckTables   bool
    CheckColumns  bool
    // ... rule-specific fields
}

type RuleValidator interface {
    Name() string
    Validate(db *models.Database, rule Rule) []ValidationResult
}

func LoadConfig(path string) (*Config, error)
func GetDefaultConfig() *Config
```

**Configuration Loading Behavior:**
- If `--rules` flag is provided but file not found: Use default configuration (don't error)
- If file exists but is invalid YAML: Return error
- Default configuration has sensible rules enabled at "warn" level
- Users can override by creating their own `.relspec-rules.yaml` file

### 3. Validators (`pkg/inspector/validators.go`)

Each validator implements rule logic:

```go
// Primary Key Validators
func validatePrimaryKeyNaming(db *models.Database, rule Rule) []ValidationResult
func validatePrimaryKeyDatatype(db *models.Database, rule Rule) []ValidationResult
func validatePrimaryKeyAutoIncrement(db *models.Database, rule Rule) []ValidationResult

// Foreign Key Validators
func validateForeignKeyColumnNaming(db *models.Database, rule Rule) []ValidationResult
func validateForeignKeyConstraintNaming(db *models.Database, rule Rule) []ValidationResult
func validateForeignKeyIndex(db *models.Database, rule Rule) []ValidationResult

// Naming Convention Validators
func validateTableNamingCase(db *models.Database, rule Rule) []ValidationResult
func validateColumnNamingCase(db *models.Database, rule Rule) []ValidationResult

// Length Validators
func validateTableNameLength(db *models.Database, rule Rule) []ValidationResult
func validateColumnNameLength(db *models.Database, rule Rule) []ValidationResult

// Reserved Keywords Validator
func validateReservedKeywords(db *models.Database, rule Rule) []ValidationResult

// Integrity Validators
func validateMissingPrimaryKey(db *models.Database, rule Rule) []ValidationResult
func validateOrphanedForeignKey(db *models.Database, rule Rule) []ValidationResult
func validateCircularDependency(db *models.Database, rule Rule) []ValidationResult

// Registry of all validators
var validators = map[string]RuleValidator{
    "primary_key_naming": primaryKeyNamingValidator{},
    // ...
}
```

### 4. Report Formatting (`pkg/inspector/report.go`)

```go
type ReportFormatter interface {
    Format(report *InspectorReport) (string, error)
}

type MarkdownFormatter struct {
    UseColors bool  // ANSI colors for terminal output
}
type JSONFormatter struct{}

func (f *MarkdownFormatter) Format(report *InspectorReport) (string, error)
func (f *JSONFormatter) Format(report *InspectorReport) (string, error)

// Helper to detect if output is a TTY (terminal)
func isTerminal(w io.Writer) bool
```

**Output Behavior:**
- Markdown format will use ANSI color codes when outputting to a terminal (TTY)
- When piped or redirected to a file, plain markdown without colors
- Colors: Red for errors, Yellow for warnings, Green for passed checks

**Markdown Format Example:**
```markdown
# RelSpec Inspector Report

**Database:** my_database
**Source Format:** pgsql
**Generated:** 2025-12-31 10:30:45

## Summary
- Rules Checked: 12
- Errors: 3
- Warnings: 5
- Passed: 4

## Violations

### Errors (3)

#### primary_key_naming
**Table:** users, **Column:** user_id
Primary key columns must start with 'id_'

#### table_name_length
**Table:** user_authentication_sessions_with_metadata
Table name exceeds maximum length (64 characters)

### Warnings (5)

#### foreign_key_index
**Table:** orders, **Column:** customer_id
Foreign keys should have indexes

...
```

**JSON Format Example:**
```json
{
  "summary": {
    "total_rules": 12,
    "rules_checked": 12,
    "error_count": 3,
    "warning_count": 5,
    "passed_count": 4
  },
  "violations": [
    {
      "rule_name": "primary_key_naming",
      "level": "error",
      "message": "Primary key columns must start with 'id_'",
      "location": "public.users.user_id",
      "context": {
        "schema": "public",
        "table": "users",
        "column": "user_id",
        "current_name": "user_id",
        "expected_pattern": "^id_"
      },
      "passed": false
    }
  ],
  "generated_at": "2025-12-31T10:30:45Z",
  "database": "my_database",
  "source_format": "pgsql"
}
```

### 5. CLI Command (`cmd/relspec/inspect.go`)

```go
var inspectCmd = &cobra.Command{
    Use:   "inspect",
    Short: "Inspect and validate database schemas against rules",
    Long:  `Read database schemas from various formats and validate against configurable rules.`,
    RunE:  runInspect,
}

func init() {
    inspectCmd.Flags().String("from", "", "Input format (dbml, pgsql, json, etc.)")
    inspectCmd.Flags().String("from-path", "", "Input file path")
    inspectCmd.Flags().String("from-conn", "", "Database connection string")
    inspectCmd.Flags().String("rules", ".relspec-rules.yaml", "Rules configuration file")
    inspectCmd.Flags().String("output-format", "markdown", "Output format (markdown, json)")
    inspectCmd.Flags().String("output", "", "Output file (default: stdout)")
    inspectCmd.Flags().String("schema", "", "Filter by schema name")
    inspectCmd.MarkFlagRequired("from")
}

func runInspect(cmd *cobra.Command, args []string) error {
    // 1. Parse flags
    // 2. Create reader (reuse pattern from convert.go)
    // 3. Read database
    // 4. Load rules config (use defaults if file not found)
    // 5. Create inspector
    // 6. Run inspection
    // 7. Detect if output is terminal (for color support)
    // 8. Format report (with/without ANSI colors)
    // 9. Write output
    // 10. Exit with appropriate code (0 if no errors, 1 if errors)
}
```

## Implementation Phases

### Phase 1: Core Infrastructure
1. Create `pkg/inspector/` package structure
2. Implement `Config` and YAML loading
3. Implement `Inspector` core with basic validation framework
4. Create CLI command skeleton

### Phase 2: Basic Validators
1. Implement naming convention validators
   - Primary key naming
   - Foreign key column naming
   - Foreign key constraint naming
   - Table/column case validation
2. Implement length validators
3. Implement reserved keywords validator (leverage `pkg/pgsql/keywords.go`)

### Phase 3: Advanced Validators
1. Implement datatype validators
2. Implement integrity validators (missing PK, orphaned FK, circular deps)
3. Implement foreign key index validator

### Phase 4: Reporting
1. Implement `InspectorReport` structure
2. Implement markdown formatter
3. Implement JSON formatter
4. Add summary statistics

### Phase 5: CLI Integration
1. Wire up CLI command with flags
2. Integrate reader factory (from convert.go pattern)
3. Add output file handling
4. Add exit code logic
5. Add progress reporting

### Phase 6: Testing & Documentation
1. Unit tests for validators
2. Integration tests with sample schemas
3. Test with all reader formats
4. Update README with inspector documentation
5. Create example rules configuration file

## Files to Create

1. `pkg/inspector/inspector.go` - Core inspector logic
2. `pkg/inspector/rules.go` - Rule definitions and config loading
3. `pkg/inspector/validators.go` - Validation implementations
4. `pkg/inspector/report.go` - Report formatting
5. `pkg/inspector/config.go` - Config utilities
6. `cmd/relspec/inspect.go` - CLI command
7. `.relspec-rules.yaml.example` - Example configuration
8. `pkg/inspector/inspector_test.go` - Tests

## Files to Modify

1. `cmd/relspec/root.go` - Register inspect command
2. `README.md` - Add inspector documentation (if requested)

## Example Usage

```bash
# Inspect a PostgreSQL database with default rules
relspec inspect --from pgsql --from-conn "postgresql://localhost/mydb"

# Inspect a DBML file with custom rules
relspec inspect --from dbml --from-path schema.dbml --rules my-rules.yaml

# Output JSON report to file
relspec inspect --from json --from-path db.json --output-format json --output report.json

# Inspect specific schema only
relspec inspect --from pgsql --from-conn "..." --schema public

# Use custom rules location
relspec inspect --from dbml --from-path schema.dbml --rules /path/to/rules.yaml
```

## Exit Codes
- 0: Success (no errors, only warnings or all passed)
- 1: Validation errors found (rules with level="enforce" failed)
- 2: Runtime error (invalid config, reader error, etc.)

## Dependencies
- Existing: `pkg/models`, `pkg/readers`, `pkg/pgsql/keywords.go`
- New: `gopkg.in/yaml.v3` for YAML parsing (may already be in go.mod)

## Design Decisions

### Confirmed Choices (from user)
1. **Example config file**: Create `.relspec-rules.yaml.example` in repository root with documented examples
2. **Missing rules file**: Use sensible built-in defaults (don't error), all rules at "warn" level by default
3. **Terminal output**: ANSI colors (red/yellow/green) when outputting to terminal, plain markdown when piped/redirected
4. **Foreign key naming**: Separate configurable rules for both FK column names and FK constraint names

### Architecture Rationale
1. **Why YAML for config?**: Human-readable, supports comments, standard for config files
2. **Why three levels (enforce/warn/off)?**: Flexibility for gradual adoption, different contexts
3. **Why markdown + JSON?**: Markdown for human review, JSON for tooling integration
4. **Why pkg/inspector?**: Follows existing package structure, separates concerns
5. **Reuse readers**: Leverage existing reader infrastructure, supports all formats automatically
6. **Exit codes**: Follow standard conventions (0=success, 1=validation fail, 2=error)

## Future Enhancements (Not in Scope)
- Auto-fix mode (automatically rename columns, etc.)
- Custom rule plugins
- HTML report format
- Rule templates for different databases
- CI/CD integration examples
- Performance metrics in report