fix(go.sum): update ResolveSpec dependency to v1.0.87
CI / build-and-test (push) Failing after 1s
Release / release (push) Failing after 19m26s

This commit is contained in:
Hein
2026-06-23 13:17:16 +02:00
parent 0227912325
commit 1adf50e3db
2436 changed files with 1078758 additions and 114 deletions
@@ -0,0 +1,218 @@
# Automatic Relation Loading Strategies
## Overview
**NEW:** The database adapters now **automatically** choose the optimal loading strategy by inspecting your model's relationship tags!
Simply use `PreloadRelation()` and the system automatically:
- Detects relationship type from Bun/GORM tags
- Uses **JOIN** for many-to-one and one-to-one (efficient, no duplication)
- Uses **separate query** for one-to-many and many-to-many (avoids duplication)
## How It Works
```go
// Just write this - the system handles the rest!
db.NewSelect().
Model(&links).
PreloadRelation("Provider"). // ✓ Auto-detects belongs-to → uses JOIN
PreloadRelation("Tags"). // ✓ Auto-detects has-many → uses separate query
Scan(ctx, &links)
```
### Detection Logic
The system inspects your model's struct tags:
**Bun models:**
```go
type Link struct {
Provider *Provider `bun:"rel:belongs-to"` // → Detected: belongs-to → JOIN
Tags []Tag `bun:"rel:has-many"` // → Detected: has-many → Separate query
}
```
**GORM models:**
```go
type Link struct {
ProviderID int
Provider *Provider `gorm:"foreignKey:ProviderID"` // → Detected: belongs-to → JOIN
Tags []Tag `gorm:"many2many:link_tags"` // → Detected: many-to-many → Separate query
}
```
**Type inference (fallback):**
- `[]Type` (slice) → has-many → Separate query
- `*Type` (pointer) → belongs-to → JOIN
- `Type` (struct) → belongs-to → JOIN
### What Gets Logged
Enable debug logging to see strategy selection:
```go
bunAdapter.EnableQueryDebug()
```
**Output:**
```
DEBUG: PreloadRelation 'Provider' detected as: belongs-to
INFO: Using JOIN strategy for belongs-to relation 'Provider'
DEBUG: PreloadRelation 'Links' detected as: has-many
DEBUG: Using separate query for has-many relation 'Links'
```
## Relationship Types
| Bun Tag | GORM Pattern | Field Type | Strategy | Why |
|---------|--------------|------------|----------|-----|
| `rel:has-many` | Slice field | `[]Type` | Separate Query | Avoids duplicating parent data |
| `rel:belongs-to` | `foreignKey:` | `*Type` | JOIN | Single parent, no duplication |
| `rel:has-one` | Single pointer | `*Type` | JOIN | One-to-one, no duplication |
| `rel:many-to-many` | `many2many:` | `[]Type` | Separate Query | Complex join, avoid cartesian |
## Manual Override
If you need to force a specific strategy, use `JoinRelation()`:
```go
// Force JOIN even for has-many (not recommended)
db.NewSelect().
Model(&providers).
JoinRelation("Links"). // Explicitly use JOIN
Scan(ctx, &providers)
```
## Examples
### Automatic Strategy Selection (Recommended)
```go
// Example 1: Loading parent provider for each link
// System detects belongs-to → uses JOIN automatically
db.NewSelect().
Model(&links).
PreloadRelation("Provider", func(q common.SelectQuery) common.SelectQuery {
return q.Where("active = ?", true)
}).
Scan(ctx, &links)
// Generated SQL: Single query with JOIN
// SELECT links.*, providers.*
// FROM links
// LEFT JOIN providers ON links.provider_id = providers.id
// WHERE providers.active = true
// Example 2: Loading child links for each provider
// System detects has-many → uses separate query automatically
db.NewSelect().
Model(&providers).
PreloadRelation("Links", func(q common.SelectQuery) common.SelectQuery {
return q.Where("active = ?", true)
}).
Scan(ctx, &providers)
// Generated SQL: Two queries
// Query 1: SELECT * FROM providers
// Query 2: SELECT * FROM links
// WHERE provider_id IN (1, 2, 3, ...)
// AND active = true
```
### Mixed Relationships
```go
type Order struct {
ID int
CustomerID int
Customer *Customer `bun:"rel:belongs-to"` // JOIN
Items []Item `bun:"rel:has-many"` // Separate
Invoice *Invoice `bun:"rel:has-one"` // JOIN
}
// All three handled optimally!
db.NewSelect().
Model(&orders).
PreloadRelation("Customer"). // → JOIN (many-to-one)
PreloadRelation("Items"). // → Separate (one-to-many)
PreloadRelation("Invoice"). // → JOIN (one-to-one)
Scan(ctx, &orders)
```
## Performance Benefits
### Before (Manual Strategy Selection)
```go
// You had to remember which to use:
.PreloadRelation("Provider") // Should I use PreloadRelation or JoinRelation?
.PreloadRelation("Links") // Which is more efficient here?
```
### After (Automatic Selection)
```go
// Just use PreloadRelation everywhere:
.PreloadRelation("Provider") // ✓ System uses JOIN automatically
.PreloadRelation("Links") // ✓ System uses separate query automatically
```
## Migration Guide
**No changes needed!** If you're already using `PreloadRelation()`, it now automatically optimizes:
```go
// Before: Always used separate query
.PreloadRelation("Provider") // Inefficient: extra round trip
// After: Automatic optimization
.PreloadRelation("Provider") // ✓ Now uses JOIN automatically!
```
## Implementation Details
### Supported Bun Tags
- `rel:has-many` → Separate query
- `rel:belongs-to` → JOIN
- `rel:has-one` → JOIN
- `rel:many-to-many` or `rel:m2m` → Separate query
### Supported GORM Patterns
- `many2many:` tag → Separate query
- `foreignKey:` tag → JOIN (belongs-to)
- `[]Type` slice without many2many → Separate query (has-many)
- `*Type` pointer with foreignKey → JOIN (belongs-to)
- `*Type` pointer without foreignKey → JOIN (has-one)
### Fallback Behavior
- `[]Type` (slice) → Separate query (safe default for collections)
- `*Type` or `Type` (single) → JOIN (safe default for single relations)
- Unknown → Separate query (safest default)
## Debugging
To see strategy selection in action:
```go
// Enable debug logging
bunAdapter.EnableQueryDebug() // or gormAdapter.EnableQueryDebug()
// Run your query
db.NewSelect().
Model(&records).
PreloadRelation("RelationName").
Scan(ctx, &records)
// Check logs for:
// - "PreloadRelation 'X' detected as: belongs-to"
// - "Using JOIN strategy for belongs-to relation 'X'"
// - Actual SQL queries executed
```
## Best Practices
1. **Use PreloadRelation() for everything** - Let the system optimize
2. **Define proper relationship tags** - Ensures correct detection
3. **Only use JoinRelation() for overrides** - When you know better than auto-detection
4. **Enable debug logging during development** - Verify optimal strategies are chosen
5. **Trust the system** - It's designed to choose correctly based on relationship type
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,176 @@
package database
import (
"context"
"database/sql"
"fmt"
_ "github.com/jackc/pgx/v5/stdlib" // PostgreSQL driver
"github.com/bitechdev/ResolveSpec/pkg/common"
)
// Example demonstrates how to use the PgSQL adapter
func ExamplePgSQLAdapter() error {
// Connect to PostgreSQL database
dsn := "postgres://username:password@localhost:5432/dbname?sslmode=disable"
db, err := sql.Open("pgx", dsn)
if err != nil {
return fmt.Errorf("failed to open database: %w", err)
}
defer db.Close()
// Create the PgSQL adapter
adapter := NewPgSQLAdapter(db)
// Enable query debugging (optional)
adapter.EnableQueryDebug()
ctx := context.Background()
// Example 1: Simple SELECT query
var results []map[string]interface{}
err = adapter.NewSelect().
Table("users").
Where("age > ?", 18).
Order("created_at DESC").
Limit(10).
Scan(ctx, &results)
if err != nil {
return fmt.Errorf("select failed: %w", err)
}
// Example 2: INSERT query
result, err := adapter.NewInsert().
Table("users").
Value("name", "John Doe").
Value("email", "john@example.com").
Value("age", 25).
Returning("id").
Exec(ctx)
if err != nil {
return fmt.Errorf("insert failed: %w", err)
}
fmt.Printf("Rows affected: %d\n", result.RowsAffected())
// Example 3: UPDATE query
result, err = adapter.NewUpdate().
Table("users").
Set("name", "Jane Doe").
Where("id = ?", 1).
Exec(ctx)
if err != nil {
return fmt.Errorf("update failed: %w", err)
}
fmt.Printf("Rows updated: %d\n", result.RowsAffected())
// Example 4: DELETE query
result, err = adapter.NewDelete().
Table("users").
Where("age < ?", 18).
Exec(ctx)
if err != nil {
return fmt.Errorf("delete failed: %w", err)
}
fmt.Printf("Rows deleted: %d\n", result.RowsAffected())
// Example 5: Using transactions
err = adapter.RunInTransaction(ctx, func(tx common.Database) error {
// Insert a new user
_, err := tx.NewInsert().
Table("users").
Value("name", "Transaction User").
Value("email", "tx@example.com").
Exec(ctx)
if err != nil {
return err
}
// Update another user
_, err = tx.NewUpdate().
Table("users").
Set("verified", true).
Where("email = ?", "tx@example.com").
Exec(ctx)
if err != nil {
return err
}
// Both operations succeed or both rollback
return nil
})
if err != nil {
return fmt.Errorf("transaction failed: %w", err)
}
// Example 6: JOIN query
err = adapter.NewSelect().
Table("users u").
Column("u.id", "u.name", "p.title as post_title").
LeftJoin("posts p ON p.user_id = u.id").
Where("u.active = ?", true).
Scan(ctx, &results)
if err != nil {
return fmt.Errorf("join query failed: %w", err)
}
// Example 7: Aggregation query
count, err := adapter.NewSelect().
Table("users").
Where("active = ?", true).
Count(ctx)
if err != nil {
return fmt.Errorf("count failed: %w", err)
}
fmt.Printf("Active users: %d\n", count)
// Example 8: Raw SQL execution
_, err = adapter.Exec(ctx, "CREATE INDEX IF NOT EXISTS idx_users_email ON users(email)")
if err != nil {
return fmt.Errorf("raw exec failed: %w", err)
}
// Example 9: Raw SQL query
var users []map[string]interface{}
err = adapter.Query(ctx, &users, "SELECT * FROM users WHERE age > $1 LIMIT $2", 18, 10)
if err != nil {
return fmt.Errorf("raw query failed: %w", err)
}
return nil
}
// User is an example model
type User struct {
ID int `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
Age int `json:"age"`
}
// TableName implements common.TableNameProvider
func (u User) TableName() string {
return "users"
}
// ExampleWithModel demonstrates using models with the PgSQL adapter
func ExampleWithModel() error {
dsn := "postgres://username:password@localhost:5432/dbname?sslmode=disable"
db, err := sql.Open("pgx", dsn)
if err != nil {
return err
}
defer db.Close()
adapter := NewPgSQLAdapter(db)
ctx := context.Background()
// Use model with adapter
user := User{}
err = adapter.NewSelect().
Model(&user).
Where("id = ?", 1).
Scan(ctx, &user)
return err
}
@@ -0,0 +1,275 @@
package database
import (
"context"
"database/sql"
_ "github.com/jackc/pgx/v5/stdlib"
"github.com/bitechdev/ResolveSpec/pkg/common"
)
// Example models for demonstrating preload functionality
// Author model - has many Posts
type Author struct {
ID int `db:"id"`
Name string `db:"name"`
Email string `db:"email"`
Posts []*Post `bun:"rel:has-many,join:id=author_id"`
}
func (a Author) TableName() string {
return "authors"
}
// Post model - belongs to Author, has many Comments
type Post struct {
ID int `db:"id"`
Title string `db:"title"`
Content string `db:"content"`
AuthorID int `db:"author_id"`
Author *Author `bun:"rel:belongs-to,join:author_id=id"`
Comments []*Comment `bun:"rel:has-many,join:id=post_id"`
}
func (p Post) TableName() string {
return "posts"
}
// Comment model - belongs to Post
type Comment struct {
ID int `db:"id"`
Content string `db:"content"`
PostID int `db:"post_id"`
Post *Post `bun:"rel:belongs-to,join:post_id=id"`
}
func (c Comment) TableName() string {
return "comments"
}
// ExamplePreload demonstrates the Preload functionality
func ExamplePreload() error {
dsn := "postgres://username:password@localhost:5432/dbname?sslmode=disable"
db, err := sql.Open("pgx", dsn)
if err != nil {
return err
}
defer db.Close()
adapter := NewPgSQLAdapter(db)
ctx := context.Background()
// Example 1: Simple Preload (uses subquery for has-many)
var authors []*Author
err = adapter.NewSelect().
Model(&Author{}).
Table("authors").
Preload("Posts"). // Load all posts for each author
Scan(ctx, &authors)
if err != nil {
return err
}
// Now authors[i].Posts will be populated with their posts
return nil
}
// ExamplePreloadRelation demonstrates smart PreloadRelation with auto-detection
func ExamplePreloadRelation() error {
dsn := "postgres://username:password@localhost:5432/dbname?sslmode=disable"
db, err := sql.Open("pgx", dsn)
if err != nil {
return err
}
defer db.Close()
adapter := NewPgSQLAdapter(db)
ctx := context.Background()
// Example 1: PreloadRelation auto-detects has-many (uses subquery)
var authors []*Author
err = adapter.NewSelect().
Model(&Author{}).
Table("authors").
PreloadRelation("Posts", func(q common.SelectQuery) common.SelectQuery {
return q.Where("published = ?", true).Order("created_at DESC")
}).
Where("active = ?", true).
Scan(ctx, &authors)
if err != nil {
return err
}
// Example 2: PreloadRelation auto-detects belongs-to (uses JOIN)
var posts []*Post
err = adapter.NewSelect().
Model(&Post{}).
Table("posts").
PreloadRelation("Author"). // Will use JOIN because it's belongs-to
Scan(ctx, &posts)
if err != nil {
return err
}
// Example 3: Nested preloads
err = adapter.NewSelect().
Model(&Author{}).
Table("authors").
PreloadRelation("Posts", func(q common.SelectQuery) common.SelectQuery {
// First load posts, then preload comments for each post
return q.Limit(10)
}).
Scan(ctx, &authors)
if err != nil {
return err
}
// Manually load nested relationships (two-level preloading)
for _, author := range authors {
if author.Posts != nil {
for _, post := range author.Posts {
var comments []*Comment
err := adapter.NewSelect().
Table("comments").
Where("post_id = ?", post.ID).
Scan(ctx, &comments)
if err == nil {
post.Comments = comments
}
}
}
}
return nil
}
// ExampleJoinRelation demonstrates explicit JOIN loading
func ExampleJoinRelation() error {
dsn := "postgres://username:password@localhost:5432/dbname?sslmode=disable"
db, err := sql.Open("pgx", dsn)
if err != nil {
return err
}
defer db.Close()
adapter := NewPgSQLAdapter(db)
ctx := context.Background()
// Example 1: Force JOIN for belongs-to relationship
var posts []*Post
err = adapter.NewSelect().
Model(&Post{}).
Table("posts").
JoinRelation("Author", func(q common.SelectQuery) common.SelectQuery {
return q.Where("active = ?", true)
}).
Scan(ctx, &posts)
if err != nil {
return err
}
// Example 2: Multiple JOINs
err = adapter.NewSelect().
Model(&Post{}).
Table("posts p").
Column("p.*", "a.name as author_name", "a.email as author_email").
LeftJoin("authors a ON a.id = p.author_id").
Where("p.published = ?", true).
Scan(ctx, &posts)
return err
}
// ExampleScanModel demonstrates ScanModel with struct destinations
func ExampleScanModel() error {
dsn := "postgres://username:password@localhost:5432/dbname?sslmode=disable"
db, err := sql.Open("pgx", dsn)
if err != nil {
return err
}
defer db.Close()
adapter := NewPgSQLAdapter(db)
ctx := context.Background()
// Example 1: Scan single struct
author := Author{}
err = adapter.NewSelect().
Model(&author).
Table("authors").
Where("id = ?", 1).
ScanModel(ctx) // ScanModel automatically uses the model set with Model()
if err != nil {
return err
}
// Example 2: Scan slice of structs
authors := []*Author{}
err = adapter.NewSelect().
Model(&authors).
Table("authors").
Where("active = ?", true).
Limit(10).
ScanModel(ctx)
return err
}
// ExampleCompleteWorkflow demonstrates a complete workflow with preloading
func ExampleCompleteWorkflow() error {
dsn := "postgres://username:password@localhost:5432/dbname?sslmode=disable"
db, err := sql.Open("pgx", dsn)
if err != nil {
return err
}
defer db.Close()
adapter := NewPgSQLAdapter(db)
adapter.EnableQueryDebug() // Enable query logging
ctx := context.Background()
// Step 1: Create an author
author := &Author{
Name: "John Doe",
Email: "john@example.com",
}
result, err := adapter.NewInsert().
Table("authors").
Value("name", author.Name).
Value("email", author.Email).
Returning("id").
Exec(ctx)
if err != nil {
return err
}
_ = result
// Step 2: Load author with all their posts
var loadedAuthor Author
err = adapter.NewSelect().
Model(&loadedAuthor).
Table("authors").
PreloadRelation("Posts", func(q common.SelectQuery) common.SelectQuery {
return q.Order("created_at DESC").Limit(5)
}).
Where("id = ?", 1).
ScanModel(ctx)
if err != nil {
return err
}
// Step 3: Update author name
_, err = adapter.NewUpdate().
Table("authors").
Set("name", "Jane Doe").
Where("id = ?", 1).
Exec(ctx)
return err
}
@@ -0,0 +1,335 @@
package database
import (
"reflect"
"strings"
"time"
"github.com/bitechdev/ResolveSpec/pkg/common"
"github.com/bitechdev/ResolveSpec/pkg/metrics"
"github.com/bitechdev/ResolveSpec/pkg/reflection"
)
const maxMetricFallbackEntityLength = 120
func recordQueryMetrics(enabled bool, operation, schema, entity, table string, startedAt time.Time, err error) {
if !enabled {
return
}
metrics.GetProvider().RecordDBQuery(
normalizeMetricOperation(operation),
normalizeMetricSchema(schema),
normalizeMetricEntity(entity, table),
normalizeMetricTable(table),
time.Since(startedAt),
err,
)
}
func normalizeMetricOperation(operation string) string {
operation = strings.ToUpper(strings.TrimSpace(operation))
if operation == "" {
return "UNKNOWN"
}
return operation
}
func normalizeMetricSchema(schema string) string {
schema = cleanMetricIdentifier(schema)
if schema == "" {
return "default"
}
return schema
}
func normalizeMetricEntity(entity, table string) string {
entity = cleanMetricIdentifier(entity)
if entity != "" {
return entity
}
table = cleanMetricIdentifier(table)
if table != "" {
return table
}
return "unknown"
}
func normalizeMetricTable(table string) string {
table = cleanMetricIdentifier(table)
if table == "" {
return "unknown"
}
return table
}
func entityNameFromModel(model interface{}, table string) string {
if model == nil {
return cleanMetricIdentifier(table)
}
modelType := reflect.TypeOf(model)
for modelType != nil && (modelType.Kind() == reflect.Pointer || modelType.Kind() == reflect.Slice || modelType.Kind() == reflect.Array) {
modelType = modelType.Elem()
}
if modelType == nil {
return cleanMetricIdentifier(table)
}
if modelType.Kind() == reflect.Struct && modelType.Name() != "" {
return reflection.ToSnakeCase(modelType.Name())
}
return cleanMetricIdentifier(table)
}
func schemaAndTableFromModel(model interface{}, driverName string) (schema, table string) {
provider, ok := tableNameProviderFromModel(model)
if !ok {
return "", ""
}
return parseTableName(provider.TableName(), driverName)
}
// tableNameProviderType is cached to avoid repeated reflection on every call.
var tableNameProviderType = reflect.TypeOf((*common.TableNameProvider)(nil)).Elem()
func tableNameProviderFromModel(model interface{}) (common.TableNameProvider, bool) {
if model == nil {
return nil, false
}
if provider, ok := model.(common.TableNameProvider); ok {
return provider, true
}
modelType := reflect.TypeOf(model)
for modelType != nil && (modelType.Kind() == reflect.Pointer || modelType.Kind() == reflect.Slice || modelType.Kind() == reflect.Array) {
modelType = modelType.Elem()
}
if modelType == nil || modelType.Kind() != reflect.Struct {
return nil, false
}
// Check whether *T implements TableNameProvider before allocating.
ptrType := reflect.PointerTo(modelType)
if !ptrType.Implements(tableNameProviderType) && !modelType.Implements(tableNameProviderType) {
return nil, false
}
modelValue := reflect.New(modelType)
if provider, ok := modelValue.Interface().(common.TableNameProvider); ok {
return provider, true
}
if provider, ok := modelValue.Elem().Interface().(common.TableNameProvider); ok {
return provider, true
}
return nil, false
}
func metricTargetFromRawQuery(query, driverName string) (operation, schema, entity, table string) {
operation = normalizeMetricOperation(firstQueryKeyword(query))
tableRef := tableFromRawQuery(query, operation)
if tableRef == "" {
return operation, "", fallbackMetricEntityFromQuery(query), "unknown"
}
schema, table = parseTableName(tableRef, driverName)
entity = cleanMetricIdentifier(table)
return operation, schema, entity, table
}
func fallbackMetricEntityFromQuery(query string) string {
query = sanitizeMetricQueryShape(query)
if query == "" {
return "unknown"
}
if len(query) > maxMetricFallbackEntityLength {
return query[:maxMetricFallbackEntityLength-3] + "..."
}
return query
}
func sanitizeMetricQueryShape(query string) string {
query = strings.TrimSpace(query)
if query == "" {
return ""
}
var out strings.Builder
for i := 0; i < len(query); {
if query[i] == '\'' {
out.WriteByte('?')
i++
for i < len(query) {
if query[i] == '\'' {
if i+1 < len(query) && query[i+1] == '\'' {
i += 2
continue
}
i++
break
}
i++
}
continue
}
if query[i] == '?' {
out.WriteByte('?')
i++
continue
}
if query[i] == '$' && i+1 < len(query) && isASCIIDigit(query[i+1]) {
out.WriteByte('?')
i++
for i < len(query) && isASCIIDigit(query[i]) {
i++
}
continue
}
if query[i] == ':' && (i == 0 || query[i-1] != ':') && i+1 < len(query) && isIdentifierStart(query[i+1]) {
out.WriteByte('?')
i++
for i < len(query) && isIdentifierPart(query[i]) {
i++
}
continue
}
if query[i] == '@' && (i == 0 || query[i-1] != '@') && i+1 < len(query) && isIdentifierStart(query[i+1]) {
out.WriteByte('?')
i++
for i < len(query) && isIdentifierPart(query[i]) {
i++
}
continue
}
if startsNumericLiteral(query, i) {
out.WriteByte('?')
i++
for i < len(query) && (isASCIIDigit(query[i]) || query[i] == '.') {
i++
}
continue
}
out.WriteByte(query[i])
i++
}
return strings.Join(strings.Fields(out.String()), " ")
}
func startsNumericLiteral(query string, idx int) bool {
if idx >= len(query) {
return false
}
start := idx
if query[idx] == '-' {
if idx+1 >= len(query) || !isASCIIDigit(query[idx+1]) {
return false
}
start++
}
if !isASCIIDigit(query[start]) {
return false
}
if idx > 0 && isIdentifierPart(query[idx-1]) {
return false
}
if start+1 < len(query) && query[start] == '0' && (query[start+1] == 'x' || query[start+1] == 'X') {
return false
}
return true
}
func isASCIIDigit(ch byte) bool {
return ch >= '0' && ch <= '9'
}
func isIdentifierStart(ch byte) bool {
return (ch >= 'a' && ch <= 'z') || (ch >= 'A' && ch <= 'Z') || ch == '_'
}
func isIdentifierPart(ch byte) bool {
return isIdentifierStart(ch) || isASCIIDigit(ch)
}
func firstQueryKeyword(query string) string {
query = strings.TrimSpace(query)
if query == "" {
return ""
}
fields := strings.Fields(query)
if len(fields) == 0 {
return ""
}
return fields[0]
}
func tableFromRawQuery(query, operation string) string {
tokens := tokenizeQuery(query)
if len(tokens) == 0 {
return ""
}
switch operation {
case "SELECT":
return tokenAfter(tokens, "FROM")
case "INSERT":
return tokenAfter(tokens, "INTO")
case "UPDATE":
return tokenAfter(tokens, "UPDATE")
case "DELETE":
return tokenAfter(tokens, "FROM")
default:
return ""
}
}
func tokenAfter(tokens []string, keyword string) string {
for idx, token := range tokens {
if strings.EqualFold(token, keyword) && idx+1 < len(tokens) {
return cleanMetricIdentifier(tokens[idx+1])
}
}
return ""
}
func tokenizeQuery(query string) []string {
replacer := strings.NewReplacer(
"\n", " ",
"\t", " ",
"(", " ",
")", " ",
",", " ",
)
return strings.Fields(replacer.Replace(query))
}
func cleanMetricIdentifier(value string) string {
value = strings.TrimSpace(value)
value = strings.Trim(value, "\"'`[]")
value = strings.TrimRight(value, ";")
return value
}
@@ -0,0 +1,132 @@
package database
import (
"context"
"database/sql"
"testing"
"github.com/stretchr/testify/require"
)
// TestHelper provides utilities for database testing
type TestHelper struct {
DB *sql.DB
Adapter *PgSQLAdapter
t *testing.T
}
// NewTestHelper creates a new test helper
func NewTestHelper(t *testing.T, db *sql.DB) *TestHelper {
return &TestHelper{
DB: db,
Adapter: NewPgSQLAdapter(db),
t: t,
}
}
// CleanupTables truncates all test tables
func (h *TestHelper) CleanupTables() {
ctx := context.Background()
tables := []string{"comments", "posts", "users"}
for _, table := range tables {
_, err := h.DB.ExecContext(ctx, "TRUNCATE TABLE "+table+" CASCADE")
require.NoError(h.t, err)
}
}
// InsertUser inserts a test user and returns the ID
func (h *TestHelper) InsertUser(name, email string, age int) int {
ctx := context.Background()
result, err := h.Adapter.NewInsert().
Table("users").
Value("name", name).
Value("email", email).
Value("age", age).
Exec(ctx)
require.NoError(h.t, err)
id, _ := result.LastInsertId()
return int(id)
}
// InsertPost inserts a test post and returns the ID
func (h *TestHelper) InsertPost(userID int, title, content string, published bool) int {
ctx := context.Background()
result, err := h.Adapter.NewInsert().
Table("posts").
Value("user_id", userID).
Value("title", title).
Value("content", content).
Value("published", published).
Exec(ctx)
require.NoError(h.t, err)
id, _ := result.LastInsertId()
return int(id)
}
// InsertComment inserts a test comment and returns the ID
func (h *TestHelper) InsertComment(postID int, content string) int {
ctx := context.Background()
result, err := h.Adapter.NewInsert().
Table("comments").
Value("post_id", postID).
Value("content", content).
Exec(ctx)
require.NoError(h.t, err)
id, _ := result.LastInsertId()
return int(id)
}
// AssertUserExists checks if a user exists by email
func (h *TestHelper) AssertUserExists(email string) {
ctx := context.Background()
exists, err := h.Adapter.NewSelect().
Table("users").
Where("email = ?", email).
Exists(ctx)
require.NoError(h.t, err)
require.True(h.t, exists, "User with email %s should exist", email)
}
// AssertUserCount asserts the number of users
func (h *TestHelper) AssertUserCount(expected int) {
ctx := context.Background()
count, err := h.Adapter.NewSelect().
Table("users").
Count(ctx)
require.NoError(h.t, err)
require.Equal(h.t, expected, count)
}
// GetUserByEmail retrieves a user by email
func (h *TestHelper) GetUserByEmail(email string) map[string]interface{} {
ctx := context.Background()
var results []map[string]interface{}
err := h.Adapter.NewSelect().
Table("users").
Where("email = ?", email).
Scan(ctx, &results)
require.NoError(h.t, err)
require.Len(h.t, results, 1, "Expected exactly one user with email %s", email)
return results[0]
}
// BeginTestTransaction starts a transaction for testing
func (h *TestHelper) BeginTestTransaction() (*PgSQLTxAdapter, func()) {
ctx := context.Background()
tx, err := h.DB.BeginTx(ctx, nil)
require.NoError(h.t, err)
adapter := &PgSQLTxAdapter{tx: tx}
cleanup := func() {
tx.Rollback()
}
return adapter, cleanup
}
@@ -0,0 +1,117 @@
package database
import (
"database/sql"
"strings"
"github.com/uptrace/bun/dialect/mssqldialect"
"github.com/uptrace/bun/dialect/pgdialect"
"github.com/uptrace/bun/dialect/sqlitedialect"
"gorm.io/driver/postgres"
"gorm.io/driver/sqlite"
"gorm.io/driver/sqlserver"
"gorm.io/gorm"
"github.com/bitechdev/ResolveSpec/pkg/logger"
)
// PostgreSQL identifier length limit (63 bytes + null terminator = 64 bytes total)
const postgresIdentifierLimit = 63
// checkAliasLength checks if a preload relation path will generate aliases that exceed PostgreSQL's limit
// Returns true if the alias is likely to be truncated
func checkAliasLength(relation string) bool {
// Bun generates aliases like: parentalias__childalias__columnname
// For nested preloads, it uses the pattern: relation1__relation2__relation3__columnname
parts := strings.Split(relation, ".")
if len(parts) <= 1 {
return false // Single level relations are fine
}
// Calculate the actual alias prefix length that Bun will generate
// Bun uses double underscores (__) between each relation level
// and converts the relation names to lowercase with underscores
aliasPrefix := strings.ToLower(strings.Join(parts, "__"))
aliasPrefixLen := len(aliasPrefix)
// We need to add 2 more underscores for the column name separator plus column name length
// Column names in the error were things like "rid_mastertype_hubtype" (23 chars)
// To be safe, assume the longest column name could be around 35 chars
maxColumnNameLen := 35
estimatedMaxLen := aliasPrefixLen + 2 + maxColumnNameLen
// Check if this would exceed PostgreSQL's identifier limit
if estimatedMaxLen > postgresIdentifierLimit {
logger.Warn("Preload relation '%s' will generate aliases up to %d chars (prefix: %d + column: %d), exceeding PostgreSQL's %d char limit",
relation, estimatedMaxLen, aliasPrefixLen, maxColumnNameLen, postgresIdentifierLimit)
return true
}
// Also check if just the prefix is getting close (within 15 chars of limit)
// This gives room for column names
if aliasPrefixLen > (postgresIdentifierLimit - 15) {
logger.Warn("Preload relation '%s' has alias prefix of %d chars, which may cause truncation with longer column names (limit: %d)",
relation, aliasPrefixLen, postgresIdentifierLimit)
return true
}
return false
}
// parseTableName splits a table name that may contain schema into separate schema and table
// For example: "public.users" -> ("public", "users")
//
// "users" -> ("", "users")
//
// For SQLite, schema.table is translated to schema_table since SQLite doesn't support schemas
// in the same way as PostgreSQL/MSSQL
func parseTableName(fullTableName, driverName string) (schema, table string) {
if idx := strings.LastIndex(fullTableName, "."); idx != -1 {
schema = fullTableName[:idx]
table = fullTableName[idx+1:]
// For SQLite, convert schema.table to schema_table
if driverName == "sqlite" || driverName == "sqlite3" {
table = schema + "_" + table
schema = ""
}
return schema, table
}
return "", fullTableName
}
// GetPostgresDialect returns a Bun PostgreSQL dialect
func GetPostgresDialect() *pgdialect.Dialect {
return pgdialect.New()
}
// GetSQLiteDialect returns a Bun SQLite dialect
func GetSQLiteDialect() *sqlitedialect.Dialect {
return sqlitedialect.New()
}
// GetMSSQLDialect returns a Bun MSSQL dialect
func GetMSSQLDialect() *mssqldialect.Dialect {
return mssqldialect.New()
}
// GetPostgresDialector returns a GORM PostgreSQL dialector
func GetPostgresDialector(db *sql.DB) gorm.Dialector {
return postgres.New(postgres.Config{
Conn: db,
})
}
// GetSQLiteDialector returns a GORM SQLite dialector
func GetSQLiteDialector(db *sql.DB) gorm.Dialector {
return sqlite.Dialector{
Conn: db,
}
}
// GetMSSQLDialector returns a GORM MSSQL dialector
func GetMSSQLDialector(db *sql.DB) gorm.Dialector {
return sqlserver.New(sqlserver.Config{
Conn: db,
})
}
@@ -0,0 +1,214 @@
package router
import (
"net/http"
"github.com/uptrace/bunrouter"
"github.com/bitechdev/ResolveSpec/pkg/common"
"github.com/bitechdev/ResolveSpec/pkg/logger"
)
// BunRouterAdapter adapts uptrace/bunrouter to work with our Router interface
type BunRouterAdapter struct {
router *bunrouter.Router
}
// NewBunRouterAdapter creates a new bunrouter adapter
func NewBunRouterAdapter(router *bunrouter.Router) *BunRouterAdapter {
return &BunRouterAdapter{router: router}
}
// NewBunRouterAdapterDefault creates a new bunrouter adapter with default router
func NewBunRouterAdapterDefault() *BunRouterAdapter {
return &BunRouterAdapter{router: bunrouter.New()}
}
func (b *BunRouterAdapter) HandleFunc(pattern string, handler common.HTTPHandlerFunc) common.RouteRegistration {
route := &BunRouterRegistration{
router: b.router,
pattern: pattern,
handler: handler,
}
return route
}
func (b *BunRouterAdapter) ServeHTTP(w common.ResponseWriter, r common.Request) {
// This method would be used when we need to serve through our interface
// For now, we'll work directly with the underlying router
w.WriteHeader(http.StatusNotImplemented)
_, err := w.Write([]byte(`{"error":"ServeHTTP not implemented - use GetBunRouter() for direct access"}`))
if err != nil {
logger.Warn("Failed to write. %v", err)
}
}
// GetBunRouter returns the underlying bunrouter for direct access
func (b *BunRouterAdapter) GetBunRouter() *bunrouter.Router {
return b.router
}
// BunRouterRegistration implements RouteRegistration for bunrouter
type BunRouterRegistration struct {
router *bunrouter.Router
pattern string
handler common.HTTPHandlerFunc
}
func (b *BunRouterRegistration) Methods(methods ...string) common.RouteRegistration {
// bunrouter handles methods differently - we'll register for each method
for _, method := range methods {
b.router.Handle(method, b.pattern, func(w http.ResponseWriter, req bunrouter.Request) error {
// Convert bunrouter.Request to our BunRouterRequest
reqAdapter := &BunRouterRequest{req: req}
respAdapter := &HTTPResponseWriter{resp: w}
b.handler(respAdapter, reqAdapter)
return nil
})
}
return b
}
func (b *BunRouterRegistration) PathPrefix(prefix string) common.RouteRegistration {
// bunrouter doesn't have PathPrefix like mux, but we can modify the pattern
newPattern := prefix + b.pattern
b.pattern = newPattern
return b
}
// BunRouterRequest adapts bunrouter.Request to our Request interface
type BunRouterRequest struct {
req bunrouter.Request
body []byte
}
// NewBunRouterRequest creates a new BunRouterRequest adapter
func NewBunRouterRequest(req bunrouter.Request) *BunRouterRequest {
return &BunRouterRequest{req: req}
}
func (b *BunRouterRequest) Method() string {
return b.req.Method
}
func (b *BunRouterRequest) URL() string {
return b.req.URL.String()
}
func (b *BunRouterRequest) Header(key string) string {
return b.req.Header.Get(key)
}
func (b *BunRouterRequest) Body() ([]byte, error) {
if b.body != nil {
return b.body, nil
}
if b.req.Body == nil {
return nil, nil
}
// Create HTTPRequest adapter and use its Body() method
httpAdapter := NewHTTPRequest(b.req.Request)
body, err := httpAdapter.Body()
if err != nil {
return nil, err
}
b.body = body
return body, nil
}
func (b *BunRouterRequest) PathParam(key string) string {
return b.req.Param(key)
}
func (b *BunRouterRequest) QueryParam(key string) string {
return b.req.URL.Query().Get(key)
}
func (b *BunRouterRequest) AllQueryParams() map[string]string {
params := make(map[string]string)
for key, values := range b.req.URL.Query() {
if len(values) > 0 {
params[key] = values[0]
}
}
return params
}
func (b *BunRouterRequest) AllHeaders() map[string]string {
headers := make(map[string]string)
for key, values := range b.req.Header {
if len(values) > 0 {
headers[key] = values[0]
}
}
return headers
}
// UnderlyingRequest returns the underlying *http.Request
// This is useful when you need to pass the request to other handlers
func (b *BunRouterRequest) UnderlyingRequest() *http.Request {
return b.req.Request
}
// StandardBunRouterAdapter creates routes compatible with standard bunrouter handlers
type StandardBunRouterAdapter struct {
*BunRouterAdapter
}
func NewStandardBunRouterAdapter() *StandardBunRouterAdapter {
return &StandardBunRouterAdapter{
BunRouterAdapter: NewBunRouterAdapterDefault(),
}
}
// RegisterRoute registers a route that works with the existing Handler
func (s *StandardBunRouterAdapter) RegisterRoute(method, pattern string, handler func(http.ResponseWriter, *http.Request, map[string]string)) {
s.router.Handle(method, pattern, func(w http.ResponseWriter, req bunrouter.Request) error {
// Extract path parameters
params := make(map[string]string)
// bunrouter doesn't provide a direct way to get all params
// You would typically access them individually with req.Param("name")
// For this example, we'll create the map based on the request context
handler(w, req.Request, params)
return nil
})
}
// RegisterRouteWithParams registers a route with explicit parameter extraction
func (s *StandardBunRouterAdapter) RegisterRouteWithParams(method, pattern string, paramNames []string, handler func(http.ResponseWriter, *http.Request, map[string]string)) {
s.router.Handle(method, pattern, func(w http.ResponseWriter, req bunrouter.Request) error {
// Extract specified path parameters
params := make(map[string]string)
for _, paramName := range paramNames {
params[paramName] = req.Param(paramName)
}
handler(w, req.Request, params)
return nil
})
}
// BunRouterConfig holds bunrouter-specific configuration
type BunRouterConfig struct {
UseStrictSlash bool
RedirectTrailingSlash bool
HandleMethodNotAllowed bool
HandleOPTIONS bool
GlobalOPTIONS http.Handler
GlobalMethodNotAllowed http.Handler
PanicHandler func(http.ResponseWriter, *http.Request, interface{})
}
// DefaultBunRouterConfig returns default bunrouter configuration
func DefaultBunRouterConfig() *BunRouterConfig {
return &BunRouterConfig{
UseStrictSlash: false,
RedirectTrailingSlash: true,
HandleMethodNotAllowed: true,
HandleOPTIONS: true,
}
}
@@ -0,0 +1,238 @@
package router
import (
"encoding/json"
"io"
"net/http"
"github.com/gorilla/mux"
"github.com/bitechdev/ResolveSpec/pkg/common"
"github.com/bitechdev/ResolveSpec/pkg/logger"
)
// MuxAdapter adapts Gorilla Mux to work with our Router interface
type MuxAdapter struct {
router *mux.Router
}
// NewMuxAdapter creates a new Mux adapter
func NewMuxAdapter(router *mux.Router) *MuxAdapter {
return &MuxAdapter{router: router}
}
func (m *MuxAdapter) HandleFunc(pattern string, handler common.HTTPHandlerFunc) common.RouteRegistration {
route := &MuxRouteRegistration{
router: m.router,
pattern: pattern,
handler: handler,
}
return route
}
func (m *MuxAdapter) ServeHTTP(w common.ResponseWriter, r common.Request) {
// This method would be used when we need to serve through our interface
// For now, we'll work directly with the underlying router
w.WriteHeader(http.StatusNotImplemented)
_, err := w.Write([]byte(`{"error":"ServeHTTP not implemented - use GetMuxRouter() for direct access"}`))
if err != nil {
logger.Warn("Failed to write. %v", err)
}
}
// MuxRouteRegistration implements RouteRegistration for Mux
type MuxRouteRegistration struct {
router *mux.Router
pattern string
handler common.HTTPHandlerFunc
route *mux.Route
}
func (m *MuxRouteRegistration) Methods(methods ...string) common.RouteRegistration {
if m.route == nil {
m.route = m.router.HandleFunc(m.pattern, func(w http.ResponseWriter, r *http.Request) {
reqAdapter := &HTTPRequest{req: r, vars: mux.Vars(r)}
respAdapter := &HTTPResponseWriter{resp: w}
m.handler(respAdapter, reqAdapter)
})
}
m.route.Methods(methods...)
return m
}
func (m *MuxRouteRegistration) PathPrefix(prefix string) common.RouteRegistration {
if m.route == nil {
m.route = m.router.HandleFunc(m.pattern, func(w http.ResponseWriter, r *http.Request) {
reqAdapter := &HTTPRequest{req: r, vars: mux.Vars(r)}
respAdapter := &HTTPResponseWriter{resp: w}
m.handler(respAdapter, reqAdapter)
})
}
m.route.PathPrefix(prefix)
return m
}
// HTTPRequest adapts standard http.Request to our Request interface
type HTTPRequest struct {
req *http.Request
vars map[string]string
body []byte
}
func NewHTTPRequest(r *http.Request) *HTTPRequest {
return &HTTPRequest{
req: r,
vars: make(map[string]string),
}
}
func (h *HTTPRequest) Method() string {
return h.req.Method
}
func (h *HTTPRequest) URL() string {
return h.req.URL.String()
}
func (h *HTTPRequest) Header(key string) string {
return h.req.Header.Get(key)
}
func (h *HTTPRequest) Body() ([]byte, error) {
if h.body != nil {
return h.body, nil
}
if h.req.Body == nil {
return nil, nil
}
defer h.req.Body.Close()
body, err := io.ReadAll(h.req.Body)
if err != nil {
return nil, err
}
h.body = body
return body, nil
}
func (h *HTTPRequest) PathParam(key string) string {
return h.vars[key]
}
func (h *HTTPRequest) QueryParam(key string) string {
return h.req.URL.Query().Get(key)
}
func (h *HTTPRequest) AllQueryParams() map[string]string {
params := make(map[string]string)
for key, values := range h.req.URL.Query() {
if len(values) > 0 {
params[key] = values[0]
}
}
return params
}
func (h *HTTPRequest) AllHeaders() map[string]string {
headers := make(map[string]string)
for key, values := range h.req.Header {
if len(values) > 0 {
headers[key] = values[0]
}
}
return headers
}
// UnderlyingRequest returns the underlying *http.Request
// This is useful when you need to pass the request to other handlers
func (h *HTTPRequest) UnderlyingRequest() *http.Request {
return h.req
}
// HTTPResponseWriter adapts our ResponseWriter interface to standard http.ResponseWriter
type HTTPResponseWriter struct {
resp http.ResponseWriter
w common.ResponseWriter //nolint:unused
status int
}
func NewHTTPResponseWriter(w http.ResponseWriter) *HTTPResponseWriter {
return &HTTPResponseWriter{resp: w}
}
func (h *HTTPResponseWriter) SetHeader(key, value string) {
h.resp.Header().Set(key, value)
}
func (h *HTTPResponseWriter) WriteHeader(statusCode int) {
h.status = statusCode
h.resp.WriteHeader(statusCode)
}
func (h *HTTPResponseWriter) Write(data []byte) (int, error) {
return h.resp.Write(data)
}
func (h *HTTPResponseWriter) WriteJSON(data interface{}) error {
h.SetHeader("Content-Type", "application/json")
enc := json.NewEncoder(h.resp)
enc.SetEscapeHTML(false)
return enc.Encode(data)
}
// UnderlyingResponseWriter returns the underlying http.ResponseWriter
// This is useful when you need to pass the response writer to other handlers
func (h *HTTPResponseWriter) UnderlyingResponseWriter() http.ResponseWriter {
return h.resp
}
// StandardMuxAdapter creates routes compatible with standard http.HandlerFunc
type StandardMuxAdapter struct {
*MuxAdapter
}
func NewStandardMuxAdapter() *StandardMuxAdapter {
return &StandardMuxAdapter{
MuxAdapter: NewMuxAdapter(mux.NewRouter()),
}
}
// RegisterRoute registers a route that works with the existing Handler
func (s *StandardMuxAdapter) RegisterRoute(pattern string, handler func(http.ResponseWriter, *http.Request, map[string]string)) *mux.Route {
return s.router.HandleFunc(pattern, func(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
handler(w, r, vars)
})
}
// GetMuxRouter returns the underlying mux router for direct access
func (s *StandardMuxAdapter) GetMuxRouter() *mux.Router {
return s.router
}
// PathParamExtractor extracts path parameters from different router types
type PathParamExtractor interface {
ExtractParams(*http.Request) map[string]string
}
// MuxParamExtractor extracts parameters from Gorilla Mux
type MuxParamExtractor struct{}
func (m MuxParamExtractor) ExtractParams(r *http.Request) map[string]string {
return mux.Vars(r)
}
// RouterConfig holds router configuration
type RouterConfig struct {
PathPrefix string
Middleware []func(http.Handler) http.Handler
ParamExtractor PathParamExtractor
}
// DefaultRouterConfig returns default router configuration
func DefaultRouterConfig() *RouterConfig {
return &RouterConfig{
PathPrefix: "",
Middleware: make([]func(http.Handler) http.Handler, 0),
ParamExtractor: MuxParamExtractor{},
}
}
+149
View File
@@ -0,0 +1,149 @@
package common
import (
"fmt"
"strings"
"github.com/bitechdev/ResolveSpec/pkg/config"
)
// CORSConfig holds CORS configuration
type CORSConfig struct {
AllowedOrigins []string
AllowedMethods []string
AllowedHeaders []string
MaxAge int
}
// DefaultCORSConfig returns a default CORS configuration suitable for HeadSpec
func DefaultCORSConfig() CORSConfig {
configManager := config.GetConfigManager()
cfg, _ := configManager.GetConfig()
hosts := make([]string, 0)
// hosts = append(hosts, "*")
_, _, ipsList := config.GetIPs()
for i := range cfg.Servers.Instances {
server := cfg.Servers.Instances[i]
if server.Port == 0 {
continue
}
hosts = append(hosts, server.ExternalURLs...)
hosts = append(hosts, fmt.Sprintf("http://%s:%d", server.Host, server.Port))
hosts = append(hosts, fmt.Sprintf("https://%s:%d", server.Host, server.Port))
hosts = append(hosts, fmt.Sprintf("http://%s:%d", "localhost", server.Port))
for _, ip := range ipsList {
hosts = append(hosts, fmt.Sprintf("http://%s:%d", ip.String(), server.Port))
hosts = append(hosts, fmt.Sprintf("https://%s:%d", ip.String(), server.Port))
}
}
return CORSConfig{
AllowedOrigins: hosts,
AllowedMethods: []string{"GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"},
AllowedHeaders: GetHeadSpecHeaders(),
MaxAge: 86400, // 24 hours
}
}
// GetHeadSpecHeaders returns all headers used by HeadSpec
func GetHeadSpecHeaders() []string {
return []string{
// Standard headers
"Content-Type",
"Authorization",
"Accept",
"Accept-Language",
"Content-Language",
// Field Selection
"X-Select-Fields",
"X-Not-Select-Fields",
"X-Clean-JSON",
// Filtering & Search
"X-FieldFilter-*",
"X-SearchFilter-*",
"X-SearchOp-*",
"X-SearchOr-*",
"X-SearchAnd-*",
"X-SearchCols",
"X-Custom-SQL-W",
"X-Custom-SQL-W-*",
"X-Custom-SQL-Or",
"X-Custom-SQL-Or-*",
// Joins & Relations
"X-Preload",
"X-Preload-*",
"X-Expand",
"X-Expand-*",
"X-Custom-SQL-Join",
"X-Custom-SQL-Join-*",
// Sorting & Pagination
"X-Sort",
"X-Sort-*",
"X-Limit",
"X-Offset",
"X-Cursor-Forward",
"X-Cursor-Backward",
// Advanced Features
"X-AdvSQL-*",
"X-CQL-Sel-*",
"X-Distinct",
"X-SkipCount",
"X-SkipCache",
"X-Fetch-RowNumber",
"X-PKRow",
// Response Format
"X-SimpleAPI",
"X-DetailAPI",
"X-Syncfusion",
"X-Single-Record-As-Object",
// Transaction Control
"X-Transaction-Atomic",
// X-Files - comprehensive JSON configuration
"X-Files",
}
}
// SetCORSHeaders sets CORS headers on a response writer
func SetCORSHeaders(w ResponseWriter, r Request, config CORSConfig) {
// Set allowed origins
// if len(config.AllowedOrigins) > 0 {
// w.SetHeader("Access-Control-Allow-Origin", strings.Join(config.AllowedOrigins, ", "))
// }
// Todo origin list parsing
w.SetHeader("Access-Control-Allow-Origin", "*")
// Set allowed methods
if len(config.AllowedMethods) > 0 {
w.SetHeader("Access-Control-Allow-Methods", strings.Join(config.AllowedMethods, ", "))
}
// Set allowed headers
// if len(config.AllowedHeaders) > 0 {
// w.SetHeader("Access-Control-Allow-Headers", strings.Join(config.AllowedHeaders, ", "))
// }
w.SetHeader("Access-Control-Allow-Headers", "*")
// Set max age
if config.MaxAge > 0 {
w.SetHeader("Access-Control-Max-Age", fmt.Sprintf("%d", config.MaxAge))
}
// Allow credentials
w.SetHeader("Access-Control-Allow-Credentials", "true")
// Expose headers that clients can read
exposeHeaders := config.AllowedHeaders
exposeHeaders = append(exposeHeaders, "Content-Range", "X-Api-Range-Total", "X-Api-Range-Size")
w.SetHeader("Access-Control-Expose-Headers", strings.Join(exposeHeaders, ", "))
}
+97
View File
@@ -0,0 +1,97 @@
package common
// Example showing how to use the common handler interfaces
// This file demonstrates the handler interface hierarchy and usage patterns
// ProcessWithAnyHandler demonstrates using the base SpecHandler interface
// which works with any handler type (resolvespec, restheadspec, or funcspec)
func ProcessWithAnyHandler(handler SpecHandler) Database {
// All handlers expose GetDatabase() through the SpecHandler interface
return handler.GetDatabase()
}
// ProcessCRUDRequest demonstrates using the CRUDHandler interface
// which works with resolvespec.Handler and restheadspec.Handler
func ProcessCRUDRequest(handler CRUDHandler, w ResponseWriter, r Request, params map[string]string) {
// Both resolvespec and restheadspec handlers implement Handle()
handler.Handle(w, r, params)
}
// ProcessMetadataRequest demonstrates getting metadata from CRUD handlers
func ProcessMetadataRequest(handler CRUDHandler, w ResponseWriter, r Request, params map[string]string) {
// Both resolvespec and restheadspec handlers implement HandleGet()
handler.HandleGet(w, r, params)
}
// Example usage patterns (not executable, just for documentation):
/*
// Example 1: Using with resolvespec.Handler
func ExampleResolveSpec() {
db := // ... get database
registry := // ... get registry
handler := resolvespec.NewHandler(db, registry)
// Can be used as SpecHandler
var specHandler SpecHandler = handler
database := specHandler.GetDatabase()
// Can be used as CRUDHandler
var crudHandler CRUDHandler = handler
crudHandler.Handle(w, r, params)
crudHandler.HandleGet(w, r, params)
}
// Example 2: Using with restheadspec.Handler
func ExampleRestHeadSpec() {
db := // ... get database
registry := // ... get registry
handler := restheadspec.NewHandler(db, registry)
// Can be used as SpecHandler
var specHandler SpecHandler = handler
database := specHandler.GetDatabase()
// Can be used as CRUDHandler
var crudHandler CRUDHandler = handler
crudHandler.Handle(w, r, params)
crudHandler.HandleGet(w, r, params)
}
// Example 3: Using with funcspec.Handler
func ExampleFuncSpec() {
db := // ... get database
handler := funcspec.NewHandler(db)
// Can be used as SpecHandler
var specHandler SpecHandler = handler
database := specHandler.GetDatabase()
// Can be used as QueryHandler
var queryHandler QueryHandler = handler
// funcspec has different methods: SqlQueryList() and SqlQuery()
// which return HTTP handler functions
}
// Example 4: Polymorphic handler processing
func ProcessHandlers(handlers []SpecHandler) {
for _, handler := range handlers {
// All handlers expose the database
db := handler.GetDatabase()
// Type switch for specific handler types
switch h := handler.(type) {
case CRUDHandler:
// This is resolvespec or restheadspec
// Can call Handle() and HandleGet()
_ = h
case QueryHandler:
// This is funcspec
// Can call SqlQueryList() and SqlQuery()
_ = h
}
}
}
*/
+309
View File
@@ -0,0 +1,309 @@
package common
import (
"fmt"
"reflect"
"strconv"
"strings"
"github.com/bitechdev/ResolveSpec/pkg/logger"
)
// ValidateAndUnwrapModelResult contains the result of model validation
type ValidateAndUnwrapModelResult struct {
ModelType reflect.Type
Model interface{}
ModelPtr interface{}
OriginalType reflect.Type
}
// ValidateAndUnwrapModel validates that a model is a struct type and unwraps
// pointers, slices, and arrays to get to the base struct type.
// Returns an error if the model is not a valid struct type.
func ValidateAndUnwrapModel(model interface{}) (*ValidateAndUnwrapModelResult, error) {
modelType := reflect.TypeOf(model)
originalType := modelType
// Unwrap pointers, slices, and arrays to get to the base struct type
for modelType != nil && (modelType.Kind() == reflect.Pointer || modelType.Kind() == reflect.Slice || modelType.Kind() == reflect.Array) {
modelType = modelType.Elem()
}
// Validate that we have a struct type
if modelType == nil || modelType.Kind() != reflect.Struct {
return nil, fmt.Errorf("model must be a struct type, got %v. Ensure you register the struct (e.g., ModelCoreAccount{}) not a slice (e.g., []*ModelCoreAccount)", originalType)
}
// If the registered model was a pointer or slice, use the unwrapped struct type
if originalType != modelType {
model = reflect.New(modelType).Elem().Interface()
}
// Create a pointer to the model type for database operations
modelPtr := reflect.New(reflect.TypeOf(model)).Interface()
return &ValidateAndUnwrapModelResult{
ModelType: modelType,
Model: model,
ModelPtr: modelPtr,
OriginalType: originalType,
}, nil
}
// ExtractTagValue extracts the value for a given key from a struct tag string.
// It handles both semicolon and comma-separated tag formats (e.g., GORM and BUN tags).
// For tags like "json:name;validate:required" it will extract "name" for key "json".
// For tags like "rel:has-many,join:table" it will extract "table" for key "join".
func ExtractTagValue(tag, key string) string {
// Split by both semicolons and commas to handle different tag formats
// We need to be smart about this - commas can be part of values
// So we'll try semicolon first, then comma if needed
separators := []string{";", ","}
for _, sep := range separators {
parts := strings.Split(tag, sep)
for _, part := range parts {
part = strings.TrimSpace(part)
if strings.HasPrefix(part, key+":") {
return strings.TrimPrefix(part, key+":")
}
}
}
return ""
}
// GetRelationshipInfo analyzes a model type and extracts relationship metadata
// for a specific relation field identified by its JSON name.
// Returns nil if the field is not found or is not a valid relationship.
func GetRelationshipInfo(modelType reflect.Type, relationName string) *RelationshipInfo {
// Ensure we have a struct type
if modelType == nil || modelType.Kind() != reflect.Struct {
logger.Warn("Cannot get relationship info from non-struct type: %v", modelType)
return nil
}
for i := 0; i < modelType.NumField(); i++ {
field := modelType.Field(i)
jsonTag := field.Tag.Get("json")
jsonName := strings.Split(jsonTag, ",")[0]
if jsonName == relationName {
gormTag := field.Tag.Get("gorm")
bunTag := field.Tag.Get("bun")
info := &RelationshipInfo{
FieldName: field.Name,
JSONName: jsonName,
}
if strings.Contains(bunTag, "rel:") || strings.Contains(bunTag, "join:") {
//bun:"rel:has-many,join:rid_hub=rid_hub_division"
if strings.Contains(bunTag, "has-many") {
info.RelationType = "hasMany"
} else if strings.Contains(bunTag, "has-one") {
info.RelationType = "hasOne"
} else if strings.Contains(bunTag, "belongs-to") {
info.RelationType = "belongsTo"
} else if strings.Contains(bunTag, "many-to-many") {
info.RelationType = "many2many"
} else {
info.RelationType = "hasOne"
}
// Extract join info
joinPart := ExtractTagValue(bunTag, "join")
if joinPart != "" && info.RelationType == "many2many" {
// For many2many, the join part is the join table name
info.JoinTable = joinPart
} else if joinPart != "" {
// For other relations, parse foreignKey and references
joinParts := strings.Split(joinPart, "=")
if len(joinParts) == 2 {
info.ForeignKey = joinParts[0]
info.References = joinParts[1]
}
}
// Get related model type
if field.Type.Kind() == reflect.Slice {
elemType := field.Type.Elem()
if elemType.Kind() == reflect.Pointer {
elemType = elemType.Elem()
}
if elemType.Kind() == reflect.Struct {
info.RelatedModel = reflect.New(elemType).Elem().Interface()
}
} else if field.Type.Kind() == reflect.Pointer || field.Type.Kind() == reflect.Struct {
elemType := field.Type
if elemType.Kind() == reflect.Pointer {
elemType = elemType.Elem()
}
if elemType.Kind() == reflect.Struct {
info.RelatedModel = reflect.New(elemType).Elem().Interface()
}
}
return info
}
// Parse GORM tag to determine relationship type and keys
if strings.Contains(gormTag, "foreignKey") {
info.ForeignKey = ExtractTagValue(gormTag, "foreignKey")
info.References = ExtractTagValue(gormTag, "references")
// Determine if it's belongsTo or hasMany/hasOne
if field.Type.Kind() == reflect.Slice {
info.RelationType = "hasMany"
// Get the element type for slice
elemType := field.Type.Elem()
if elemType.Kind() == reflect.Pointer {
elemType = elemType.Elem()
}
if elemType.Kind() == reflect.Struct {
info.RelatedModel = reflect.New(elemType).Elem().Interface()
}
} else if field.Type.Kind() == reflect.Pointer || field.Type.Kind() == reflect.Struct {
info.RelationType = "belongsTo"
elemType := field.Type
if elemType.Kind() == reflect.Pointer {
elemType = elemType.Elem()
}
if elemType.Kind() == reflect.Struct {
info.RelatedModel = reflect.New(elemType).Elem().Interface()
}
}
} else if strings.Contains(gormTag, "many2many") {
info.RelationType = "many2many"
info.JoinTable = ExtractTagValue(gormTag, "many2many")
// Get the element type for many2many (always slice)
if field.Type.Kind() == reflect.Slice {
elemType := field.Type.Elem()
if elemType.Kind() == reflect.Pointer {
elemType = elemType.Elem()
}
if elemType.Kind() == reflect.Struct {
info.RelatedModel = reflect.New(elemType).Elem().Interface()
}
}
} else {
// Field has no GORM relationship tags, so it's not a relation
return nil
}
return info
}
}
return nil
}
// RelationPathToBunAlias converts a relation path (e.g., "Order.Customer") to a Bun alias format.
// It converts to lowercase and replaces dots with double underscores.
// For example: "Order.Customer" -> "order__customer"
func RelationPathToBunAlias(relationPath string) string {
if relationPath == "" {
return ""
}
// Convert to lowercase and replace dots with double underscores
alias := strings.ToLower(relationPath)
alias = strings.ReplaceAll(alias, ".", "__")
return alias
}
// ReplaceTableReferencesInSQL replaces references to a base table name in a SQL expression
// with the appropriate alias for the current preload level.
// For example, if baseTableName is "mastertaskitem" and targetAlias is "mal__mal",
// it will replace "mastertaskitem.rid_mastertaskitem" with "mal__mal.rid_mastertaskitem"
func ReplaceTableReferencesInSQL(sqlExpr, baseTableName, targetAlias string) string {
if sqlExpr == "" || baseTableName == "" || targetAlias == "" {
return sqlExpr
}
// Replace both quoted and unquoted table references
// Handle patterns like: tablename.column, "tablename".column, tablename."column", "tablename"."column"
// Pattern 1: tablename.column (unquoted)
result := strings.ReplaceAll(sqlExpr, baseTableName+".", targetAlias+".")
// Pattern 2: "tablename".column or "tablename"."column" (quoted table name)
result = strings.ReplaceAll(result, "\""+baseTableName+"\".", "\""+targetAlias+"\".")
return result
}
// GetTableNameFromModel extracts the table name from a model.
// It checks the bun tag first, then falls back to converting the struct name to snake_case.
func GetTableNameFromModel(model interface{}) string {
if model == nil {
return ""
}
modelType := reflect.TypeOf(model)
// Unwrap pointers
for modelType != nil && modelType.Kind() == reflect.Pointer {
modelType = modelType.Elem()
}
if modelType == nil || modelType.Kind() != reflect.Struct {
return ""
}
// Look for bun tag on embedded BaseModel
for i := 0; i < modelType.NumField(); i++ {
field := modelType.Field(i)
if field.Anonymous {
bunTag := field.Tag.Get("bun")
if strings.HasPrefix(bunTag, "table:") {
return strings.TrimPrefix(bunTag, "table:")
}
}
}
// Fallback: convert struct name to lowercase (simple heuristic)
// This handles cases like "MasterTaskItem" -> "mastertaskitem"
return strings.ToLower(modelType.Name())
}
// ConvertSliceForBun converts []interface{} values to PostgreSQL array literal strings.
// BUN's fallback appender for []interface{} is JSON encoding, which produces "[]" —
// invalid PostgreSQL array syntax. PostgreSQL expects "{}" for empty arrays and
// "{elem1,elem2}" for non-empty ones. All other value types are returned unchanged.
func ConvertSliceForBun(value interface{}) interface{} {
arr, ok := value.([]interface{})
if !ok {
return value
}
if len(arr) == 0 {
return "{}"
}
parts := make([]string, len(arr))
for i, elem := range arr {
switch e := elem.(type) {
case string:
needsQuote := e == "" || strings.ContainsAny(e, `,"\\{}`+"\t\n\r ")
if needsQuote {
e = strings.ReplaceAll(e, `\`, `\\`)
e = strings.ReplaceAll(e, `"`, `""`)
parts[i] = `"` + e + `"`
} else {
parts[i] = e
}
case float64:
if e == float64(int64(e)) {
parts[i] = strconv.FormatInt(int64(e), 10)
} else {
parts[i] = strconv.FormatFloat(e, 'f', -1, 64)
}
case bool:
if e {
parts[i] = "t"
} else {
parts[i] = "f"
}
case nil:
parts[i] = "NULL"
default:
parts[i] = fmt.Sprintf("%v", e)
}
}
return "{" + strings.Join(parts, ",") + "}"
}
+311
View File
@@ -0,0 +1,311 @@
package common
import (
"context"
"encoding/json"
"io"
"net/http"
)
// Database interface designed to work with both GORM and Bun
type Database interface {
// Core query operations
NewSelect() SelectQuery
NewInsert() InsertQuery
NewUpdate() UpdateQuery
NewDelete() DeleteQuery
// Raw SQL execution
Exec(ctx context.Context, query string, args ...interface{}) (Result, error)
Query(ctx context.Context, dest interface{}, query string, args ...interface{}) error
// Transaction support
BeginTx(ctx context.Context) (Database, error)
CommitTx(ctx context.Context) error
RollbackTx(ctx context.Context) error
RunInTransaction(ctx context.Context, fn func(Database) error) error
// GetUnderlyingDB returns the underlying database connection
// For GORM, this returns *gorm.DB
// For Bun, this returns *bun.DB
// This is useful for provider-specific features like PostgreSQL NOTIFY/LISTEN
GetUnderlyingDB() interface{}
// DriverName returns the canonical name of the underlying database driver.
// Possible values: "postgres", "sqlite", "mssql", "mysql".
// All adapters normalise vendor-specific strings (e.g. Bun's "pg", GORM's
// "sqlserver") to the values above before returning.
DriverName() string
}
// SelectQuery interface for building SELECT queries (compatible with both GORM and Bun)
type SelectQuery interface {
Model(model interface{}) SelectQuery
Table(table string) SelectQuery
Column(columns ...string) SelectQuery
ColumnExpr(query string, args ...interface{}) SelectQuery
Where(query string, args ...interface{}) SelectQuery
WhereOr(query string, args ...interface{}) SelectQuery
Join(query string, args ...interface{}) SelectQuery
LeftJoin(query string, args ...interface{}) SelectQuery
Preload(relation string, conditions ...interface{}) SelectQuery
PreloadRelation(relation string, apply ...func(SelectQuery) SelectQuery) SelectQuery
JoinRelation(relation string, apply ...func(SelectQuery) SelectQuery) SelectQuery
Order(order string) SelectQuery
OrderExpr(order string, args ...interface{}) SelectQuery
Limit(n int) SelectQuery
Offset(n int) SelectQuery
Group(group string) SelectQuery
Having(having string, args ...interface{}) SelectQuery
// Execution methods
Scan(ctx context.Context, dest interface{}) error
ScanModel(ctx context.Context) error
Count(ctx context.Context) (int, error)
Exists(ctx context.Context) (bool, error)
}
// InsertQuery interface for building INSERT queries
type InsertQuery interface {
Model(model interface{}) InsertQuery
Table(table string) InsertQuery
Value(column string, value interface{}) InsertQuery
OnConflict(action string) InsertQuery
Returning(columns ...string) InsertQuery
// Execution
Exec(ctx context.Context) (Result, error)
Scan(ctx context.Context, dest interface{}) error
}
// UpdateQuery interface for building UPDATE queries
type UpdateQuery interface {
Model(model interface{}) UpdateQuery
Table(table string) UpdateQuery
Set(column string, value interface{}) UpdateQuery
SetMap(values map[string]interface{}) UpdateQuery
Where(query string, args ...interface{}) UpdateQuery
Returning(columns ...string) UpdateQuery
// Execution
Exec(ctx context.Context) (Result, error)
}
// DeleteQuery interface for building DELETE queries
type DeleteQuery interface {
Model(model interface{}) DeleteQuery
Table(table string) DeleteQuery
Where(query string, args ...interface{}) DeleteQuery
// Execution
Exec(ctx context.Context) (Result, error)
}
// Result interface for query execution results
type Result interface {
RowsAffected() int64
LastInsertId() (int64, error)
}
// ModelRegistry manages model registration and retrieval
type ModelRegistry interface {
RegisterModel(name string, model interface{}) error
GetModel(name string) (interface{}, error)
GetAllModels() map[string]interface{}
GetModelByEntity(schema, entity string) (interface{}, error)
}
// Router interface for HTTP router abstraction
type Router interface {
HandleFunc(pattern string, handler HTTPHandlerFunc) RouteRegistration
ServeHTTP(w ResponseWriter, r Request)
}
// RouteRegistration allows method chaining for route configuration
type RouteRegistration interface {
Methods(methods ...string) RouteRegistration
PathPrefix(prefix string) RouteRegistration
}
// Request interface abstracts HTTP request
type Request interface {
Method() string
URL() string
Header(key string) string
AllHeaders() map[string]string // Get all headers as a map
Body() ([]byte, error)
PathParam(key string) string
QueryParam(key string) string
AllQueryParams() map[string]string // Get all query parameters as a map
UnderlyingRequest() *http.Request // Get the underlying *http.Request for forwarding to other handlers
}
// ResponseWriter interface abstracts HTTP response
type ResponseWriter interface {
SetHeader(key, value string)
WriteHeader(statusCode int)
Write(data []byte) (int, error)
WriteJSON(data interface{}) error
UnderlyingResponseWriter() http.ResponseWriter // Get the underlying http.ResponseWriter for forwarding to other handlers
}
// HTTPHandlerFunc type for HTTP handlers
type HTTPHandlerFunc func(ResponseWriter, Request)
// WrapHTTPRequest wraps standard http.ResponseWriter and *http.Request into common interfaces
func WrapHTTPRequest(w http.ResponseWriter, r *http.Request) (ResponseWriter, Request) {
return &StandardResponseWriter{w: w}, &StandardRequest{r: r}
}
// StandardResponseWriter adapts http.ResponseWriter to ResponseWriter interface
type StandardResponseWriter struct {
w http.ResponseWriter
status int
}
func (s *StandardResponseWriter) SetHeader(key, value string) {
s.w.Header().Set(key, value)
}
func (s *StandardResponseWriter) WriteHeader(statusCode int) {
s.status = statusCode
s.w.WriteHeader(statusCode)
}
func (s *StandardResponseWriter) Write(data []byte) (int, error) {
return s.w.Write(data)
}
func (s *StandardResponseWriter) WriteJSON(data interface{}) error {
s.SetHeader("Content-Type", "application/json")
enc := json.NewEncoder(s.w)
enc.SetEscapeHTML(false)
return enc.Encode(data)
}
func (s *StandardResponseWriter) UnderlyingResponseWriter() http.ResponseWriter {
return s.w
}
// StandardRequest adapts *http.Request to Request interface
type StandardRequest struct {
r *http.Request
body []byte
}
func (s *StandardRequest) Method() string {
return s.r.Method
}
func (s *StandardRequest) URL() string {
return s.r.URL.String()
}
func (s *StandardRequest) Header(key string) string {
return s.r.Header.Get(key)
}
func (s *StandardRequest) AllHeaders() map[string]string {
headers := make(map[string]string)
for key, values := range s.r.Header {
if len(values) > 0 {
headers[key] = values[0]
}
}
return headers
}
func (s *StandardRequest) Body() ([]byte, error) {
if s.body != nil {
return s.body, nil
}
if s.r.Body == nil {
return nil, nil
}
defer s.r.Body.Close()
body, err := io.ReadAll(s.r.Body)
if err != nil {
return nil, err
}
s.body = body
return body, nil
}
func (s *StandardRequest) PathParam(key string) string {
// Standard http.Request doesn't have path params
// This should be set by the router
return ""
}
func (s *StandardRequest) QueryParam(key string) string {
return s.r.URL.Query().Get(key)
}
func (s *StandardRequest) AllQueryParams() map[string]string {
params := make(map[string]string)
for key, values := range s.r.URL.Query() {
if len(values) > 0 {
params[key] = values[0]
}
}
return params
}
func (s *StandardRequest) UnderlyingRequest() *http.Request {
return s.r
}
// TableNameProvider interface for models that provide table names
type TableNameProvider interface {
TableName() string
}
type TableAliasProvider interface {
TableAlias() string
}
// PrimaryKeyNameProvider interface for models that provide primary key column names
type PrimaryKeyNameProvider interface {
GetIDName() string
}
// SchemaProvider interface for models that provide schema names
type SchemaProvider interface {
SchemaName() string
}
// SpecHandler interface represents common functionality across all spec handlers
// This is the base interface implemented by:
// - resolvespec.Handler: Handles CRUD operations via request body with explicit operation field
// - restheadspec.Handler: Handles CRUD operations via HTTP methods (GET/POST/PUT/DELETE)
// - funcspec.Handler: Handles custom SQL query execution with dynamic parameters
//
// The interface hierarchy is:
//
// SpecHandler (base)
// ├── CRUDHandler (resolvespec, restheadspec)
// └── QueryHandler (funcspec)
type SpecHandler interface {
// GetDatabase returns the underlying database connection
GetDatabase() Database
}
// CRUDHandler interface for handlers that support CRUD operations
// This is implemented by resolvespec.Handler and restheadspec.Handler
type CRUDHandler interface {
SpecHandler
// Handle processes API requests through router-agnostic interface
Handle(w ResponseWriter, r Request, params map[string]string)
// HandleGet processes GET requests for metadata
HandleGet(w ResponseWriter, r Request, params map[string]string)
}
// QueryHandler interface for handlers that execute SQL queries
// This is implemented by funcspec.Handler
// Note: funcspec uses standard http.ResponseWriter and *http.Request instead of common interfaces
type QueryHandler interface {
SpecHandler
// Methods are defined in funcspec package due to different function signature requirements
}
+645
View File
@@ -0,0 +1,645 @@
package common
import (
"context"
"fmt"
"reflect"
"strings"
"github.com/bitechdev/ResolveSpec/pkg/logger"
"github.com/bitechdev/ResolveSpec/pkg/reflection"
)
// CRUDRequestProvider interface for models that provide CRUD request strings
type CRUDRequestProvider interface {
GetRequest() string
}
// RelationshipInfoProvider interface for handlers that can provide relationship info
type RelationshipInfoProvider interface {
GetRelationshipInfo(modelType reflect.Type, relationName string) *RelationshipInfo
}
// NestedCUDProcessor handles recursive processing of nested object graphs
type NestedCUDProcessor struct {
db Database
registry ModelRegistry
relationshipHelper RelationshipInfoProvider
}
// NewNestedCUDProcessor creates a new nested CUD processor
func NewNestedCUDProcessor(db Database, registry ModelRegistry, relationshipHelper RelationshipInfoProvider) *NestedCUDProcessor {
return &NestedCUDProcessor{
db: db,
registry: registry,
relationshipHelper: relationshipHelper,
}
}
// ProcessResult contains the result of processing a CUD operation
type ProcessResult struct {
ID interface{} // The ID of the processed record
AffectedRows int64 // Number of rows affected
Data map[string]interface{} // The processed data
RelationData map[string]interface{} // Data from processed relations
}
// ProcessNestedCUD recursively processes nested object graphs for Create, Update, Delete operations
// with automatic foreign key resolution
func (p *NestedCUDProcessor) ProcessNestedCUD(
ctx context.Context,
operation string, // "insert", "update", or "delete"
data map[string]interface{},
model interface{},
parentIDs map[string]interface{}, // Parent IDs for foreign key resolution
tableName string,
) (*ProcessResult, error) {
logger.Info("Processing nested CUD: operation=%s, table=%s", operation, tableName)
result := &ProcessResult{
Data: make(map[string]interface{}),
RelationData: make(map[string]interface{}),
}
// Check if data has a _request field that overrides the operation
if requestOp := p.extractCRUDRequest(data); requestOp != "" {
logger.Debug("Found _request override: %s", requestOp)
operation = requestOp
}
// Get model type for reflection
modelType := reflect.TypeOf(model)
for modelType != nil && (modelType.Kind() == reflect.Pointer || modelType.Kind() == reflect.Slice || modelType.Kind() == reflect.Array) {
modelType = modelType.Elem()
}
if modelType == nil || modelType.Kind() != reflect.Struct {
logger.Error("Invalid model type: operation=%s, table=%s, modelType=%v, expected struct", operation, tableName, modelType)
return nil, fmt.Errorf("model must be a struct type, got %v", modelType)
}
// Separate relation fields from regular fields
relationFields := make(map[string]*RelationshipInfo)
regularData := make(map[string]interface{})
for key, value := range data {
// Skip _request field in actual data processing
if key == "_request" {
continue
}
// Check if this field is a relation
relInfo := p.relationshipHelper.GetRelationshipInfo(modelType, key)
if relInfo != nil {
relationFields[key] = relInfo
result.RelationData[key] = value
} else {
regularData[key] = value
}
}
// Filter regularData to only include fields that exist in the model,
// and translate JSON keys to their actual database column names.
regularData = p.filterValidFields(regularData, model)
// Inject parent IDs for foreign key resolution
p.injectForeignKeys(regularData, modelType, parentIDs)
// Get the primary key name for this model
pkName := reflection.GetPrimaryKeyName(model)
// Check if we have any data to process (besides _request)
hasData := len(regularData) > 0
// Process based on operation
switch strings.ToLower(operation) {
case "insert", "create", "add":
// Only perform insert if we have data to insert
if hasData {
id, err := p.processInsert(ctx, regularData, tableName)
if err != nil {
logger.Error("Insert failed for table=%s, data=%+v, error=%v", tableName, regularData, err)
return nil, fmt.Errorf("insert failed: %w", err)
}
result.ID = id
result.AffectedRows = 1
result.Data = regularData
// Re-select the inserted row so result.Data reflects DB-generated defaults.
if row, err := p.processSelect(ctx, tableName, id); err != nil {
logger.Warn("Select after insert failed: table=%s, id=%v, error=%v", tableName, id, err)
} else if len(row) > 0 {
result.Data = row
}
// Process child relations after parent insert (to get parent ID)
if err := p.processChildRelations(ctx, "insert", id, relationFields, result.RelationData, modelType, parentIDs); err != nil {
logger.Error("Failed to process child relations after insert: table=%s, parentID=%v, relations=%+v, error=%v", tableName, id, relationFields, err)
return nil, fmt.Errorf("failed to process child relations: %w", err)
}
} else {
logger.Debug("Skipping insert for %s - no data columns besides _request", tableName)
}
case "update", "change", "modify":
// Only perform update if we have data to update
if reflection.IsEmptyValue(data[pkName]) {
logger.Warn("Skipping update for %s - no primary key", tableName)
return result, nil
}
if hasData {
rows, err := p.processUpdate(ctx, regularData, tableName, data[pkName])
if err != nil {
logger.Error("Update failed for table=%s, id=%v, data=%+v, error=%v", tableName, data[pkName], regularData, err)
return nil, fmt.Errorf("update failed: %w", err)
}
result.ID = data[pkName]
result.AffectedRows = rows
result.Data = regularData
// Re-select the updated row so result.Data reflects current DB state.
if row, err := p.processSelect(ctx, tableName, result.ID); err != nil {
logger.Warn("Select after update failed: table=%s, id=%v, error=%v", tableName, result.ID, err)
} else if len(row) > 0 {
result.Data = row
}
// Process child relations for update
if err := p.processChildRelations(ctx, "update", data[pkName], relationFields, result.RelationData, modelType, parentIDs); err != nil {
logger.Error("Failed to process child relations after update: table=%s, parentID=%v, relations=%+v, error=%v", tableName, data[pkName], regularData, err)
return nil, fmt.Errorf("failed to process child relations: %w", err)
}
} else {
logger.Debug("Skipping update for %s - no data columns besides _request", tableName)
result.ID = data[pkName]
}
case "delete", "remove":
if reflection.IsEmptyValue(data[pkName]) {
logger.Warn("Skipping delete for %s - no primary key", tableName)
return result, nil
}
// Process child relations first (for referential integrity)
if err := p.processChildRelations(ctx, "delete", data[pkName], relationFields, result.RelationData, modelType, parentIDs); err != nil {
logger.Error("Failed to process child relations before delete: table=%s, id=%v, relations=%+v, error=%v", tableName, data[pkName], relationFields, err)
return nil, fmt.Errorf("failed to process child relations: %w", err)
}
rows, err := p.processDelete(ctx, tableName, data[pkName])
if err != nil {
logger.Error("Delete failed for table=%s, id=%v, error=%v", tableName, data[pkName], err)
return nil, fmt.Errorf("delete failed: %w", err)
}
result.ID = data[pkName]
result.AffectedRows = rows
result.Data = regularData
default:
logger.Error("Unsupported operation: %s for table=%s", operation, tableName)
return nil, fmt.Errorf("unsupported operation: %s", operation)
}
logger.Info("Nested CUD completed: operation=%s, id=%v, rows=%d", operation, result.ID, result.AffectedRows)
return result, nil
}
// extractCRUDRequest extracts the request field from data if present
func (p *NestedCUDProcessor) extractCRUDRequest(data map[string]interface{}) string {
if request, ok := data["_request"]; ok {
if requestStr, ok := request.(string); ok {
return strings.ToLower(strings.TrimSpace(requestStr))
}
}
return ""
}
// filterValidFields filters input data to only include fields that exist in the model,
// and translates JSON key names to their actual database column names.
// For example, a field tagged `json:"_changed_date" bun:"changed_date"` will be
// included in the result as "changed_date", not "_changed_date".
func (p *NestedCUDProcessor) filterValidFields(data map[string]interface{}, model interface{}) map[string]interface{} {
if len(data) == 0 {
return data
}
modelType := reflect.TypeOf(model)
for modelType != nil && (modelType.Kind() == reflect.Pointer || modelType.Kind() == reflect.Slice || modelType.Kind() == reflect.Array) {
modelType = modelType.Elem()
}
if modelType == nil || modelType.Kind() != reflect.Struct {
return data
}
// Build a mapping from JSON key -> DB column name for all writable fields.
// This both validates which fields belong to the model and translates their names
// to the correct column names for use in SQL insert/update queries.
jsonToDBCol := reflection.BuildJSONToDBColumnMap(modelType)
filteredData := make(map[string]interface{})
for key, value := range data {
dbColName, exists := jsonToDBCol[key]
if exists {
filteredData[dbColName] = value
} else {
logger.Debug("Skipping invalid field '%s' - not found in model %v", key, modelType)
}
}
return filteredData
}
// injectForeignKeys injects parent IDs into data for foreign key fields.
// data is expected to be keyed by DB column names (as returned by filterValidFields).
func (p *NestedCUDProcessor) injectForeignKeys(data map[string]interface{}, modelType reflect.Type, parentIDs map[string]interface{}) {
if len(parentIDs) == 0 {
return
}
pkCol := reflection.GetPrimaryKeyName(reflect.New(modelType).Interface())
for parentKey, parentID := range parentIDs {
dbColNames := reflection.GetForeignKeyColumn(modelType, parentKey)
if len(dbColNames) == 0 {
// No explicit tag found — fall back to naming convention by scanning scalar fields.
for i := 0; i < modelType.NumField(); i++ {
field := modelType.Field(i)
jsonName := strings.Split(field.Tag.Get("json"), ",")[0]
if strings.EqualFold(jsonName, "rid"+parentKey) ||
strings.EqualFold(jsonName, "rid_"+parentKey) ||
strings.EqualFold(jsonName, "id_"+parentKey) ||
strings.EqualFold(jsonName, parentKey+"_id") ||
strings.EqualFold(jsonName, parentKey+"id") ||
strings.EqualFold(field.Name, parentKey+"ID") {
dbColNames = []string{reflection.GetColumnName(field)}
break
}
}
}
for _, dbColName := range dbColNames {
if pkCol != "" && strings.EqualFold(dbColName, pkCol) {
continue
}
if _, exists := data[dbColName]; !exists {
logger.Debug("Injecting foreign key: %s = %v", dbColName, parentID)
data[dbColName] = parentID
}
}
}
}
// processInsert handles insert operation
func (p *NestedCUDProcessor) processInsert(
ctx context.Context,
data map[string]interface{},
tableName string,
) (interface{}, error) {
logger.Debug("Inserting into %s with data: %+v", tableName, data)
query := p.db.NewInsert().Table(tableName)
for key, value := range data {
query = query.Value(key, ConvertSliceForBun(value))
}
pkName := reflection.GetPrimaryKeyName(tableName)
query = query.Returning(pkName)
var id interface{}
if err := query.Scan(ctx, &id); err != nil {
logger.Error("Insert execution failed: table=%s, data=%+v, error=%v", tableName, data, err)
return nil, fmt.Errorf("insert exec failed: %w", err)
}
logger.Debug("Insert successful, ID: %v", id)
return id, nil
}
// processSelect fetches the row identified by id from tableName into a flat map.
// Used to populate result.Data with the actual DB state after insert/update.
func (p *NestedCUDProcessor) processSelect(ctx context.Context, tableName string, id interface{}) (map[string]interface{}, error) {
pkName := reflection.GetPrimaryKeyName(tableName)
var row map[string]interface{}
if err := p.db.NewSelect().
Table(tableName).
Where(fmt.Sprintf("%s = ?", QuoteIdent(pkName)), id).
Scan(ctx, &row); err != nil {
return nil, fmt.Errorf("select after write failed: %w", err)
}
return row, nil
}
// processUpdate handles update operation
func (p *NestedCUDProcessor) processUpdate(
ctx context.Context,
data map[string]interface{},
tableName string,
id interface{},
) (int64, error) {
if id == nil {
logger.Error("Update requires an ID: table=%s, data=%+v", tableName, data)
return 0, fmt.Errorf("update requires an ID")
}
logger.Debug("Updating %s with ID %v, data: %+v", tableName, id, data)
query := p.db.NewUpdate().Table(tableName).SetMap(data).Where(fmt.Sprintf("%s = ?", QuoteIdent(reflection.GetPrimaryKeyName(tableName))), id)
result, err := query.Exec(ctx)
if err != nil {
logger.Error("Update execution failed: table=%s, id=%v, data=%+v, error=%v", tableName, id, data, err)
return 0, fmt.Errorf("update exec failed: %w", err)
}
rows := result.RowsAffected()
logger.Debug("Update successful, rows affected: %d", rows)
return rows, nil
}
// processDelete handles delete operation
func (p *NestedCUDProcessor) processDelete(ctx context.Context, tableName string, id interface{}) (int64, error) {
if id == nil {
logger.Error("Delete requires an ID: table=%s", tableName)
return 0, fmt.Errorf("delete requires an ID")
}
logger.Debug("Deleting from %s with ID %v", tableName, id)
query := p.db.NewDelete().Table(tableName).Where(fmt.Sprintf("%s = ?", QuoteIdent(reflection.GetPrimaryKeyName(tableName))), id)
result, err := query.Exec(ctx)
if err != nil {
logger.Error("Delete execution failed: table=%s, id=%v, error=%v", tableName, id, err)
return 0, fmt.Errorf("delete exec failed: %w", err)
}
rows := result.RowsAffected()
logger.Debug("Delete successful, rows affected: %d", rows)
return rows, nil
}
// processChildRelations recursively processes child relations
func (p *NestedCUDProcessor) processChildRelations(
ctx context.Context,
operation string,
parentID interface{},
relationFields map[string]*RelationshipInfo,
relationData map[string]interface{},
parentModelType reflect.Type,
incomingParentIDs map[string]interface{}, // IDs from all ancestors
) error {
for relationName, relInfo := range relationFields {
relationValue, exists := relationData[relationName]
if !exists || relationValue == nil {
continue
}
logger.Debug("Processing relation: %s, type: %s", relationName, relInfo.RelationType)
// Get the related model
field, found := parentModelType.FieldByName(relInfo.FieldName)
if !found {
logger.Error("Field %s not found in model type %v for relation %s", relInfo.FieldName, parentModelType, relationName)
continue
}
// Get the model type for the relation
relatedModelType := field.Type
if relatedModelType.Kind() == reflect.Slice {
relatedModelType = relatedModelType.Elem()
}
if relatedModelType.Kind() == reflect.Pointer {
relatedModelType = relatedModelType.Elem()
}
// Create an instance of the related model
relatedModel := reflect.New(relatedModelType).Elem().Interface()
// Get table name for related model
relatedTableName := p.getTableNameForModel(relatedModel, relInfo.JSONName)
// Prepare parent IDs for foreign key injection
// Start by copying all incoming parent IDs (from ancestors)
parentIDs := make(map[string]interface{})
for k, v := range incomingParentIDs {
parentIDs[k] = v
}
logger.Debug("Inherited %d parent IDs from ancestors: %+v", len(incomingParentIDs), incomingParentIDs)
// Add the current parent's primary key to the parentIDs map
// This ensures nested children have access to all ancestor IDs
if parentID != nil && parentModelType != nil {
// Get the parent model's primary key field name
parentPKFieldName := reflection.GetPrimaryKeyName(parentModelType)
if parentPKFieldName != "" {
// Get the JSON name for the primary key field
parentPKJSONName := reflection.GetJSONNameForField(parentModelType, parentPKFieldName)
baseName := ""
if len(parentPKJSONName) > 1 {
baseName = parentPKJSONName
} else {
// Add parent's PK to the map using the base model name
baseName = strings.TrimSuffix(parentPKFieldName, "ID")
baseName = strings.TrimSuffix(strings.ToLower(baseName), "_id")
if baseName == "" {
baseName = "parent"
}
}
parentIDs[baseName] = parentID
logger.Debug("Added current parent PK to parentIDs map: %s=%v (from field %s)", baseName, parentID, parentPKFieldName)
}
}
// Also add the foreign key reference if specified
if relInfo.ForeignKey != "" && parentID != nil {
// Extract the base name from foreign key (e.g., "DepartmentID" -> "Department")
baseName := strings.TrimSuffix(relInfo.ForeignKey, "ID")
baseName = strings.TrimSuffix(strings.ToLower(baseName), "_id")
// Only add if different from what we already added
if _, exists := parentIDs[baseName]; !exists {
parentIDs[baseName] = parentID
logger.Debug("Added foreign key to parentIDs map: %s=%v (from FK %s)", baseName, parentID, relInfo.ForeignKey)
}
}
logger.Debug("Final parentIDs map for relation %s: %+v", relationName, parentIDs)
// Determine which field name to use for setting parent ID in child data
// Priority: Use foreign key field name if specified
var foreignKeyFieldName string
if relInfo.ForeignKey != "" {
// For has-many/has-one: join:parentCol=childCol
// ForeignKey = parent side, References = child side (where we actually set the value)
childField := relInfo.ForeignKey
if (relInfo.RelationType == "hasMany" || relInfo.RelationType == "hasOne") && relInfo.References != "" {
childField = relInfo.References
}
foreignKeyFieldName = reflection.GetJSONNameForField(relatedModelType, childField)
if foreignKeyFieldName == "" {
foreignKeyFieldName = strings.ToLower(childField)
}
logger.Debug("Using foreign key field for direct assignment: %s (from FK %s -> child %s)", foreignKeyFieldName, relInfo.ForeignKey, childField)
}
// Get the primary key name for the child model to avoid overwriting it in recursive relationships
childPKName := reflection.GetPrimaryKeyName(relatedModel)
childPKFieldName := reflection.GetJSONNameForField(relatedModelType, childPKName)
if childPKFieldName == "" {
childPKFieldName = strings.ToLower(childPKName)
}
logger.Debug("Processing relation with foreignKeyField=%s, childPK=%s", foreignKeyFieldName, childPKFieldName)
// Process based on relation type and data structure
switch v := relationValue.(type) {
case map[string]interface{}:
// Single related object - directly set foreign key if specified
// IMPORTANT: In recursive relationships, don't overwrite the primary key
if parentID != nil && foreignKeyFieldName != "" && foreignKeyFieldName != childPKFieldName {
v[foreignKeyFieldName] = parentID
logger.Debug("Set foreign key in single relation: %s=%v", foreignKeyFieldName, parentID)
} else if foreignKeyFieldName == childPKFieldName {
logger.Debug("Skipping foreign key assignment - same as primary key (recursive relationship): %s", foreignKeyFieldName)
}
_, err := p.ProcessNestedCUD(ctx, operation, v, relatedModel, parentIDs, relatedTableName)
if err != nil {
logger.Error("Failed to process single relation: name=%s, table=%s, operation=%s, parentID=%v, data=%+v, error=%v",
relationName, relatedTableName, operation, parentID, v, err)
return fmt.Errorf("failed to process relation %s: %w", relationName, err)
}
case []interface{}:
// Multiple related objects
for i, item := range v {
if itemMap, ok := item.(map[string]interface{}); ok {
// Directly set foreign key if specified
// IMPORTANT: In recursive relationships, don't overwrite the primary key
if parentID != nil && foreignKeyFieldName != "" && foreignKeyFieldName != childPKFieldName {
itemMap[foreignKeyFieldName] = parentID
logger.Debug("Set foreign key in relation array[%d]: %s=%v", i, foreignKeyFieldName, parentID)
} else if foreignKeyFieldName == childPKFieldName {
logger.Debug("Skipping foreign key assignment in array[%d] - same as primary key (recursive relationship): %s", i, foreignKeyFieldName)
}
_, err := p.ProcessNestedCUD(ctx, operation, itemMap, relatedModel, parentIDs, relatedTableName)
if err != nil {
logger.Error("Failed to process relation array item: name=%s[%d], table=%s, operation=%s, parentID=%v, data=%+v, error=%v",
relationName, i, relatedTableName, operation, parentID, itemMap, err)
return fmt.Errorf("failed to process relation %s[%d]: %w", relationName, i, err)
}
} else {
logger.Warn("Relation array item is not a map: name=%s[%d], type=%T", relationName, i, item)
}
}
case []map[string]interface{}:
// Multiple related objects (typed slice)
for i, itemMap := range v {
// Directly set foreign key if specified
// IMPORTANT: In recursive relationships, don't overwrite the primary key
if parentID != nil && foreignKeyFieldName != "" && foreignKeyFieldName != childPKFieldName {
itemMap[foreignKeyFieldName] = parentID
logger.Debug("Set foreign key in relation typed array[%d]: %s=%v", i, foreignKeyFieldName, parentID)
} else if foreignKeyFieldName == childPKFieldName {
logger.Debug("Skipping foreign key assignment in typed array[%d] - same as primary key (recursive relationship): %s", i, foreignKeyFieldName)
}
_, err := p.ProcessNestedCUD(ctx, operation, itemMap, relatedModel, parentIDs, relatedTableName)
if err != nil {
logger.Error("Failed to process relation typed array item: name=%s[%d], table=%s, operation=%s, parentID=%v, data=%+v, error=%v",
relationName, i, relatedTableName, operation, parentID, itemMap, err)
return fmt.Errorf("failed to process relation %s[%d]: %w", relationName, i, err)
}
}
default:
logger.Error("Unsupported relation data type: name=%s, type=%T, value=%+v", relationName, relationValue, relationValue)
}
}
return nil
}
// getTableNameForModel gets the table name for a model
func (p *NestedCUDProcessor) getTableNameForModel(model interface{}, defaultName string) string {
if provider, ok := model.(TableNameProvider); ok {
tableName := provider.TableName()
if tableName != "" {
return tableName
}
}
return defaultName
}
// ShouldUseNestedProcessor determines if we should use nested CUD processing
// It recursively checks if the data contains:
// 1. A _request field at any level, OR
// 2. Nested relations that themselves contain further nested relations or _request fields
// This ensures nested processing is only used when there are deeply nested operations
func ShouldUseNestedProcessor(data map[string]interface{}, model interface{}, relationshipHelper RelationshipInfoProvider) bool {
return shouldUseNestedProcessorDepth(data, model, relationshipHelper, 0)
}
// shouldUseNestedProcessorDepth is the internal recursive implementation with depth tracking
func shouldUseNestedProcessorDepth(data map[string]interface{}, model interface{}, relationshipHelper RelationshipInfoProvider, depth int) bool {
// Check for _request field
if _, hasCRUDRequest := data["_request"]; hasCRUDRequest {
return true
}
// Get model type
modelType := reflect.TypeOf(model)
for modelType != nil && (modelType.Kind() == reflect.Pointer || modelType.Kind() == reflect.Slice || modelType.Kind() == reflect.Array) {
modelType = modelType.Elem()
}
if modelType == nil || modelType.Kind() != reflect.Struct {
return false
}
// Check if data contains any fields that are relations (nested objects or arrays)
for key, value := range data {
// Skip _request and regular scalar fields
if key == "_request" {
continue
}
// Check if this field is a relation in the model
relInfo := relationshipHelper.GetRelationshipInfo(modelType, key)
if relInfo != nil {
// Check if the value is actually nested data (object or array)
switch v := value.(type) {
case map[string]interface{}, []interface{}, []map[string]interface{}:
// If we're already at a nested level (depth > 0) and found a relation,
// that means we have multi-level nesting, so return true
if depth > 0 {
return true
}
// At depth 0, recurse to check if the nested data has further nesting
switch typedValue := v.(type) {
case map[string]interface{}:
if shouldUseNestedProcessorDepth(typedValue, relInfo.RelatedModel, relationshipHelper, depth+1) {
return true
}
case []interface{}:
for _, item := range typedValue {
if itemMap, ok := item.(map[string]interface{}); ok {
if shouldUseNestedProcessorDepth(itemMap, relInfo.RelatedModel, relationshipHelper, depth+1) {
return true
}
}
}
case []map[string]interface{}:
for _, itemMap := range typedValue {
if shouldUseNestedProcessorDepth(itemMap, relInfo.RelatedModel, relationshipHelper, depth+1) {
return true
}
}
}
}
}
}
return false
}
File diff suppressed because it is too large Load Diff
+153
View File
@@ -0,0 +1,153 @@
package common
// SQLError wraps a database error together with the SQL that caused it,
// so callers can surface the query in API error responses for easier debugging.
type SQLError struct {
Err error
SQL string
}
func (e *SQLError) Error() string { return e.Err.Error() }
func (e *SQLError) Unwrap() error { return e.Err }
// WrapSQLError wraps err with the given SQL. If err is nil it returns nil.
func WrapSQLError(err error, sql string) error {
if err == nil {
return nil
}
return &SQLError{Err: err, SQL: sql}
}
type RequestBody struct {
Operation string `json:"operation"`
Data interface{} `json:"data"`
ID *int64 `json:"id"`
Options RequestOptions `json:"options"`
}
type RequestOptions struct {
Preload []PreloadOption `json:"preload"`
Columns []string `json:"columns"`
OmitColumns []string `json:"omit_columns"`
Filters []FilterOption `json:"filters"`
Sort []SortOption `json:"sort"`
Limit *int `json:"limit"`
Offset *int `json:"offset"`
CustomOperators []CustomOperator `json:"customOperators"`
ComputedColumns []ComputedColumn `json:"computedColumns"`
Parameters []Parameter `json:"parameters"`
// Cursor pagination
CursorForward string `json:"cursor_forward"`
CursorBackward string `json:"cursor_backward"`
FetchRowNumber *string `json:"fetch_row_number"`
// Join table aliases (used for validation of prefixed columns in filters/sorts)
// Not serialized to JSON as it's internal validation state
JoinAliases []string `json:"-"`
}
type Parameter struct {
Name string `json:"name"`
Value string `json:"value"`
Sequence *int `json:"sequence"`
}
type PreloadOption struct {
Relation string `json:"relation"`
TableName string `json:"table_name"` // Actual database table name (e.g., "mastertaskitem")
Columns []string `json:"columns"`
OmitColumns []string `json:"omit_columns"`
Sort []SortOption `json:"sort"`
Filters []FilterOption `json:"filters"`
Where string `json:"where"`
Limit *int `json:"limit"`
Offset *int `json:"offset"`
Updatable *bool `json:"updateable"` // if true, the relation can be updated
ComputedQL map[string]string `json:"computed_ql"` // Computed columns as SQL expressions
Recursive bool `json:"recursive"` // if true, preload recursively up to 5 levels
// Relationship keys from XFiles - used to build proper foreign key filters
PrimaryKey string `json:"primary_key"` // Primary key of the related table
RelatedKey string `json:"related_key"` // For child tables: column in child that references parent
ForeignKey string `json:"foreign_key"` // For parent tables: column in current table that references parent
RecursiveChildKey string `json:"recursive_child_key"` // For recursive tables: FK column used for recursion (e.g., "rid_parentmastertaskitem")
// Custom SQL JOINs from XFiles - used when preload needs additional joins
SqlJoins []string `json:"sql_joins"` // Custom SQL JOIN clauses
JoinAliases []string `json:"join_aliases"` // Extracted table aliases from SqlJoins for validation
}
type FilterOption struct {
Column string `json:"column"`
Operator string `json:"operator"`
Value interface{} `json:"value"`
LogicOperator string `json:"logic_operator"` // "AND" or "OR" - how this filter combines with previous filters
}
type SortOption struct {
Column string `json:"column"`
Direction string `json:"direction"`
}
type CustomOperator struct {
Name string `json:"name"`
SQL string `json:"sql"`
}
type ComputedColumn struct {
Name string `json:"name"`
Expression string `json:"expression"`
}
// Response structures
type Response struct {
Success bool `json:"success"`
Data interface{} `json:"data"`
Metadata *Metadata `json:"metadata,omitempty"`
Error *APIError `json:"error,omitempty"`
}
type Metadata struct {
Total int64 `json:"total"`
Count int64 `json:"count"`
Filtered int64 `json:"filtered"`
Limit int `json:"limit"`
Offset int `json:"offset"`
RowNumber *int64 `json:"row_number,omitempty"`
}
type APIError struct {
Code string `json:"code"`
Message string `json:"message"`
Details interface{} `json:"details,omitempty"`
Detail string `json:"detail,omitempty"`
SQL string `json:"sql,omitempty"`
}
type Column struct {
Name string `json:"name"`
Type string `json:"type"`
IsNullable bool `json:"is_nullable"`
IsPrimary bool `json:"is_primary"`
IsUnique bool `json:"is_unique"`
HasIndex bool `json:"has_index"`
}
type TableMetadata struct {
Schema string `json:"schema"`
Table string `json:"table"`
Columns []Column `json:"columns"`
Relations []string `json:"relations"`
}
// RelationshipInfo contains information about a model relationship
type RelationshipInfo struct {
FieldName string `json:"field_name"`
JSONName string `json:"json_name"`
RelationType string `json:"relation_type"` // "belongsTo", "hasMany", "hasOne", "many2many"
ForeignKey string `json:"foreign_key"`
References string `json:"references"`
JoinTable string `json:"join_table"`
RelatedModel interface{} `json:"related_model"`
}
+431
View File
@@ -0,0 +1,431 @@
package common
import (
"fmt"
"reflect"
"sort"
"strings"
"github.com/bitechdev/ResolveSpec/pkg/logger"
"github.com/bitechdev/ResolveSpec/pkg/reflection"
)
// ColumnValidator validates column names against a model's fields
type ColumnValidator struct {
validColumns map[string]bool
model interface{}
}
// NewColumnValidator creates a new column validator for a given model
func NewColumnValidator(model interface{}) *ColumnValidator {
validator := &ColumnValidator{
validColumns: make(map[string]bool),
model: model,
}
validator.buildValidColumns()
return validator
}
// buildValidColumns extracts all valid column names from the model using reflection
func (v *ColumnValidator) buildValidColumns() {
modelType := reflect.TypeOf(v.model)
// Unwrap pointers, slices, and arrays to get to the base struct type
for modelType != nil && (modelType.Kind() == reflect.Pointer || modelType.Kind() == reflect.Slice || modelType.Kind() == reflect.Array) {
modelType = modelType.Elem()
}
// Validate that we have a struct type
if modelType == nil || modelType.Kind() != reflect.Struct {
return
}
// Extract column names from struct fields
for i := 0; i < modelType.NumField(); i++ {
field := modelType.Field(i)
if !field.IsExported() || field.Anonymous {
continue
}
// Get column name from bun, gorm, or json tag
columnName := v.getColumnName(field)
if columnName != "" && columnName != "-" {
v.validColumns[strings.ToLower(columnName)] = true
}
}
}
// getColumnName extracts the column name from a struct field's tags
// Supports both Bun and GORM tags
func (v *ColumnValidator) getColumnName(field reflect.StructField) string {
// First check Bun tag for column name
bunTag := field.Tag.Get("bun")
if bunTag != "" && bunTag != "-" {
parts := strings.Split(bunTag, ",")
// The first part is usually the column name
columnName := strings.TrimSpace(parts[0])
if columnName != "" && columnName != "-" {
return columnName
}
}
// Check GORM tag for column name
gormTag := field.Tag.Get("gorm")
if strings.Contains(gormTag, "column:") {
parts := strings.Split(gormTag, ";")
for _, part := range parts {
part = strings.TrimSpace(part)
if strings.HasPrefix(part, "column:") {
return strings.TrimPrefix(part, "column:")
}
}
}
// Fall back to JSON tag
jsonTag := field.Tag.Get("json")
if jsonTag != "" && jsonTag != "-" {
// Extract just the name part (before any comma)
jsonName := strings.Split(jsonTag, ",")[0]
return jsonName
}
// Fall back to field name in lowercase (snake_case conversion would be better)
return strings.ToLower(field.Name)
}
// ValidateColumn validates a single column name
// Returns nil if valid, error if invalid
// Columns prefixed with "cql" (case insensitive) are always valid
// Handles PostgreSQL JSON operators (-> and ->>)
func (v *ColumnValidator) ValidateColumn(column string) error {
// Allow empty columns
if column == "" {
return nil
}
// Allow columns prefixed with "cql" (case insensitive) for computed columns
if strings.HasPrefix(strings.ToLower(column), "cql") {
return nil
}
// Extract source column name (remove JSON operators like ->> or ->)
sourceColumn := reflection.ExtractSourceColumn(column)
// Check if column exists in model
if _, exists := v.validColumns[strings.ToLower(sourceColumn)]; !exists {
return fmt.Errorf("invalid column '%s': column does not exist in model", column)
}
return nil
}
// IsValidColumn checks if a column is valid
// Returns true if valid, false if invalid
func (v *ColumnValidator) IsValidColumn(column string) bool {
return v.ValidateColumn(column) == nil
}
// Columns returns all valid column names known to this validator
func (v *ColumnValidator) Columns() []string {
cols := make([]string, 0, len(v.validColumns))
for col := range v.validColumns {
cols = append(cols, col)
}
sort.Strings(cols)
return cols
}
// FilterValidColumns filters a list of columns, returning only valid ones
// Logs warnings for any invalid columns
func (v *ColumnValidator) FilterValidColumns(columns []string) []string {
if len(columns) == 0 {
return columns
}
validColumns := make([]string, 0, len(columns))
for _, col := range columns {
if v.IsValidColumn(col) {
validColumns = append(validColumns, col)
} else {
logger.Warn("Invalid column '%s' filtered out: column does not exist in model", col)
}
}
return validColumns
}
// ValidateColumns validates multiple column names
// Returns error with details about all invalid columns
func (v *ColumnValidator) ValidateColumns(columns []string) error {
var invalidColumns []string
for _, column := range columns {
if err := v.ValidateColumn(column); err != nil {
invalidColumns = append(invalidColumns, column)
}
}
if len(invalidColumns) > 0 {
return fmt.Errorf("invalid columns: %s", strings.Join(invalidColumns, ", "))
}
return nil
}
// ValidateRequestOptions validates all column references in RequestOptions
func (v *ColumnValidator) ValidateRequestOptions(options RequestOptions) error {
// Validate Columns
if err := v.ValidateColumns(options.Columns); err != nil {
return fmt.Errorf("in select columns: %w", err)
}
// Validate OmitColumns
if err := v.ValidateColumns(options.OmitColumns); err != nil {
return fmt.Errorf("in omit columns: %w", err)
}
// Validate Filter columns
for _, filter := range options.Filters {
if err := v.ValidateColumn(filter.Column); err != nil {
return fmt.Errorf("in filter: %w", err)
}
}
// Validate Sort columns
for _, sort := range options.Sort {
if err := v.ValidateColumn(sort.Column); err != nil {
return fmt.Errorf("in sort: %w", err)
}
}
// Validate Preload columns (if specified)
for idx := range options.Preload {
preload := options.Preload[idx]
// Note: We don't validate the relation name itself, as it's a relationship
// Only validate columns if specified for the preload
if err := v.ValidateColumns(preload.Columns); err != nil {
return fmt.Errorf("in preload '%s' columns: %w", preload.Relation, err)
}
if err := v.ValidateColumns(preload.OmitColumns); err != nil {
return fmt.Errorf("in preload '%s' omit columns: %w", preload.Relation, err)
}
// Validate filter columns in preload
for _, filter := range preload.Filters {
if err := v.ValidateColumn(filter.Column); err != nil {
return fmt.Errorf("in preload '%s' filter: %w", preload.Relation, err)
}
}
}
return nil
}
// FilterRequestOptions filters all column references in RequestOptions
// Returns a new RequestOptions with only valid columns, logging warnings for invalid ones
func (v *ColumnValidator) FilterRequestOptions(options RequestOptions) RequestOptions {
filtered := options
// Filter Columns
filtered.Columns = v.FilterValidColumns(options.Columns)
// Filter OmitColumns
filtered.OmitColumns = v.FilterValidColumns(options.OmitColumns)
// Filter Filter columns
validFilters := make([]FilterOption, 0, len(options.Filters))
for _, filter := range options.Filters {
if strings.EqualFold(filter.Column, "all") {
allCols := v.Columns()
if len(filtered.Columns) > 0 {
allCols = filtered.Columns
}
for _, col := range allCols {
expanded := filter
expanded.Column = col
expanded.LogicOperator = "OR"
validFilters = append(validFilters, expanded)
}
} else if v.IsValidColumn(filter.Column) {
validFilters = append(validFilters, filter)
} else {
logger.Warn("Invalid column in filter '%s' removed", filter.Column)
}
}
filtered.Filters = validFilters
// Filter Sort columns
validSorts := make([]SortOption, 0, len(options.Sort))
for _, sort := range options.Sort {
if v.IsValidColumn(sort.Column) {
validSorts = append(validSorts, sort)
} else {
foundJoin := false
for _, j := range options.JoinAliases {
if strings.Contains(sort.Column, j) {
foundJoin = true
break
}
}
if foundJoin {
validSorts = append(validSorts, sort)
continue
}
if strings.HasPrefix(sort.Column, "(") && strings.HasSuffix(sort.Column, ")") {
// Allow sort by expression/subquery, but validate for security
if IsSafeSortExpression(sort.Column) {
validSorts = append(validSorts, sort)
} else {
logger.Warn("Unsafe sort expression '%s' removed", sort.Column)
}
} else {
logger.Warn("Invalid column in sort '%s' removed", sort.Column)
}
}
}
filtered.Sort = validSorts
// Filter Preload columns
validPreloads := make([]PreloadOption, 0, len(options.Preload))
modelType := reflect.TypeOf(v.model)
if modelType != nil && modelType.Kind() == reflect.Pointer {
modelType = modelType.Elem()
}
for idx := range options.Preload {
preload := options.Preload[idx]
filteredPreload := preload
// Use the related model's validator for preload columns/filters/sorts
preloadValidator := v
if modelType != nil {
if relInfo := GetRelationshipInfo(modelType, preload.Relation); relInfo != nil && relInfo.RelatedModel != nil {
preloadValidator = NewColumnValidator(relInfo.RelatedModel)
}
}
filteredPreload.Columns = preloadValidator.FilterValidColumns(preload.Columns)
filteredPreload.OmitColumns = preloadValidator.FilterValidColumns(preload.OmitColumns)
// Preserve SqlJoins and JoinAliases for preloads with custom joins
filteredPreload.SqlJoins = preload.SqlJoins
filteredPreload.JoinAliases = preload.JoinAliases
// Filter preload filters
validPreloadFilters := make([]FilterOption, 0, len(preload.Filters))
for _, filter := range preload.Filters {
if preloadValidator.IsValidColumn(filter.Column) {
validPreloadFilters = append(validPreloadFilters, filter)
} else {
// Check if the filter column references a joined table alias
foundJoin := false
for _, alias := range preload.JoinAliases {
if strings.Contains(filter.Column, alias) {
foundJoin = true
break
}
}
if foundJoin {
validPreloadFilters = append(validPreloadFilters, filter)
} else {
logger.Warn("Invalid column in preload '%s' filter '%s' removed", preload.Relation, filter.Column)
}
}
}
filteredPreload.Filters = validPreloadFilters
// Filter preload sort columns
validPreloadSorts := make([]SortOption, 0, len(preload.Sort))
for _, sort := range preload.Sort {
if preloadValidator.IsValidColumn(sort.Column) {
validPreloadSorts = append(validPreloadSorts, sort)
} else if strings.HasPrefix(sort.Column, "(") && strings.HasSuffix(sort.Column, ")") {
// Allow sort by expression/subquery, but validate for security
if IsSafeSortExpression(sort.Column) {
validPreloadSorts = append(validPreloadSorts, sort)
} else {
logger.Warn("Unsafe sort expression in preload '%s' removed: '%s'", preload.Relation, sort.Column)
}
} else {
logger.Warn("Invalid column in preload '%s' sort '%s' removed", preload.Relation, sort.Column)
}
}
filteredPreload.Sort = validPreloadSorts
validPreloads = append(validPreloads, filteredPreload)
}
filtered.Preload = validPreloads
// Clear JoinAliases - this is an internal validation field and should not be persisted
filtered.JoinAliases = nil
return filtered
}
// IsSafeSortExpression validates that a sort expression (enclosed in brackets) is safe
// and doesn't contain SQL injection attempts or dangerous commands
func IsSafeSortExpression(expr string) bool {
if expr == "" {
return false
}
// Expression must be enclosed in brackets
expr = strings.TrimSpace(expr)
if !strings.HasPrefix(expr, "(") || !strings.HasSuffix(expr, ")") {
return false
}
// Remove outer brackets for content validation
expr = expr[1 : len(expr)-1]
expr = strings.TrimSpace(expr)
// Convert to lowercase for checking dangerous keywords
exprLower := strings.ToLower(expr)
// Check for dangerous SQL commands that should never be in a sort expression
dangerousKeywords := []string{
"drop ", "delete ", "insert ", "update ", "alter ", "create ",
"truncate ", "exec ", "execute ", "grant ", "revoke ",
"into ", "values ", "set ", "shutdown", "xp_",
}
for _, keyword := range dangerousKeywords {
if strings.Contains(exprLower, keyword) {
logger.Warn("Dangerous SQL keyword '%s' detected in sort expression: %s", keyword, expr)
return false
}
}
// Check for SQL comment attempts
if strings.Contains(expr, "--") || strings.Contains(expr, "/*") || strings.Contains(expr, "*/") {
logger.Warn("SQL comment detected in sort expression: %s", expr)
return false
}
// Check for semicolon (command separator)
if strings.Contains(expr, ";") {
logger.Warn("Command separator (;) detected in sort expression: %s", expr)
return false
}
// Expression appears safe
return true
}
// GetValidColumns returns a list of all valid column names for debugging purposes
func (v *ColumnValidator) GetValidColumns() []string {
columns := make([]string, 0, len(v.validColumns))
for col := range v.validColumns {
columns = append(columns, col)
}
return columns
}
func QuoteIdent(qualifier string) string {
return `"` + strings.ReplaceAll(qualifier, `"`, `""`) + `"`
}
func QuoteLiteral(value string) string {
return `'` + strings.ReplaceAll(value, `'`, `''`) + `'`
}