ResolveSpec/pkg/middleware/README.md

# Middleware Package

HTTP middleware utilities for security and performance.

## Table of Contents

1. [Rate Limiting](#rate-limiting)
2. [Request Size Limits](#request-size-limits)
3. [Input Sanitization](#input-sanitization)

---

## Rate Limiting

Production-grade rate limiting using token bucket algorithm.

### Quick Start

```go
import "github.com/bitechdev/ResolveSpec/pkg/middleware"

// Create rate limiter: 100 requests per second, burst of 20
rateLimiter := middleware.NewRateLimiter(100, 20)

// Apply to all routes
router.Use(rateLimiter.Middleware)
```

### Basic Usage

```go
package main

import (
    "log"
    "net/http"

    "github.com/bitechdev/ResolveSpec/pkg/middleware"
    "github.com/gorilla/mux"
)

func main() {
    router := mux.NewRouter()

    // Rate limit: 10 requests per second, burst of 5
    rateLimiter := middleware.NewRateLimiter(10, 5)
    router.Use(rateLimiter.Middleware)

    router.HandleFunc("/api/data", dataHandler)

    log.Fatal(http.ListenAndServe(":8080", router))
}
```

### Custom Key Extraction

By default, rate limiting is per IP address. Customize the key:

```go
// Rate limit by User ID from header
keyFunc := func(r *http.Request) string {
    userID := r.Header.Get("X-User-ID")
    if userID == "" {
        return r.RemoteAddr // Fallback to IP
    }
    return "user:" + userID
}

router.Use(rateLimiter.MiddlewareWithKeyFunc(keyFunc))
```

### Advanced Key Functions

**By API Key:**

```go
keyFunc := func(r *http.Request) string {
    apiKey := r.Header.Get("X-API-Key")
    if apiKey == "" {
        return r.RemoteAddr
    }
    return "api:" + apiKey
}
```

**By Authenticated User:**

```go
keyFunc := func(r *http.Request) string {
    // Extract from JWT or session
    user := getUserFromContext(r.Context())
    if user != nil {
        return "user:" + user.ID
    }
    return r.RemoteAddr
}
```

**By Path + User:**

```go
keyFunc := func(r *http.Request) string {
    user := getUserFromContext(r.Context())
    if user != nil {
        return fmt.Sprintf("user:%s:path:%s", user.ID, r.URL.Path)
    }
    return r.URL.Path + ":" + r.RemoteAddr
}
```

### Different Limits Per Route

```go
func main() {
    router := mux.NewRouter()

    // Public endpoints: 10 rps
    publicLimiter := middleware.NewRateLimiter(10, 5)

    // API endpoints: 100 rps
    apiLimiter := middleware.NewRateLimiter(100, 20)

    // Admin endpoints: 1000 rps
    adminLimiter := middleware.NewRateLimiter(1000, 50)

    // Apply different limiters to subrouters
    publicRouter := router.PathPrefix("/public").Subrouter()
    publicRouter.Use(publicLimiter.Middleware)

    apiRouter := router.PathPrefix("/api").Subrouter()
    apiRouter.Use(apiLimiter.Middleware)

    adminRouter := router.PathPrefix("/admin").Subrouter()
    adminRouter.Use(adminLimiter.Middleware)
}
```

### Rate Limit Response

When rate limited, clients receive:

```http
HTTP/1.1 429 Too Many Requests
Content-Type: text/plain

```

### Configuration Examples

**Tight Rate Limit (Anti-abuse):**

```go
// 1 request per second, burst of 3
rateLimiter := middleware.NewRateLimiter(1, 3)
```

**Moderate Rate Limit (Standard API):**

```go
// 100 requests per second, burst of 20
rateLimiter := middleware.NewRateLimiter(100, 20)
```

**Generous Rate Limit (Internal Services):**

```go
// 1000 requests per second, burst of 100
rateLimiter := middleware.NewRateLimiter(1000, 100)
```

**Time-based Limits:**

```go
// 60 requests per minute = 1 request per second
rateLimiter := middleware.NewRateLimiter(1, 10)

// 1000 requests per hour ≈ 0.28 requests per second
rateLimiter := middleware.NewRateLimiter(0.28, 50)
```

### Understanding Burst

The burst parameter allows short bursts above the rate:

```go
// Rate: 10 rps, Burst: 5
// Allows up to 5 requests immediately, then 10/second
rateLimiter := middleware.NewRateLimiter(10, 5)
```

**Bucket fills at rate:** 10 tokens/second
**Bucket capacity:** 5 tokens
**Request consumes:** 1 token

**Example traffic pattern:**
- T=0s: 5 requests → ✅ All allowed (burst)
- T=0.1s: 1 request → ❌ Denied (bucket empty)
- T=0.5s: 1 request → ✅ Allowed (bucket refilled 0.5 tokens)
- T=1s: 1 request → ✅ Allowed (bucket has ~1 token)

### Cleanup Behavior

The rate limiter automatically cleans up inactive limiters every 5 minutes to prevent memory leaks.

### Performance Characteristics

- **Memory**: ~100 bytes per active limiter
- **Throughput**: >1M requests/second
- **Latency**: <1μs per request
- **Concurrency**: Lock-free for rate checks

### Production Deployment

**With Reverse Proxy:**

```go
// Use X-Forwarded-For or X-Real-IP
keyFunc := func(r *http.Request) string {
    // Check proxy headers first
    if ip := r.Header.Get("X-Forwarded-For"); ip != "" {
        return strings.Split(ip, ",")[0]
    }
    if ip := r.Header.Get("X-Real-IP"); ip != "" {
        return ip
    }
    return r.RemoteAddr
}

router.Use(rateLimiter.MiddlewareWithKeyFunc(keyFunc))
```

**Environment-based Configuration:**

```go
import "os"

func getRateLimiter() *middleware.RateLimiter {
    rps := getEnvFloat("RATE_LIMIT_RPS", 100)
    burst := getEnvInt("RATE_LIMIT_BURST", 20)
    return middleware.NewRateLimiter(rps, burst)
}
```

### Testing Rate Limits

```bash
# Send 10 requests rapidly
for i in {1..10}; do
  curl -w "Status: %{http_code}\n" http://localhost:8080/api/data
done
```

**Expected output:**
```
Status: 200  # Request 1-5 (within burst)
Status: 200
Status: 200
Status: 200
Status: 200
Status: 429  # Request 6-10 (rate limited)
Status: 429
Status: 429
Status: 429
Status: 429
```

### Complete Example

```go
package main

import (
    "encoding/json"
    "log"
    "net/http"
    "os"
    "strconv"

    "github.com/bitechdev/ResolveSpec/pkg/middleware"
    "github.com/gorilla/mux"
)

func main() {
    // Configuration from environment
    rps, _ := strconv.ParseFloat(os.Getenv("RATE_LIMIT_RPS"), 64)
    if rps == 0 {
        rps = 100 // Default
    }

    burst, _ := strconv.Atoi(os.Getenv("RATE_LIMIT_BURST"))
    if burst == 0 {
        burst = 20 // Default
    }

    // Create rate limiter
    rateLimiter := middleware.NewRateLimiter(rps, burst)

    // Custom key extraction
    keyFunc := func(r *http.Request) string {
        // Try API key first
        if apiKey := r.Header.Get("X-API-Key"); apiKey != "" {
            return "api:" + apiKey
        }
        // Try authenticated user
        if userID := r.Header.Get("X-User-ID"); userID != "" {
            return "user:" + userID
        }
        // Fall back to IP
        if ip := r.Header.Get("X-Forwarded-For"); ip != "" {
            return ip
        }
        return r.RemoteAddr
    }

    // Create router
    router := mux.NewRouter()

    // Apply rate limiting
    router.Use(rateLimiter.MiddlewareWithKeyFunc(keyFunc))

    // Routes
    router.HandleFunc("/api/data", dataHandler)
    router.HandleFunc("/health", healthHandler)

    log.Printf("Starting server with rate limit: %.1f rps, burst: %d", rps, burst)
    log.Fatal(http.ListenAndServe(":8080", router))
}

func dataHandler(w http.ResponseWriter, r *http.Request) {
    json.NewEncoder(w).Encode(map[string]string{
        "message": "Data endpoint",
    })
}

func healthHandler(w http.ResponseWriter, r *http.Request) {
    w.WriteHeader(http.StatusOK)
    w.Write([]byte("OK"))
}
```

## Best Practices

1. **Set Appropriate Limits**: Consider your backend capacity
   - Database: Can it handle X queries/second?
   - External APIs: What are their rate limits?
   - Server resources: CPU, memory, connections

2. **Use Burst Wisely**: Allow legitimate traffic spikes
   - Too low: Reject valid bursts
   - Too high: Allow abuse

3. **Monitor Rate Limits**: Track how often limits are hit
   ```go
   // Log rate limit events
   if rateLimited {
       log.Printf("Rate limited: %s", clientKey)
   }
   ```

4. **Provide Feedback**: Include rate limit headers (future enhancement)
   ```http
   X-RateLimit-Limit: 100
   X-RateLimit-Remaining: 95
   X-RateLimit-Reset: 1640000000
   ```

5. **Tiered Limits**: Different limits for different user tiers
   ```go
   func getRateLimiter(userTier string) *middleware.RateLimiter {
       switch userTier {
       case "premium":
           return middleware.NewRateLimiter(1000, 100)
       case "standard":
           return middleware.NewRateLimiter(100, 20)
       default:
           return middleware.NewRateLimiter(10, 5)
       }
   }
   ```

---

## Request Size Limits

Protect against oversized request bodies with configurable size limits.

### Quick Start

```go
import "github.com/bitechdev/ResolveSpec/pkg/middleware"

// Default: 10MB limit
sizeLimiter := middleware.NewRequestSizeLimiter(0)
router.Use(sizeLimiter.Middleware)
```

### Custom Size Limit

```go
// 5MB limit
sizeLimiter := middleware.NewRequestSizeLimiter(5 * 1024 * 1024)
router.Use(sizeLimiter.Middleware)

// Or use constants
sizeLimiter := middleware.NewRequestSizeLimiter(middleware.Size5MB)
```

### Available Size Constants

```go
middleware.Size1MB    // 1 MB
middleware.Size5MB    // 5 MB
middleware.Size10MB   // 10 MB (default)
middleware.Size50MB   // 50 MB
middleware.Size100MB  // 100 MB
```

### Different Limits Per Route

```go
func main() {
    router := mux.NewRouter()

    // File upload endpoint: 50MB
    uploadLimiter := middleware.NewRequestSizeLimiter(middleware.Size50MB)
    uploadRouter := router.PathPrefix("/upload").Subrouter()
    uploadRouter.Use(uploadLimiter.Middleware)

    // API endpoints: 1MB
    apiLimiter := middleware.NewRequestSizeLimiter(middleware.Size1MB)
    apiRouter := router.PathPrefix("/api").Subrouter()
    apiRouter.Use(apiLimiter.Middleware)
}
```

### Dynamic Size Limits

```go
// Custom size based on request
sizeFunc := func(r *http.Request) int64 {
    // Premium users get 50MB
    if isPremiumUser(r) {
        return middleware.Size50MB
    }
    // Free users get 5MB
    return middleware.Size5MB
}

router.Use(sizeLimiter.MiddlewareWithCustomSize(sizeFunc))
```

**By Content-Type:**

```go
sizeFunc := func(r *http.Request) int64 {
    contentType := r.Header.Get("Content-Type")

    switch {
    case strings.Contains(contentType, "multipart/form-data"):
        return middleware.Size50MB // File uploads
    case strings.Contains(contentType, "application/json"):
        return middleware.Size1MB // JSON APIs
    default:
        return middleware.Size10MB // Default
    }
}
```

### Error Response

When size limit exceeded:

```http
HTTP/1.1 413 Request Entity Too Large
X-Max-Request-Size: 10485760

http: request body too large
```

### Complete Example

```go
package main

import (
    "log"
    "net/http"

    "github.com/bitechdev/ResolveSpec/pkg/middleware"
    "github.com/gorilla/mux"
)

func main() {
    router := mux.NewRouter()

    // API routes: 1MB limit
    api := router.PathPrefix("/api").Subrouter()
    apiLimiter := middleware.NewRequestSizeLimiter(middleware.Size1MB)
    api.Use(apiLimiter.Middleware)
    api.HandleFunc("/users", createUserHandler).Methods("POST")

    // Upload routes: 50MB limit
    upload := router.PathPrefix("/upload").Subrouter()
    uploadLimiter := middleware.NewRequestSizeLimiter(middleware.Size50MB)
    upload.Use(uploadLimiter.Middleware)
    upload.HandleFunc("/file", uploadFileHandler).Methods("POST")

    log.Fatal(http.ListenAndServe(":8080", router))
}
```

---

## Input Sanitization

Protect against XSS, injection attacks, and malicious input.

### Quick Start

```go
import "github.com/bitechdev/ResolveSpec/pkg/middleware"

// Default sanitizer (safe defaults)
sanitizer := middleware.DefaultSanitizer()
router.Use(sanitizer.Middleware)
```

### Sanitizer Types

**Default Sanitizer (Recommended):**

```go
sanitizer := middleware.DefaultSanitizer()
// ✓ Escapes HTML entities
// ✓ Removes null bytes
// ✓ Removes control characters
// ✓ Blocks XSS patterns (script tags, event handlers)
// ✗ Does not strip HTML (allows legitimate content)
```

**Strict Sanitizer:**

```go
sanitizer := middleware.StrictSanitizer()
// ✓ All default features
// ✓ Strips ALL HTML tags
// ✓ Max string length: 10,000 chars
```

### Custom Configuration

```go
sanitizer := &middleware.Sanitizer{
    StripHTML:          true,  // Remove HTML tags
    EscapeHTML:         false, // Don't escape (already stripped)
    RemoveNullBytes:    true,  // Remove \x00
    RemoveControlChars: true,  // Remove dangerous control chars
    MaxStringLength:    5000,  // Limit to 5000 chars

    // Block patterns (regex)
    BlockPatterns: []*regexp.Regexp{
        regexp.MustCompile(`(?i)<script`),
        regexp.MustCompile(`(?i)javascript:`),
    },

    // Custom sanitization function
    CustomSanitizer: func(s string) string {
        // Your custom logic
        return strings.ToLower(s)
    },
}

router.Use(sanitizer.Middleware)
```

### What Gets Sanitized

**Automatic (via middleware):**
- Query parameters
- Headers (User-Agent, Referer, X-Forwarded-For, X-Real-IP)

**Manual (in your handler):**
- Request body (JSON, form data)
- Database queries
- File names

### Manual Sanitization

**String Values:**

```go
sanitizer := middleware.DefaultSanitizer()

// Sanitize user input
username := sanitizer.Sanitize(r.FormValue("username"))
email := sanitizer.Sanitize(r.FormValue("email"))
```

**Map/JSON Data:**

```go
var data map[string]interface{}
json.Unmarshal(body, &data)

// Sanitize all string values recursively
sanitizedData := sanitizer.SanitizeMap(data)
```

**Nested Structures:**

```go
type User struct {
    Name    string
    Email   string
    Bio     string
    Profile map[string]interface{}
}

// After unmarshaling
user.Name = sanitizer.Sanitize(user.Name)
user.Email = sanitizer.Sanitize(user.Email)
user.Bio = sanitizer.Sanitize(user.Bio)
user.Profile = sanitizer.SanitizeMap(user.Profile)
```

### Specialized Sanitizers

**Filenames:**

```go
import "github.com/bitechdev/ResolveSpec/pkg/middleware"

filename := middleware.SanitizeFilename(uploadedFilename)
// Removes: .., /, \, null bytes
// Limits: 255 characters
```

**Emails:**

```go
email := middleware.SanitizeEmail(" USER@EXAMPLE.COM ")
// Result: "user@example.com"
// Trims, lowercases, removes null bytes
```

**URLs:**

```go
url := middleware.SanitizeURL(userInput)
// Blocks: javascript:, data: protocols
// Removes: null bytes
```

### Blocked Patterns (Default)

The default sanitizer blocks:

1. **Script tags**: `<script>...</script>`
2. **JavaScript protocol**: `javascript:alert(1)`
3. **Event handlers**: `onclick="..."`, `onerror="..."`
4. **Iframes**: `<iframe src="...">`
5. **Objects**: `<object data="...">`
6. **Embeds**: `<embed src="...">`

### Security Best Practices

**1. Layer Defense:**

```go
// Layer 1: Middleware (query params, headers)
router.Use(sanitizer.Middleware)

// Layer 2: Input validation (in handler)
func createUserHandler(w http.ResponseWriter, r *http.Request) {
    var user User
    json.NewDecoder(r.Body).Decode(&user)

    // Sanitize
    user.Name = sanitizer.Sanitize(user.Name)
    user.Email = middleware.SanitizeEmail(user.Email)

    // Validate
    if !isValidEmail(user.Email) {
        http.Error(w, "Invalid email", 400)
        return
    }

    // Use parameterized queries (prevents SQL injection)
    db.Exec("INSERT INTO users (name, email) VALUES (?, ?)",
        user.Name, user.Email)
}
```

**2. Context-Aware Sanitization:**

```go
// HTML content (user posts, comments)
sanitizer := middleware.StrictSanitizer()
post.Content = sanitizer.Sanitize(post.Content)

// Structured data (JSON API)
sanitizer := middleware.DefaultSanitizer()
data = sanitizer.SanitizeMap(jsonData)

// Search queries (preserve special chars)
query = middleware.SanitizeFilename(searchTerm) // Light sanitization
```

**3. Output Encoding:**

```go
// When rendering HTML
import "html/template"

tmpl := template.Must(template.New("page").Parse(`
    <h1>{{.Title}}</h1>
    <p>{{.Content}}</p>
`))

// template.HTML automatically escapes
tmpl.Execute(w, data)
```

### Complete Example

```go
package main

import (
    "encoding/json"
    "log"
    "net/http"

    "github.com/bitechdev/ResolveSpec/pkg/middleware"
    "github.com/gorilla/mux"
)

func main() {
    router := mux.NewRouter()

    // Apply sanitization middleware
    sanitizer := middleware.DefaultSanitizer()
    router.Use(sanitizer.Middleware)

    router.HandleFunc("/api/users", createUserHandler).Methods("POST")

    log.Fatal(http.ListenAndServe(":8080", router))
}

func createUserHandler(w http.ResponseWriter, r *http.Request) {
    sanitizer := middleware.DefaultSanitizer()

    var user struct {
        Name  string `json:"name"`
        Email string `json:"email"`
        Bio   string `json:"bio"`
    }

    if err := json.NewDecoder(r.Body).Decode(&user); err != nil {
        http.Error(w, "Invalid JSON", 400)
        return
    }

    // Sanitize inputs
    user.Name = sanitizer.Sanitize(user.Name)
    user.Email = middleware.SanitizeEmail(user.Email)
    user.Bio = sanitizer.Sanitize(user.Bio)

    // Validate
    if len(user.Name) == 0 || len(user.Email) == 0 {
        http.Error(w, "Name and email required", 400)
        return
    }

    // Save to database (use parameterized queries!)
    // db.Exec("INSERT INTO users (name, email, bio) VALUES (?, ?, ?)",
    //     user.Name, user.Email, user.Bio)

    w.WriteHeader(http.StatusCreated)
    json.NewEncoder(w).Encode(map[string]string{
        "status": "created",
    })
}
```

### Testing Sanitization

```bash
# Test XSS prevention
curl -X POST http://localhost:8080/api/users \
  -H "Content-Type: application/json" \
  -d '{
    "name": "<script>alert(1)</script>John",
    "email": "test@example.com",
    "bio": "My bio with <iframe src=\"evil.com\"></iframe>"
  }'

# Script tags and iframes should be removed
```

### Performance

- **Overhead**: <1ms per request for typical payloads
- **Regex compilation**: Done once at initialization
- **Safe for production**: Minimal performance impact