Compare commits

..

30 Commits

Author SHA1 Message Date
Hein
82d84435f2 Fixed version release auto inc
Some checks failed
Build , Vet Test, and Lint / Run Vet Tests (1.24.x) (push) Successful in -25m36s
Build , Vet Test, and Lint / Run Vet Tests (1.23.x) (push) Successful in -25m5s
Build , Vet Test, and Lint / Lint Code (push) Successful in -25m16s
Build , Vet Test, and Lint / Build (push) Successful in -25m29s
Tests / Unit Tests (push) Failing after -25m48s
Tests / Integration Tests (push) Failing after -25m53s
2025-12-31 09:41:23 +02:00
Bitech Systems
b99b08430e Merge pull request #16 from bitechdev/fix-funcapi
fix: 🔒 Fixed funcapi possible injections
2025-12-31 09:38:44 +02:00
Hein
fae9a082bd refactor: 🚨 linting issues 2025-12-31 09:37:12 +02:00
Hein Puth (Warkanum)
191822b91c Merge pull request #17 from bitechdev/copilot/sub-pr-16
fix: Comprehensive SQL injection protection in funcspec handlers
2025-12-31 09:35:49 +02:00
copilot-swe-agent[bot]
a6a17d019f fix: Optimized regex performance and added backslash escaping for LIKE patterns
- Added backslash escaping to LIKE pattern sanitization
- Optimized ValidSQL select mode with single pre-compiled regex
- All funcspec tests pass (except pre-existing TestReplaceMetaVariables)

Co-authored-by: warkanum <208308+warkanum@users.noreply.github.com>
2025-12-31 07:28:16 +00:00
copilot-swe-agent[bot]
a7cc42044b fix: Improved SQL injection protection with proper handling
- Fixed IN clause to conditionally quote only string values (not numeric)
- Fixed LIKE pattern sanitization to preserve wildcards while preventing injection
- Improved dangerous pattern removal with case-insensitive regex while preserving case
- All funcspec tests now pass (except pre-existing TestReplaceMetaVariables)

Co-authored-by: warkanum <208308+warkanum@users.noreply.github.com>
2025-12-31 07:24:56 +00:00
copilot-swe-agent[bot]
8cdc353029 fix: Fixed SQL quoting for empty/zero values and sanitized match filter
- Sanitize colval immediately after extraction in match= filter
- Fixed empty/zero value handling to use proper SQL literals (0 vs '')
- Applied proper quoting for string vs numeric comparisons
- Fixed x-fieldfilter handlers for proper value handling

Co-authored-by: warkanum <208308+warkanum@users.noreply.github.com>
2025-12-31 07:22:42 +00:00
copilot-swe-agent[bot]
6528e94297 fix: Improved SQL injection protections based on code review
- Fixed backslash escaping order in colvalue mode
- Added proper quoting for IN clause values
- Simplified dangerous pattern matching with case-insensitive approach
- All funcspec tests pass (except pre-existing test failure)

Co-authored-by: warkanum <208308+warkanum@users.noreply.github.com>
2025-12-31 07:20:56 +00:00
copilot-swe-agent[bot]
f711bf38d2 fix: Enhanced SQL injection protection in funcspec
- Added sanitization for path parameters in mergePathParams
- Added sanitization for query parameters with p- prefix in mergeQueryParams
- Added sanitization for header parameters in mergeHeaderParams
- Fixed IN clause to sanitize all values individually
- Improved ValidSQL function with better escaping and more injection patterns
- Added backslash escaping to colvalue mode
- Extended dangerous keyword list in select mode

Co-authored-by: warkanum <208308+warkanum@users.noreply.github.com>
2025-12-31 07:19:53 +00:00
copilot-swe-agent[bot]
44356d8750 Initial plan 2025-12-31 07:11:57 +00:00
Hein
caf85cf558 fix: 🔒 Fixed funcapi possible injections 2025-12-31 09:09:16 +02:00
Hein Puth (Warkanum)
2e1547ec65 Merge pull request #15 from bitechdev/feature-staticweb
Some checks failed
Build , Vet Test, and Lint / Run Vet Tests (1.24.x) (push) Successful in -21m25s
Build , Vet Test, and Lint / Run Vet Tests (1.23.x) (push) Successful in -21m19s
Build , Vet Test, and Lint / Build (push) Successful in -25m26s
Build , Vet Test, and Lint / Lint Code (push) Successful in -24m49s
Tests / Unit Tests (push) Failing after -25m46s
Tests / Integration Tests (push) Failing after -25m47s
feature: staticweb
2025-12-30 17:49:33 +02:00
Hein
49cdc6f17b fix: lint issues after merge 2025-12-30 17:46:33 +02:00
Hein
0bd653820c Merge branch 'main' of https://github.com/bitechdev/ResolveSpec into feature-staticweb 2025-12-30 17:45:17 +02:00
Hein
9209193157 fix: lint issues and docs 2025-12-30 17:44:57 +02:00
Hein Puth (Warkanum)
b8c44c5a99 Merge pull request #14 from bitechdev/copilot/fix-api-null-response
Fix API returning null instead of empty array when no records found
2025-12-30 17:35:02 +02:00
Hein
28fd88fff1 staticweb package for easier static web server hosting 2025-12-30 17:31:07 +02:00
copilot-swe-agent[bot]
be38341383 Fix formatting issues with gofmt
- Removed trailing whitespace
- Fixed tab/space alignment in struct definitions
- All tests still passing

Co-authored-by: warkanum <208308+warkanum@users.noreply.github.com>
2025-12-30 15:30:23 +00:00
copilot-swe-agent[bot]
fab744b878 Add clarifying comments about X-No-Data-Found header timing
- Added comments explaining why X-No-Data-Found is set before normalization
- Header reflects database query result, not final response format
- Clarifies that normalizeResultArray doesn't affect header logic
- All tests passing

Co-authored-by: warkanum <208308+warkanum@users.noreply.github.com>
2025-12-30 14:04:16 +00:00
copilot-swe-agent[bot]
5ad2bd3a78 Improve test robustness - use explicit flag instead of string comparison
- Changed test to use shouldBeEmptyArr flag instead of hardcoded name comparison
- Makes test more maintainable and less fragile
- All tests still passing

Co-authored-by: warkanum <208308+warkanum@users.noreply.github.com>
2025-12-30 14:02:42 +00:00
copilot-swe-agent[bot]
333fe158e9 Address code review feedback - improve data length calculation clarity
- Simplified data length calculation logic in sendFormattedResponse
- Simplified data length calculation logic in sendResponseWithOptions
- Calculate dataLen after nil conversion for clarity and consistency
- All tests still passing

Co-authored-by: warkanum <208308+warkanum@users.noreply.github.com>
2025-12-30 14:01:20 +00:00
copilot-swe-agent[bot]
2a2d351ad4 Fix API returning null with 200 code when no records found
- Modified handleRead to always return empty array [] instead of null when no ID provided
- Added X-No-Data-Found header when result count is 0
- Updated normalizeResultArray to keep empty arrays as arrays instead of converting to empty objects
- Updated sendFormattedResponse and sendResponseWithOptions to handle empty data properly
- All responses now return 200 OK instead of 206 Partial Content when no data found
- Added comprehensive tests to verify the fix

Co-authored-by: warkanum <208308+warkanum@users.noreply.github.com>
2025-12-30 13:57:15 +00:00
copilot-swe-agent[bot]
e918c49b84 Initial plan 2025-12-30 13:51:07 +00:00
Hein Puth (Warkanum)
41e4956510 Merge pull request #12 from bitechdev/copilot/fix-prefix-event-issue
[WIP] Fix prefix addition in where queries and xfiles options
2025-12-30 15:38:35 +02:00
copilot-swe-agent[bot]
8e8c3c6de6 Refactor: Extract common logic from stripOuterParentheses functions
Co-authored-by: warkanum <208308+warkanum@users.noreply.github.com>
2025-12-30 13:36:29 +00:00
copilot-swe-agent[bot]
aa9b7312f6 Fix AddTablePrefixToColumns to handle parenthesized AND conditions correctly
Co-authored-by: warkanum <208308+warkanum@users.noreply.github.com>
2025-12-30 13:31:18 +00:00
copilot-swe-agent[bot]
dca43b0e05 Initial analysis: identified bug in AddTablePrefixToColumns
Co-authored-by: warkanum <208308+warkanum@users.noreply.github.com>
2025-12-30 13:26:37 +00:00
copilot-swe-agent[bot]
6f368bbce5 Initial plan 2025-12-30 13:18:17 +00:00
Hein Puth (Warkanum)
8704cee941 Merge pull request #9 from bitechdev/websocketspec
feature: Websocketspec and mqtt spec
2025-12-30 15:02:59 +02:00
Hein Puth (Warkanum)
4ce5afe0ac Merge pull request #10 from bitechdev/copilot/sub-pr-9
Add WebSocketSpec and MQTTSpec real-time protocol implementations
2025-12-30 14:50:35 +02:00
27 changed files with 5486 additions and 877 deletions

View File

@@ -13,14 +13,22 @@ test-integration:
# Run all tests (unit + integration) # Run all tests (unit + integration)
test: test-unit test-integration test: test-unit test-integration
release-version: ## Create and push a release with specific version (use: make release-version VERSION=v1.2.3) release-version: ## Create and push a release with specific version (use: make release-version VERSION=v1.2.3 or make release-version to auto-increment)
@if [ -z "$(VERSION)" ]; then \ @if [ -z "$(VERSION)" ]; then \
echo "Error: VERSION is required. Usage: make release-version VERSION=v1.2.3"; \ latest_tag=$$(git describe --tags --abbrev=0 2>/dev/null || echo "v0.0.0"); \
exit 1; \ echo "No VERSION specified. Last version: $$latest_tag"; \
fi version_num=$$(echo "$$latest_tag" | sed 's/^v//'); \
@version="$(VERSION)"; \ major=$$(echo "$$version_num" | cut -d. -f1); \
if ! echo "$$version" | grep -q "^v"; then \ minor=$$(echo "$$version_num" | cut -d. -f2); \
version="v$$version"; \ patch=$$(echo "$$version_num" | cut -d. -f3); \
new_patch=$$((patch + 1)); \
version="v$$major.$$minor.$$new_patch"; \
echo "Auto-incrementing to: $$version"; \
else \
version="$(VERSION)"; \
if ! echo "$$version" | grep -q "^v"; then \
version="v$$version"; \
fi; \
fi; \ fi; \
echo "Creating release: $$version"; \ echo "Creating release: $$version"; \
latest_tag=$$(git describe --tags --abbrev=0 2>/dev/null || echo ""); \ latest_tag=$$(git describe --tags --abbrev=0 2>/dev/null || echo ""); \

989
README.md

File diff suppressed because it is too large Load Diff

View File

@@ -234,35 +234,52 @@ func stripOuterParentheses(s string) string {
s = strings.TrimSpace(s) s = strings.TrimSpace(s)
for { for {
if len(s) < 2 || s[0] != '(' || s[len(s)-1] != ')' { stripped, wasStripped := stripOneMatchingOuterParen(s)
if !wasStripped {
return s return s
} }
s = stripped
}
}
// Check if these parentheses match (i.e., they're the outermost pair) // stripOneOuterParentheses removes only one level of matching outer parentheses from a string
depth := 0 // Unlike stripOuterParentheses, this only strips once, preserving nested parentheses
matched := false func stripOneOuterParentheses(s string) string {
for i := 0; i < len(s); i++ { stripped, _ := stripOneMatchingOuterParen(strings.TrimSpace(s))
switch s[i] { return stripped
case '(': }
depth++
case ')': // stripOneMatchingOuterParen is a helper that strips one matching pair of outer parentheses
depth-- // Returns the stripped string and a boolean indicating if stripping occurred
if depth == 0 && i == len(s)-1 { func stripOneMatchingOuterParen(s string) (string, bool) {
matched = true if len(s) < 2 || s[0] != '(' || s[len(s)-1] != ')' {
} else if depth == 0 { return s, false
// Found a closing paren before the end, so outer parens don't match }
return s
} // Check if these parentheses match (i.e., they're the outermost pair)
depth := 0
matched := false
for i := 0; i < len(s); i++ {
switch s[i] {
case '(':
depth++
case ')':
depth--
if depth == 0 && i == len(s)-1 {
matched = true
} else if depth == 0 {
// Found a closing paren before the end, so outer parens don't match
return s, false
} }
} }
if !matched {
return s
}
// Strip the outer parentheses and continue
s = strings.TrimSpace(s[1 : len(s)-1])
} }
if !matched {
return s, false
}
// Strip the outer parentheses
return strings.TrimSpace(s[1 : len(s)-1]), true
} }
// splitByAND splits a WHERE clause by AND operators (case-insensitive) // splitByAND splits a WHERE clause by AND operators (case-insensitive)
@@ -683,8 +700,8 @@ func AddTablePrefixToColumns(where string, tableName string) string {
// - No valid column reference is found // - No valid column reference is found
// - The column doesn't exist in the table (when validColumns is provided) // - The column doesn't exist in the table (when validColumns is provided)
func addPrefixToSingleCondition(cond string, tableName string, validColumns map[string]bool) string { func addPrefixToSingleCondition(cond string, tableName string, validColumns map[string]bool) string {
// Strip outer grouping parentheses to get to the actual condition // Strip one level of outer grouping parentheses to get to the actual condition
strippedCond := stripOuterParentheses(cond) strippedCond := stripOneOuterParentheses(cond)
// Skip SQL literals and trivial conditions (true, false, null, 1=1, etc.) // Skip SQL literals and trivial conditions (true, false, null, 1=1, etc.)
if IsSQLExpression(strippedCond) || IsTrivialCondition(strippedCond) { if IsSQLExpression(strippedCond) || IsTrivialCondition(strippedCond) {
@@ -692,6 +709,34 @@ func addPrefixToSingleCondition(cond string, tableName string, validColumns map[
return cond return cond
} }
// After stripping outer parentheses, check if there are multiple AND-separated conditions
// at the top level. If so, split and process each separately to avoid incorrectly
// treating "true AND status" as a single column name.
subConditions := splitByAND(strippedCond)
if len(subConditions) > 1 {
// Multiple conditions found - process each separately
logger.Debug("Found %d sub-conditions after stripping parentheses, processing separately", len(subConditions))
processedConditions := make([]string, 0, len(subConditions))
for _, subCond := range subConditions {
// Recursively process each sub-condition
processed := addPrefixToSingleCondition(subCond, tableName, validColumns)
processedConditions = append(processedConditions, processed)
}
result := strings.Join(processedConditions, " AND ")
// Preserve original outer parentheses if they existed
if cond != strippedCond {
result = "(" + result + ")"
}
return result
}
// If we stripped parentheses and still have more parentheses, recursively process
if cond != strippedCond && strings.HasPrefix(strippedCond, "(") && strings.HasSuffix(strippedCond, ")") {
// Recursively handle nested parentheses
processed := addPrefixToSingleCondition(strippedCond, tableName, validColumns)
return "(" + processed + ")"
}
// Extract the left side of the comparison (before the operator) // Extract the left side of the comparison (before the operator)
columnRef := extractLeftSideOfComparison(strippedCond) columnRef := extractLeftSideOfComparison(strippedCond)
if columnRef == "" { if columnRef == "" {

View File

@@ -658,3 +658,76 @@ func TestSanitizeWhereClauseWithModel(t *testing.T) {
}) })
} }
} }
func TestAddTablePrefixToColumns_ComplexConditions(t *testing.T) {
tests := []struct {
name string
where string
tableName string
expected string
}{
{
name: "Parentheses with true AND condition - should not prefix true",
where: "(true AND status = 'active')",
tableName: "mastertask",
expected: "(true AND mastertask.status = 'active')",
},
{
name: "Parentheses with multiple conditions including true",
where: "(true AND status = 'active' AND id > 5)",
tableName: "mastertask",
expected: "(true AND mastertask.status = 'active' AND mastertask.id > 5)",
},
{
name: "Nested parentheses with true",
where: "((true AND status = 'active'))",
tableName: "mastertask",
expected: "((true AND mastertask.status = 'active'))",
},
{
name: "Mixed: false AND valid conditions",
where: "(false AND name = 'test')",
tableName: "mastertask",
expected: "(false AND mastertask.name = 'test')",
},
{
name: "Mixed: null AND valid conditions",
where: "(null AND status = 'active')",
tableName: "mastertask",
expected: "(null AND mastertask.status = 'active')",
},
{
name: "Multiple true conditions in parentheses",
where: "(true AND true AND status = 'active')",
tableName: "mastertask",
expected: "(true AND true AND mastertask.status = 'active')",
},
{
name: "Simple true without parens - should not prefix",
where: "true",
tableName: "mastertask",
expected: "true",
},
{
name: "Simple condition without parens - should prefix",
where: "status = 'active'",
tableName: "mastertask",
expected: "mastertask.status = 'active'",
},
{
name: "Unregistered table with true - should not prefix true",
where: "(true AND status = 'active')",
tableName: "unregistered_table",
expected: "(true AND unregistered_table.status = 'active')",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := AddTablePrefixToColumns(tt.where, tt.tableName)
if result != tt.expected {
t.Errorf("AddTablePrefixToColumns(%q, %q) = %q; want %q", tt.where, tt.tableName, result, tt.expected)
}
})
}
}

View File

@@ -84,7 +84,7 @@ func (h *Handler) SqlQueryList(sqlquery string, options SqlQueryOptions) HTTPFun
// Create local copy to avoid modifying the captured parameter across requests // Create local copy to avoid modifying the captured parameter across requests
sqlquery := sqlquery sqlquery := sqlquery
ctx, cancel := context.WithTimeout(r.Context(), 900*time.Second) ctx, cancel := context.WithTimeout(r.Context(), 15*time.Minute)
defer cancel() defer cancel()
var dbobjlist []map[string]interface{} var dbobjlist []map[string]interface{}
@@ -423,7 +423,7 @@ func (h *Handler) SqlQuery(sqlquery string, options SqlQueryOptions) HTTPFuncTyp
// Create local copy to avoid modifying the captured parameter across requests // Create local copy to avoid modifying the captured parameter across requests
sqlquery := sqlquery sqlquery := sqlquery
ctx, cancel := context.WithTimeout(r.Context(), 600*time.Second) ctx, cancel := context.WithTimeout(r.Context(), 15*time.Minute)
defer cancel() defer cancel()
propQry := make(map[string]string) propQry := make(map[string]string)
@@ -522,10 +522,17 @@ func (h *Handler) SqlQuery(sqlquery string, options SqlQueryOptions) HTTPFuncTyp
if strings.HasPrefix(kLower, "x-fieldfilter-") { if strings.HasPrefix(kLower, "x-fieldfilter-") {
colname := strings.ReplaceAll(kLower, "x-fieldfilter-", "") colname := strings.ReplaceAll(kLower, "x-fieldfilter-", "")
if strings.Contains(strings.ToLower(sqlquery), colname) { if strings.Contains(strings.ToLower(sqlquery), colname) {
if val == "" || val == "0" { switch val {
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("COALESCE(%s, 0) = %s", ValidSQL(colname, "colname"), ValidSQL(val, "colvalue"))) case "0":
} else { sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("COALESCE(%s, 0) = 0", ValidSQL(colname, "colname")))
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("%s = %s", ValidSQL(colname, "colname"), ValidSQL(val, "colvalue"))) case "":
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("(%[1]s = '' OR %[1]s IS NULL)", ValidSQL(colname, "colname")))
default:
if IsNumeric(val) {
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("%s = %s", ValidSQL(colname, "colname"), ValidSQL(val, "colvalue")))
} else {
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("%s = '%s'", ValidSQL(colname, "colname"), ValidSQL(val, "colvalue")))
}
} }
} }
} }
@@ -662,7 +669,10 @@ func (h *Handler) mergePathParams(r *http.Request, sqlquery string, variables ma
for k, v := range pathVars { for k, v := range pathVars {
kword := fmt.Sprintf("[%s]", k) kword := fmt.Sprintf("[%s]", k)
if strings.Contains(sqlquery, kword) { if strings.Contains(sqlquery, kword) {
sqlquery = strings.ReplaceAll(sqlquery, kword, fmt.Sprintf("%v", v)) // Sanitize the value before replacing
vStr := fmt.Sprintf("%v", v)
sanitized := ValidSQL(vStr, "colvalue")
sqlquery = strings.ReplaceAll(sqlquery, kword, sanitized)
} }
variables[k] = v variables[k] = v
@@ -690,7 +700,9 @@ func (h *Handler) mergeQueryParams(r *http.Request, sqlquery string, variables m
// Replace in SQL if placeholder exists // Replace in SQL if placeholder exists
if strings.Contains(sqlquery, kword) && len(val) > 0 { if strings.Contains(sqlquery, kword) && len(val) > 0 {
if strings.HasPrefix(parmk, "p-") { if strings.HasPrefix(parmk, "p-") {
sqlquery = strings.ReplaceAll(sqlquery, kword, val) // Sanitize the parameter value before replacing
sanitized := ValidSQL(val, "colvalue")
sqlquery = strings.ReplaceAll(sqlquery, kword, sanitized)
} }
} }
@@ -702,15 +714,36 @@ func (h *Handler) mergeQueryParams(r *http.Request, sqlquery string, variables m
// Apply filters if allowed // Apply filters if allowed
if allowFilter && len(parmk) > 1 && strings.Contains(strings.ToLower(sqlquery), strings.ToLower(parmk)) { if allowFilter && len(parmk) > 1 && strings.Contains(strings.ToLower(sqlquery), strings.ToLower(parmk)) {
if len(parmv) > 1 { if len(parmv) > 1 {
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("%s IN (%s)", ValidSQL(parmk, "colname"), strings.Join(parmv, ","))) // Sanitize each value in the IN clause with appropriate quoting
sanitizedValues := make([]string, len(parmv))
for i, v := range parmv {
if IsNumeric(v) {
// Numeric values don't need quotes
sanitizedValues[i] = ValidSQL(v, "colvalue")
} else {
// String values need quotes
sanitized := ValidSQL(v, "colvalue")
sanitizedValues[i] = fmt.Sprintf("'%s'", sanitized)
}
}
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("%s IN (%s)", ValidSQL(parmk, "colname"), strings.Join(sanitizedValues, ",")))
} else { } else {
if strings.Contains(val, "match=") { if strings.Contains(val, "match=") {
colval := strings.ReplaceAll(val, "match=", "") colval := strings.ReplaceAll(val, "match=", "")
// Escape single quotes and backslashes for LIKE patterns
// But don't escape wildcards % and _ which are intentional
colval = strings.ReplaceAll(colval, "\\", "\\\\")
colval = strings.ReplaceAll(colval, "'", "''")
if colval != "*" { if colval != "*" {
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("%s ILIKE '%%%s%%'", ValidSQL(parmk, "colname"), ValidSQL(colval, "colvalue"))) sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("%s ILIKE '%%%s%%'", ValidSQL(parmk, "colname"), colval))
} }
} else if val == "" || val == "0" { } else if val == "" || val == "0" {
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("(%[1]s = %[2]s OR %[1]s IS NULL)", ValidSQL(parmk, "colname"), ValidSQL(val, "colvalue"))) // For empty/zero values, treat as literal 0 or empty string with quotes
if val == "0" {
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("(%[1]s = 0 OR %[1]s IS NULL)", ValidSQL(parmk, "colname")))
} else {
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("(%[1]s = '' OR %[1]s IS NULL)", ValidSQL(parmk, "colname")))
}
} else { } else {
if IsNumeric(val) { if IsNumeric(val) {
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("%s = %s", ValidSQL(parmk, "colname"), ValidSQL(val, "colvalue"))) sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("%s = %s", ValidSQL(parmk, "colname"), ValidSQL(val, "colvalue")))
@@ -743,16 +776,25 @@ func (h *Handler) mergeHeaderParams(r *http.Request, sqlquery string, variables
kword := fmt.Sprintf("[%s]", k) kword := fmt.Sprintf("[%s]", k)
if strings.Contains(sqlquery, kword) { if strings.Contains(sqlquery, kword) {
sqlquery = strings.ReplaceAll(sqlquery, kword, val) // Sanitize the header value before replacing
sanitized := ValidSQL(val, "colvalue")
sqlquery = strings.ReplaceAll(sqlquery, kword, sanitized)
} }
// Handle special headers // Handle special headers
if strings.Contains(k, "x-fieldfilter-") { if strings.Contains(k, "x-fieldfilter-") {
colname := strings.ReplaceAll(k, "x-fieldfilter-", "") colname := strings.ReplaceAll(k, "x-fieldfilter-", "")
if val == "" || val == "0" { switch val {
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("COALESCE(%s, 0) = %s", ValidSQL(colname, "colname"), ValidSQL(val, "colvalue"))) case "0":
} else { sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("COALESCE(%s, 0) = 0", ValidSQL(colname, "colname")))
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("%s = %s", ValidSQL(colname, "colname"), ValidSQL(val, "colvalue"))) case "":
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("(%[1]s = '' OR %[1]s IS NULL)", ValidSQL(colname, "colname")))
default:
if IsNumeric(val) {
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("%s = %s", ValidSQL(colname, "colname"), ValidSQL(val, "colvalue")))
} else {
sqlquery = sqlQryWhere(sqlquery, fmt.Sprintf("%s = '%s'", ValidSQL(colname, "colname"), ValidSQL(val, "colvalue")))
}
} }
} }
@@ -782,12 +824,15 @@ func (h *Handler) mergeHeaderParams(r *http.Request, sqlquery string, variables
func (h *Handler) replaceMetaVariables(sqlquery string, r *http.Request, userCtx *security.UserContext, metainfo map[string]interface{}, variables map[string]interface{}) string { func (h *Handler) replaceMetaVariables(sqlquery string, r *http.Request, userCtx *security.UserContext, metainfo map[string]interface{}, variables map[string]interface{}) string {
if strings.Contains(sqlquery, "[p_meta_default]") { if strings.Contains(sqlquery, "[p_meta_default]") {
data, _ := json.Marshal(metainfo) data, _ := json.Marshal(metainfo)
sqlquery = strings.ReplaceAll(sqlquery, "[p_meta_default]", fmt.Sprintf("'%s'::jsonb", string(data))) dataStr := strings.ReplaceAll(string(data), "$META$", "/*META*/")
sqlquery = strings.ReplaceAll(sqlquery, "[p_meta_default]", fmt.Sprintf("$META$%s$META$::jsonb", dataStr))
} }
if strings.Contains(sqlquery, "[json_variables]") { if strings.Contains(sqlquery, "[json_variables]") {
data, _ := json.Marshal(variables) data, _ := json.Marshal(variables)
sqlquery = strings.ReplaceAll(sqlquery, "[json_variables]", fmt.Sprintf("'%s'::jsonb", string(data))) dataStr := strings.ReplaceAll(string(data), "$VAR$", "/*VAR*/")
sqlquery = strings.ReplaceAll(sqlquery, "[json_variables]", fmt.Sprintf("$VAR$%s$VAR$::jsonb", dataStr))
} }
if strings.Contains(sqlquery, "[rid_user]") { if strings.Contains(sqlquery, "[rid_user]") {
@@ -795,7 +840,7 @@ func (h *Handler) replaceMetaVariables(sqlquery string, r *http.Request, userCtx
} }
if strings.Contains(sqlquery, "[user]") { if strings.Contains(sqlquery, "[user]") {
sqlquery = strings.ReplaceAll(sqlquery, "[user]", fmt.Sprintf("'%s'", userCtx.UserName)) sqlquery = strings.ReplaceAll(sqlquery, "[user]", fmt.Sprintf("$USR$%s$USR$", strings.ReplaceAll(userCtx.UserName, "$USR$", "/*USR*/")))
} }
if strings.Contains(sqlquery, "[rid_session]") { if strings.Contains(sqlquery, "[rid_session]") {
@@ -806,7 +851,7 @@ func (h *Handler) replaceMetaVariables(sqlquery string, r *http.Request, userCtx
} }
if strings.Contains(sqlquery, "[method]") { if strings.Contains(sqlquery, "[method]") {
sqlquery = strings.ReplaceAll(sqlquery, "[method]", r.Method) sqlquery = strings.ReplaceAll(sqlquery, "[method]", fmt.Sprintf("$M$%s$M$", strings.ReplaceAll(r.Method, "$M$", "/*M*/")))
} }
if strings.Contains(sqlquery, "[post_body]") { if strings.Contains(sqlquery, "[post_body]") {
@@ -819,7 +864,7 @@ func (h *Handler) replaceMetaVariables(sqlquery string, r *http.Request, userCtx
} }
} }
} }
sqlquery = strings.ReplaceAll(sqlquery, "[post_body]", fmt.Sprintf("'%s'", bodystr)) sqlquery = strings.ReplaceAll(sqlquery, "[post_body]", fmt.Sprintf("$PBODY$%s$PBODY$", strings.ReplaceAll(bodystr, "$PBODY$", "/*PBODY*/")))
} }
return sqlquery return sqlquery
@@ -859,19 +904,23 @@ func ValidSQL(input, mode string) string {
reg := regexp.MustCompile(`[^a-zA-Z0-9_\.]`) reg := regexp.MustCompile(`[^a-zA-Z0-9_\.]`)
return reg.ReplaceAllString(input, "") return reg.ReplaceAllString(input, "")
case "colvalue": case "colvalue":
// For column values, escape single quotes // For column values, escape single quotes and backslashes
return strings.ReplaceAll(input, "'", "''") // Note: Backslashes must be escaped first, then single quotes
result := strings.ReplaceAll(input, "\\", "\\\\")
result = strings.ReplaceAll(result, "'", "''")
return result
case "select": case "select":
// For SELECT clauses, be more permissive but still safe // For SELECT clauses, be more permissive but still safe
// Remove semicolons and common SQL injection patterns // Remove semicolons and common SQL injection patterns (case-insensitive)
dangerous := []string{";", "--", "/*", "*/", "xp_", "sp_", "DROP ", "DELETE ", "TRUNCATE ", "UPDATE ", "INSERT "} dangerous := []string{
result := input ";", "--", "/\\*", "\\*/", "xp_", "sp_",
for _, d := range dangerous { "drop ", "delete ", "truncate ", "update ", "insert ",
result = strings.ReplaceAll(result, d, "") "exec ", "execute ", "union ", "declare ", "alter ", "create ",
result = strings.ReplaceAll(result, strings.ToLower(d), "")
result = strings.ReplaceAll(result, strings.ToUpper(d), "")
} }
return result // Build a single regex pattern with all dangerous keywords
pattern := "(?i)(" + strings.Join(dangerous, "|") + ")"
re := regexp.MustCompile(pattern)
return re.ReplaceAllString(input, "")
default: default:
return input return input
} }

703
pkg/resolvespec/README.md Normal file
View File

@@ -0,0 +1,703 @@
# ResolveSpec - Body-Based REST API
ResolveSpec provides a REST API where query options are passed in the JSON request body. This approach offers GraphQL-like flexibility while maintaining RESTful principles, making it ideal for complex queries and operations.
## Features
* **Body-Based Querying**: All query options passed via JSON request body
* **Lifecycle Hooks**: Before/after hooks for create, read, update, delete operations
* **Cursor Pagination**: Efficient cursor-based pagination with complex sorting
* **Offset Pagination**: Traditional limit/offset pagination support
* **Advanced Filtering**: Multiple operators, AND/OR logic, and custom SQL
* **Relationship Preloading**: Load related entities with custom column selection and filters
* **Recursive CRUD**: Automatically handle nested object graphs with foreign key resolution
* **Computed Columns**: Define virtual columns with SQL expressions
* **Database-Agnostic**: Works with GORM, Bun, or custom database adapters
* **Router-Agnostic**: Integrates with any HTTP router through standard interfaces
* **Type-Safe**: Strong type validation and conversion
## Quick Start
### Setup with GORM
```go
import "github.com/bitechdev/ResolveSpec/pkg/resolvespec"
import "github.com/gorilla/mux"
// Create handler
handler := resolvespec.NewHandlerWithGORM(db)
// IMPORTANT: Register models BEFORE setting up routes
handler.registry.RegisterModel("core.users", &User{})
handler.registry.RegisterModel("core.posts", &Post{})
// Setup routes
router := mux.NewRouter()
resolvespec.SetupMuxRoutes(router, handler, nil)
// Start server
http.ListenAndServe(":8080", router)
```
### Setup with Bun ORM
```go
import "github.com/bitechdev/ResolveSpec/pkg/resolvespec"
import "github.com/uptrace/bun"
// Create handler with Bun
handler := resolvespec.NewHandlerWithBun(bunDB)
// Register models
handler.registry.RegisterModel("core.users", &User{})
// Setup routes (same as GORM)
router := mux.NewRouter()
resolvespec.SetupMuxRoutes(router, handler, nil)
```
## Basic Usage
### Simple Read Request
```http
POST /core/users HTTP/1.1
Content-Type: application/json
```
### With Preloading
```http
POST /core/users HTTP/1.1
Content-Type: application/json
```
## Request Structure
### Request Format
```json
{
"operation": "read|create|update|delete",
"data": {
// For create/update operations
},
"options": {
"columns": [...],
"preload": [...],
"filters": [...],
"sort": [...],
"limit": number,
"offset": number,
"cursor_forward": "string",
"cursor_backward": "string",
"customOperators": [...],
"computedColumns": [...]
}
}
```
### Operations
| Operation | Description | Requires Data | Requires ID |
|-----------|-------------|---------------|-------------|
| `read` | Fetch records | No | Optional (single record) |
| `create` | Create new record(s) | Yes | No |
| `update` | Update existing record(s) | Yes | Yes (in URL) |
| `delete` | Delete record(s) | No | Yes (in URL) |
### Options Fields
| Field | Type | Description | Example |
|-------|------|-------------|---------|
| `columns` | `[]string` | Columns to select | `["id", "name", "email"]` |
| `preload` | `[]PreloadConfig` | Relations to load | See [Preloading](#preloading) |
| `filters` | `[]Filter` | Filter conditions | See [Filtering](#filtering) |
| `sort` | `[]Sort` | Sort criteria | `[{"column": "created_at", "direction": "desc"}]` |
| `limit` | `int` | Max records to return | `50` |
| `offset` | `int` | Number of records to skip | `100` |
| `cursor_forward` | `string` | Cursor for next page | `"12345"` |
| `cursor_backward` | `string` | Cursor for previous page | `"12300"` |
| `customOperators` | `[]CustomOperator` | Custom SQL conditions | See [Custom Operators](#custom-operators) |
| `computedColumns` | `[]ComputedColumn` | Virtual columns | See [Computed Columns](#computed-columns) |
## Filtering
### Available Operators
| Operator | Description | Example |
|----------|-------------|---------|
| `eq` | Equal | `{"column": "status", "operator": "eq", "value": "active"}` |
| `neq` | Not Equal | `{"column": "status", "operator": "neq", "value": "deleted"}` |
| `gt` | Greater Than | `{"column": "age", "operator": "gt", "value": 18}` |
| `gte` | Greater Than or Equal | `{"column": "age", "operator": "gte", "value": 18}` |
| `lt` | Less Than | `{"column": "price", "operator": "lt", "value": 100}` |
| `lte` | Less Than or Equal | `{"column": "price", "operator": "lte", "value": 100}` |
| `like` | LIKE pattern | `{"column": "name", "operator": "like", "value": "%john%"}` |
| `ilike` | Case-insensitive LIKE | `{"column": "email", "operator": "ilike", "value": "%@example.com"}` |
| `in` | IN clause | `{"column": "status", "operator": "in", "value": ["active", "pending"]}` |
| `contains` | Contains string | `{"column": "description", "operator": "contains", "value": "important"}` |
| `startswith` | Starts with string | `{"column": "name", "operator": "startswith", "value": "John"}` |
| `endswith` | Ends with string | `{"column": "email", "operator": "endswith", "value": "@example.com"}` |
| `between` | Between (exclusive) | `{"column": "age", "operator": "between", "value": [18, 65]}` |
| `betweeninclusive` | Between (inclusive) | `{"column": "price", "operator": "betweeninclusive", "value": [10, 100]}` |
| `empty` | IS NULL or empty | `{"column": "deleted_at", "operator": "empty"}` |
| `notempty` | IS NOT NULL | `{"column": "email", "operator": "notempty"}` |
### Complex Filtering Example
```json
{
"operation": "read",
"options": {
"filters": [
{
"column": "status",
"operator": "eq",
"value": "active"
},
{
"column": "age",
"operator": "gte",
"value": 18
},
{
"column": "email",
"operator": "ilike",
"value": "%@company.com"
}
]
}
}
```
## Preloading
Load related entities with custom configuration:
```json
{
"operation": "read",
"options": {
"columns": ["id", "name", "email"],
"preload": [
{
"relation": "posts",
"columns": ["id", "title", "created_at"],
"filters": [
{
"column": "status",
"operator": "eq",
"value": "published"
}
],
"sort": [
{
"column": "created_at",
"direction": "desc"
}
],
"limit": 5
},
{
"relation": "profile",
"columns": ["bio", "website"]
}
]
}
}
```
## Cursor Pagination
Efficient pagination for large datasets:
### First Request (No Cursor)
```json
{
"operation": "read",
"options": {
"sort": [
{
"column": "created_at",
"direction": "desc"
},
{
"column": "id",
"direction": "asc"
}
],
"limit": 50
}
}
```
### Next Page (Forward Cursor)
```json
{
"operation": "read",
"options": {
"sort": [
{
"column": "created_at",
"direction": "desc"
},
{
"column": "id",
"direction": "asc"
}
],
"limit": 50,
"cursor_forward": "12345"
}
}
```
### Previous Page (Backward Cursor)
```json
{
"operation": "read",
"options": {
"sort": [
{
"column": "created_at",
"direction": "desc"
},
{
"column": "id",
"direction": "asc"
}
],
"limit": 50,
"cursor_backward": "12300"
}
}
```
**Benefits over offset pagination**:
* Consistent results when data changes
* Better performance for large offsets
* Prevents "skipped" or duplicate records
* Works with complex sort expressions
## Recursive CRUD Operations
Automatically handle nested object graphs with intelligent foreign key resolution.
### Creating Nested Objects
```json
{
"operation": "create",
"data": {
"name": "John Doe",
"email": "john@example.com",
"posts": [
{
"title": "My First Post",
"content": "Hello World",
"tags": [
{"name": "tech"},
{"name": "programming"}
]
},
{
"title": "Second Post",
"content": "More content"
}
],
"profile": {
"bio": "Software Developer",
"website": "https://example.com"
}
}
}
```
### Per-Record Operation Control with `_request`
Control individual operations for each nested record:
```json
{
"operation": "update",
"data": {
"name": "John Updated",
"posts": [
{
"_request": "insert",
"title": "New Post",
"content": "Fresh content"
},
{
"_request": "update",
"id": 456,
"title": "Updated Post Title"
},
{
"_request": "delete",
"id": 789
}
]
}
}
```
**Supported `_request` values**:
* `insert` - Create a new related record
* `update` - Update an existing related record
* `delete` - Delete a related record
* `upsert` - Create if doesn't exist, update if exists
**How It Works**:
1. Automatic foreign key resolution - parent IDs propagate to children
2. Recursive processing - handles nested relationships at any depth
3. Transaction safety - all operations execute atomically
4. Relationship detection - automatically detects belongsTo, hasMany, hasOne, many2many
5. Flexible operations - mix create, update, and delete in one request
## Computed Columns
Define virtual columns using SQL expressions:
```json
{
"operation": "read",
"options": {
"columns": ["id", "first_name", "last_name"],
"computedColumns": [
{
"name": "full_name",
"expression": "CONCAT(first_name, ' ', last_name)"
},
{
"name": "age_years",
"expression": "EXTRACT(YEAR FROM AGE(birth_date))"
}
]
}
}
```
## Custom Operators
Add custom SQL conditions when needed:
```json
{
"operation": "read",
"options": {
"customOperators": [
{
"condition": "LOWER(email) LIKE ?",
"values": ["%@example.com"]
},
{
"condition": "created_at > NOW() - INTERVAL '7 days'"
}
]
}
}
```
## Lifecycle Hooks
Register hooks for all CRUD operations:
```go
import "github.com/bitechdev/ResolveSpec/pkg/resolvespec"
// Create handler
handler := resolvespec.NewHandlerWithGORM(db)
// Register a before-read hook (e.g., for authorization)
handler.Hooks().Register(resolvespec.BeforeRead, func(ctx *resolvespec.HookContext) error {
// Check permissions
if !userHasPermission(ctx.Context, ctx.Entity) {
return fmt.Errorf("unauthorized access to %s", ctx.Entity)
}
// Modify query options
if ctx.Options.Limit == nil || *ctx.Options.Limit > 100 {
ctx.Options.Limit = ptr(100) // Enforce max limit
}
return nil
})
// Register an after-read hook (e.g., for data transformation)
handler.Hooks().Register(resolvespec.AfterRead, func(ctx *resolvespec.HookContext) error {
// Transform or filter results
if users, ok := ctx.Result.([]User); ok {
for i := range users {
users[i].Email = maskEmail(users[i].Email)
}
}
return nil
})
// Register a before-create hook (e.g., for validation)
handler.Hooks().Register(resolvespec.BeforeCreate, func(ctx *resolvespec.HookContext) error {
// Validate data
if user, ok := ctx.Data.(*User); ok {
if user.Email == "" {
return fmt.Errorf("email is required")
}
// Add timestamps
user.CreatedAt = time.Now()
}
return nil
})
```
**Available Hook Types**:
* `BeforeRead`, `AfterRead`
* `BeforeCreate`, `AfterCreate`
* `BeforeUpdate`, `AfterUpdate`
* `BeforeDelete`, `AfterDelete`
**HookContext** provides:
* `Context`: Request context
* `Handler`: Access to handler, database, and registry
* `Schema`, `Entity`, `TableName`: Request info
* `Model`: The registered model type
* `Options`: Parsed request options (filters, sorting, etc.)
* `ID`: Record ID (for single-record operations)
* `Data`: Request data (for create/update)
* `Result`: Operation result (for after hooks)
* `Writer`: Response writer (allows hooks to modify response)
## Model Registration
```go
type User struct {
ID uint `json:"id" gorm:"primaryKey"`
Name string `json:"name"`
Email string `json:"email"`
Status string `json:"status"`
CreatedAt time.Time `json:"created_at"`
Posts []Post `json:"posts,omitempty" gorm:"foreignKey:UserID"`
Profile *Profile `json:"profile,omitempty" gorm:"foreignKey:UserID"`
}
type Post struct {
ID uint `json:"id" gorm:"primaryKey"`
UserID uint `json:"user_id"`
Title string `json:"title"`
Content string `json:"content"`
Status string `json:"status"`
CreatedAt time.Time `json:"created_at"`
Tags []Tag `json:"tags,omitempty" gorm:"many2many:post_tags"`
}
// Schema.Table format
handler.registry.RegisterModel("core.users", &User{})
handler.registry.RegisterModel("core.posts", &Post{})
```
## Complete Example
```go
package main
import (
"log"
"net/http"
"github.com/bitechdev/ResolveSpec/pkg/resolvespec"
"github.com/gorilla/mux"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
type User struct {
ID uint `json:"id" gorm:"primaryKey"`
Name string `json:"name"`
Email string `json:"email"`
Status string `json:"status"`
Posts []Post `json:"posts,omitempty" gorm:"foreignKey:UserID"`
}
type Post struct {
ID uint `json:"id" gorm:"primaryKey"`
UserID uint `json:"user_id"`
Title string `json:"title"`
Content string `json:"content"`
Status string `json:"status"`
}
func main() {
// Connect to database
db, err := gorm.Open(postgres.Open("your-connection-string"), &gorm.Config{})
if err != nil {
log.Fatal(err)
}
// Create handler
handler := resolvespec.NewHandlerWithGORM(db)
// Register models
handler.registry.RegisterModel("core.users", &User{})
handler.registry.RegisterModel("core.posts", &Post{})
// Add hooks
handler.Hooks().Register(resolvespec.BeforeRead, func(ctx *resolvespec.HookContext) error {
log.Printf("Reading %s", ctx.Entity)
return nil
})
// Setup routes
router := mux.NewRouter()
resolvespec.SetupMuxRoutes(router, handler, nil)
// Start server
log.Println("Server starting on :8080")
log.Fatal(http.ListenAndServe(":8080", router))
}
```
## Testing
ResolveSpec is designed for testability:
```go
import (
"bytes"
"encoding/json"
"net/http/httptest"
"testing"
)
func TestUserRead(t *testing.T) {
handler := resolvespec.NewHandlerWithGORM(testDB)
handler.registry.RegisterModel("core.users", &User{})
reqBody := map[string]interface{}{
"operation": "read",
"options": map[string]interface{}{
"columns": []string{"id", "name"},
"limit": 10,
},
}
body, _ := json.Marshal(reqBody)
req := httptest.NewRequest("POST", "/core/users", bytes.NewReader(body))
rec := httptest.NewRecorder()
// Test your handler...
}
```
## Router Integration
### Gorilla Mux
```go
router := mux.NewRouter()
resolvespec.SetupMuxRoutes(router, handler, nil)
```
### BunRouter
```go
router := bunrouter.New()
resolvespec.SetupBunRouterWithResolveSpec(router, handler)
```
### Custom Routers
```go
// Implement custom integration using common.Request and common.ResponseWriter
router.POST("/:schema/:entity", func(w http.ResponseWriter, r *http.Request) {
params := extractParams(r) // Your param extraction logic
reqAdapter := router.NewHTTPRequest(r)
respAdapter := router.NewHTTPResponseWriter(w)
handler.Handle(respAdapter, reqAdapter, params)
})
```
## Response Format
### Success Response
```json
{
"success": true,
"data": [...],
"metadata": {
"total": 100,
"filtered": 50,
"limit": 10,
"offset": 0
}
}
```
### Error Response
```json
{
"success": false,
"error": {
"code": "validation_error",
"message": "Invalid request",
"details": "..."
}
}
```
## See Also
* [Main README](../../README.md) - ResolveSpec overview
* [RestHeadSpec Package](../restheadspec/README.md) - Header-based API
* [StaticWeb Package](../server/staticweb/README.md) - Static file server
## License
This package is part of ResolveSpec and is licensed under the MIT License.
```
## Response Format
### Success Response
```json
{
"success": true,
"data": [...],
"metadata": {
"total": 100,
"filtered": 50,
"limit": 10,
"offset": 0
}
}
```
### Error Response
```json
{
"success": false,
"error": {
"code": "validation_error",
"message": "Invalid request",
"details": "..."
}
}
```
## See Also
* [Main README](../../README.md) - ResolveSpec overview
* [RestHeadSpec Package](../restheadspec/README.md) - Header-based API
* [StaticWeb Package](../server/staticweb/README.md) - Static file server
## License
This package is part of ResolveSpec and is licensed under the MIT License.

445
pkg/restheadspec/README.md Normal file
View File

@@ -0,0 +1,445 @@
# RestHeadSpec - Header-Based REST API
RestHeadSpec provides a REST API where all query options are passed via HTTP headers instead of the request body. This provides cleaner separation between data and metadata, making it ideal for GET requests and RESTful architectures.
## Features
* **Header-Based Querying**: All query options via HTTP headers
* **Lifecycle Hooks**: Before/after hooks for create, read, update, delete operations
* **Cursor Pagination**: Efficient cursor-based pagination with complex sorting
* **Advanced Filtering**: Field filters, search operators, AND/OR logic
* **Multiple Response Formats**: Simple, detailed, and Syncfusion-compatible responses
* **Single Record as Object**: Automatically return single-element arrays as objects (default)
* **Base64 Support**: Base64-encoded header values for complex queries
* **Type-Aware Filtering**: Automatic type detection and conversion
* **CORS Support**: Comprehensive CORS headers for cross-origin requests
* **OPTIONS Method**: Full OPTIONS support for CORS preflight
## Quick Start
### Setup with GORM
```go
import "github.com/bitechdev/ResolveSpec/pkg/restheadspec"
import "github.com/gorilla/mux"
// Create handler
handler := restheadspec.NewHandlerWithGORM(db)
// IMPORTANT: Register models BEFORE setting up routes
handler.Registry.RegisterModel("public.users", &User{})
handler.Registry.RegisterModel("public.posts", &Post{})
// Setup routes
router := mux.NewRouter()
restheadspec.SetupMuxRoutes(router, handler, nil)
// Start server
http.ListenAndServe(":8080", router)
```
### Setup with Bun ORM
```go
import "github.com/bitechdev/ResolveSpec/pkg/restheadspec"
import "github.com/uptrace/bun"
// Create handler with Bun
handler := restheadspec.NewHandlerWithBun(bunDB)
// Register models
handler.Registry.RegisterModel("public.users", &User{})
// Setup routes (same as GORM)
router := mux.NewRouter()
restheadspec.SetupMuxRoutes(router, handler, nil)
```
## Basic Usage
### Simple GET Request
```http
GET /public/users HTTP/1.1
Host: api.example.com
X-Select-Fields: id,name,email
X-FieldFilter-Status: active
X-Sort: -created_at
X-Limit: 50
```
### With Preloading
```http
GET /public/users HTTP/1.1
X-Select-Fields: id,name,email,department_id
X-Preload: department:id,name
X-FieldFilter-Status: active
X-Limit: 50
```
## Common Headers
| Header | Description | Example |
|--------|-------------|---------|
| `X-Select-Fields` | Columns to include | `id,name,email` |
| `X-Not-Select-Fields` | Columns to exclude | `password,internal_notes` |
| `X-FieldFilter-{col}` | Exact match filter | `X-FieldFilter-Status: active` |
| `X-SearchFilter-{col}` | Fuzzy search (ILIKE) | `X-SearchFilter-Name: john` |
| `X-SearchOp-{op}-{col}` | Filter with operator | `X-SearchOp-Gte-Age: 18` |
| `X-Preload` | Preload relations | `posts:id,title` |
| `X-Sort` | Sort columns | `-created_at,+name` |
| `X-Limit` | Limit results | `50` |
| `X-Offset` | Offset for pagination | `100` |
| `X-Clean-JSON` | Remove null/empty fields | `true` |
| `X-Single-Record-As-Object` | Return single records as objects | `false` |
**Available Operators**: `eq`, `neq`, `gt`, `gte`, `lt`, `lte`, `contains`, `startswith`, `endswith`, `between`, `betweeninclusive`, `in`, `empty`, `notempty`
For complete header documentation, see [HEADERS.md](HEADERS.md).
## Lifecycle Hooks
RestHeadSpec supports lifecycle hooks for all CRUD operations:
```go
import "github.com/bitechdev/ResolveSpec/pkg/restheadspec"
// Create handler
handler := restheadspec.NewHandlerWithGORM(db)
// Register a before-read hook (e.g., for authorization)
handler.Hooks.Register(restheadspec.BeforeRead, func(ctx *restheadspec.HookContext) error {
// Check permissions
if !userHasPermission(ctx.Context, ctx.Entity) {
return fmt.Errorf("unauthorized access to %s", ctx.Entity)
}
// Modify query options
ctx.Options.Limit = ptr(100) // Enforce max limit
return nil
})
// Register an after-read hook (e.g., for data transformation)
handler.Hooks.Register(restheadspec.AfterRead, func(ctx *restheadspec.HookContext) error {
// Transform or filter results
if users, ok := ctx.Result.([]User); ok {
for i := range users {
users[i].Email = maskEmail(users[i].Email)
}
}
return nil
})
// Register a before-create hook (e.g., for validation)
handler.Hooks.Register(restheadspec.BeforeCreate, func(ctx *restheadspec.HookContext) error {
// Validate data
if user, ok := ctx.Data.(*User); ok {
if user.Email == "" {
return fmt.Errorf("email is required")
}
// Add timestamps
user.CreatedAt = time.Now()
}
return nil
})
```
**Available Hook Types**:
* `BeforeRead`, `AfterRead`
* `BeforeCreate`, `AfterCreate`
* `BeforeUpdate`, `AfterUpdate`
* `BeforeDelete`, `AfterDelete`
**HookContext** provides:
* `Context`: Request context
* `Handler`: Access to handler, database, and registry
* `Schema`, `Entity`, `TableName`: Request info
* `Model`: The registered model type
* `Options`: Parsed request options (filters, sorting, etc.)
* `ID`: Record ID (for single-record operations)
* `Data`: Request data (for create/update)
* `Result`: Operation result (for after hooks)
* `Writer`: Response writer (allows hooks to modify response)
## Cursor Pagination
RestHeadSpec supports efficient cursor-based pagination for large datasets:
```http
GET /public/posts HTTP/1.1
X-Sort: -created_at,+id
X-Limit: 50
X-Cursor-Forward: <cursor_token>
```
**How it works**:
1. First request returns results + cursor token in response
2. Subsequent requests use `X-Cursor-Forward` or `X-Cursor-Backward`
3. Cursor maintains consistent ordering even with data changes
4. Supports complex multi-column sorting
**Benefits over offset pagination**:
* Consistent results when data changes
* Better performance for large offsets
* Prevents "skipped" or duplicate records
* Works with complex sort expressions
**Example with hooks**:
```go
// Enable cursor pagination in a hook
handler.Hooks.Register(restheadspec.BeforeRead, func(ctx *restheadspec.HookContext) error {
// For large tables, enforce cursor pagination
if ctx.Entity == "posts" && ctx.Options.Offset != nil && *ctx.Options.Offset > 1000 {
return fmt.Errorf("use cursor pagination for large offsets")
}
return nil
})
```
## Response Formats
RestHeadSpec supports multiple response formats:
**1. Simple Format** (`X-SimpleApi: true`):
```json
[
{ "id": 1, "name": "John" },
{ "id": 2, "name": "Jane" }
]
```
**2. Detail Format** (`X-DetailApi: true`, default):
```json
{
"success": true,
"data": [...],
"metadata": {
"total": 100,
"filtered": 100,
"limit": 50,
"offset": 0
}
}
```
**3. Syncfusion Format** (`X-Syncfusion: true`):
```json
{
"result": [...],
"count": 100
}
```
## Single Record as Object (Default Behavior)
By default, RestHeadSpec automatically converts single-element arrays into objects for cleaner API responses.
**Default behavior (enabled)**:
```http
GET /public/users/123
```
```json
{
"success": true,
"data": { "id": 123, "name": "John", "email": "john@example.com" }
}
```
**To disable** (force arrays):
```http
GET /public/users/123
X-Single-Record-As-Object: false
```
```json
{
"success": true,
"data": [{ "id": 123, "name": "John", "email": "john@example.com" }]
}
```
**How it works**:
* When a query returns exactly **one record**, it's returned as an object
* When a query returns **multiple records**, they're returned as an array
* Set `X-Single-Record-As-Object: false` to always receive arrays
* Works with all response formats (simple, detail, syncfusion)
* Applies to both read operations and create/update returning clauses
## CORS & OPTIONS Support
RestHeadSpec includes comprehensive CORS support for cross-origin requests:
**OPTIONS Method**:
```http
OPTIONS /public/users HTTP/1.1
```
Returns metadata with appropriate CORS headers:
```http
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Headers: Content-Type, Authorization, X-Select-Fields, X-FieldFilter-*, ...
Access-Control-Max-Age: 86400
Access-Control-Allow-Credentials: true
```
**Key Features**:
* OPTIONS returns model metadata (same as GET metadata endpoint)
* All HTTP methods include CORS headers automatically
* OPTIONS requests don't require authentication (CORS preflight)
* Supports all HeadSpec custom headers (`X-Select-Fields`, `X-FieldFilter-*`, etc.)
* 24-hour max age to reduce preflight requests
**Configuration**:
```go
import "github.com/bitechdev/ResolveSpec/pkg/common"
// Get default CORS config
corsConfig := common.DefaultCORSConfig()
// Customize if needed
corsConfig.AllowedOrigins = []string{"https://example.com"}
corsConfig.AllowedMethods = []string{"GET", "POST", "PUT", "DELETE", "OPTIONS"}
```
## Advanced Features
### Base64 Encoding
For complex header values, use base64 encoding:
```http
GET /public/users HTTP/1.1
X-Select-Fields-Base64: aWQsbmFtZSxlbWFpbA==
```
### AND/OR Logic
Combine multiple filters with AND/OR logic:
```http
GET /public/users HTTP/1.1
X-FieldFilter-Status: active
X-SearchOp-Gte-Age: 18
X-Filter-Logic: AND
```
### Complex Preloading
Load nested relationships:
```http
GET /public/users HTTP/1.1
X-Preload: posts:id,title,comments:id,text,author:name
```
## Model Registration
```go
type User struct {
ID uint `json:"id" gorm:"primaryKey"`
Name string `json:"name"`
Email string `json:"email"`
Posts []Post `json:"posts,omitempty" gorm:"foreignKey:UserID"`
}
// Schema.Table format
handler.Registry.RegisterModel("public.users", &User{})
```
## Complete Example
```go
package main
import (
"log"
"net/http"
"github.com/bitechdev/ResolveSpec/pkg/restheadspec"
"github.com/gorilla/mux"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
type User struct {
ID uint `json:"id" gorm:"primaryKey"`
Name string `json:"name"`
Email string `json:"email"`
Status string `json:"status"`
}
func main() {
// Connect to database
db, err := gorm.Open(postgres.Open("your-connection-string"), &gorm.Config{})
if err != nil {
log.Fatal(err)
}
// Create handler
handler := restheadspec.NewHandlerWithGORM(db)
// Register models
handler.Registry.RegisterModel("public.users", &User{})
// Add hooks
handler.Hooks.Register(restheadspec.BeforeRead, func(ctx *restheadspec.HookContext) error {
log.Printf("Reading %s", ctx.Entity)
return nil
})
// Setup routes
router := mux.NewRouter()
restheadspec.SetupMuxRoutes(router, handler, nil)
// Start server
log.Println("Server starting on :8080")
log.Fatal(http.ListenAndServe(":8080", router))
}
```
## Testing
RestHeadSpec is designed for testability:
```go
import (
"net/http/httptest"
"testing"
)
func TestUserRead(t *testing.T) {
handler := restheadspec.NewHandlerWithGORM(testDB)
handler.Registry.RegisterModel("public.users", &User{})
req := httptest.NewRequest("GET", "/public/users", nil)
req.Header.Set("X-Select-Fields", "id,name")
req.Header.Set("X-Limit", "10")
rec := httptest.NewRecorder()
// Test your handler...
}
```
## See Also
* [HEADERS.md](HEADERS.md) - Complete header reference
* [Main README](../../README.md) - ResolveSpec overview
* [ResolveSpec Package](../resolvespec/README.md) - Body-based API
* [StaticWeb Package](../server/staticweb/README.md) - Static file server
## License
This package is part of ResolveSpec and is licensed under the MIT License.

View File

@@ -0,0 +1,193 @@
package restheadspec
import (
"encoding/json"
"net/http"
"strings"
"testing"
"github.com/bitechdev/ResolveSpec/pkg/common"
)
// Test that normalizeResultArray returns empty array when no records found without ID
func TestNormalizeResultArray_EmptyArrayWhenNoID(t *testing.T) {
handler := &Handler{}
tests := []struct {
name string
input interface{}
shouldBeEmptyArr bool
}{
{
name: "nil should return empty array",
input: nil,
shouldBeEmptyArr: true,
},
{
name: "empty slice should return empty array",
input: []*EmptyTestModel{},
shouldBeEmptyArr: true,
},
{
name: "single element should return the element",
input: []*EmptyTestModel{{ID: 1, Name: "test"}},
shouldBeEmptyArr: false,
},
{
name: "multiple elements should return the slice",
input: []*EmptyTestModel{
{ID: 1, Name: "test1"},
{ID: 2, Name: "test2"},
},
shouldBeEmptyArr: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := handler.normalizeResultArray(tt.input)
// For cases that should return empty array
if tt.shouldBeEmptyArr {
emptyArr, ok := result.([]interface{})
if !ok {
t.Errorf("Expected empty array []interface{}{}, got %T: %v", result, result)
return
}
if len(emptyArr) != 0 {
t.Errorf("Expected empty array with length 0, got length %d", len(emptyArr))
}
// Verify it serializes to [] and not null
jsonBytes, err := json.Marshal(result)
if err != nil {
t.Errorf("Failed to marshal result: %v", err)
return
}
if string(jsonBytes) != "[]" {
t.Errorf("Expected JSON '[]', got '%s'", string(jsonBytes))
}
}
})
}
}
// Test that sendFormattedResponse adds X-No-Data-Found header
func TestSendFormattedResponse_NoDataFoundHeader(t *testing.T) {
handler := &Handler{}
// Mock ResponseWriter
mockWriter := &MockTestResponseWriter{
headers: make(map[string]string),
}
metadata := &common.Metadata{
Total: 0,
Count: 0,
Filtered: 0,
Limit: 10,
Offset: 0,
}
options := ExtendedRequestOptions{
RequestOptions: common.RequestOptions{},
}
// Test with empty data
emptyData := []interface{}{}
handler.sendFormattedResponse(mockWriter, emptyData, metadata, options)
// Check if X-No-Data-Found header was set
if mockWriter.headers["X-No-Data-Found"] != "true" {
t.Errorf("Expected X-No-Data-Found header to be 'true', got '%s'", mockWriter.headers["X-No-Data-Found"])
}
// Verify the body is an empty array
if mockWriter.body == nil {
t.Error("Expected body to be set, got nil")
} else {
bodyBytes, err := json.Marshal(mockWriter.body)
if err != nil {
t.Errorf("Failed to marshal body: %v", err)
}
// The body should be wrapped in a Response object with "data" field
bodyStr := string(bodyBytes)
if !strings.Contains(bodyStr, `"data":[]`) && !strings.Contains(bodyStr, `"result":[]`) {
t.Errorf("Expected body to contain empty array, got: %s", bodyStr)
}
}
}
// Test that sendResponseWithOptions adds X-No-Data-Found header
func TestSendResponseWithOptions_NoDataFoundHeader(t *testing.T) {
handler := &Handler{}
// Mock ResponseWriter
mockWriter := &MockTestResponseWriter{
headers: make(map[string]string),
}
metadata := &common.Metadata{}
options := &ExtendedRequestOptions{}
// Test with nil data
handler.sendResponseWithOptions(mockWriter, nil, metadata, options)
// Check if X-No-Data-Found header was set
if mockWriter.headers["X-No-Data-Found"] != "true" {
t.Errorf("Expected X-No-Data-Found header to be 'true', got '%s'", mockWriter.headers["X-No-Data-Found"])
}
// Check status code is 200
if mockWriter.statusCode != 200 {
t.Errorf("Expected status code 200, got %d", mockWriter.statusCode)
}
// Verify the body is an empty array
if mockWriter.body == nil {
t.Error("Expected body to be set, got nil")
} else {
bodyBytes, err := json.Marshal(mockWriter.body)
if err != nil {
t.Errorf("Failed to marshal body: %v", err)
}
bodyStr := string(bodyBytes)
if bodyStr != "[]" {
t.Errorf("Expected body to be '[]', got: %s", bodyStr)
}
}
}
// MockTestResponseWriter for testing
type MockTestResponseWriter struct {
headers map[string]string
statusCode int
body interface{}
}
func (m *MockTestResponseWriter) SetHeader(key, value string) {
m.headers[key] = value
}
func (m *MockTestResponseWriter) WriteHeader(statusCode int) {
m.statusCode = statusCode
}
func (m *MockTestResponseWriter) Write(data []byte) (int, error) {
return len(data), nil
}
func (m *MockTestResponseWriter) WriteJSON(data interface{}) error {
m.body = data
return nil
}
func (m *MockTestResponseWriter) UnderlyingResponseWriter() http.ResponseWriter {
return nil
}
// EmptyTestModel for testing
type EmptyTestModel struct {
ID int64 `json:"id"`
Name string `json:"name"`
}

View File

@@ -2143,12 +2143,22 @@ func (h *Handler) sendResponse(w common.ResponseWriter, data interface{}, metada
// sendResponseWithOptions sends a response with optional formatting // sendResponseWithOptions sends a response with optional formatting
func (h *Handler) sendResponseWithOptions(w common.ResponseWriter, data interface{}, metadata *common.Metadata, options *ExtendedRequestOptions) { func (h *Handler) sendResponseWithOptions(w common.ResponseWriter, data interface{}, metadata *common.Metadata, options *ExtendedRequestOptions) {
w.SetHeader("Content-Type", "application/json") w.SetHeader("Content-Type", "application/json")
// Handle nil data - convert to empty array
if data == nil { if data == nil {
data = map[string]interface{}{} data = []interface{}{}
w.WriteHeader(http.StatusPartialContent)
} else {
w.WriteHeader(http.StatusOK)
} }
// Calculate data length after nil conversion
dataLen := reflection.Len(data)
// Add X-No-Data-Found header when no records were found
if dataLen == 0 {
w.SetHeader("X-No-Data-Found", "true")
}
w.WriteHeader(http.StatusOK)
// Normalize single-record arrays to objects if requested // Normalize single-record arrays to objects if requested
if options != nil && options.SingleRecordAsObject { if options != nil && options.SingleRecordAsObject {
data = h.normalizeResultArray(data) data = h.normalizeResultArray(data)
@@ -2165,7 +2175,7 @@ func (h *Handler) sendResponseWithOptions(w common.ResponseWriter, data interfac
// Returns the single element if data is a slice/array with exactly one element, otherwise returns data unchanged // Returns the single element if data is a slice/array with exactly one element, otherwise returns data unchanged
func (h *Handler) normalizeResultArray(data interface{}) interface{} { func (h *Handler) normalizeResultArray(data interface{}) interface{} {
if data == nil { if data == nil {
return map[string]interface{}{} return []interface{}{}
} }
// Use reflection to check if data is a slice or array // Use reflection to check if data is a slice or array
@@ -2180,15 +2190,15 @@ func (h *Handler) normalizeResultArray(data interface{}) interface{} {
// Return the single element // Return the single element
return dataValue.Index(0).Interface() return dataValue.Index(0).Interface()
} else if dataValue.Len() == 0 { } else if dataValue.Len() == 0 {
// Return empty object instead of empty array // Keep empty array as empty array, don't convert to empty object
return map[string]interface{}{} return []interface{}{}
} }
} }
if dataValue.Kind() == reflect.String { if dataValue.Kind() == reflect.String {
str := dataValue.String() str := dataValue.String()
if str == "" || str == "null" { if str == "" || str == "null" {
return map[string]interface{}{} return []interface{}{}
} }
} }
@@ -2199,16 +2209,25 @@ func (h *Handler) normalizeResultArray(data interface{}) interface{} {
func (h *Handler) sendFormattedResponse(w common.ResponseWriter, data interface{}, metadata *common.Metadata, options ExtendedRequestOptions) { func (h *Handler) sendFormattedResponse(w common.ResponseWriter, data interface{}, metadata *common.Metadata, options ExtendedRequestOptions) {
// Normalize single-record arrays to objects if requested // Normalize single-record arrays to objects if requested
httpStatus := http.StatusOK httpStatus := http.StatusOK
// Handle nil data - convert to empty array
if data == nil { if data == nil {
data = map[string]interface{}{} data = []interface{}{}
httpStatus = http.StatusPartialContent
} else {
dataLen := reflection.Len(data)
if dataLen == 0 {
httpStatus = http.StatusPartialContent
}
} }
// Calculate data length after nil conversion
// Note: This is done BEFORE normalization because X-No-Data-Found indicates
// whether data was found in the database, not the final response format
dataLen := reflection.Len(data)
// Add X-No-Data-Found header when no records were found
if dataLen == 0 {
w.SetHeader("X-No-Data-Found", "true")
}
// Apply normalization after header is set
// normalizeResultArray may convert single-element arrays to objects,
// but the X-No-Data-Found header reflects the original query result
if options.SingleRecordAsObject { if options.SingleRecordAsObject {
data = h.normalizeResultArray(data) data = h.normalizeResultArray(data)
} }

View File

@@ -0,0 +1,439 @@
# StaticWeb - Interface-Driven Static File Server
StaticWeb is a flexible, interface-driven Go package for serving static files over HTTP. It supports multiple filesystem backends (local, zip, embedded) and provides pluggable policies for caching, MIME types, and fallback strategies.
## Features
- **Router-Agnostic**: Works with any HTTP router through standard `http.Handler`
- **Multiple Filesystem Providers**: Local directories, zip files, embedded filesystems
- **Pluggable Policies**: Customizable cache, MIME type, and fallback strategies
- **Thread-Safe**: Safe for concurrent use
- **Resource Management**: Proper lifecycle management with `Close()` methods
- **Extensible**: Easy to add new providers and policies
## Installation
```bash
go get github.com/bitechdev/ResolveSpec/pkg/server/staticweb
```
## Quick Start
### Basic Usage
Serve files from a local directory:
```go
import "github.com/bitechdev/ResolveSpec/pkg/server/staticweb"
// Create service
service := staticweb.NewService(nil)
// Mount a local directory
provider, _ := staticweb.LocalProvider("./public")
service.Mount(staticweb.MountConfig{
URLPrefix: "/static",
Provider: provider,
})
// Use with any router
router.PathPrefix("/").Handler(service.Handler())
```
### Single Page Application (SPA)
Serve an SPA with HTML fallback routing:
```go
service := staticweb.NewService(nil)
provider, _ := staticweb.LocalProvider("./dist")
service.Mount(staticweb.MountConfig{
URLPrefix: "/",
Provider: provider,
FallbackStrategy: staticweb.HTMLFallback("index.html"),
})
// API routes take precedence (registered first)
router.HandleFunc("/api/users", usersHandler)
router.HandleFunc("/api/posts", postsHandler)
// Static files handle all other routes
router.PathPrefix("/").Handler(service.Handler())
```
## Filesystem Providers
### Local Directory
Serve files from a local filesystem directory:
```go
provider, err := staticweb.LocalProvider("/var/www/static")
```
### Zip File
Serve files from a zip archive:
```go
provider, err := staticweb.ZipProvider("./static.zip")
```
### Embedded Filesystem
Serve files from Go's embedded filesystem:
```go
//go:embed assets
var assets embed.FS
// Direct embedded FS
provider, err := staticweb.EmbedProvider(&assets, "")
// Or from a zip file within embedded FS
provider, err := staticweb.EmbedProvider(&assets, "assets.zip")
```
## Cache Policies
### Simple Cache
Single TTL for all files:
```go
cachePolicy := staticweb.SimpleCache(3600) // 1 hour
```
### Extension-Based Cache
Different TTLs per file type:
```go
rules := map[string]int{
".html": 3600, // 1 hour
".js": 86400, // 1 day
".css": 86400, // 1 day
".png": 604800, // 1 week
}
cachePolicy := staticweb.ExtensionCache(rules, 3600) // default 1 hour
```
### No Cache
Disable caching entirely:
```go
cachePolicy := staticweb.NoCache()
```
## Fallback Strategies
### HTML Fallback
Serve index.html for non-asset requests (SPA routing):
```go
fallback := staticweb.HTMLFallback("index.html")
```
### Extension-Based Fallback
Skip fallback for known static assets:
```go
fallback := staticweb.DefaultExtensionFallback("index.html")
```
Custom extensions:
```go
staticExts := []string{".js", ".css", ".png", ".jpg"}
fallback := staticweb.ExtensionFallback(staticExts, "index.html")
```
## Configuration
### Service Configuration
```go
config := &staticweb.ServiceConfig{
DefaultCacheTime: 3600,
DefaultMIMETypes: map[string]string{
".webp": "image/webp",
".wasm": "application/wasm",
},
}
service := staticweb.NewService(config)
```
### Mount Configuration
```go
service.Mount(staticweb.MountConfig{
URLPrefix: "/static",
Provider: provider,
CachePolicy: cachePolicy, // Optional
MIMEResolver: mimeResolver, // Optional
FallbackStrategy: fallbackStrategy, // Optional
})
```
## Advanced Usage
### Multiple Mount Points
Serve different directories at different URL prefixes with different policies:
```go
service := staticweb.NewService(nil)
// Long-lived assets
assetsProvider, _ := staticweb.LocalProvider("./assets")
service.Mount(staticweb.MountConfig{
URLPrefix: "/assets",
Provider: assetsProvider,
CachePolicy: staticweb.SimpleCache(604800), // 1 week
})
// Frequently updated HTML
htmlProvider, _ := staticweb.LocalProvider("./public")
service.Mount(staticweb.MountConfig{
URLPrefix: "/",
Provider: htmlProvider,
CachePolicy: staticweb.SimpleCache(300), // 5 minutes
})
```
### Custom MIME Types
```go
mimeResolver := staticweb.DefaultMIMEResolver()
mimeResolver.RegisterMIMEType(".webp", "image/webp")
mimeResolver.RegisterMIMEType(".wasm", "application/wasm")
service.Mount(staticweb.MountConfig{
URLPrefix: "/static",
Provider: provider,
MIMEResolver: mimeResolver,
})
```
### Resource Cleanup
Always close the service when done:
```go
service := staticweb.NewService(nil)
defer service.Close()
// ... mount and use service ...
```
Or unmount individual mount points:
```go
service.Unmount("/static")
```
### Reloading/Refreshing Content
Reload providers to pick up changes from the underlying filesystem. This is particularly useful for zip files in development:
```go
// When zip file or directory contents change
err := service.Reload()
if err != nil {
log.Printf("Failed to reload: %v", err)
}
```
Providers that support reloading:
- **ZipFSProvider**: Reopens the zip file to pick up changes
- **LocalFSProvider**: Refreshes the directory view (automatically picks up changes)
- **EmbedFSProvider**: Not reloadable (embedded at compile time)
You can also reload individual providers:
```go
if reloadable, ok := provider.(staticweb.ReloadableProvider); ok {
err := reloadable.Reload()
if err != nil {
log.Printf("Failed to reload: %v", err)
}
}
```
**Development Workflow Example:**
```go
service := staticweb.NewService(nil)
provider, _ := staticweb.ZipProvider("./dist.zip")
service.Mount(staticweb.MountConfig{
URLPrefix: "/app",
Provider: provider,
})
// In development, reload when dist.zip is rebuilt
go func() {
watcher := fsnotify.NewWatcher()
watcher.Add("./dist.zip")
for range watcher.Events {
log.Println("Reloading static files...")
if err := service.Reload(); err != nil {
log.Printf("Reload failed: %v", err)
}
}
}()
```
## Router Integration
### Gorilla Mux
```go
router := mux.NewRouter()
router.HandleFunc("/api/users", usersHandler)
router.PathPrefix("/").Handler(service.Handler())
```
### Standard http.ServeMux
```go
http.Handle("/api/", apiHandler)
http.Handle("/", service.Handler())
```
### BunRouter
```go
router.GET("/api/users", usersHandler)
router.GET("/*path", bunrouter.HTTPHandlerFunc(service.Handler()))
```
## Architecture
### Core Interfaces
#### FileSystemProvider
Abstracts the source of files:
```go
type FileSystemProvider interface {
Open(name string) (fs.File, error)
Close() error
Type() string
}
```
Implementations:
- `LocalFSProvider` - Local directories
- `ZipFSProvider` - Zip archives
- `EmbedFSProvider` - Embedded filesystems
#### CachePolicy
Defines caching behavior:
```go
type CachePolicy interface {
GetCacheTime(path string) int
GetCacheHeaders(path string) map[string]string
}
```
Implementations:
- `SimpleCachePolicy` - Single TTL
- `ExtensionBasedCachePolicy` - Per-extension TTL
- `NoCachePolicy` - Disable caching
#### FallbackStrategy
Handles missing files:
```go
type FallbackStrategy interface {
ShouldFallback(path string) bool
GetFallbackPath(path string) string
}
```
Implementations:
- `NoFallback` - Return 404
- `HTMLFallbackStrategy` - SPA routing
- `ExtensionBasedFallback` - Skip known assets
#### MIMETypeResolver
Determines Content-Type:
```go
type MIMETypeResolver interface {
GetMIMEType(path string) string
RegisterMIMEType(extension, mimeType string)
}
```
Implementations:
- `DefaultMIMEResolver` - Common web types
- `ConfigurableMIMEResolver` - Custom mappings
## Testing
### Mock Providers
```go
import staticwebtesting "github.com/bitechdev/ResolveSpec/pkg/server/staticweb/testing"
provider := staticwebtesting.NewMockProvider(map[string][]byte{
"index.html": []byte("<html>test</html>"),
"app.js": []byte("console.log('test')"),
})
service.Mount(staticweb.MountConfig{
URLPrefix: "/",
Provider: provider,
})
```
### Test Helpers
```go
req := httptest.NewRequest("GET", "/index.html", nil)
rec := httptest.NewRecorder()
service.Handler().ServeHTTP(rec, req)
// Assert response
assert.Equal(t, 200, rec.Code)
```
## Future Features
The interface-driven design allows for easy extensibility:
### Planned Providers
- **HTTPFSProvider**: Fetch files from remote HTTP servers with local caching
- **S3FSProvider**: Serve files from S3-compatible storage
- **CompositeProvider**: Fallback chain across multiple providers
- **MemoryProvider**: In-memory filesystem for testing
### Planned Policies
- **TimedCachePolicy**: Different cache times by time of day
- **ConditionalCachePolicy**: Smart cache based on file size/type
- **RegexFallbackStrategy**: Pattern-based routing
## License
See the main repository for license information.
## Contributing
Contributions are welcome! The interface-driven design makes it easy to add new providers and policies without modifying existing code.

View File

@@ -0,0 +1,99 @@
package staticweb
import (
"embed"
"fmt"
"github.com/bitechdev/ResolveSpec/pkg/server/staticweb/policies"
"github.com/bitechdev/ResolveSpec/pkg/server/staticweb/providers"
)
// ServiceConfig configures the static file service.
type ServiceConfig struct {
// DefaultCacheTime is the default cache duration in seconds.
// Used when a mount point doesn't specify a custom CachePolicy.
// Default: 172800 (48 hours)
DefaultCacheTime int
// DefaultMIMETypes is a map of file extensions to MIME types.
// These are added to the default MIME resolver.
// Extensions should include the leading dot (e.g., ".webp").
DefaultMIMETypes map[string]string
}
// DefaultServiceConfig returns a ServiceConfig with sensible defaults.
func DefaultServiceConfig() *ServiceConfig {
return &ServiceConfig{
DefaultCacheTime: 172800, // 48 hours
DefaultMIMETypes: make(map[string]string),
}
}
// Validate checks if the ServiceConfig is valid.
func (c *ServiceConfig) Validate() error {
if c.DefaultCacheTime < 0 {
return fmt.Errorf("DefaultCacheTime cannot be negative")
}
return nil
}
// Helper constructor functions for providers
// LocalProvider creates a FileSystemProvider for a local directory.
func LocalProvider(path string) (FileSystemProvider, error) {
return providers.NewLocalFSProvider(path)
}
// ZipProvider creates a FileSystemProvider for a zip file.
func ZipProvider(zipPath string) (FileSystemProvider, error) {
return providers.NewZipFSProvider(zipPath)
}
// EmbedProvider creates a FileSystemProvider for an embedded filesystem.
// If zipFile is empty, the embedded FS is used directly.
// If zipFile is specified, it's treated as a path to a zip file within the embedded FS.
// The embedFS parameter can be any fs.FS, but is typically *embed.FS.
func EmbedProvider(embedFS *embed.FS, zipFile string) (FileSystemProvider, error) {
return providers.NewEmbedFSProvider(embedFS, zipFile)
}
// Policy constructor functions
// SimpleCache creates a simple cache policy with the given TTL in seconds.
func SimpleCache(seconds int) CachePolicy {
return policies.NewSimpleCachePolicy(seconds)
}
// ExtensionCache creates an extension-based cache policy.
// rules maps file extensions (with leading dot) to cache times in seconds.
// defaultTime is used for files that don't match any rule.
func ExtensionCache(rules map[string]int, defaultTime int) CachePolicy {
return policies.NewExtensionBasedCachePolicy(rules, defaultTime)
}
// NoCache creates a cache policy that disables all caching.
func NoCache() CachePolicy {
return policies.NewNoCachePolicy()
}
// HTMLFallback creates a fallback strategy for SPAs that serves the given index file.
func HTMLFallback(indexFile string) FallbackStrategy {
return policies.NewHTMLFallbackStrategy(indexFile)
}
// ExtensionFallback creates an extension-based fallback strategy.
// staticExtensions is a list of file extensions that should NOT use fallback.
// fallbackPath is the file to serve when fallback is triggered.
func ExtensionFallback(staticExtensions []string, fallbackPath string) FallbackStrategy {
return policies.NewExtensionBasedFallback(staticExtensions, fallbackPath)
}
// DefaultExtensionFallback creates an extension-based fallback with common web asset extensions.
func DefaultExtensionFallback(fallbackPath string) FallbackStrategy {
return policies.NewDefaultExtensionBasedFallback(fallbackPath)
}
// DefaultMIMEResolver creates a MIME resolver with common web file types.
func DefaultMIMEResolver() MIMETypeResolver {
return policies.NewDefaultMIMEResolver()
}

View File

@@ -0,0 +1,60 @@
package staticweb_test
import (
"fmt"
"github.com/bitechdev/ResolveSpec/pkg/server/staticweb"
staticwebtesting "github.com/bitechdev/ResolveSpec/pkg/server/staticweb/testing"
)
// Example_reload demonstrates reloading content when files change.
func Example_reload() {
service := staticweb.NewService(nil)
// Create a provider
provider := staticwebtesting.NewMockProvider(map[string][]byte{
"version.txt": []byte("v1.0.0"),
})
service.Mount(staticweb.MountConfig{
URLPrefix: "/static",
Provider: provider,
})
// Simulate updating the file
provider.AddFile("version.txt", []byte("v2.0.0"))
// Reload to pick up changes (in real usage with zip files)
err := service.Reload()
if err != nil {
fmt.Printf("Failed to reload: %v\n", err)
} else {
fmt.Println("Successfully reloaded static files")
}
// Output: Successfully reloaded static files
}
// Example_reloadZip demonstrates reloading a zip file provider.
func Example_reloadZip() {
service := staticweb.NewService(nil)
// In production, you would use:
// provider, _ := staticweb.ZipProvider("./dist.zip")
// For this example, we use a mock
provider := staticwebtesting.NewMockProvider(map[string][]byte{
"app.js": []byte("console.log('v1')"),
})
service.Mount(staticweb.MountConfig{
URLPrefix: "/app",
Provider: provider,
})
fmt.Println("Serving from zip file")
// When the zip file is updated, call Reload()
// service.Reload()
// Output: Serving from zip file
}

View File

@@ -0,0 +1,138 @@
package staticweb_test
import (
"fmt"
"net/http"
"github.com/bitechdev/ResolveSpec/pkg/server/staticweb"
staticwebtesting "github.com/bitechdev/ResolveSpec/pkg/server/staticweb/testing"
"github.com/gorilla/mux"
)
// Example_basic demonstrates serving files from a local directory.
func Example_basic() {
service := staticweb.NewService(nil)
// Using mock provider for example purposes
provider := staticwebtesting.NewMockProvider(map[string][]byte{
"index.html": []byte("<html>test</html>"),
})
_ = service.Mount(staticweb.MountConfig{
URLPrefix: "/static",
Provider: provider,
})
router := mux.NewRouter()
router.PathPrefix("/").Handler(service.Handler())
fmt.Println("Serving files from ./public at /static")
// Output: Serving files from ./public at /static
}
// Example_spa demonstrates an SPA with HTML fallback routing.
func Example_spa() {
service := staticweb.NewService(nil)
// Using mock provider for example purposes
provider := staticwebtesting.NewMockProvider(map[string][]byte{
"index.html": []byte("<html>app</html>"),
})
_ = service.Mount(staticweb.MountConfig{
URLPrefix: "/",
Provider: provider,
FallbackStrategy: staticweb.HTMLFallback("index.html"),
})
router := mux.NewRouter()
// API routes take precedence
router.HandleFunc("/api/users", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("users"))
})
// Static files handle all other routes
router.PathPrefix("/").Handler(service.Handler())
fmt.Println("SPA with fallback to index.html")
// Output: SPA with fallback to index.html
}
// Example_multiple demonstrates multiple mount points with different policies.
func Example_multiple() {
service := staticweb.NewService(&staticweb.ServiceConfig{
DefaultCacheTime: 3600,
})
// Assets with long cache (using mock for example)
assetsProvider := staticwebtesting.NewMockProvider(map[string][]byte{
"app.js": []byte("console.log('test')"),
})
service.Mount(staticweb.MountConfig{
URLPrefix: "/assets",
Provider: assetsProvider,
CachePolicy: staticweb.SimpleCache(604800), // 1 week
})
// HTML with short cache (using mock for example)
htmlProvider := staticwebtesting.NewMockProvider(map[string][]byte{
"index.html": []byte("<html>test</html>"),
})
service.Mount(staticweb.MountConfig{
URLPrefix: "/",
Provider: htmlProvider,
CachePolicy: staticweb.SimpleCache(300), // 5 minutes
})
fmt.Println("Multiple mount points configured")
// Output: Multiple mount points configured
}
// Example_zip demonstrates serving from a zip file (concept).
func Example_zip() {
service := staticweb.NewService(nil)
// For actual usage, you would use:
// provider, err := staticweb.ZipProvider("./static.zip")
// For this example, we use a mock
provider := staticwebtesting.NewMockProvider(map[string][]byte{
"file.txt": []byte("content"),
})
service.Mount(staticweb.MountConfig{
URLPrefix: "/static",
Provider: provider,
})
fmt.Println("Serving from zip file")
// Output: Serving from zip file
}
// Example_extensionCache demonstrates extension-based caching.
func Example_extensionCache() {
service := staticweb.NewService(nil)
// Using mock provider for example purposes
provider := staticwebtesting.NewMockProvider(map[string][]byte{
"index.html": []byte("<html>test</html>"),
"app.js": []byte("console.log('test')"),
})
// Different cache times per file type
cacheRules := map[string]int{
".html": 3600, // 1 hour
".js": 86400, // 1 day
".css": 86400, // 1 day
".png": 604800, // 1 week
}
service.Mount(staticweb.MountConfig{
URLPrefix: "/",
Provider: provider,
CachePolicy: staticweb.ExtensionCache(cacheRules, 3600), // default 1 hour
})
fmt.Println("Extension-based caching configured")
// Output: Extension-based caching configured
}

View File

@@ -0,0 +1,130 @@
package staticweb
import (
"io/fs"
"net/http"
)
// FileSystemProvider abstracts the source of files (local, zip, embedded, future: http, s3)
// Implementations must be safe for concurrent use.
type FileSystemProvider interface {
// Open opens the named file.
// The name is always a slash-separated path relative to the filesystem root.
Open(name string) (fs.File, error)
// Close releases any resources held by the provider.
// After Close is called, the provider should not be used.
Close() error
// Type returns the provider type (e.g., "local", "zip", "embed", "http", "s3").
// This is primarily for debugging and logging purposes.
Type() string
}
// ReloadableProvider is an optional interface that providers can implement
// to support reloading/refreshing their content.
// This is useful for development workflows where the underlying files may change.
type ReloadableProvider interface {
FileSystemProvider
// Reload refreshes the provider's content from the underlying source.
// For zip files, this reopens the zip archive.
// For local directories, this refreshes the filesystem view.
// Returns an error if the reload fails.
Reload() error
}
// CachePolicy defines how files should be cached by browsers and proxies.
// Implementations must be safe for concurrent use.
type CachePolicy interface {
// GetCacheTime returns the cache duration in seconds for the given path.
// A value of 0 means no caching.
// A negative value can be used to indicate browser should revalidate.
GetCacheTime(path string) int
// GetCacheHeaders returns additional cache-related HTTP headers for the given path.
// Common headers include "Cache-Control", "Expires", "ETag", etc.
// Returns nil if no additional headers are needed.
GetCacheHeaders(path string) map[string]string
}
// MIMETypeResolver determines the Content-Type for files.
// Implementations must be safe for concurrent use.
type MIMETypeResolver interface {
// GetMIMEType returns the MIME type for the given file path.
// Returns empty string if the MIME type cannot be determined.
GetMIMEType(path string) string
// RegisterMIMEType registers a custom MIME type for the given file extension.
// The extension should include the leading dot (e.g., ".webp").
RegisterMIMEType(extension, mimeType string)
}
// FallbackStrategy handles requests for files that don't exist.
// This is commonly used for Single Page Applications (SPAs) that use client-side routing.
// Implementations must be safe for concurrent use.
type FallbackStrategy interface {
// ShouldFallback determines if a fallback should be attempted for the given path.
// Returns true if the request should be handled by fallback logic.
ShouldFallback(path string) bool
// GetFallbackPath returns the path to serve instead of the originally requested path.
// This is only called if ShouldFallback returns true.
GetFallbackPath(path string) string
}
// MountConfig configures a single mount point.
// A mount point connects a URL prefix to a filesystem provider with optional policies.
type MountConfig struct {
// URLPrefix is the URL path prefix where the filesystem should be mounted.
// Must start with "/" (e.g., "/static", "/", "/assets").
// Requests starting with this prefix will be handled by this mount point.
URLPrefix string
// Provider is the filesystem provider that supplies the files.
// Required.
Provider FileSystemProvider
// CachePolicy determines how files should be cached.
// If nil, the service's default cache policy is used.
CachePolicy CachePolicy
// MIMEResolver determines Content-Type headers for files.
// If nil, the service's default MIME resolver is used.
MIMEResolver MIMETypeResolver
// FallbackStrategy handles requests for missing files.
// If nil, no fallback is performed and 404 responses are returned.
FallbackStrategy FallbackStrategy
}
// StaticFileService manages multiple mount points and serves static files.
// The service is safe for concurrent use.
type StaticFileService interface {
// Mount adds a new mount point with the given configuration.
// Returns an error if the URLPrefix is already mounted or if the config is invalid.
Mount(config MountConfig) error
// Unmount removes the mount point at the given URL prefix.
// Returns an error if no mount point exists at that prefix.
// Automatically calls Close() on the provider to release resources.
Unmount(urlPrefix string) error
// ListMounts returns a sorted list of all mounted URL prefixes.
ListMounts() []string
// Reload reinitializes all filesystem providers.
// This can be used to pick up changes in the underlying filesystems.
// Not all providers may support reloading.
Reload() error
// Close releases all resources held by the service and all mounted providers.
// After Close is called, the service should not be used.
Close() error
// Handler returns an http.Handler that serves static files from all mount points.
// The handler performs longest-prefix matching to find the appropriate mount point.
// If no mount point matches, the handler returns without writing a response,
// allowing other handlers (like API routes) to process the request.
Handler() http.Handler
}

View File

@@ -0,0 +1,235 @@
package staticweb
import (
"fmt"
"io"
"io/fs"
"net/http"
"path"
"strings"
)
// mountPoint represents a mounted filesystem at a specific URL prefix.
type mountPoint struct {
urlPrefix string
provider FileSystemProvider
cachePolicy CachePolicy
mimeResolver MIMETypeResolver
fallbackStrategy FallbackStrategy
fileServer http.Handler
}
// newMountPoint creates a new mount point with the given configuration.
func newMountPoint(config MountConfig, defaults *ServiceConfig) (*mountPoint, error) {
if config.URLPrefix == "" {
return nil, fmt.Errorf("URLPrefix cannot be empty")
}
if !strings.HasPrefix(config.URLPrefix, "/") {
return nil, fmt.Errorf("URLPrefix must start with /")
}
if config.Provider == nil {
return nil, fmt.Errorf("provider cannot be nil")
}
mp := &mountPoint{
urlPrefix: config.URLPrefix,
provider: config.Provider,
cachePolicy: config.CachePolicy,
mimeResolver: config.MIMEResolver,
fallbackStrategy: config.FallbackStrategy,
}
// Apply defaults if policies are not specified
if mp.cachePolicy == nil && defaults != nil {
mp.cachePolicy = defaultCachePolicy(defaults.DefaultCacheTime)
}
if mp.mimeResolver == nil {
mp.mimeResolver = defaultMIMEResolver()
}
// Create an http.FileServer for serving files
mp.fileServer = http.FileServer(http.FS(config.Provider))
return mp, nil
}
// ServeHTTP handles HTTP requests for files in this mount point.
func (m *mountPoint) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Strip the URL prefix to get the file path
filePath := strings.TrimPrefix(r.URL.Path, m.urlPrefix)
if filePath == "" {
filePath = "/"
}
// Clean the path
filePath = path.Clean(filePath)
// Try to open the file
file, err := m.provider.Open(strings.TrimPrefix(filePath, "/"))
if err != nil {
// File doesn't exist - check if we should use fallback
if m.fallbackStrategy != nil && m.fallbackStrategy.ShouldFallback(filePath) {
fallbackPath := m.fallbackStrategy.GetFallbackPath(filePath)
file, err = m.provider.Open(strings.TrimPrefix(fallbackPath, "/"))
if err == nil {
// Successfully opened fallback file
defer file.Close()
m.serveFile(w, r, fallbackPath, file)
return
}
}
// No fallback or fallback failed - return 404
http.NotFound(w, r)
return
}
defer file.Close()
// Serve the file
m.serveFile(w, r, filePath, file)
}
// serveFile serves a single file with appropriate headers.
func (m *mountPoint) serveFile(w http.ResponseWriter, r *http.Request, filePath string, file fs.File) {
// Get file info
stat, err := file.Stat()
if err != nil {
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
return
}
// If it's a directory, try to serve index.html
if stat.IsDir() {
indexPath := path.Join(filePath, "index.html")
indexFile, err := m.provider.Open(strings.TrimPrefix(indexPath, "/"))
if err != nil {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
defer indexFile.Close()
indexStat, err := indexFile.Stat()
if err != nil {
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
return
}
filePath = indexPath
stat = indexStat
file = indexFile
}
// Set Content-Type header using MIME resolver
if m.mimeResolver != nil {
if mimeType := m.mimeResolver.GetMIMEType(filePath); mimeType != "" {
w.Header().Set("Content-Type", mimeType)
}
}
// Apply cache policy
if m.cachePolicy != nil {
headers := m.cachePolicy.GetCacheHeaders(filePath)
for key, value := range headers {
w.Header().Set(key, value)
}
}
// Serve the content
if seeker, ok := file.(interface {
io.ReadSeeker
}); ok {
http.ServeContent(w, r, stat.Name(), stat.ModTime(), seeker)
} else {
// If the file doesn't support seeking, we need to read it all into memory
data, err := fs.ReadFile(m.provider, strings.TrimPrefix(filePath, "/"))
if err != nil {
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
return
}
http.ServeContent(w, r, stat.Name(), stat.ModTime(), strings.NewReader(string(data)))
}
}
// Close releases resources held by the mount point.
func (m *mountPoint) Close() error {
if m.provider != nil {
return m.provider.Close()
}
return nil
}
// defaultCachePolicy creates a default simple cache policy.
func defaultCachePolicy(cacheTime int) CachePolicy {
// Import the policies package type - we'll need to use the concrete type
// For now, create a simple inline implementation
return &simpleCachePolicy{cacheTime: cacheTime}
}
// simpleCachePolicy is a simple inline implementation of CachePolicy
type simpleCachePolicy struct {
cacheTime int
}
func (p *simpleCachePolicy) GetCacheTime(path string) int {
return p.cacheTime
}
func (p *simpleCachePolicy) GetCacheHeaders(path string) map[string]string {
if p.cacheTime <= 0 {
return map[string]string{
"Cache-Control": "no-cache, no-store, must-revalidate",
"Pragma": "no-cache",
"Expires": "0",
}
}
return map[string]string{
"Cache-Control": fmt.Sprintf("public, max-age=%d", p.cacheTime),
}
}
// defaultMIMEResolver creates a default MIME resolver.
func defaultMIMEResolver() MIMETypeResolver {
// Import the policies package type - we'll need to use the concrete type
// For now, create a simple inline implementation
return &simpleMIMEResolver{
types: map[string]string{
".js": "application/javascript",
".mjs": "application/javascript",
".cjs": "application/javascript",
".css": "text/css",
".html": "text/html",
".htm": "text/html",
".json": "application/json",
".png": "image/png",
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
".gif": "image/gif",
".svg": "image/svg+xml",
".ico": "image/x-icon",
".txt": "text/plain",
},
}
}
// simpleMIMEResolver is a simple inline implementation of MIMETypeResolver
type simpleMIMEResolver struct {
types map[string]string
}
func (r *simpleMIMEResolver) GetMIMEType(filePath string) string {
ext := strings.ToLower(path.Ext(filePath))
if mimeType, ok := r.types[ext]; ok {
return mimeType
}
return ""
}
func (r *simpleMIMEResolver) RegisterMIMEType(extension, mimeType string) {
if !strings.HasPrefix(extension, ".") {
extension = "." + extension
}
r.types[strings.ToLower(extension)] = mimeType
}

View File

@@ -0,0 +1,592 @@
# StaticWeb Package Interface-Driven Refactoring Plan
## Overview
Refactor `pkg/server/staticweb` to be interface-driven, router-agnostic, and maintainable. This is a breaking change that replaces the existing API with a cleaner design.
## User Requirements
- ✅ Work with any server (not just Gorilla mux)
- ✅ Serve static files from zip files or directories
- ✅ Support embedded, local filesystems
- ✅ Interface-driven and maintainable architecture
- ✅ Struct-based configuration
- ✅ Breaking changes acceptable
- 🔮 Remote HTTP/HTTPS and S3 support (future feature)
## Design Principles
1. **Interface-first**: Define clear interfaces for all abstractions
2. **Composition over inheritance**: Combine small, focused components
3. **Router-agnostic**: Return standard `http.Handler` for universal compatibility
4. **Configurable policies**: Extract hardcoded behavior into pluggable strategies
5. **Resource safety**: Proper lifecycle management with `Close()` methods
6. **Testability**: Mock-friendly interfaces with clear boundaries
7. **Extensibility**: Easy to add new providers (HTTP, S3, etc.) in future
## Architecture Overview
### Core Interfaces (pkg/server/staticweb/interfaces.go)
```go
// FileSystemProvider abstracts file sources (local, zip, embedded, future: http, s3)
type FileSystemProvider interface {
Open(name string) (fs.File, error)
Close() error
Type() string
}
// CachePolicy defines caching behavior
type CachePolicy interface {
GetCacheTime(path string) int
GetCacheHeaders(path string) map[string]string
}
// MIMETypeResolver determines content types
type MIMETypeResolver interface {
GetMIMEType(path string) string
RegisterMIMEType(extension, mimeType string)
}
// FallbackStrategy handles missing files
type FallbackStrategy interface {
ShouldFallback(path string) bool
GetFallbackPath(path string) string
}
// StaticFileService manages mount points
type StaticFileService interface {
Mount(config MountConfig) error
Unmount(urlPrefix string) error
ListMounts() []string
Reload() error
Close() error
Handler() http.Handler // Router-agnostic integration
}
```
### Configuration (pkg/server/staticweb/config.go)
```go
// MountConfig configures a single mount point
type MountConfig struct {
URLPrefix string
Provider FileSystemProvider
CachePolicy CachePolicy // Optional, uses default if nil
MIMEResolver MIMETypeResolver // Optional, uses default if nil
FallbackStrategy FallbackStrategy // Optional, no fallback if nil
}
// ServiceConfig configures the service
type ServiceConfig struct {
DefaultCacheTime int // Default: 48 hours
DefaultMIMETypes map[string]string // Additional MIME types
}
```
## Implementation Plan
### Step 1: Create Core Interfaces
**File**: `/home/hein/hein/dev/ResolveSpec/pkg/server/staticweb/interfaces.go` (NEW)
Define all interfaces listed above. This establishes the contract for all components.
### Step 2: Implement Default Policies
**Directory**: `/home/hein/hein/dev/ResolveSpec/pkg/server/staticweb/policies/` (NEW)
**File**: `policies/cache.go`
- `SimpleCachePolicy` - Single TTL for all files
- `ExtensionBasedCachePolicy` - Different TTL per file extension
- `NoCachePolicy` - Disables caching
**File**: `policies/mime.go`
- `DefaultMIMEResolver` - Standard web MIME types + stdlib
- `ConfigurableMIMEResolver` - User-defined mappings
- Migrate hardcoded MIME types from `InitMimeTypes()` (lines 60-76)
**File**: `policies/fallback.go`
- `NoFallback` - Returns 404 for missing files
- `HTMLFallbackStrategy` - SPA routing (serves index.html)
- `ExtensionBasedFallback` - Current behavior (checks extensions)
- Migrate logic from `StaticHTMLFallbackHandler()` (lines 241-285)
### Step 3: Implement FileSystem Providers
**Directory**: `/home/hein/hein/dev/ResolveSpec/pkg/server/staticweb/providers/` (NEW)
**File**: `providers/local.go`
```go
type LocalFSProvider struct {
path string
fs fs.FS
}
func NewLocalFSProvider(path string) (*LocalFSProvider, error)
func (l *LocalFSProvider) Open(name string) (fs.File, error)
func (l *LocalFSProvider) Close() error
func (l *LocalFSProvider) Type() string
```
**File**: `providers/zip.go`
```go
type ZipFSProvider struct {
zipPath string
zipReader *zip.ReadCloser
zipFS *zipfs.ZipFS
mu sync.RWMutex
}
func NewZipFSProvider(zipPath string) (*ZipFSProvider, error)
func (z *ZipFSProvider) Open(name string) (fs.File, error)
func (z *ZipFSProvider) Close() error
func (z *ZipFSProvider) Type() string
```
- Integrates with existing `pkg/server/zipfs/zipfs.go`
- Manages zip file lifecycle properly
**File**: `providers/embed.go`
```go
type EmbedFSProvider struct {
embedFS *embed.FS
zipFile string // Optional: path within embedded FS to zip file
zipReader *zip.ReadCloser
fs fs.FS
mu sync.RWMutex
}
func NewEmbedFSProvider(embedFS *embed.FS, zipFile string) (*EmbedFSProvider, error)
func (e *EmbedFSProvider) Open(name string) (fs.File, error)
func (e *EmbedFSProvider) Close() error
func (e *EmbedFSProvider) Type() string
```
### Step 4: Implement Mount Point
**File**: `/home/hein/hein/dev/ResolveSpec/pkg/server/staticweb/mount.go` (NEW)
```go
type mountPoint struct {
urlPrefix string
provider FileSystemProvider
cachePolicy CachePolicy
mimeResolver MIMETypeResolver
fallbackStrategy FallbackStrategy
}
func newMountPoint(config MountConfig, defaults *ServiceConfig) (*mountPoint, error)
func (m *mountPoint) ServeHTTP(w http.ResponseWriter, r *http.Request)
func (m *mountPoint) Close() error
```
**Key behaviors**:
- Strips URL prefix before passing to provider
- Applies cache headers via `CachePolicy`
- Sets Content-Type via `MIMETypeResolver`
- Falls back via `FallbackStrategy` if file not found
- Integrates with `http.FileServer()` for actual serving
### Step 5: Implement Service
**File**: `/home/hein/hein/dev/ResolveSpec/pkg/server/staticweb/service.go` (NEW)
```go
type service struct {
mounts map[string]*mountPoint // urlPrefix -> mount
config *ServiceConfig
mu sync.RWMutex
}
func NewService(config *ServiceConfig) StaticFileService
func (s *service) Mount(config MountConfig) error
func (s *service) Unmount(urlPrefix string) error
func (s *service) ListMounts() []string
func (s *service) Reload() error
func (s *service) Close() error
func (s *service) Handler() http.Handler
```
**Handler Implementation**:
- Performs longest-prefix matching to find mount point
- Delegates to mount point's `ServeHTTP()`
- Returns silently if no match (allows API routes to handle)
- Thread-safe with `sync.RWMutex`
### Step 6: Create Configuration Helpers
**File**: `/home/hein/hein/dev/ResolveSpec/pkg/server/staticweb/config.go` (NEW)
```go
// Default configurations
func DefaultServiceConfig() *ServiceConfig
func DefaultCachePolicy() CachePolicy
func DefaultMIMEResolver() MIMETypeResolver
// Helper constructors
func LocalProvider(path string) FileSystemProvider
func ZipProvider(zipPath string) FileSystemProvider
func EmbedProvider(embedFS *embed.FS, zipFile string) FileSystemProvider
// Policy constructors
func SimpleCache(seconds int) CachePolicy
func ExtensionCache(rules map[string]int) CachePolicy
func HTMLFallback(indexFile string) FallbackStrategy
func ExtensionFallback(staticExtensions []string) FallbackStrategy
```
### Step 7: Update/Remove Existing Files
**REMOVE**: `/home/hein/hein/dev/ResolveSpec/pkg/server/staticweb/staticweb.go`
- All functionality migrated to new interface-based design
- No backward compatibility needed per user request
**KEEP**: `/home/hein/hein/dev/ResolveSpec/pkg/server/zipfs/zipfs.go`
- Still used by `ZipFSProvider`
- Already implements `fs.FS` interface correctly
### Step 8: Create Examples and Tests
**File**: `/home/hein/hein/dev/ResolveSpec/pkg/server/staticweb/example_test.go` (NEW)
```go
func ExampleService_basic() { /* Serve local directory */ }
func ExampleService_spa() { /* SPA with fallback */ }
func ExampleService_multiple() { /* Multiple mount points */ }
func ExampleService_zip() { /* Serve from zip file */ }
func ExampleService_embedded() { /* Serve from embedded zip */ }
```
**File**: `/home/hein/hein/dev/ResolveSpec/pkg/server/staticweb/service_test.go` (NEW)
- Test mount/unmount operations
- Test longest-prefix matching
- Test concurrent access
- Test resource cleanup
**File**: `/home/hein/hein/dev/ResolveSpec/pkg/server/staticweb/providers/providers_test.go` (NEW)
- Test each provider implementation
- Test resource cleanup
- Test error handling
**File**: `/home/hein/hein/dev/ResolveSpec/pkg/server/staticweb/testing/mocks.go` (NEW)
- `MockFileSystemProvider` - In-memory file storage
- `MockCachePolicy` - Configurable cache behavior
- `MockMIMEResolver` - Custom MIME mappings
- Test helpers for common scenarios
### Step 9: Create Documentation
**File**: `/home/hein/hein/dev/ResolveSpec/pkg/server/staticweb/README.md` (NEW)
Document:
- Quick start examples
- Interface overview
- Provider implementations
- Policy customization
- Router integration patterns
- Migration guide from old API
- Future features roadmap
## Key Improvements Over Current Implementation
### 1. Router-Agnostic Design
**Before**: Coupled to Gorilla mux via `RegisterRoutes(*mux.Router)`
**After**: Returns `http.Handler`, works with any router
```go
// Works with Gorilla Mux
muxRouter.PathPrefix("/").Handler(service.Handler())
// Works with standard http.ServeMux
http.Handle("/", service.Handler())
// Works with any http.Handler-compatible router
```
### 2. Configurable Behaviors
**Before**: Hardcoded MIME types, cache times, file extensions
**After**: Pluggable policies
```go
// Custom cache per file type
cachePolicy := ExtensionCache(map[string]int{
".html": 3600, // 1 hour
".js": 86400, // 1 day
".css": 86400, // 1 day
".png": 604800, // 1 week
})
// Custom fallback logic
fallback := HTMLFallback("index.html")
service.Mount(MountConfig{
URLPrefix: "/",
Provider: LocalProvider("./dist"),
CachePolicy: cachePolicy,
FallbackStrategy: fallback,
})
```
### 3. Better Resource Management
**Before**: Manual zip file cleanup, easy to leak resources
**After**: Proper lifecycle with `Close()` on all components
```go
defer service.Close() // Cleans up all providers
```
### 4. Testability
**Before**: Hard to test, coupled to filesystem
**After**: Mock providers for testing
```go
mockProvider := testing.NewInMemoryProvider(map[string][]byte{
"index.html": []byte("<html>test</html>"),
})
service.Mount(MountConfig{
URLPrefix: "/",
Provider: mockProvider,
})
```
### 5. Extensibility
**Before**: Need to modify code to support new file sources
**After**: Implement `FileSystemProvider` interface
```go
// Future: Add HTTP provider without changing core code
type HTTPFSProvider struct { /* ... */ }
func (h *HTTPFSProvider) Open(name string) (fs.File, error) { /* ... */ }
func (h *HTTPFSProvider) Close() error { /* ... */ }
func (h *HTTPFSProvider) Type() string { return "http" }
```
## Future Features (To Implement Later)
### Remote HTTP/HTTPS Provider
**File**: `providers/http.go` (FUTURE)
Serve static files from remote HTTP servers with local caching:
```go
type HTTPFSProvider struct {
baseURL string
httpClient *http.Client
cache LocalCache // Optional disk/memory cache
cacheTTL time.Duration
mu sync.RWMutex
}
// Example usage
service.Mount(MountConfig{
URLPrefix: "/cdn",
Provider: HTTPProvider("https://cdn.example.com/assets"),
})
```
**Features**:
- Fetch files from remote URLs on-demand
- Local cache to reduce remote requests
- Configurable TTL and cache eviction
- HEAD request support for metadata
- Retry logic and timeout handling
- Support for authentication headers
### S3-Compatible Provider
**File**: `providers/s3.go` (FUTURE)
Serve static files from S3, MinIO, or S3-compatible storage:
```go
type S3FSProvider struct {
bucket string
prefix string
region string
client *s3.Client
cache LocalCache
mu sync.RWMutex
}
// Example usage
service.Mount(MountConfig{
URLPrefix: "/media",
Provider: S3Provider("my-bucket", "static/", "us-east-1"),
})
```
**Features**:
- List and fetch objects from S3 buckets
- Support for AWS S3, MinIO, DigitalOcean Spaces, etc.
- IAM role or credential-based authentication
- Optional local caching layer
- Efficient metadata retrieval
- Support for presigned URLs
### Other Future Providers
- **GitProvider**: Serve files from Git repositories
- **MemoryProvider**: In-memory filesystem for testing/temporary files
- **ProxyProvider**: Proxy to another static file server
- **CompositeProvider**: Fallback chain across multiple providers
### Advanced Cache Policies (FUTURE)
- **TimedCachePolicy**: Different cache times by time of day
- **UserAgentCachePolicy**: Cache based on client type
- **ConditionalCachePolicy**: Smart cache based on file size/type
- **DistributedCachePolicy**: Shared cache across service instances
### Advanced Fallback Strategies (FUTURE)
- **RegexFallbackStrategy**: Pattern-based routing
- **I18nFallbackStrategy**: Language-based file resolution
- **VersionedFallbackStrategy**: A/B testing support
## Critical Files Summary
### Files to CREATE (in order):
1. `pkg/server/staticweb/interfaces.go` - Core contracts
2. `pkg/server/staticweb/config.go` - Configuration structs and helpers
3. `pkg/server/staticweb/policies/cache.go` - Cache policy implementations
4. `pkg/server/staticweb/policies/mime.go` - MIME resolver implementations
5. `pkg/server/staticweb/policies/fallback.go` - Fallback strategy implementations
6. `pkg/server/staticweb/providers/local.go` - Local directory provider
7. `pkg/server/staticweb/providers/zip.go` - Zip file provider
8. `pkg/server/staticweb/providers/embed.go` - Embedded filesystem provider
9. `pkg/server/staticweb/mount.go` - Mount point implementation
10. `pkg/server/staticweb/service.go` - Main service implementation
11. `pkg/server/staticweb/testing/mocks.go` - Test helpers
12. `pkg/server/staticweb/service_test.go` - Service tests
13. `pkg/server/staticweb/providers/providers_test.go` - Provider tests
14. `pkg/server/staticweb/example_test.go` - Example code
15. `pkg/server/staticweb/README.md` - Documentation
### Files to REMOVE:
1. `pkg/server/staticweb/staticweb.go` - Replaced by new design
### Files to KEEP:
1. `pkg/server/zipfs/zipfs.go` - Used by ZipFSProvider
### Files for FUTURE (not in this refactoring):
1. `pkg/server/staticweb/providers/http.go` - HTTP/HTTPS remote provider
2. `pkg/server/staticweb/providers/s3.go` - S3-compatible storage provider
3. `pkg/server/staticweb/providers/composite.go` - Fallback chain provider
## Example Usage After Refactoring
### Basic Static Site
```go
service := staticweb.NewService(nil) // Use defaults
err := service.Mount(staticweb.MountConfig{
URLPrefix: "/static",
Provider: staticweb.LocalProvider("./public"),
})
muxRouter.PathPrefix("/").Handler(service.Handler())
```
### SPA with API Routes
```go
service := staticweb.NewService(nil)
service.Mount(staticweb.MountConfig{
URLPrefix: "/",
Provider: staticweb.LocalProvider("./dist"),
FallbackStrategy: staticweb.HTMLFallback("index.html"),
})
// API routes take precedence (registered first)
muxRouter.HandleFunc("/api/users", usersHandler)
muxRouter.HandleFunc("/api/posts", postsHandler)
// Static files handle all other routes
muxRouter.PathPrefix("/").Handler(service.Handler())
```
### Multiple Mount Points with Different Policies
```go
service := staticweb.NewService(&staticweb.ServiceConfig{
DefaultCacheTime: 3600,
})
// Assets with long cache
service.Mount(staticweb.MountConfig{
URLPrefix: "/assets",
Provider: staticweb.LocalProvider("./assets"),
CachePolicy: staticweb.SimpleCache(604800), // 1 week
})
// HTML with short cache
service.Mount(staticweb.MountConfig{
URLPrefix: "/",
Provider: staticweb.LocalProvider("./public"),
CachePolicy: staticweb.SimpleCache(300), // 5 minutes
})
router.PathPrefix("/").Handler(service.Handler())
```
### Embedded Files from Zip
```go
//go:embed assets.zip
var assetsZip embed.FS
service := staticweb.NewService(nil)
service.Mount(staticweb.MountConfig{
URLPrefix: "/static",
Provider: staticweb.EmbedProvider(&assetsZip, "assets.zip"),
})
router.PathPrefix("/").Handler(service.Handler())
```
### Future: CDN Fallback (when HTTP provider is implemented)
```go
// Primary CDN with local fallback
service.Mount(staticweb.MountConfig{
URLPrefix: "/static",
Provider: staticweb.CompositeProvider(
staticweb.HTTPProvider("https://cdn.example.com/assets"),
staticweb.LocalProvider("./public/assets"),
),
})
```
## Testing Strategy
### Unit Tests
- Each provider implementation independently
- Each policy implementation independently
- Mount point request handling
- Service mount/unmount operations
### Integration Tests
- Full request flow through service
- Multiple mount points
- Longest-prefix matching
- Resource cleanup
### Example Tests
- Executable examples in `example_test.go`
- Demonstrate common usage patterns
## Migration Impact
### Breaking Changes
- Complete API redesign (acceptable per user)
- Package not currently used in codebase (no migration needed)
- New consumers will use new API from start
### Future Extensibility
The interface-driven design allows future additions without breaking changes:
- Add HTTPFSProvider by implementing `FileSystemProvider`
- Add S3FSProvider by implementing `FileSystemProvider`
- Add custom cache policies by implementing `CachePolicy`
- Add custom fallback strategies by implementing `FallbackStrategy`
## Implementation Order
1. Interfaces (foundation)
2. Configuration (API surface)
3. Policies (pluggable behavior)
4. Providers (filesystem abstraction)
5. Mount Point (request handling)
6. Service (orchestration)
7. Tests (validation)
8. Documentation (usage)
9. Remove old code (cleanup)
This order ensures each layer builds on tested, working components.
---

View File

@@ -0,0 +1,103 @@
package policies
import (
"fmt"
"path"
"strings"
)
// SimpleCachePolicy implements a basic cache policy with a single TTL for all files.
type SimpleCachePolicy struct {
cacheTime int // Cache duration in seconds
}
// NewSimpleCachePolicy creates a new SimpleCachePolicy with the given cache time in seconds.
func NewSimpleCachePolicy(cacheTimeSeconds int) *SimpleCachePolicy {
return &SimpleCachePolicy{
cacheTime: cacheTimeSeconds,
}
}
// GetCacheTime returns the cache duration for any file.
func (p *SimpleCachePolicy) GetCacheTime(filePath string) int {
return p.cacheTime
}
// GetCacheHeaders returns the Cache-Control header for the given file.
func (p *SimpleCachePolicy) GetCacheHeaders(filePath string) map[string]string {
if p.cacheTime <= 0 {
return map[string]string{
"Cache-Control": "no-cache, no-store, must-revalidate",
"Pragma": "no-cache",
"Expires": "0",
}
}
return map[string]string{
"Cache-Control": fmt.Sprintf("public, max-age=%d", p.cacheTime),
}
}
// ExtensionBasedCachePolicy implements a cache policy that varies by file extension.
type ExtensionBasedCachePolicy struct {
rules map[string]int // Extension -> cache time in seconds
defaultTime int // Default cache time for unmatched extensions
}
// NewExtensionBasedCachePolicy creates a new ExtensionBasedCachePolicy.
// rules maps file extensions (with leading dot, e.g., ".js") to cache times in seconds.
// defaultTime is used for files that don't match any rule.
func NewExtensionBasedCachePolicy(rules map[string]int, defaultTime int) *ExtensionBasedCachePolicy {
return &ExtensionBasedCachePolicy{
rules: rules,
defaultTime: defaultTime,
}
}
// GetCacheTime returns the cache duration based on the file extension.
func (p *ExtensionBasedCachePolicy) GetCacheTime(filePath string) int {
ext := strings.ToLower(path.Ext(filePath))
if cacheTime, ok := p.rules[ext]; ok {
return cacheTime
}
return p.defaultTime
}
// GetCacheHeaders returns cache headers based on the file extension.
func (p *ExtensionBasedCachePolicy) GetCacheHeaders(filePath string) map[string]string {
cacheTime := p.GetCacheTime(filePath)
if cacheTime <= 0 {
return map[string]string{
"Cache-Control": "no-cache, no-store, must-revalidate",
"Pragma": "no-cache",
"Expires": "0",
}
}
return map[string]string{
"Cache-Control": fmt.Sprintf("public, max-age=%d", cacheTime),
}
}
// NoCachePolicy implements a cache policy that disables all caching.
type NoCachePolicy struct{}
// NewNoCachePolicy creates a new NoCachePolicy.
func NewNoCachePolicy() *NoCachePolicy {
return &NoCachePolicy{}
}
// GetCacheTime always returns 0 (no caching).
func (p *NoCachePolicy) GetCacheTime(filePath string) int {
return 0
}
// GetCacheHeaders returns headers that disable caching.
func (p *NoCachePolicy) GetCacheHeaders(filePath string) map[string]string {
return map[string]string{
"Cache-Control": "no-cache, no-store, must-revalidate",
"Pragma": "no-cache",
"Expires": "0",
}
}

View File

@@ -0,0 +1,159 @@
package policies
import (
"path"
"strings"
)
// NoFallback implements a fallback strategy that never falls back.
// All requests for missing files will result in 404 responses.
type NoFallback struct{}
// NewNoFallback creates a new NoFallback strategy.
func NewNoFallback() *NoFallback {
return &NoFallback{}
}
// ShouldFallback always returns false.
func (f *NoFallback) ShouldFallback(filePath string) bool {
return false
}
// GetFallbackPath returns an empty string (never called since ShouldFallback returns false).
func (f *NoFallback) GetFallbackPath(filePath string) string {
return ""
}
// HTMLFallbackStrategy implements a fallback strategy for Single Page Applications (SPAs).
// It serves a specified HTML file (typically index.html) for non-file requests.
type HTMLFallbackStrategy struct {
indexFile string
}
// NewHTMLFallbackStrategy creates a new HTMLFallbackStrategy.
// indexFile is the path to the HTML file to serve (e.g., "index.html", "/index.html").
func NewHTMLFallbackStrategy(indexFile string) *HTMLFallbackStrategy {
return &HTMLFallbackStrategy{
indexFile: indexFile,
}
}
// ShouldFallback returns true for requests that don't look like static assets.
func (f *HTMLFallbackStrategy) ShouldFallback(filePath string) bool {
// Always fall back unless it looks like a static asset
return !f.isStaticAsset(filePath)
}
// GetFallbackPath returns the index file path.
func (f *HTMLFallbackStrategy) GetFallbackPath(filePath string) string {
return f.indexFile
}
// isStaticAsset checks if the path looks like a static asset (has a file extension).
func (f *HTMLFallbackStrategy) isStaticAsset(filePath string) bool {
return path.Ext(filePath) != ""
}
// ExtensionBasedFallback implements a fallback strategy that skips fallback for known static file extensions.
// This is the behavior from the original StaticHTMLFallbackHandler.
type ExtensionBasedFallback struct {
staticExtensions map[string]bool
fallbackPath string
}
// NewExtensionBasedFallback creates a new ExtensionBasedFallback strategy.
// staticExtensions is a list of file extensions (with leading dot) that should NOT use fallback.
// fallbackPath is the file to serve when fallback is triggered.
func NewExtensionBasedFallback(staticExtensions []string, fallbackPath string) *ExtensionBasedFallback {
extMap := make(map[string]bool)
for _, ext := range staticExtensions {
if !strings.HasPrefix(ext, ".") {
ext = "." + ext
}
extMap[strings.ToLower(ext)] = true
}
return &ExtensionBasedFallback{
staticExtensions: extMap,
fallbackPath: fallbackPath,
}
}
// NewDefaultExtensionBasedFallback creates an ExtensionBasedFallback with common web asset extensions.
// This matches the behavior of the original StaticHTMLFallbackHandler.
func NewDefaultExtensionBasedFallback(fallbackPath string) *ExtensionBasedFallback {
return NewExtensionBasedFallback([]string{
".js", ".css", ".png", ".svg", ".ico", ".json",
".jpg", ".jpeg", ".gif", ".woff", ".woff2", ".ttf", ".eot",
}, fallbackPath)
}
// ShouldFallback returns true if the file path doesn't have a static asset extension.
func (f *ExtensionBasedFallback) ShouldFallback(filePath string) bool {
ext := strings.ToLower(path.Ext(filePath))
// If it's a known static extension, don't fallback
if f.staticExtensions[ext] {
return false
}
// Otherwise, try fallback
return true
}
// GetFallbackPath returns the configured fallback path.
func (f *ExtensionBasedFallback) GetFallbackPath(filePath string) string {
return f.fallbackPath
}
// HTMLExtensionFallback implements a fallback strategy that appends .html to paths.
// This tries to serve {path}.html for missing files.
type HTMLExtensionFallback struct {
staticExtensions map[string]bool
}
// NewHTMLExtensionFallback creates a new HTMLExtensionFallback strategy.
func NewHTMLExtensionFallback(staticExtensions []string) *HTMLExtensionFallback {
extMap := make(map[string]bool)
for _, ext := range staticExtensions {
if !strings.HasPrefix(ext, ".") {
ext = "." + ext
}
extMap[strings.ToLower(ext)] = true
}
return &HTMLExtensionFallback{
staticExtensions: extMap,
}
}
// ShouldFallback returns true if the path doesn't have a static extension or .html.
func (f *HTMLExtensionFallback) ShouldFallback(filePath string) bool {
ext := strings.ToLower(path.Ext(filePath))
// If it's a known static extension, don't fallback
if f.staticExtensions[ext] {
return false
}
// If it already has .html, don't fallback
if ext == ".html" || ext == ".htm" {
return false
}
return true
}
// GetFallbackPath returns the path with .html appended.
func (f *HTMLExtensionFallback) GetFallbackPath(filePath string) string {
cleanPath := path.Clean(filePath)
if !strings.HasSuffix(filePath, "/") {
cleanPath = strings.TrimRight(cleanPath, "/")
}
if !strings.HasSuffix(strings.ToLower(cleanPath), ".html") {
return cleanPath + ".html"
}
return cleanPath
}

View File

@@ -0,0 +1,245 @@
package policies
import (
"mime"
"path"
"strings"
"sync"
)
// DefaultMIMEResolver implements a MIME type resolver using Go's standard mime package
// and a set of common web file type mappings.
type DefaultMIMEResolver struct {
customTypes map[string]string
mu sync.RWMutex
}
// NewDefaultMIMEResolver creates a new DefaultMIMEResolver with common web MIME types.
func NewDefaultMIMEResolver() *DefaultMIMEResolver {
resolver := &DefaultMIMEResolver{
customTypes: make(map[string]string),
}
// JavaScript & TypeScript
resolver.RegisterMIMEType(".js", "application/javascript")
resolver.RegisterMIMEType(".mjs", "application/javascript")
resolver.RegisterMIMEType(".cjs", "application/javascript")
resolver.RegisterMIMEType(".ts", "text/typescript")
resolver.RegisterMIMEType(".tsx", "text/tsx")
resolver.RegisterMIMEType(".jsx", "text/jsx")
// CSS & Styling
resolver.RegisterMIMEType(".css", "text/css")
resolver.RegisterMIMEType(".scss", "text/x-scss")
resolver.RegisterMIMEType(".sass", "text/x-sass")
resolver.RegisterMIMEType(".less", "text/x-less")
// HTML & XML
resolver.RegisterMIMEType(".html", "text/html")
resolver.RegisterMIMEType(".htm", "text/html")
resolver.RegisterMIMEType(".xml", "application/xml")
resolver.RegisterMIMEType(".xhtml", "application/xhtml+xml")
// Images - Raster
resolver.RegisterMIMEType(".png", "image/png")
resolver.RegisterMIMEType(".jpg", "image/jpeg")
resolver.RegisterMIMEType(".jpeg", "image/jpeg")
resolver.RegisterMIMEType(".gif", "image/gif")
resolver.RegisterMIMEType(".webp", "image/webp")
resolver.RegisterMIMEType(".avif", "image/avif")
resolver.RegisterMIMEType(".bmp", "image/bmp")
resolver.RegisterMIMEType(".tiff", "image/tiff")
resolver.RegisterMIMEType(".tif", "image/tiff")
resolver.RegisterMIMEType(".ico", "image/x-icon")
resolver.RegisterMIMEType(".cur", "image/x-icon")
// Images - Vector
resolver.RegisterMIMEType(".svg", "image/svg+xml")
resolver.RegisterMIMEType(".svgz", "image/svg+xml")
// Fonts
resolver.RegisterMIMEType(".woff", "font/woff")
resolver.RegisterMIMEType(".woff2", "font/woff2")
resolver.RegisterMIMEType(".ttf", "font/ttf")
resolver.RegisterMIMEType(".otf", "font/otf")
resolver.RegisterMIMEType(".eot", "application/vnd.ms-fontobject")
// Audio
resolver.RegisterMIMEType(".mp3", "audio/mpeg")
resolver.RegisterMIMEType(".wav", "audio/wav")
resolver.RegisterMIMEType(".ogg", "audio/ogg")
resolver.RegisterMIMEType(".oga", "audio/ogg")
resolver.RegisterMIMEType(".m4a", "audio/mp4")
resolver.RegisterMIMEType(".aac", "audio/aac")
resolver.RegisterMIMEType(".flac", "audio/flac")
resolver.RegisterMIMEType(".opus", "audio/opus")
resolver.RegisterMIMEType(".weba", "audio/webm")
// Video
resolver.RegisterMIMEType(".mp4", "video/mp4")
resolver.RegisterMIMEType(".webm", "video/webm")
resolver.RegisterMIMEType(".ogv", "video/ogg")
resolver.RegisterMIMEType(".avi", "video/x-msvideo")
resolver.RegisterMIMEType(".mpeg", "video/mpeg")
resolver.RegisterMIMEType(".mpg", "video/mpeg")
resolver.RegisterMIMEType(".mov", "video/quicktime")
resolver.RegisterMIMEType(".wmv", "video/x-ms-wmv")
resolver.RegisterMIMEType(".flv", "video/x-flv")
resolver.RegisterMIMEType(".mkv", "video/x-matroska")
resolver.RegisterMIMEType(".m4v", "video/mp4")
// Data & Configuration
resolver.RegisterMIMEType(".json", "application/json")
resolver.RegisterMIMEType(".xml", "application/xml")
resolver.RegisterMIMEType(".yml", "application/yaml")
resolver.RegisterMIMEType(".yaml", "application/yaml")
resolver.RegisterMIMEType(".toml", "application/toml")
resolver.RegisterMIMEType(".ini", "text/plain")
resolver.RegisterMIMEType(".conf", "text/plain")
resolver.RegisterMIMEType(".config", "text/plain")
// Documents
resolver.RegisterMIMEType(".pdf", "application/pdf")
resolver.RegisterMIMEType(".doc", "application/msword")
resolver.RegisterMIMEType(".docx", "application/vnd.openxmlformats-officedocument.wordprocessingml.document")
resolver.RegisterMIMEType(".xls", "application/vnd.ms-excel")
resolver.RegisterMIMEType(".xlsx", "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet")
resolver.RegisterMIMEType(".ppt", "application/vnd.ms-powerpoint")
resolver.RegisterMIMEType(".pptx", "application/vnd.openxmlformats-officedocument.presentationml.presentation")
resolver.RegisterMIMEType(".odt", "application/vnd.oasis.opendocument.text")
resolver.RegisterMIMEType(".ods", "application/vnd.oasis.opendocument.spreadsheet")
resolver.RegisterMIMEType(".odp", "application/vnd.oasis.opendocument.presentation")
// Archives
resolver.RegisterMIMEType(".zip", "application/zip")
resolver.RegisterMIMEType(".tar", "application/x-tar")
resolver.RegisterMIMEType(".gz", "application/gzip")
resolver.RegisterMIMEType(".bz2", "application/x-bzip2")
resolver.RegisterMIMEType(".7z", "application/x-7z-compressed")
resolver.RegisterMIMEType(".rar", "application/vnd.rar")
// Text files
resolver.RegisterMIMEType(".txt", "text/plain")
resolver.RegisterMIMEType(".md", "text/markdown")
resolver.RegisterMIMEType(".markdown", "text/markdown")
resolver.RegisterMIMEType(".csv", "text/csv")
resolver.RegisterMIMEType(".log", "text/plain")
// Source code (for syntax highlighting in browsers)
resolver.RegisterMIMEType(".c", "text/x-c")
resolver.RegisterMIMEType(".cpp", "text/x-c++")
resolver.RegisterMIMEType(".h", "text/x-c")
resolver.RegisterMIMEType(".hpp", "text/x-c++")
resolver.RegisterMIMEType(".go", "text/x-go")
resolver.RegisterMIMEType(".py", "text/x-python")
resolver.RegisterMIMEType(".java", "text/x-java")
resolver.RegisterMIMEType(".rs", "text/x-rust")
resolver.RegisterMIMEType(".rb", "text/x-ruby")
resolver.RegisterMIMEType(".php", "text/x-php")
resolver.RegisterMIMEType(".sh", "text/x-shellscript")
resolver.RegisterMIMEType(".bash", "text/x-shellscript")
resolver.RegisterMIMEType(".sql", "text/x-sql")
resolver.RegisterMIMEType(".template.sql", "text/plain")
resolver.RegisterMIMEType(".upg", "text/plain")
// Web Assembly
resolver.RegisterMIMEType(".wasm", "application/wasm")
// Manifest & Service Worker
resolver.RegisterMIMEType(".webmanifest", "application/manifest+json")
resolver.RegisterMIMEType(".manifest", "text/cache-manifest")
// 3D Models
resolver.RegisterMIMEType(".gltf", "model/gltf+json")
resolver.RegisterMIMEType(".glb", "model/gltf-binary")
resolver.RegisterMIMEType(".obj", "model/obj")
resolver.RegisterMIMEType(".stl", "model/stl")
// Other common web assets
resolver.RegisterMIMEType(".map", "application/json") // Source maps
resolver.RegisterMIMEType(".swf", "application/x-shockwave-flash")
resolver.RegisterMIMEType(".apk", "application/vnd.android.package-archive")
resolver.RegisterMIMEType(".dmg", "application/x-apple-diskimage")
resolver.RegisterMIMEType(".exe", "application/x-msdownload")
resolver.RegisterMIMEType(".iso", "application/x-iso9660-image")
return resolver
}
// GetMIMEType returns the MIME type for the given file path.
// It first checks custom registered types, then falls back to Go's mime.TypeByExtension.
func (r *DefaultMIMEResolver) GetMIMEType(filePath string) string {
ext := strings.ToLower(path.Ext(filePath))
// Check custom types first
r.mu.RLock()
if mimeType, ok := r.customTypes[ext]; ok {
r.mu.RUnlock()
return mimeType
}
r.mu.RUnlock()
// Fall back to standard library
if mimeType := mime.TypeByExtension(ext); mimeType != "" {
return mimeType
}
// Return empty string if unknown
return ""
}
// RegisterMIMEType registers a custom MIME type for the given file extension.
func (r *DefaultMIMEResolver) RegisterMIMEType(extension, mimeType string) {
if !strings.HasPrefix(extension, ".") {
extension = "." + extension
}
r.mu.Lock()
r.customTypes[strings.ToLower(extension)] = mimeType
r.mu.Unlock()
}
// ConfigurableMIMEResolver implements a MIME type resolver with user-defined mappings only.
// It does not use any default mappings.
type ConfigurableMIMEResolver struct {
types map[string]string
mu sync.RWMutex
}
// NewConfigurableMIMEResolver creates a new ConfigurableMIMEResolver with the given mappings.
func NewConfigurableMIMEResolver(types map[string]string) *ConfigurableMIMEResolver {
resolver := &ConfigurableMIMEResolver{
types: make(map[string]string),
}
for ext, mimeType := range types {
resolver.RegisterMIMEType(ext, mimeType)
}
return resolver
}
// GetMIMEType returns the MIME type for the given file path.
func (r *ConfigurableMIMEResolver) GetMIMEType(filePath string) string {
ext := strings.ToLower(path.Ext(filePath))
r.mu.RLock()
defer r.mu.RUnlock()
if mimeType, ok := r.types[ext]; ok {
return mimeType
}
return ""
}
// RegisterMIMEType registers a MIME type for the given file extension.
func (r *ConfigurableMIMEResolver) RegisterMIMEType(extension, mimeType string) {
if !strings.HasPrefix(extension, ".") {
extension = "." + extension
}
r.mu.Lock()
r.types[strings.ToLower(extension)] = mimeType
r.mu.Unlock()
}

View File

@@ -0,0 +1,119 @@
package providers
import (
"archive/zip"
"bytes"
"embed"
"fmt"
"io"
"io/fs"
"sync"
"github.com/bitechdev/ResolveSpec/pkg/server/zipfs"
)
// EmbedFSProvider serves files from an embedded filesystem.
// It supports both direct embedded directories and embedded zip files.
type EmbedFSProvider struct {
embedFS *embed.FS
zipFile string // Optional: path within embedded FS to zip file
zipReader *zip.Reader
fs fs.FS
mu sync.RWMutex
}
// NewEmbedFSProvider creates a new EmbedFSProvider.
// If zipFile is empty, the embedded FS is used directly.
// If zipFile is specified, it's treated as a path to a zip file within the embedded FS.
func NewEmbedFSProvider(embedFS fs.FS, zipFile string) (*EmbedFSProvider, error) {
if embedFS == nil {
return nil, fmt.Errorf("embedded filesystem cannot be nil")
}
// Try to cast to *embed.FS for tracking purposes
var embedFSPtr *embed.FS
if efs, ok := embedFS.(*embed.FS); ok {
embedFSPtr = efs
}
provider := &EmbedFSProvider{
embedFS: embedFSPtr,
zipFile: zipFile,
}
// If zipFile is specified, open it as a zip archive
if zipFile != "" {
// Read the zip file from the embedded FS
// We need to check if the FS supports ReadFile
var data []byte
var err error
if readFileFS, ok := embedFS.(interface{ ReadFile(string) ([]byte, error) }); ok {
data, err = readFileFS.ReadFile(zipFile)
} else {
// Fall back to Open and reading
file, openErr := embedFS.Open(zipFile)
if openErr != nil {
return nil, fmt.Errorf("failed to open embedded zip file: %w", openErr)
}
defer file.Close()
data, err = io.ReadAll(file)
}
if err != nil {
return nil, fmt.Errorf("failed to read embedded zip file: %w", err)
}
// Create a zip reader from the data
reader := bytes.NewReader(data)
zipReader, err := zip.NewReader(reader, int64(len(data)))
if err != nil {
return nil, fmt.Errorf("failed to create zip reader: %w", err)
}
provider.zipReader = zipReader
provider.fs = zipfs.NewZipFS(zipReader)
} else {
// Use the embedded FS directly
provider.fs = embedFS
}
return provider, nil
}
// Open opens the named file from the embedded filesystem.
func (p *EmbedFSProvider) Open(name string) (fs.File, error) {
p.mu.RLock()
defer p.mu.RUnlock()
if p.fs == nil {
return nil, fmt.Errorf("embedded filesystem is closed")
}
return p.fs.Open(name)
}
// Close releases any resources held by the provider.
// For embedded filesystems, this is mostly a no-op since Go manages the lifecycle.
func (p *EmbedFSProvider) Close() error {
p.mu.Lock()
defer p.mu.Unlock()
// Clear references to allow garbage collection
p.fs = nil
p.zipReader = nil
return nil
}
// Type returns "embed" or "embed-zip" depending on the configuration.
func (p *EmbedFSProvider) Type() string {
if p.zipFile != "" {
return "embed-zip"
}
return "embed"
}
// ZipFile returns the path to the zip file within the embedded FS, if any.
func (p *EmbedFSProvider) ZipFile() string {
return p.zipFile
}

View File

@@ -0,0 +1,80 @@
package providers
import (
"fmt"
"io/fs"
"os"
"path/filepath"
)
// LocalFSProvider serves files from a local directory.
type LocalFSProvider struct {
path string
fs fs.FS
}
// NewLocalFSProvider creates a new LocalFSProvider for the given directory path.
// The path must be an absolute path to an existing directory.
func NewLocalFSProvider(path string) (*LocalFSProvider, error) {
// Validate that the path exists and is a directory
info, err := os.Stat(path)
if err != nil {
return nil, fmt.Errorf("failed to stat directory: %w", err)
}
if !info.IsDir() {
return nil, fmt.Errorf("path is not a directory: %s", path)
}
// Convert to absolute path
absPath, err := filepath.Abs(path)
if err != nil {
return nil, fmt.Errorf("failed to get absolute path: %w", err)
}
return &LocalFSProvider{
path: absPath,
fs: os.DirFS(absPath),
}, nil
}
// Open opens the named file from the local directory.
func (p *LocalFSProvider) Open(name string) (fs.File, error) {
return p.fs.Open(name)
}
// Close releases any resources held by the provider.
// For local filesystem, this is a no-op since os.DirFS doesn't hold resources.
func (p *LocalFSProvider) Close() error {
return nil
}
// Type returns "local".
func (p *LocalFSProvider) Type() string {
return "local"
}
// Path returns the absolute path to the directory being served.
func (p *LocalFSProvider) Path() string {
return p.path
}
// Reload refreshes the filesystem view.
// For local directories, os.DirFS automatically picks up changes,
// so this recreates the DirFS to ensure a fresh view.
func (p *LocalFSProvider) Reload() error {
// Verify the directory still exists
info, err := os.Stat(p.path)
if err != nil {
return fmt.Errorf("failed to stat directory: %w", err)
}
if !info.IsDir() {
return fmt.Errorf("path is no longer a directory: %s", p.path)
}
// Recreate the DirFS
p.fs = os.DirFS(p.path)
return nil
}

View File

@@ -0,0 +1,393 @@
package providers
import (
"archive/zip"
"bytes"
"io"
"io/fs"
"os"
"path/filepath"
"testing"
"time"
)
func TestLocalFSProvider(t *testing.T) {
// Create a temporary directory with test files
tmpDir := t.TempDir()
testFile := filepath.Join(tmpDir, "test.txt")
if err := os.WriteFile(testFile, []byte("test content"), 0644); err != nil {
t.Fatal(err)
}
provider, err := NewLocalFSProvider(tmpDir)
if err != nil {
t.Fatalf("Failed to create provider: %v", err)
}
defer provider.Close()
// Test opening a file
file, err := provider.Open("test.txt")
if err != nil {
t.Fatalf("Failed to open file: %v", err)
}
defer file.Close()
// Read the file
data, err := io.ReadAll(file)
if err != nil {
t.Fatalf("Failed to read file: %v", err)
}
if string(data) != "test content" {
t.Errorf("Expected 'test content', got %q", string(data))
}
// Test type
if provider.Type() != "local" {
t.Errorf("Expected type 'local', got %q", provider.Type())
}
}
func TestZipFSProvider(t *testing.T) {
// Create a temporary zip file
tmpDir := t.TempDir()
zipPath := filepath.Join(tmpDir, "test.zip")
// Create zip file with test content
zipFile, err := os.Create(zipPath)
if err != nil {
t.Fatal(err)
}
zipWriter := zip.NewWriter(zipFile)
fileWriter, err := zipWriter.Create("test.txt")
if err != nil {
t.Fatal(err)
}
_, err = fileWriter.Write([]byte("zip content"))
if err != nil {
t.Fatal(err)
}
if err := zipWriter.Close(); err != nil {
t.Fatal(err)
}
if err := zipFile.Close(); err != nil {
t.Fatal(err)
}
// Test the provider
provider, err := NewZipFSProvider(zipPath)
if err != nil {
t.Fatalf("Failed to create provider: %v", err)
}
defer provider.Close()
// Test opening a file
file, err := provider.Open("test.txt")
if err != nil {
t.Fatalf("Failed to open file: %v", err)
}
defer file.Close()
// Read the file
data, err := io.ReadAll(file)
if err != nil {
t.Fatalf("Failed to read file: %v", err)
}
if string(data) != "zip content" {
t.Errorf("Expected 'zip content', got %q", string(data))
}
// Test type
if provider.Type() != "zip" {
t.Errorf("Expected type 'zip', got %q", provider.Type())
}
}
func TestZipFSProviderReload(t *testing.T) {
// Create a temporary zip file
tmpDir := t.TempDir()
zipPath := filepath.Join(tmpDir, "test.zip")
// Helper to create zip with content
createZip := func(content string) {
zipFile, err := os.Create(zipPath)
if err != nil {
t.Fatal(err)
}
defer zipFile.Close()
zipWriter := zip.NewWriter(zipFile)
fileWriter, err := zipWriter.Create("test.txt")
if err != nil {
t.Fatal(err)
}
_, err = fileWriter.Write([]byte(content))
if err != nil {
t.Fatal(err)
}
if err := zipWriter.Close(); err != nil {
t.Fatal(err)
}
}
// Create initial zip
createZip("original content")
// Test the provider
provider, err := NewZipFSProvider(zipPath)
if err != nil {
t.Fatalf("Failed to create provider: %v", err)
}
defer provider.Close()
// Read initial content
file, err := provider.Open("test.txt")
if err != nil {
t.Fatalf("Failed to open file: %v", err)
}
data, _ := io.ReadAll(file)
file.Close()
if string(data) != "original content" {
t.Errorf("Expected 'original content', got %q", string(data))
}
// Update the zip file
createZip("updated content")
// Reload the provider
if err := provider.Reload(); err != nil {
t.Fatalf("Failed to reload: %v", err)
}
// Read updated content
file, err = provider.Open("test.txt")
if err != nil {
t.Fatalf("Failed to open file after reload: %v", err)
}
data, _ = io.ReadAll(file)
file.Close()
if string(data) != "updated content" {
t.Errorf("Expected 'updated content', got %q", string(data))
}
}
func TestLocalFSProviderReload(t *testing.T) {
// Create a temporary directory with test files
tmpDir := t.TempDir()
testFile := filepath.Join(tmpDir, "test.txt")
if err := os.WriteFile(testFile, []byte("original"), 0644); err != nil {
t.Fatal(err)
}
provider, err := NewLocalFSProvider(tmpDir)
if err != nil {
t.Fatalf("Failed to create provider: %v", err)
}
defer provider.Close()
// Read initial content
file, err := provider.Open("test.txt")
if err != nil {
t.Fatalf("Failed to open file: %v", err)
}
data, _ := io.ReadAll(file)
file.Close()
if string(data) != "original" {
t.Errorf("Expected 'original', got %q", string(data))
}
// Update the file
if err := os.WriteFile(testFile, []byte("updated"), 0644); err != nil {
t.Fatal(err)
}
// Reload the provider
if err := provider.Reload(); err != nil {
t.Fatalf("Failed to reload: %v", err)
}
// Read updated content
file, err = provider.Open("test.txt")
if err != nil {
t.Fatalf("Failed to open file after reload: %v", err)
}
data, _ = io.ReadAll(file)
file.Close()
if string(data) != "updated" {
t.Errorf("Expected 'updated', got %q", string(data))
}
}
func TestEmbedFSProvider(t *testing.T) {
// Test with a mock embed.FS
mockFS := &mockEmbedFS{
files: map[string][]byte{
"test.txt": []byte("test content"),
},
}
provider, err := NewEmbedFSProvider(mockFS, "")
if err != nil {
t.Fatalf("Failed to create provider: %v", err)
}
defer provider.Close()
// Test type
if provider.Type() != "embed" {
t.Errorf("Expected type 'embed', got %q", provider.Type())
}
// Test opening a file
file, err := provider.Open("test.txt")
if err != nil {
t.Fatalf("Failed to open file: %v", err)
}
defer file.Close()
// Read the file
data, err := io.ReadAll(file)
if err != nil {
t.Fatalf("Failed to read file: %v", err)
}
if string(data) != "test content" {
t.Errorf("Expected 'test content', got %q", string(data))
}
}
func TestEmbedFSProviderWithZip(t *testing.T) {
// Create an embedded-like FS with a zip file
// For simplicity, we'll use a mock embed.FS
tmpDir := t.TempDir()
zipPath := filepath.Join(tmpDir, "test.zip")
// Create zip file
zipFile, err := os.Create(zipPath)
if err != nil {
t.Fatal(err)
}
zipWriter := zip.NewWriter(zipFile)
fileWriter, err := zipWriter.Create("test.txt")
if err != nil {
t.Fatal(err)
}
_, err = fileWriter.Write([]byte("embedded zip content"))
if err != nil {
t.Fatal(err)
}
zipWriter.Close()
zipFile.Close()
// Read the zip file
zipData, err := os.ReadFile(zipPath)
if err != nil {
t.Fatal(err)
}
// Create a mock embed.FS
mockFS := &mockEmbedFS{
files: map[string][]byte{
"test.zip": zipData,
},
}
provider, err := NewEmbedFSProvider(mockFS, "test.zip")
if err != nil {
t.Fatalf("Failed to create provider: %v", err)
}
defer provider.Close()
// Test opening a file
file, err := provider.Open("test.txt")
if err != nil {
t.Fatalf("Failed to open file: %v", err)
}
defer file.Close()
// Read the file
data, err := io.ReadAll(file)
if err != nil {
t.Fatalf("Failed to read file: %v", err)
}
if string(data) != "embedded zip content" {
t.Errorf("Expected 'embedded zip content', got %q", string(data))
}
// Test type
if provider.Type() != "embed-zip" {
t.Errorf("Expected type 'embed-zip', got %q", provider.Type())
}
}
// mockEmbedFS is a mock embed.FS for testing
type mockEmbedFS struct {
files map[string][]byte
}
func (m *mockEmbedFS) Open(name string) (fs.File, error) {
data, ok := m.files[name]
if !ok {
return nil, os.ErrNotExist
}
return &mockFile{
name: name,
reader: bytes.NewReader(data),
size: int64(len(data)),
}, nil
}
func (m *mockEmbedFS) ReadFile(name string) ([]byte, error) {
data, ok := m.files[name]
if !ok {
return nil, os.ErrNotExist
}
return data, nil
}
type mockFile struct {
name string
reader *bytes.Reader
size int64
}
func (f *mockFile) Stat() (fs.FileInfo, error) {
return &mockFileInfo{name: f.name, size: f.size}, nil
}
func (f *mockFile) Read(p []byte) (int, error) {
return f.reader.Read(p)
}
func (f *mockFile) Close() error {
return nil
}
type mockFileInfo struct {
name string
size int64
}
func (fi *mockFileInfo) Name() string { return fi.name }
func (fi *mockFileInfo) Size() int64 { return fi.size }
func (fi *mockFileInfo) Mode() fs.FileMode { return 0644 }
func (fi *mockFileInfo) ModTime() time.Time { return time.Now() }
func (fi *mockFileInfo) IsDir() bool { return false }
func (fi *mockFileInfo) Sys() interface{} { return nil }

View File

@@ -0,0 +1,102 @@
package providers
import (
"archive/zip"
"fmt"
"io/fs"
"path/filepath"
"sync"
"github.com/bitechdev/ResolveSpec/pkg/server/zipfs"
)
// ZipFSProvider serves files from a zip file.
type ZipFSProvider struct {
zipPath string
zipReader *zip.ReadCloser
zipFS *zipfs.ZipFS
mu sync.RWMutex
}
// NewZipFSProvider creates a new ZipFSProvider for the given zip file path.
func NewZipFSProvider(zipPath string) (*ZipFSProvider, error) {
// Convert to absolute path
absPath, err := filepath.Abs(zipPath)
if err != nil {
return nil, fmt.Errorf("failed to get absolute path: %w", err)
}
// Open the zip file
zipReader, err := zip.OpenReader(absPath)
if err != nil {
return nil, fmt.Errorf("failed to open zip file: %w", err)
}
return &ZipFSProvider{
zipPath: absPath,
zipReader: zipReader,
zipFS: zipfs.NewZipFS(&zipReader.Reader),
}, nil
}
// Open opens the named file from the zip archive.
func (p *ZipFSProvider) Open(name string) (fs.File, error) {
p.mu.RLock()
defer p.mu.RUnlock()
if p.zipFS == nil {
return nil, fmt.Errorf("zip filesystem is closed")
}
return p.zipFS.Open(name)
}
// Close releases resources held by the zip reader.
func (p *ZipFSProvider) Close() error {
p.mu.Lock()
defer p.mu.Unlock()
if p.zipReader != nil {
err := p.zipReader.Close()
p.zipReader = nil
p.zipFS = nil
return err
}
return nil
}
// Type returns "zip".
func (p *ZipFSProvider) Type() string {
return "zip"
}
// Path returns the absolute path to the zip file being served.
func (p *ZipFSProvider) Path() string {
return p.zipPath
}
// Reload reopens the zip file to pick up any changes.
// This is useful in development when the zip file is updated.
func (p *ZipFSProvider) Reload() error {
p.mu.Lock()
defer p.mu.Unlock()
// Close the existing zip reader if open
if p.zipReader != nil {
if err := p.zipReader.Close(); err != nil {
return fmt.Errorf("failed to close old zip reader: %w", err)
}
}
// Reopen the zip file
zipReader, err := zip.OpenReader(p.zipPath)
if err != nil {
return fmt.Errorf("failed to reopen zip file: %w", err)
}
p.zipReader = zipReader
p.zipFS = zipfs.NewZipFS(&zipReader.Reader)
return nil
}

View File

@@ -0,0 +1,189 @@
package staticweb
import (
"fmt"
"net/http"
"sort"
"strings"
"sync"
)
// service implements the StaticFileService interface.
type service struct {
mounts map[string]*mountPoint // urlPrefix -> mount point
config *ServiceConfig
mu sync.RWMutex
}
// NewService creates a new static file service with the given configuration.
// If config is nil, default configuration is used.
func NewService(config *ServiceConfig) StaticFileService {
if config == nil {
config = DefaultServiceConfig()
}
return &service{
mounts: make(map[string]*mountPoint),
config: config,
}
}
// Mount adds a new mount point with the given configuration.
func (s *service) Mount(config MountConfig) error {
// Validate the config
if err := s.validateMountConfig(config); err != nil {
return err
}
s.mu.Lock()
defer s.mu.Unlock()
// Check if the prefix is already mounted
if _, exists := s.mounts[config.URLPrefix]; exists {
return fmt.Errorf("mount point already exists at %s", config.URLPrefix)
}
// Create the mount point
mp, err := newMountPoint(config, s.config)
if err != nil {
return fmt.Errorf("failed to create mount point: %w", err)
}
// Add to the map
s.mounts[config.URLPrefix] = mp
return nil
}
// Unmount removes the mount point at the given URL prefix.
func (s *service) Unmount(urlPrefix string) error {
s.mu.Lock()
defer s.mu.Unlock()
mp, exists := s.mounts[urlPrefix]
if !exists {
return fmt.Errorf("no mount point exists at %s", urlPrefix)
}
// Close the mount point to release resources
if err := mp.Close(); err != nil {
return fmt.Errorf("failed to close mount point: %w", err)
}
// Remove from the map
delete(s.mounts, urlPrefix)
return nil
}
// ListMounts returns a sorted list of all mounted URL prefixes.
func (s *service) ListMounts() []string {
s.mu.RLock()
defer s.mu.RUnlock()
prefixes := make([]string, 0, len(s.mounts))
for prefix := range s.mounts {
prefixes = append(prefixes, prefix)
}
sort.Strings(prefixes)
return prefixes
}
// Reload reinitializes all filesystem providers that support reloading.
// This is useful when the underlying files have changed (e.g., zip file updated).
// Providers that implement ReloadableProvider will be reloaded.
func (s *service) Reload() error {
s.mu.RLock()
defer s.mu.RUnlock()
var errors []error
// Reload all mount points that support it
for prefix, mp := range s.mounts {
if reloadable, ok := mp.provider.(ReloadableProvider); ok {
if err := reloadable.Reload(); err != nil {
errors = append(errors, fmt.Errorf("%s: %w", prefix, err))
}
}
}
// Return combined errors if any
if len(errors) > 0 {
return fmt.Errorf("errors while reloading providers: %v", errors)
}
return nil
}
// Close releases all resources held by the service.
func (s *service) Close() error {
s.mu.Lock()
defer s.mu.Unlock()
var errors []error
// Close all mount points
for prefix, mp := range s.mounts {
if err := mp.Close(); err != nil {
errors = append(errors, fmt.Errorf("%s: %w", prefix, err))
}
}
// Clear the map
s.mounts = make(map[string]*mountPoint)
// Return combined errors if any
if len(errors) > 0 {
return fmt.Errorf("errors while closing mount points: %v", errors)
}
return nil
}
// Handler returns an http.Handler that serves static files from all mount points.
func (s *service) Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
s.mu.RLock()
defer s.mu.RUnlock()
// Find the best matching mount point using longest-prefix matching
var bestMatch *mountPoint
var bestPrefix string
for prefix, mp := range s.mounts {
if strings.HasPrefix(r.URL.Path, prefix) {
if len(prefix) > len(bestPrefix) {
bestMatch = mp
bestPrefix = prefix
}
}
}
// If no mount point matches, return without writing a response
// This allows other handlers (like API routes) to process the request
if bestMatch == nil {
return
}
// Serve the file from the matched mount point
bestMatch.ServeHTTP(w, r)
})
}
// validateMountConfig validates the mount configuration.
func (s *service) validateMountConfig(config MountConfig) error {
if config.URLPrefix == "" {
return fmt.Errorf("URLPrefix cannot be empty")
}
if !strings.HasPrefix(config.URLPrefix, "/") {
return fmt.Errorf("URLPrefix must start with /")
}
if config.Provider == nil {
return fmt.Errorf("provider cannot be nil")
}
return nil
}

View File

@@ -0,0 +1,257 @@
package staticweb
import (
"net/http"
"net/http/httptest"
"testing"
staticwebtesting "github.com/bitechdev/ResolveSpec/pkg/server/staticweb/testing"
)
func TestServiceMount(t *testing.T) {
service := NewService(nil)
provider := staticwebtesting.NewMockProvider(map[string][]byte{
"index.html": []byte("<html>test</html>"),
})
err := service.Mount(MountConfig{
URLPrefix: "/test",
Provider: provider,
})
if err != nil {
t.Fatalf("Failed to mount: %v", err)
}
mounts := service.ListMounts()
if len(mounts) != 1 || mounts[0] != "/test" {
t.Errorf("Expected [/test], got %v", mounts)
}
}
func TestServiceUnmount(t *testing.T) {
service := NewService(nil)
provider := staticwebtesting.NewMockProvider(map[string][]byte{
"index.html": []byte("<html>test</html>"),
})
service.Mount(MountConfig{
URLPrefix: "/test",
Provider: provider,
})
err := service.Unmount("/test")
if err != nil {
t.Fatalf("Failed to unmount: %v", err)
}
mounts := service.ListMounts()
if len(mounts) != 0 {
t.Errorf("Expected empty list, got %v", mounts)
}
}
func TestServiceHandler(t *testing.T) {
service := NewService(nil)
provider := staticwebtesting.NewMockProvider(map[string][]byte{
"index.html": []byte("<html>test</html>"),
"app.js": []byte("console.log('test')"),
})
err := service.Mount(MountConfig{
URLPrefix: "/static",
Provider: provider,
})
if err != nil {
t.Fatalf("Failed to mount: %v", err)
}
tests := []struct {
name string
path string
expectedStatus int
expectedBody string
}{
{
name: "serve index.html",
path: "/static/index.html",
expectedStatus: http.StatusOK,
expectedBody: "<html>test</html>",
},
{
name: "serve app.js",
path: "/static/app.js",
expectedStatus: http.StatusOK,
expectedBody: "console.log('test')",
},
{
name: "non-existent file",
path: "/static/nonexistent.html",
expectedStatus: http.StatusNotFound,
expectedBody: "",
},
{
name: "non-matching prefix returns nothing",
path: "/api/test",
expectedStatus: http.StatusOK, // Handler returns without writing
expectedBody: "",
},
}
handler := service.Handler()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req := httptest.NewRequest("GET", tt.path, nil)
rec := httptest.NewRecorder()
handler.ServeHTTP(rec, req)
// For non-matching prefix, handler doesn't write anything
if tt.path == "/api/test" {
if rec.Code != 200 || rec.Body.Len() != 0 {
t.Errorf("Expected no response, got status %d with body length %d", rec.Code, rec.Body.Len())
}
return
}
if rec.Code != tt.expectedStatus {
t.Errorf("Expected status %d, got %d", tt.expectedStatus, rec.Code)
}
if tt.expectedBody != "" && rec.Body.String() != tt.expectedBody {
t.Errorf("Expected body %q, got %q", tt.expectedBody, rec.Body.String())
}
})
}
}
func TestServiceLongestPrefixMatching(t *testing.T) {
service := NewService(nil)
// Mount at /
provider1 := staticwebtesting.NewMockProvider(map[string][]byte{
"index.html": []byte("root"),
})
// Mount at /static
provider2 := staticwebtesting.NewMockProvider(map[string][]byte{
"index.html": []byte("static"),
})
service.Mount(MountConfig{
URLPrefix: "/",
Provider: provider1,
})
service.Mount(MountConfig{
URLPrefix: "/static",
Provider: provider2,
})
handler := service.Handler()
tests := []struct {
path string
expectedBody string
}{
{"/index.html", "root"},
{"/static/index.html", "static"},
}
for _, tt := range tests {
t.Run(tt.path, func(t *testing.T) {
req := httptest.NewRequest("GET", tt.path, nil)
rec := httptest.NewRecorder()
handler.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Errorf("Expected status 200, got %d", rec.Code)
}
if rec.Body.String() != tt.expectedBody {
t.Errorf("Expected body %q, got %q", tt.expectedBody, rec.Body.String())
}
})
}
}
func TestServiceClose(t *testing.T) {
service := NewService(nil)
provider := staticwebtesting.NewMockProvider(map[string][]byte{
"index.html": []byte("<html>test</html>"),
})
service.Mount(MountConfig{
URLPrefix: "/test",
Provider: provider,
})
err := service.Close()
if err != nil {
t.Fatalf("Failed to close service: %v", err)
}
mounts := service.ListMounts()
if len(mounts) != 0 {
t.Errorf("Expected empty list after close, got %v", mounts)
}
}
func TestServiceReload(t *testing.T) {
service := NewService(nil)
// Create a mock provider that supports reload
provider := staticwebtesting.NewMockProvider(map[string][]byte{
"index.html": []byte("original"),
})
service.Mount(MountConfig{
URLPrefix: "/test",
Provider: provider,
})
handler := service.Handler()
// Test initial content
req := httptest.NewRequest("GET", "/test/index.html", nil)
rec := httptest.NewRecorder()
handler.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Errorf("Expected status 200, got %d", rec.Code)
}
if rec.Body.String() != "original" {
t.Errorf("Expected body 'original', got %q", rec.Body.String())
}
// Update the provider's content
provider.AddFile("index.html", []byte("updated"))
// The content is already updated since we're using a mock
// In a real scenario with zip files, you'd call Reload() here
err := service.Reload()
if err != nil {
t.Fatalf("Failed to reload service: %v", err)
}
// Test updated content
req = httptest.NewRequest("GET", "/test/index.html", nil)
rec = httptest.NewRecorder()
handler.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Errorf("Expected status 200, got %d", rec.Code)
}
if rec.Body.String() != "updated" {
t.Errorf("Expected body 'updated', got %q", rec.Body.String())
}
}

View File

@@ -0,0 +1,231 @@
package testing
import (
"bytes"
"fmt"
"io"
"io/fs"
"os"
"path"
"strings"
"sync"
"time"
)
// MockFileSystemProvider is an in-memory filesystem provider for testing.
type MockFileSystemProvider struct {
files map[string][]byte
closed bool
mu sync.RWMutex
}
// NewMockProvider creates a new in-memory provider with the given files.
// Keys should be slash-separated paths (e.g., "index.html", "assets/app.js").
func NewMockProvider(files map[string][]byte) *MockFileSystemProvider {
return &MockFileSystemProvider{
files: files,
}
}
// Open opens a file from the in-memory filesystem.
func (m *MockFileSystemProvider) Open(name string) (fs.File, error) {
m.mu.RLock()
defer m.mu.RUnlock()
if m.closed {
return nil, fmt.Errorf("provider is closed")
}
// Remove leading slash if present
name = strings.TrimPrefix(name, "/")
data, ok := m.files[name]
if !ok {
return nil, os.ErrNotExist
}
return &mockFile{
name: path.Base(name),
data: data,
}, nil
}
// Close marks the provider as closed.
func (m *MockFileSystemProvider) Close() error {
m.mu.Lock()
defer m.mu.Unlock()
m.closed = true
return nil
}
// Type returns "mock".
func (m *MockFileSystemProvider) Type() string {
return "mock"
}
// AddFile adds a file to the in-memory filesystem.
func (m *MockFileSystemProvider) AddFile(name string, data []byte) {
m.mu.Lock()
defer m.mu.Unlock()
name = strings.TrimPrefix(name, "/")
m.files[name] = data
}
// RemoveFile removes a file from the in-memory filesystem.
func (m *MockFileSystemProvider) RemoveFile(name string) {
m.mu.Lock()
defer m.mu.Unlock()
name = strings.TrimPrefix(name, "/")
delete(m.files, name)
}
// mockFile implements fs.File for in-memory files.
type mockFile struct {
name string
data []byte
reader *bytes.Reader
offset int64
}
func (f *mockFile) Stat() (fs.FileInfo, error) {
return &mockFileInfo{
name: f.name,
size: int64(len(f.data)),
}, nil
}
func (f *mockFile) Read(p []byte) (int, error) {
if f.reader == nil {
f.reader = bytes.NewReader(f.data)
if f.offset > 0 {
f.reader.Seek(f.offset, io.SeekStart)
}
}
n, err := f.reader.Read(p)
f.offset += int64(n)
return n, err
}
func (f *mockFile) Seek(offset int64, whence int) (int64, error) {
if f.reader == nil {
f.reader = bytes.NewReader(f.data)
}
pos, err := f.reader.Seek(offset, whence)
f.offset = pos
return pos, err
}
func (f *mockFile) Close() error {
return nil
}
// mockFileInfo implements fs.FileInfo.
type mockFileInfo struct {
name string
size int64
}
func (fi *mockFileInfo) Name() string { return fi.name }
func (fi *mockFileInfo) Size() int64 { return fi.size }
func (fi *mockFileInfo) Mode() fs.FileMode { return 0644 }
func (fi *mockFileInfo) ModTime() time.Time { return time.Now() }
func (fi *mockFileInfo) IsDir() bool { return false }
func (fi *mockFileInfo) Sys() interface{} { return nil }
// MockCachePolicy is a configurable cache policy for testing.
type MockCachePolicy struct {
CacheTime int
Headers map[string]string
}
// NewMockCachePolicy creates a new mock cache policy.
func NewMockCachePolicy(cacheTime int) *MockCachePolicy {
return &MockCachePolicy{
CacheTime: cacheTime,
Headers: make(map[string]string),
}
}
// GetCacheTime returns the configured cache time.
func (p *MockCachePolicy) GetCacheTime(path string) int {
return p.CacheTime
}
// GetCacheHeaders returns the configured headers.
func (p *MockCachePolicy) GetCacheHeaders(path string) map[string]string {
if p.Headers != nil {
return p.Headers
}
return map[string]string{
"Cache-Control": fmt.Sprintf("public, max-age=%d", p.CacheTime),
}
}
// MockMIMEResolver is a configurable MIME resolver for testing.
type MockMIMEResolver struct {
types map[string]string
mu sync.RWMutex
}
// NewMockMIMEResolver creates a new mock MIME resolver.
func NewMockMIMEResolver() *MockMIMEResolver {
return &MockMIMEResolver{
types: make(map[string]string),
}
}
// GetMIMEType returns the MIME type for the given path.
func (r *MockMIMEResolver) GetMIMEType(path string) string {
r.mu.RLock()
defer r.mu.RUnlock()
ext := strings.ToLower(path[strings.LastIndex(path, "."):])
if mimeType, ok := r.types[ext]; ok {
return mimeType
}
return "application/octet-stream"
}
// RegisterMIMEType registers a MIME type.
func (r *MockMIMEResolver) RegisterMIMEType(extension, mimeType string) {
r.mu.Lock()
defer r.mu.Unlock()
if !strings.HasPrefix(extension, ".") {
extension = "." + extension
}
r.types[strings.ToLower(extension)] = mimeType
}
// MockFallbackStrategy is a configurable fallback strategy for testing.
type MockFallbackStrategy struct {
ShouldFallbackFunc func(path string) bool
FallbackPathFunc func(path string) string
}
// NewMockFallbackStrategy creates a new mock fallback strategy.
func NewMockFallbackStrategy(shouldFallback func(string) bool, fallbackPath func(string) string) *MockFallbackStrategy {
return &MockFallbackStrategy{
ShouldFallbackFunc: shouldFallback,
FallbackPathFunc: fallbackPath,
}
}
// ShouldFallback returns whether fallback should be used.
func (s *MockFallbackStrategy) ShouldFallback(path string) bool {
if s.ShouldFallbackFunc != nil {
return s.ShouldFallbackFunc(path)
}
return false
}
// GetFallbackPath returns the fallback path.
func (s *MockFallbackStrategy) GetFallbackPath(path string) string {
if s.FallbackPathFunc != nil {
return s.FallbackPathFunc(path)
}
return "index.html"
}

112
pkg/server/zipfs/zipfs.go Normal file
View File

@@ -0,0 +1,112 @@
package zipfs
import (
"archive/zip"
"fmt"
"io"
"io/fs"
"os"
)
type ZipFS struct {
*zip.Reader
}
func NewZipFS(r *zip.Reader) *ZipFS {
return &ZipFS{r}
}
func (z *ZipFS) Open(name string) (fs.File, error) {
for _, f := range z.File {
if f.Name == name {
rc, err := f.Open()
if err != nil {
return nil, err
}
return &ZipFile{f, rc, 0}, nil
}
}
return nil, os.ErrNotExist
}
type ZipFile struct {
*zip.File
rc io.ReadCloser
offset int64
}
func (f *ZipFile) Stat() (fs.FileInfo, error) {
if f.File != nil {
return f.FileInfo(), nil
}
return nil, fmt.Errorf("no file")
}
func (f *ZipFile) Close() error {
if f.rc != nil {
return f.rc.Close()
}
return nil
}
func (f *ZipFile) Read(b []byte) (int, error) {
if f.rc == nil {
var err error
f.rc, err = f.Open()
if err != nil {
return 0, err
}
}
n, err := f.rc.Read(b)
f.offset += int64(n)
if err == io.EOF {
f.rc.Close()
f.rc = nil
}
return n, err
}
func (f *ZipFile) Seek(offset int64, whence int) (int64, error) {
if f.rc != nil {
f.rc.Close()
f.rc = nil
}
switch whence {
case io.SeekStart:
if offset < 0 {
return 0, &fs.PathError{Op: "seek", Path: f.Name, Err: fmt.Errorf("negative position")}
}
f.offset = offset
case io.SeekCurrent:
if f.offset+offset < 0 {
return 0, &fs.PathError{Op: "seek", Path: f.Name, Err: fmt.Errorf("negative position")}
}
f.offset += offset
case io.SeekEnd:
size := int64(f.UncompressedSize64)
if size+offset < 0 {
return 0, &fs.PathError{Op: "seek", Path: f.Name, Err: fmt.Errorf("negative position")}
}
f.offset = size + offset
}
return f.offset, nil
}
/*
func main() {
r, err := zip.OpenReader("path/to/your.zip")
if err != nil {
log.Fatal(err)
}
defer r.Close()
fs := NewZipFS(&r.Reader)
file, err := fs.Open(path.Join("path", "to", "file"))
if err != nil {
log.Fatal(err)
}
defer file.Close()
// Now you can use 'file' as a fs.File
}
*/