wdevs/whatshooked
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Event logging

Browse Source
This commit is contained in:
Warky
2025-12-29 06:01:04 +02:00
parent 2b1b77334a
commit 09a12560d3
11 changed files with 733 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

249
EVENT_LOGGER.md Normal file
View File

@@ -0,0 +1,249 @@
# Event Logger Configuration
The event logger allows you to persist all system events to various storage targets for auditing, debugging, and analytics.
## Configuration
Add the `event_logger` section to your `config.json`:
```json
{
"event_logger": {
"enabled": true,
"targets": ["file", "sqlite", "postgres"],
"file_dir": "./data/events",
"table_name": "event_logs"
},
"database": {
"type": "postgres",
"host": "localhost",
"port": 5432,
"username": "whatshooked",
"password": "your_password_here",
"database": "whatshooked",
"sqlite_path": "./data/events.db"
}
}
```
## Configuration Options
### event_logger
- **enabled** (boolean): Enable or disable event logging
- Default: `false`
- **targets** (array): List of storage targets to use. Options:
- `"file"` - Store events as JSON files in organized directories
- `"sqlite"` - Store events in a local SQLite database
- `"postgres"` or `"postgresql"` - Store events in PostgreSQL database
- **file_dir** (string): Base directory for file-based event storage
- Default: `"./data/events"`
- Events are organized as: `{file_dir}/{event_type}/{YYYYMMDD}/{HH_MM_SS}_{event_type}.json`
- **table_name** (string): Database table name for storing events
- Default: `"event_logs"`
### database
Database configuration is shared with the event logger when using `sqlite` or `postgres` targets.
For **SQLite**:
- `sqlite_path`: Path to SQLite database file (e.g., `"./data/events.db"`)
- If not specified, defaults to `"./data/events.db"`
For **PostgreSQL**:
- `type`: `"postgres"`
- `host`: Database host (e.g., `"localhost"`)
- `port`: Database port (e.g., `5432`)
- `username`: Database username
- `password`: Database password
- `database`: Database name
## Storage Targets
### File Target
Events are stored as JSON files in an organized directory structure:
```
./data/events/
├── message.received/
│ ├── 20231225/
│ │ ├── 14_30_45_message.received.json
│ │ ├── 14_31_12_message.received.json
│ │ └── 14_32_00_message.received.json
│ └── 20231226/
│ └── 09_15_30_message.received.json
├── message.sent/
│ └── 20231225/
│ └── 14_30_50_message.sent.json
└── whatsapp.connected/
└── 20231225/
└── 14_00_00_whatsapp.connected.json
```
Each file contains the complete event data:
```json
{
"type": "message.received",
"timestamp": "2023-12-25T14:30:45Z",
"data": {
"account_id": "acc1",
"from": "+1234567890",
"message": "Hello, world!",
...
}
}
```
### SQLite Target
Events are stored in a SQLite database with the following schema:
```sql
CREATE TABLE event_logs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
event_type TEXT NOT NULL,
timestamp DATETIME NOT NULL,
data TEXT NOT NULL,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX idx_event_logs_type_timestamp
ON event_logs(event_type, timestamp);
```
### PostgreSQL Target
Events are stored in PostgreSQL with JSONB support for efficient querying:
```sql
CREATE TABLE event_logs (
id SERIAL PRIMARY KEY,
event_type VARCHAR(100) NOT NULL,
timestamp TIMESTAMP NOT NULL,
data JSONB NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX idx_event_logs_type_timestamp
ON event_logs(event_type, timestamp);
```
## Event Types
The following event types are logged:
### WhatsApp Connection Events
- `whatsapp.connected`
- `whatsapp.disconnected`
- `whatsapp.pair.success`
- `whatsapp.pair.failed`
- `whatsapp.qr.code`
- `whatsapp.qr.timeout`
- `whatsapp.qr.error`
- `whatsapp.pair.event`
### Message Events
- `message.received`
- `message.sent`
- `message.failed`
- `message.delivered`
- `message.read`
### Hook Events
- `hook.triggered`
- `hook.success`
- `hook.failed`
## Examples
### Enable File Logging Only
```json
{
"event_logger": {
"enabled": true,
"targets": ["file"],
"file_dir": "./logs/events"
}
}
```
### Enable SQLite Logging Only
```json
{
"event_logger": {
"enabled": true,
"targets": ["sqlite"]
},
"database": {
"sqlite_path": "./data/whatshooked.db"
}
}
```
### Enable Multiple Targets
```json
{
"event_logger": {
"enabled": true,
"targets": ["file", "sqlite", "postgres"],
"file_dir": "./data/events",
"table_name": "event_logs"
},
"database": {
"type": "postgres",
"host": "localhost",
"port": 5432,
"username": "whatshooked",
"password": "securepassword",
"database": "whatshooked",
"sqlite_path": "./data/events.db"
}
}
```
## Querying Events
### SQLite Query Examples
```sql
-- Get all message events from the last 24 hours
SELECT * FROM event_logs
WHERE event_type LIKE 'message.%'
AND timestamp > datetime('now', '-1 day')
ORDER BY timestamp DESC;
-- Count events by type
SELECT event_type, COUNT(*) as count
FROM event_logs
GROUP BY event_type
ORDER BY count DESC;
```
### PostgreSQL Query Examples
```sql
-- Get all message events with specific sender
SELECT * FROM event_logs
WHERE event_type = 'message.received'
AND data->>'from' = '+1234567890'
ORDER BY timestamp DESC;
-- Find events with specific data fields
SELECT event_type, timestamp, data
FROM event_logs
WHERE data @> '{"account_id": "acc1"}'
ORDER BY timestamp DESC
LIMIT 100;
```
## Performance Considerations
- **File Target**: Suitable for low to medium event volumes. Easy to backup and archive.
- **SQLite Target**: Good for single-server deployments. Supports moderate event volumes.
- **PostgreSQL Target**: Best for high-volume deployments and complex queries. Supports concurrent access.
You can enable multiple targets simultaneously to balance immediate access (file) with queryability (database).

2
TODO.md
View File

@@ -7,5 +7,5 @@
- [ ] Whatsapp Business API support add
- [ ] Optional Postgres server connection for Whatsmeo
- [ ] Optional Postgres server,database for event saving and hook registration
- [ ] Optional Event logging into directory for each type
- [✔️] Optional Event logging into directory for each type
- [ ] MQTT Support for events (To connect it to home assistant, have to prototype. Incoming message/outgoing messages)

25
cmd/server/main.go
View File

@@ -12,6 +12,7 @@ import (
"time"
"git.warky.dev/wdevs/whatshooked/internal/config"
"git.warky.dev/wdevs/whatshooked/internal/eventlogger"
"git.warky.dev/wdevs/whatshooked/internal/events"
"git.warky.dev/wdevs/whatshooked/internal/hooks"
"git.warky.dev/wdevs/whatshooked/internal/logging"
@@ -31,6 +32,7 @@ type Server struct {
hookMgr *hooks.Manager
httpServer *http.Server
eventBus *events.EventBus
eventLogger *eventlogger.Logger
}
// resolveConfigPath determines the config file path to use
@@ -101,6 +103,21 @@ func main() {
hookMgr: hooks.NewManager(eventBus),
}
// Initialize event logger if enabled
if cfg.EventLogger.Enabled && len(cfg.EventLogger.Targets) > 0 {
evtLogger, err := eventlogger.NewLogger(cfg.EventLogger, cfg.Database)
if err != nil {
logging.Error("Failed to initialize event logger", "error", err)
} else {
srv.eventLogger = evtLogger
// Subscribe to all events
srv.eventBus.SubscribeAll(func(event events.Event) {
srv.eventLogger.Log(event)
})
logging.Info("Event logger initialized", "targets", cfg.EventLogger.Targets)
}
}
// Load hooks
srv.hookMgr.LoadHooks(cfg.Hooks)
@@ -142,6 +159,14 @@ func main() {
}
srv.whatsappMgr.DisconnectAll()
// Close event logger
if srv.eventLogger != nil {
if err := srv.eventLogger.Close(); err != nil {
logging.Error("Failed to close event logger", "error", err)
}
}
logging.Info("Server stopped")
}

18
config.example.json
View File

@@ -78,5 +78,23 @@
"mode": "link",
"base_url": "http://localhost:8080"
},
"database": {
"type": "postgres",
"host": "localhost",
"port": 5432,
"username": "whatshooked",
"password": "your_password_here",
"database": "whatshooked",
"sqlite_path": "./data/events.db"
},
"event_logger": {
"enabled": false,
"targets": [
"file",
"sqlite"
],
"file_dir": "./data/events",
"table_name": "event_logs"
},
"log_level": "info"
}

1
go.mod
View File

@@ -3,6 +3,7 @@ module git.warky.dev/wdevs/whatshooked
go 1.25.5
require (
github.com/lib/pq v1.10.9
github.com/mattn/go-sqlite3 v1.14.32
github.com/mdp/qrterminal/v3 v3.2.1
github.com/spf13/cobra v1.10.2

2
go.sum
View File

@@ -33,6 +33,8 @@ github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=

47
internal/config/config.go
View File

@@ -7,12 +7,13 @@ import (
// Config represents the application configuration
type Config struct {
Server ServerConfig `json:"server"`
WhatsApp []WhatsAppConfig `json:"whatsapp"`
Hooks []Hook `json:"hooks"`
Database DatabaseConfig `json:"database,omitempty"`
Media MediaConfig `json:"media"`
LogLevel string `json:"log_level"`
Server ServerConfig `json:"server"`
WhatsApp []WhatsAppConfig `json:"whatsapp"`
Hooks []Hook `json:"hooks"`
Database DatabaseConfig `json:"database,omitempty"`
Media MediaConfig `json:"media"`
EventLogger EventLoggerConfig `json:"event_logger,omitempty"`
LogLevel string `json:"log_level"`
}
// ServerConfig holds server-specific configuration
@@ -47,12 +48,13 @@ type Hook struct {
// DatabaseConfig holds database connection information
type DatabaseConfig struct {
Type string `json:"type"`
Host string `json:"host"`
Port int `json:"port"`
Username string `json:"username"`
Password string `json:"password"`
Database string `json:"database"`
Type string `json:"type"`
Host string `json:"host"`
Port int `json:"port"`
Username string `json:"username"`
Password string `json:"password"`
Database string `json:"database"`
SQLitePath string `json:"sqlite_path,omitempty"` // Path to SQLite database file
}
// MediaConfig holds media storage and delivery configuration
@@ -62,6 +64,18 @@ type MediaConfig struct {
BaseURL string `json:"base_url,omitempty"` // Base URL for media links
}
// EventLoggerConfig holds event logging configuration
type EventLoggerConfig struct {
Enabled bool `json:"enabled"`
Targets []string `json:"targets"` // "file", "sqlite", "postgres"
// File-based logging
FileDir string `json:"file_dir,omitempty"` // Base directory for event files
// Database logging (uses main Database config for connection)
TableName string `json:"table_name,omitempty"` // Table name for event logs (default: "event_logs")
}
// Load reads configuration from a file
func Load(path string) (*Config, error) {
data, err := os.ReadFile(path)
@@ -90,6 +104,15 @@ func Load(path string) (*Config, error) {
if cfg.Media.Mode == "" {
cfg.Media.Mode = "link"
}
if cfg.EventLogger.FileDir == "" {
cfg.EventLogger.FileDir = "./data/events"
}
if cfg.EventLogger.TableName == "" {
cfg.EventLogger.TableName = "event_logs"
}
if cfg.Database.SQLitePath == "" {
cfg.Database.SQLitePath = "./data/events.db"
}
return &cfg, nil
}

102
internal/eventlogger/eventlogger.go Normal file
View File

@@ -0,0 +1,102 @@
package eventlogger
import (
"fmt"
"strings"
"sync"
"git.warky.dev/wdevs/whatshooked/internal/config"
"git.warky.dev/wdevs/whatshooked/internal/events"
"git.warky.dev/wdevs/whatshooked/internal/logging"
)
// Logger handles event logging to multiple targets
type Logger struct {
config config.EventLoggerConfig
dbConfig config.DatabaseConfig
targets []Target
mu sync.Mutex
}
// Target represents a logging target
type Target interface {
Log(event events.Event) error
Close() error
}
// NewLogger creates a new event logger
func NewLogger(cfg config.EventLoggerConfig, dbConfig config.DatabaseConfig) (*Logger, error) {
logger := &Logger{
config: cfg,
dbConfig: dbConfig,
targets: make([]Target, 0),
}
// Initialize targets based on configuration
for _, targetType := range cfg.Targets {
switch strings.ToLower(targetType) {
case "file":
fileTarget, err := NewFileTarget(cfg.FileDir)
if err != nil {
logging.Error("Failed to initialize file target", "error", err)
continue
}
logger.targets = append(logger.targets, fileTarget)
logging.Info("Event logger file target initialized", "dir", cfg.FileDir)
case "sqlite":
sqliteTarget, err := NewSQLiteTarget(dbConfig, cfg.TableName)
if err != nil {
logging.Error("Failed to initialize SQLite target", "error", err)
continue
}
logger.targets = append(logger.targets, sqliteTarget)
logging.Info("Event logger SQLite target initialized")
case "postgres", "postgresql":
postgresTarget, err := NewPostgresTarget(dbConfig, cfg.TableName)
if err != nil {
logging.Error("Failed to initialize PostgreSQL target", "error", err)
continue
}
logger.targets = append(logger.targets, postgresTarget)
logging.Info("Event logger PostgreSQL target initialized")
default:
logging.Error("Unknown event logger target type", "type", targetType)
}
}
return logger, nil
}
// Log logs an event to all configured targets
func (l *Logger) Log(event events.Event) {
l.mu.Lock()
defer l.mu.Unlock()
for _, target := range l.targets {
if err := target.Log(event); err != nil {
logging.Error("Failed to log event", "target", fmt.Sprintf("%T", target), "error", err)
}
}
}
// Close closes all logging targets
func (l *Logger) Close() error {
l.mu.Lock()
defer l.mu.Unlock()
var errors []string
for _, target := range l.targets {
if err := target.Close(); err != nil {
errors = append(errors, err.Error())
}
}
if len(errors) > 0 {
return fmt.Errorf("errors closing targets: %s", strings.Join(errors, "; "))
}
return nil
}

69
internal/eventlogger/file_target.go Normal file
View File

@@ -0,0 +1,69 @@
package eventlogger
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"sync"
"git.warky.dev/wdevs/whatshooked/internal/events"
)
// FileTarget logs events to organized file structure
type FileTarget struct {
baseDir string
mu sync.Mutex
}
// NewFileTarget creates a new file-based logging target
func NewFileTarget(baseDir string) (*FileTarget, error) {
// Create base directory if it doesn't exist
if err := os.MkdirAll(baseDir, 0755); err != nil {
return nil, fmt.Errorf("failed to create event log directory: %w", err)
}
return &FileTarget{
baseDir: baseDir,
}, nil
}
// Log writes an event to a file
func (ft *FileTarget) Log(event events.Event) error {
ft.mu.Lock()
defer ft.mu.Unlock()
// Create directory structure: baseDir/[type]/[YYYYMMDD]/
eventType := string(event.Type)
dateDir := event.Timestamp.Format("20060102")
dirPath := filepath.Join(ft.baseDir, eventType, dateDir)
if err := os.MkdirAll(dirPath, 0755); err != nil {
return fmt.Errorf("failed to create event directory: %w", err)
}
// Create filename: [hh24_mi_ss]_[type].json
filename := fmt.Sprintf("%s_%s.json",
event.Timestamp.Format("15_04_05"),
eventType,
)
filePath := filepath.Join(dirPath, filename)
// Marshal event to JSON
data, err := json.MarshalIndent(event, "", " ")
if err != nil {
return fmt.Errorf("failed to marshal event: %w", err)
}
// Write to file
if err := os.WriteFile(filePath, data, 0644); err != nil {
return fmt.Errorf("failed to write event file: %w", err)
}
return nil
}
// Close closes the file target (no-op for file target)
func (ft *FileTarget) Close() error {
return nil
}

120
internal/eventlogger/postgres_target.go Normal file
View File

@@ -0,0 +1,120 @@
package eventlogger
import (
"context"
"database/sql"
"encoding/json"
"fmt"
"sync"
"time"
"git.warky.dev/wdevs/whatshooked/internal/config"
"git.warky.dev/wdevs/whatshooked/internal/events"
_ "github.com/lib/pq" // PostgreSQL driver
)
// PostgresTarget logs events to PostgreSQL database
type PostgresTarget struct {
db *sql.DB
tableName string
mu sync.Mutex
}
// NewPostgresTarget creates a new PostgreSQL logging target
func NewPostgresTarget(dbConfig config.DatabaseConfig, tableName string) (*PostgresTarget, error) {
// Build connection string
connStr := fmt.Sprintf("host=%s port=%d user=%s password=%s dbname=%s sslmode=disable",
dbConfig.Host,
dbConfig.Port,
dbConfig.Username,
dbConfig.Password,
dbConfig.Database,
)
// Open PostgreSQL connection
db, err := sql.Open("postgres", connStr)
if err != nil {
return nil, fmt.Errorf("failed to open PostgreSQL database: %w", err)
}
// Test connection
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := db.PingContext(ctx); err != nil {
db.Close()
return nil, fmt.Errorf("failed to connect to PostgreSQL: %w", err)
}
target := &PostgresTarget{
db: db,
tableName: tableName,
}
// Create table if it doesn't exist
if err := target.createTable(); err != nil {
db.Close()
return nil, err
}
return target, nil
}
// createTable creates the event logs table if it doesn't exist
func (pt *PostgresTarget) createTable() error {
query := fmt.Sprintf(`
CREATE TABLE IF NOT EXISTS %s (
id SERIAL PRIMARY KEY,
event_type VARCHAR(100) NOT NULL,
timestamp TIMESTAMP NOT NULL,
data JSONB NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
`, pt.tableName)
if _, err := pt.db.Exec(query); err != nil {
return fmt.Errorf("failed to create table: %w", err)
}
// Create index on event_type and timestamp
indexQuery := fmt.Sprintf(`
CREATE INDEX IF NOT EXISTS idx_%s_type_timestamp
ON %s(event_type, timestamp)
`, pt.tableName, pt.tableName)
if _, err := pt.db.Exec(indexQuery); err != nil {
return fmt.Errorf("failed to create index: %w", err)
}
return nil
}
// Log writes an event to PostgreSQL database
func (pt *PostgresTarget) Log(event events.Event) error {
pt.mu.Lock()
defer pt.mu.Unlock()
// Marshal event data to JSON
data, err := json.Marshal(event.Data)
if err != nil {
return fmt.Errorf("failed to marshal event data: %w", err)
}
query := fmt.Sprintf(`
INSERT INTO %s (event_type, timestamp, data)
VALUES ($1, $2, $3)
`, pt.tableName)
_, err = pt.db.Exec(query, string(event.Type), event.Timestamp, string(data))
if err != nil {
return fmt.Errorf("failed to insert event: %w", err)
}
return nil
}
// Close closes the PostgreSQL database connection
func (pt *PostgresTarget) Close() error {
return pt.db.Close()
}

111
internal/eventlogger/sqlite_target.go Normal file
View File

@@ -0,0 +1,111 @@
package eventlogger
import (
"database/sql"
"encoding/json"
"fmt"
"os"
"path/filepath"
"sync"
"git.warky.dev/wdevs/whatshooked/internal/config"
"git.warky.dev/wdevs/whatshooked/internal/events"
_ "github.com/mattn/go-sqlite3" // SQLite driver
)
// SQLiteTarget logs events to SQLite database
type SQLiteTarget struct {
db *sql.DB
tableName string
mu sync.Mutex
}
// NewSQLiteTarget creates a new SQLite logging target
func NewSQLiteTarget(dbConfig config.DatabaseConfig, tableName string) (*SQLiteTarget, error) {
// Use the SQLite path from config (defaults to "./data/events.db")
dbPath := dbConfig.SQLitePath
// Create directory if needed
dir := filepath.Dir(dbPath)
if err := os.MkdirAll(dir, 0755); err != nil {
return nil, fmt.Errorf("failed to create database directory: %w", err)
}
// Open SQLite connection
db, err := sql.Open("sqlite3", dbPath)
if err != nil {
return nil, fmt.Errorf("failed to open SQLite database: %w", err)
}
target := &SQLiteTarget{
db: db,
tableName: tableName,
}
// Create table if it doesn't exist
if err := target.createTable(); err != nil {
db.Close()
return nil, err
}
return target, nil
}
// createTable creates the event logs table if it doesn't exist
func (st *SQLiteTarget) createTable() error {
query := fmt.Sprintf(`
CREATE TABLE IF NOT EXISTS %s (
id INTEGER PRIMARY KEY AUTOINCREMENT,
event_type TEXT NOT NULL,
timestamp DATETIME NOT NULL,
data TEXT NOT NULL,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
)
`, st.tableName)
if _, err := st.db.Exec(query); err != nil {
return fmt.Errorf("failed to create table: %w", err)
}
// Create index on event_type and timestamp
indexQuery := fmt.Sprintf(`
CREATE INDEX IF NOT EXISTS idx_%s_type_timestamp
ON %s(event_type, timestamp)
`, st.tableName, st.tableName)
if _, err := st.db.Exec(indexQuery); err != nil {
return fmt.Errorf("failed to create index: %w", err)
}
return nil
}
// Log writes an event to SQLite database
func (st *SQLiteTarget) Log(event events.Event) error {
st.mu.Lock()
defer st.mu.Unlock()
// Marshal event data to JSON
data, err := json.Marshal(event.Data)
if err != nil {
return fmt.Errorf("failed to marshal event data: %w", err)
}
query := fmt.Sprintf(`
INSERT INTO %s (event_type, timestamp, data)
VALUES (?, ?, ?)
`, st.tableName)
_, err = st.db.Exec(query, string(event.Type), event.Timestamp, string(data))
if err != nil {
return fmt.Errorf("failed to insert event: %w", err)
}
return nil
}
// Close closes the SQLite database connection
func (st *SQLiteTarget) Close() error {
return st.db.Close()
}