@@ -57,22 +57,6 @@ go run cmd/app/main.go
**[Full Documentation](docs_wiki/)** - Comprehensive guides and references
-### Core Documentation
-- **[Configuration Guide](docs_wiki/CONFIGURATION_GUIDE.md)** - Complete configuration reference
-- **[API Response Structure](docs_wiki/API_RESPONSE_STRUCTURE.md)** - Standard response formats
-- **[Architecture Diagrams](docs_wiki/ARCHITECTURE_DIAGRAMS.md)** - System design and flow diagrams
-- **[Service Implementation](docs_wiki/SERVICE_IMPLEMENTATION.md)** - How to add new services
-
-### Infrastructure & Integration
-- **[Integration Guide](docs_wiki/INTEGRATION_GUIDE.md)** - Redis, PostgreSQL, Kafka, MinIO setup
-- **[Build Scripts](docs_wiki/BUILD_SCRIPTS.md)** - Production deployment automation
-- **[Package Management](docs_wiki/CHANGE_PACKAGE_SCRIPTS.md)** - Module renaming tools
-
-### Security & Features
-- **[API Encryption](docs_wiki/ENCRYPTION_API.md)** - End-to-end encryption
-- **[API Obfuscation](docs_wiki/API_OBFUSCATION.md)** - Data obfuscation mechanisms
-- **[TUI Implementation](docs_wiki/TUI_IMPLEMENTATION.md)** - Terminal interface details
-
## Project Structure
```
@@ -111,4 +95,4 @@ Apache License Version 2.0: [LICENSE](LICENSE)
---
-**Built with 🐸 using Go, Echo, Alpine.js, Tailwind CSS**
+**Built using Go, Echo, Alpine.js, Tailwind CSS**
diff --git a/cmd/app/main.go b/cmd/app/main.go
index 393cf79..5cce144 100644
--- a/cmd/app/main.go
+++ b/cmd/app/main.go
@@ -7,13 +7,13 @@ import (
"net/url"
"os"
"os/signal"
+ "stackyard/config"
+ "stackyard/internal/monitoring"
+ "stackyard/internal/server"
+ "stackyard/pkg/logger"
+ "stackyard/pkg/tui"
+ "stackyard/pkg/utils"
"syscall"
- "test-go/config"
- "test-go/internal/monitoring"
- "test-go/internal/server"
- "test-go/pkg/logger"
- "test-go/pkg/tui"
- "test-go/pkg/utils"
"time"
)
@@ -193,7 +193,7 @@ func runWithTUI(cfg *config.Config, bannerText string, broadcaster *monitoring.L
liveTUI.AddLog("info", "Server starting on port "+cfg.Server.Port)
liveTUI.AddLog("info", "Environment: "+cfg.App.Env)
- // Start Server in background
+ // Start Server in background - infrastructure will be initialized by the server
srv := server.New(cfg, l, broadcaster)
go func() {
liveTUI.AddLog("info", "HTTP server listening...")
diff --git a/docs_wiki/API_OBFUSCATION.md b/docs_wiki/API_OBFUSCATION.md
deleted file mode 100644
index 100de8c..0000000
--- a/docs_wiki/API_OBFUSCATION.md
+++ /dev/null
@@ -1,72 +0,0 @@
-# API Obfuscation Mechanism
-
-## Overview
-
-The API Obfuscation feature is designed to obscure JSON data in transit between the backend and the frontend. This adds a layer of stealth to the monitoring system, making traffic analysis more difficult for casual observers. The system uses Base64 encoding for the response body of specific API endpoints.
-
-## Configuration
-
-Obfuscation is controlled via the `config.yaml` file.
-
-```yaml
-monitoring:
- obfuscate_api: true # Set to true to enable, false to disable
-```
-
-If enabled, the backend will automatically encode eligible API responses. If disabled, the backend serves standard JSON, and the frontend transparently handles the standard response.
-
-## Backend Implementation
-
-The core logic resides in `internal/monitoring/middleware/obfuscator.go`.
-
-### Middleware Logic
-
-1. **Scope**: The middleware intercepts requests starting with `/api/`.
-2. **Exclusions**: The following paths are explicitly excluded from obfuscation to support streaming or static content:
- * `/api/logs` (SSE stream)
- * `/api/cpu` (SSE stream)
- * `/api/user/photos` (Binary/Static)
-3. **Content Negotiation**: The middleware only processes responses where the `Content-Type` includes `application/json`. `text/event-stream` is skipped.
-4. **Encoding**: The response body is read into a buffer, encoded using Standard Base64 (padding with `=`), and written back to the response.
-5. **Headers**:
- * `X-Obfuscated: true` is set to indicate the response is encoded.
- * `Content-Length` is updated to reflect the size of the encoded body to prevent truncation or keep-alive issues.
-
-### Code Reference
-
-See `Obfuscator` function in `internal/monitoring/middleware/obfuscator.go`.
-
-## Frontend Implementation
-
-The frontend handles de-obfuscation transparently using a global `window.fetch` interceptor in `web/monitoring/assets/js/app.js`.
-
-### Interceptor Strategy
-
-The interceptor wraps the native `fetch` API and applies a "Parse First, Decode Second" strategy to ensure robustness.
-
-1. **Check Content Type**: Ignores non-JSON and `text/event-stream` responses.
-2. **Strategy 1: Try Parse**:
- * Attempts to `JSON.parse()` the response body directly.
- * If successful, the response is standard JSON (not obfuscated). It returns the original response.
-3. **Strategy 2: Try Decode (Fallback)**:
- * If parsing fails (typical for Base64 strings), it attempts to decode the body.
- * **Normalization**: Replaces URL-safe characters (`-` to `+`, `_` to `/`) and removes whitespace.
- * **Padding**: Ensures the string length is a multiple of 4 by adding `=` padding.
- * **Decoding**: Uses `atob()` and `TextDecoder` (UTF-8) to convert the Base64 string back to text.
- * **Verification**: Attempts to `JSON.parse()` the decoded string.
- * If valid JSON is found, a new `Response` object is created with the decoded content and returned.
-4. **Final Fallback**:
- * If both strategies fail, the original body is returned.
-
-This approach ensures the frontend continues to work seamlessly whether obfuscation is enabled or disabled, or if headers are stripped by proxies.
-
-## Troubleshooting
-
-### "SyntaxError: Unexpected token 'e', ..."
-This error occurs when the frontend tries to parse the raw Base64 string as JSON. This usually indicates the interceptor failed to detect or decode the obfuscated response. The "Parse First, Decode Second" strategy resolves this by specifically catching parse errors and triggering the decode logic.
-
-### Truncated Data / Network Errors
-If the `Content-Length` header does not match the actual body size (e.g., if the body size changed due to encoding but the header remained the original size), browsers may truncate the response. The middleware explicitly sets the correct `Content-Length` of the encoded body.
-
-### CORS and Headers
-The `X-Obfuscated` header is exposed in the CORS configuration (`internal/monitoring/server.go`) to allow the frontend to detect encryption status explicitly, though the heuristic fallback logic ensures functionality even if this header is stripped.
diff --git a/docs_wiki/API_RESPONSE_STRUCTURE.md b/docs_wiki/API_RESPONSE_STRUCTURE.md
deleted file mode 100644
index 3c35313..0000000
--- a/docs_wiki/API_RESPONSE_STRUCTURE.md
+++ /dev/null
@@ -1,332 +0,0 @@
-# API Response Structure
-
-## Overview
-
-This document describes the standardized request/response structure for the Echo service.
-
-## Response Format
-
-All API responses follow a consistent structure using the `response.Response` type.
-
-### Success Response
-
-```json
-{
- "success": true,
- "message": "Optional success message",
- "data": {
- // Your response data here
- },
- "meta": {
- "page": 1,
- "per_page": 10,
- "total": 100,
- "total_pages": 10
- },
- "timestamp": 1672531200
-}
-```
-
-### Error Response
-
-```json
-{
- "success": false,
- "error": {
- "code": "ERROR_CODE",
- "message": "Human-readable error message",
- "details": {
- "field": "Additional error details"
- }
- },
- "timestamp": 1672531200
-}
-```
-
-## Usage Examples
-
-### Basic Success Response
-
-```go
-import (
- "test-go/pkg/response"
- "github.com/labstack/echo/v4"
-)
-
-func GetUser(c echo.Context) error {
- user := map[string]string{
- "id": "123",
- "name": "John Doe",
- }
-
- return response.Success(c, user, "User retrieved successfully")
-}
-```
-
-### Paginated Response
-
-```go
-func GetUsers(c echo.Context) error {
- // Parse pagination from query
- var pagination response.PaginationRequest
- if err := c.Bind(&pagination); err != nil {
- return response.BadRequest(c, "Invalid pagination parameters")
- }
-
- // Get data (example)
- users := []User{} // Your users data
- total := int64(100)
-
- // Calculate meta
- meta := response.CalculateMeta(
- pagination.GetPage(),
- pagination.GetPerPage(),
- total,
- )
-
- return response.SuccessWithMeta(c, users, meta, "Users retrieved")
-}
-```
-
-### Error Responses
-
-```go
-// Bad Request
-if err := validate(data); err != nil {
- return response.BadRequest(c, "Invalid data", map[string]interface{}{
- "validation_errors": err,
- })
-}
-
-// Not Found
-user := findUser(id)
-if user == nil {
- return response.NotFound(c, "User not found")
-}
-
-// Unauthorized
-if !isAuthenticated {
- return response.Unauthorized(c, "Invalid credentials")
-}
-
-// Internal Server Error
-if err := processData(); err != nil {
- return response.InternalServerError(c, "Failed to process data")
-}
-```
-
-### Request Validation
-
-```go
-import (
- "test-go/pkg/request"
- "test-go/pkg/response"
-)
-
-type CreateUserRequest struct {
- Username string `json:"username" validate:"required,username"`
- Email string `json:"email" validate:"required,email"`
- Age int `json:"age" validate:"required,gte=18,lte=100"`
- Phone string `json:"phone" validate:"required,phone"`
-}
-
-func CreateUser(c echo.Context) error {
- var req CreateUserRequest
-
- // Bind and validate in one call
- if err := request.Bind(c, &req); err != nil {
- if validationErr, ok := err.(*request.ValidationError); ok {
- return response.ValidationError(c, "Validation failed", validationErr.GetFieldErrors())
- }
- return response.BadRequest(c, err.Error())
- }
-
- // Process the valid request
- user := createUser(req)
-
- return response.Created(c, user, "User created successfully")
-}
-```
-
-## Available Response Helpers
-
-### Success Responses
-- `response.Success(c, data, message)` - 200 OK
-- `response.SuccessWithMeta(c, data, meta, message)` - 200 OK with metadata
-- `response.Created(c, data, message)` - 201 Created
-- `response.NoContent(c)` - 204 No Content
-
-### Error Responses
-- `response.BadRequest(c, message, details)` - 400 Bad Request
-- `response.Unauthorized(c, message)` - 401 Unauthorized
-- `response.Forbidden(c, message)` - 403 Forbidden
-- `response.NotFound(c, message)` - 404 Not Found
-- `response.Conflict(c, message, details)` - 409 Conflict
-- `response.ValidationError(c, message, details)` - 422 Unprocessable Entity
-- `response.InternalServerError(c, message)` - 500 Internal Server Error
-- `response.ServiceUnavailable(c, message)` - 503 Service Unavailable
-- `response.Error(c, statusCode, errorCode, message, details)` - Custom error
-
-## Pagination
-
-The `PaginationRequest` struct provides convenient methods:
-
-```go
-type PaginationRequest struct {
- Page int `query:"page" json:"page"`
- PerPage int `query:"per_page" json:"per_page"`
- Sort string `query:"sort" json:"sort,omitempty"`
- Order string `query:"order" json:"order,omitempty"`
-}
-
-// Methods
-pagination.GetPage() // Returns page (default: 1)
-pagination.GetPerPage() // Returns per_page (default: 10, max: 100)
-pagination.GetOffset() // Calculates offset for DB queries
-pagination.GetOrder() // Returns order (default: "desc")
-```
-
-## Request Validation
-
-### Built-in Validators
-- `required` - Field must not be empty
-- `email` - Valid email format
-- `min`, `max` - String length or numeric range
-- `gte`, `lte` - Greater/less than or equal
-- `oneof` - Value must be one of the specified options
-
-### Custom Validators
-- `phone` - Valid phone number format
-- `username` - Alphanumeric username (3-20 chars)
-
-### Common Request Structs
-
-```go
-// ID Request
-type IDRequest struct {
- ID string `param:"id" validate:"required"`
-}
-
-// Search Request
-type SearchRequest struct {
- Query string `query:"q" json:"query"`
- Filter map[string]string `query:"filter" json:"filter,omitempty"`
- Page int `query:"page" json:"page"`
- Limit int `query:"limit" json:"limit"`
-}
-
-// Date Range Request
-type DateRangeRequest struct {
- StartDate string `query:"start_date" json:"start_date"`
- EndDate string `query:"end_date" json:"end_date"`
-}
-
-// Sort Request
-type SortRequest struct {
- SortBy string `query:"sort_by" json:"sort_by"`
- SortOrder string `query:"sort_order" json:"sort_order"`
-}
-```
-
-## Best Practices
-
-1. **Always use the response helpers** - Don't manually construct response objects
-2. **Include meaningful error codes** - Make errors machine-readable
-3. **Provide context in error messages** - Help developers debug issues
-4. **Use validation** - Validate all incoming requests
-5. **Return appropriate status codes** - Follow HTTP standards
-6. **Include timestamps** - All responses include Unix timestamps
-7. **Use pagination** - For list endpoints, always support pagination
-8. **Keep responses consistent** - All endpoints should follow the same structure
-
-## Exposed Endpoints (Service A)
-- `GET /api/v1/users` - List users
-- `GET /api/v1/users/:id` - Get user details
-- `POST /api/v1/users` - Create user
-- `PUT /api/v1/users/:id` - Update user
-- `DELETE /api/v1/users/:id` - Delete user
-
-## Example Complete Handler
-
-```go
-package modules
-
-import (
- "test-go/pkg/request"
- "test-go/pkg/response"
- "github.com/labstack/echo/v4"
-)
-
-type CreateTaskRequest struct {
- Title string `json:"title" validate:"required,min=3,max=100"`
- Description string `json:"description" validate:"max=500"`
- Priority string `json:"priority" validate:"required,oneof=low medium high"`
- DueDate string `json:"due_date"`
-}
-
-type TaskResponse struct {
- ID string `json:"id"`
- Title string `json:"title"`
- Description string `json:"description"`
- Priority string `json:"priority"`
- Status string `json:"status"`
- CreatedAt int64 `json:"created_at"`
-}
-
-func CreateTask(c echo.Context) error {
- // Bind and validate
- var req CreateTaskRequest
- if err := request.Bind(c, &req); err != nil {
- if validationErr, ok := err.(*request.ValidationError); ok {
- return response.ValidationError(c, "Validation failed", validationErr.GetFieldErrors())
- }
- return response.BadRequest(c, err.Error())
- }
-
- // Business logic
- task, err := saveTask(req)
- if err != nil {
- return response.InternalServerError(c, "Failed to create task")
- }
-
- // Return success
- return response.Created(c, task, "Task created successfully")
-}
-
-func GetTasks(c echo.Context) error {
- // Parse pagination
- var pagination response.PaginationRequest
- c.Bind(&pagination)
-
- // Get tasks from database
- tasks, total, err := fetchTasks(pagination.GetOffset(), pagination.GetPerPage())
- if err != nil {
- return response.InternalServerError(c, "Failed to fetch tasks")
- }
-
- // Return with metadata
- meta := response.CalculateMeta(pagination.GetPage(), pagination.GetPerPage(), total)
- return response.SuccessWithMeta(c, tasks, meta)
-}
-
-func GetTask(c echo.Context) error {
- id := c.Param("id")
-
- task := findTask(id)
- if task == nil {
- return response.NotFound(c, "Task not found")
- }
-
- return response.Success(c, task)
-}
-
-func DeleteTask(c echo.Context) error {
- id := c.Param("id")
-
- if err := deleteTask(id); err != nil {
- return response.InternalServerError(c, "Failed to delete task")
- }
-
- return response.NoContent(c)
-}
-```
diff --git a/docs_wiki/ARCHITECTURE.md b/docs_wiki/ARCHITECTURE.md
new file mode 100644
index 0000000..d80a8e9
--- /dev/null
+++ b/docs_wiki/ARCHITECTURE.md
@@ -0,0 +1,522 @@
+# Architecture Overview
+
+This document provides a high-level overview of Stackyard's architecture, design decisions, and key concepts. Understanding this foundation will help you build effectively with the framework.
+
+## System Overview
+
+Stackyard is a modular, service-oriented Go application framework built on top of the Echo web framework. It emphasizes clean architecture, dependency injection, and production readiness with comprehensive monitoring and infrastructure integrations.
+
+## Core Architecture Principles
+
+### 1. Clean Architecture
+
+Stackyard follows clean architecture principles with clear separation of concerns:
+
+```
+┌─────────────────────────────────────┐
+│ Delivery Layer │
+│ (HTTP Handlers, Middleware) │
+├─────────────────────────────────────┤
+│ Use Case Layer │
+│ (Business Logic, Services) │
+├─────────────────────────────────────┤
+│ Infrastructure Layer │
+│ (Databases, External APIs, Utils) │
+└─────────────────────────────────────┘
+```
+
+**Benefits:**
+- **Testability**: Each layer can be tested independently
+- **Maintainability**: Changes in one layer don't affect others
+- **Flexibility**: Easy to swap implementations (e.g., different databases)
+
+### 2. Service-Oriented Design
+
+Applications are built as **composable services** that can be enabled/disabled via configuration:
+
+- **Modularity**: Services encapsulate related functionality
+- **Independence**: Services can be developed and deployed separately
+- **Configuration-Driven**: Runtime behavior controlled by `config.yaml`
+- **Dependency Injection**: Services receive dependencies through constructors
+
+### 3. Infrastructure Abstraction
+
+All external dependencies are abstracted through **infrastructure managers**:
+
+```go
+// Abstract interface
+type DatabaseManager interface {
+ Connect() error
+ Query(query string, args ...interface{}) (interface{}, error)
+ Close() error
+}
+
+// Concrete implementation
+type PostgresManager struct {
+ db *sqlx.DB
+ config PostgresConfig
+}
+```
+
+**Benefits:**
+- **Testability**: Easy to mock infrastructure in tests
+- **Flexibility**: Can swap implementations without changing business logic
+- **Consistency**: All infrastructure follows the same patterns
+
+## Key Components
+
+### Application Structure
+
+```
+stackyard/
+├── cmd/app/ # Application entry point
+├── config/ # Configuration management
+├── internal/ # Private application code
+│ ├── middleware/ # HTTP middleware
+│ ├── monitoring/ # Monitoring system
+│ ├── server/ # HTTP server setup
+│ └── services/ # Business services
+│ └── modules/ # Service implementations
+├── pkg/ # Public reusable packages
+│ ├── infrastructure/# External service integrations
+│ ├── logger/ # Logging utilities
+│ ├── request/ # Request handling & validation
+│ ├── response/ # Standardized API responses
+│ └── utils/ # General utilities
+├── scripts/ # Build and deployment scripts
+└── web/ # Static web assets
+```
+
+### Request Flow
+
+```
+1. HTTP Request → 2. Echo Router → 3. Middleware → 4. Handler → 5. Response
+
+ ↓ ↓ ↓ ↓ ↓
+ Client Routing Auth/Logging Business Logic JSON
+ (Browser/Mobile) (URL matching) (Validation) (Services) (Response)
+```
+
+### Service Registration
+
+Services are registered dynamically through a **service registry**:
+
+```go
+// Service interface
+type Service interface {
+ Name() string // Human-readable name
+ RegisterRoutes(*echo.Group) // Register HTTP routes
+ Enabled() bool // Service activation status
+ Endpoints() []string // API endpoints list
+}
+
+// Registration
+registry := services.NewServiceRegistry()
+registry.Register(modules.NewUserService(config))
+registry.Register(modules.NewProductService(config))
+registry.Boot(echoInstance) // Wire up all services
+```
+
+## Infrastructure Managers
+
+### Database Managers
+
+Stackyard supports multiple database types through abstracted managers:
+
+#### PostgreSQL Manager
+- **Multi-tenant support**: Dynamic database switching
+- **GORM integration**: Full ORM capabilities with auto-migration
+- **Connection pooling**: Efficient connection management
+- **Async operations**: Non-blocking database operations
+
+#### MongoDB Manager
+- **Document database**: NoSQL capabilities
+- **Multi-tenant**: Database-level isolation
+- **Aggregation pipelines**: Complex data processing
+- **Async operations**: Worker pool-based execution
+
+#### Redis Manager
+- **Caching**: High-performance key-value storage
+- **Pub/Sub**: Message broadcasting capabilities
+- **Batch operations**: Efficient bulk operations
+- **Async execution**: Worker pool processing
+
+### Message Queue Managers
+
+#### Kafka Manager
+- **Event streaming**: High-throughput message processing
+- **Consumer groups**: Load balancing and fault tolerance
+- **Topic management**: Dynamic topic creation and configuration
+- **Async operations**: Non-blocking message publishing
+
+### Object Storage Managers
+
+#### MinIO Manager
+- **S3-compatible**: AWS S3 API compatibility
+- **File uploads**: Efficient multipart upload handling
+- **Access control**: Bucket and object permissions
+- **Async operations**: Background file processing
+
+### Monitoring & Analytics
+
+#### Grafana Manager
+- **Dashboard creation**: Programmatic dashboard generation
+- **Data source integration**: Connect various data sources
+- **Annotation support**: Timeline event marking
+- **Health monitoring**: Service status tracking
+
+## Async Architecture
+
+### Worker Pools
+
+All infrastructure operations use **worker pools** for concurrency control:
+
+```go
+type WorkerPool struct {
+ workers int
+ jobQueue chan func()
+ stopChan chan struct{}
+ stopped chan struct{}
+}
+
+// Usage
+result := manager.AsyncOperation(ctx, data)
+// Operation runs in worker pool, returns immediately
+value, err := result.Wait() // Block until complete
+```
+
+**Benefits:**
+- **Resource control**: Limit concurrent operations
+- **Performance**: Prevent resource exhaustion
+- **Reliability**: Graceful error handling and recovery
+
+### Async Results
+
+Operations return **AsyncResult** types for flexible execution:
+
+```go
+type AsyncResult[T any] struct {
+ Value T
+ Error error
+ Done chan struct{}
+}
+
+// Synchronous usage
+result := manager.GetUserAsync(ctx, userID)
+user, err := result.Wait()
+
+// Timeout support
+user, err := result.WaitWithTimeout(5 * time.Second)
+
+// Non-blocking check
+if result.IsDone() {
+ user, err := result.Wait()
+ // Process result
+}
+```
+
+## Configuration System
+
+### Hierarchical Configuration
+
+Configuration is managed through a **hierarchical YAML structure**:
+
+```yaml
+app: # Application-level settings
+ name: "MyApp"
+ debug: true
+
+server: # HTTP server configuration
+ port: "8080"
+
+services: # Service enable/disable flags
+ user_service: true
+ product_service: false
+
+postgres: # Infrastructure-specific config
+ enabled: true
+ connections:
+ - name: "primary"
+ host: "localhost"
+ database: "myapp"
+```
+
+### Environment Override
+
+Configuration can be overridden with **environment variables**:
+
+```bash
+export APP_DEBUG=false
+export SERVER_PORT=3000
+export POSTGRES_PASSWORD=prod-password
+```
+
+### Validation & Defaults
+
+Configuration is **validated at startup** with sensible defaults:
+
+```go
+type Config struct {
+ App AppConfig `yaml:"app"`
+ Server ServerConfig `yaml:"server"`
+ Postgres PostgresConfig `yaml:"postgres" validate:"required_if=Enabled true"`
+}
+
+func (c *Config) Validate() error {
+ // Custom validation logic
+ return validate.Struct(c)
+}
+```
+
+## API Design Patterns
+
+### Standardized Responses
+
+All API responses follow a **consistent JSON structure**:
+
+```json
+{
+ "success": true,
+ "message": "Operation completed",
+ "data": { /* response data */ },
+ "meta": { /* pagination metadata */ },
+ "timestamp": 1642598400
+}
+```
+
+### Request Validation
+
+Requests are validated using **struct tags** with automatic error formatting:
+
+```go
+type CreateUserRequest struct {
+ Name string `json:"name" validate:"required,min=2,max=50"`
+ Email string `json:"email" validate:"required,email"`
+ Age int `json:"age" validate:"gte=18,lte=120"`
+}
+
+func (h *Handler) createUser(c echo.Context) error {
+ var req CreateUserRequest
+ if err := request.Bind(c, &req); err != nil {
+ return err // Automatic validation error response
+ }
+ // Process valid request...
+}
+```
+
+### Error Handling
+
+Errors are handled consistently with **standardized error codes**:
+
+```go
+// Automatic error responses
+return response.NotFound(c, "User not found")
+return response.BadRequest(c, "Invalid input")
+return response.InternalServerError(c, "Database error")
+```
+
+## Security Architecture
+
+### Authentication & Authorization
+
+- **API Key authentication**: Simple key-based auth
+- **Session management**: Secure session handling
+- **Role-based access**: Permission-based authorization
+
+### Data Protection
+
+- **API Obfuscation**: Base64 encoding for data in transit
+- **Encryption**: AES-256-GCM encryption for sensitive data
+- **Input validation**: Comprehensive request validation
+
+### Infrastructure Security
+
+- **Connection encryption**: TLS for database connections
+- **Secure defaults**: Conservative security settings
+- **Audit logging**: Comprehensive operation logging
+
+## Monitoring & Observability
+
+### Web Dashboard
+
+The monitoring dashboard provides:
+- **Real-time metrics**: System resource usage
+- **API monitoring**: Endpoint performance and errors
+- **Service health**: Individual service status
+- **Log viewing**: Real-time application logs
+
+### Terminal UI (TUI)
+
+The TUI provides:
+- **Boot sequence visualization**: Service initialization status
+- **Live log monitoring**: Real-time log streaming with filtering
+- **Interactive controls**: Keyboard shortcuts for navigation
+- **System monitoring**: Resource usage and performance metrics
+
+### Health Checks
+
+Comprehensive health endpoints:
+- **Application health**: `/health`
+- **Infrastructure health**: `/health/infrastructure`
+- **Service-specific health**: `/health/{service}`
+
+## Build & Deployment
+
+### Multi-Stage Docker Builds
+
+```dockerfile
+FROM golang:1.21-alpine AS builder
+# Build stage - compile application
+
+FROM alpine:latest AS runtime
+# Runtime stage - minimal production image
+
+FROM gcr.io/distroless/static AS ultra-minimal
+# Ultra-minimal production image
+```
+
+### Build Scripts
+
+Cross-platform build scripts provide:
+- **Binary compilation**: Optimized Go builds
+- **Asset bundling**: Include static assets in binary
+- **Backup management**: Automated backup of previous builds
+- **Code obfuscation**: Optional binary obfuscation
+
+### Deployment Options
+
+- **Binary deployment**: Direct server deployment
+- **Docker containers**: Containerized deployment
+- **Kubernetes**: Orchestrated deployment
+- **Serverless**: Function-as-a-service deployment
+
+## Performance Characteristics
+
+### Concurrency Model
+
+- **Goroutines**: Lightweight thread management
+- **Worker pools**: Controlled concurrency for I/O operations
+- **Async processing**: Non-blocking request handling
+- **Connection pooling**: Efficient resource utilization
+
+### Caching Strategy
+
+- **Multi-level caching**: Memory, Redis, CDN
+- **Cache invalidation**: TTL-based and manual invalidation
+- **Cache warming**: Pre-population of frequently accessed data
+
+### Database Optimization
+
+- **Connection pooling**: Efficient database connection management
+- **Query optimization**: Index usage and query planning
+- **Batch operations**: Bulk data operations
+- **Read/write splitting**: Separate read and write databases
+
+## Scalability Features
+
+### Horizontal Scaling
+
+- **Stateless design**: Services can be scaled independently
+- **Load balancing**: Distribute requests across instances
+- **Database sharding**: Horizontal database scaling
+- **Caching layers**: Reduce database load
+
+### Vertical Scaling
+
+- **Resource optimization**: Efficient memory and CPU usage
+- **Async processing**: Handle high concurrency
+- **Connection pooling**: Optimize external service connections
+
+## Development Workflow
+
+### Local Development
+
+```bash
+# Quick start
+go run cmd/app/main.go
+
+# With custom config
+go run cmd/app/main.go -config=config.dev.yaml
+
+# Enable debug logging
+export APP_DEBUG=true
+go run cmd/app/main.go
+```
+
+### Testing Strategy
+
+- **Unit tests**: Individual component testing
+- **Integration tests**: End-to-end API testing
+- **Performance tests**: Load and stress testing
+- **Security tests**: Penetration testing and vulnerability scanning
+
+### CI/CD Pipeline
+
+```yaml
+# GitHub Actions example
+- name: Test
+ run: go test ./...
+
+- name: Build
+ run: ./scripts/build.sh
+
+- name: Docker Build
+ run: ./scripts/docker_build.sh
+
+- name: Deploy
+ run: kubectl apply -f k8s/
+```
+
+## Best Practices
+
+### Code Organization
+
+1. **Service boundaries**: Clear separation of business logic
+2. **Dependency injection**: Constructor-based dependency injection
+3. **Interface segregation**: Small, focused interfaces
+4. **Error handling**: Consistent error handling patterns
+
+### Performance
+
+1. **Async operations**: Use async for I/O operations
+2. **Caching**: Implement appropriate caching strategies
+3. **Pagination**: Always paginate large datasets
+4. **Monitoring**: Monitor performance metrics
+
+### Security
+
+1. **Input validation**: Validate all user inputs
+2. **Secure defaults**: Conservative security settings
+3. **Regular updates**: Keep dependencies updated
+4. **Audit logging**: Log security-relevant events
+
+## Migration & Extensibility
+
+### Adding New Services
+
+1. **Implement Service interface**
+2. **Register in service registry**
+3. **Configure via config.yaml**
+4. **Add tests and documentation**
+
+### Infrastructure Extensions
+
+1. **Create infrastructure manager**
+2. **Implement async operations**
+3. **Add configuration support**
+4. **Update dependency injection**
+
+### API Extensions
+
+1. **Add new endpoints**
+2. **Implement request/response types**
+3. **Add validation rules**
+4. **Update API documentation**
+
+## Conclusion
+
+Stackyard's architecture emphasizes **modularity**, **scalability**, and **maintainability** through clean architecture principles, service-oriented design, and comprehensive infrastructure abstractions. The framework provides a solid foundation for building production-ready applications while maintaining developer productivity and code quality.
+
+The combination of **async processing**, **dependency injection**, and **configuration-driven behavior** makes Stackyard suitable for applications ranging from simple APIs to complex, multi-tenant SaaS platforms.
+
+For detailed implementation guides, see the **[Development Guide](DEVELOPMENT.md)**. For complete API reference, see the **[API Reference](REFERENCE.md)**.
diff --git a/docs_wiki/ARCHITECTURE_DIAGRAMS.md b/docs_wiki/ARCHITECTURE_DIAGRAMS.md
deleted file mode 100644
index 4f52983..0000000
--- a/docs_wiki/ARCHITECTURE_DIAGRAMS.md
+++ /dev/null
@@ -1,245 +0,0 @@
-# Request/Response Flow Architecture
-
-```mermaid
-flowchart TB
- Client[Client Request]
- Handler[Handler Function]
- Bind[request.Bind]
- Validate[request.Validate]
- Logic[Business Logic]
- Success[response.Success]
- Error[response.Error]
- Response[JSON Response]
-
- Client -->|HTTP Request| Handler
- Handler --> Bind
- Bind -->|Parse JSON| Validate
- Validate -->|Invalid| Error
- Validate -->|Valid| Logic
- Logic -->|Success| Success
- Logic -->|Error| Error
- Success --> Response
- Error --> Response
- Response -->|JSON| Client
-
- style Client fill:#e1f5ff
- style Response fill:#e1f5ff
- style Success fill:#d4edda
- style Error fill:#f8d7da
- style Logic fill:#fff3cd
-```
-
-## Response Structure
-
-```mermaid
-classDiagram
- class Response {
- +bool success
- +string message
- +interface{} data
- +ErrorDetail error
- +Meta meta
- +int64 timestamp
- }
-
- class ErrorDetail {
- +string code
- +string message
- +map details
- }
-
- class Meta {
- +int page
- +int per_page
- +int64 total
- +int total_pages
- +map extra
- }
-
- Response --> ErrorDetail
- Response --> Meta
-```
-
-## Request Validation Flow
-
-```mermaid
-sequenceDiagram
- participant C as Client
- participant H as Handler
- participant B as request.Bind
- participant V as Validator
- participant R as response
-
- C->>H: POST /api/v1/users
- H->>B: Bind(context, &req)
- B->>B: Parse JSON
- B->>V: Validate(req)
-
- alt Validation Failed
- V-->>B: ValidationError
- B-->>H: return error
- H->>R: ValidationError(...)
- R-->>C: 422 with error details
- else Validation Success
- V-->>B: nil
- B-->>H: nil
- H->>H: Process request
- H->>R: Success(data)
- R-->>C: 200 with data
- end
-```
-
-## Async Infrastructure Flow
-
-```mermaid
-graph TD
- A[HTTP Request] --> B[Handler]
- B --> C[Async Operation]
- C --> D[Worker Pool]
- D --> E[Infrastructure]
- E --> F[Result Channel]
- F --> G[Response]
-
- style A fill:#e1f5ff
- style C fill:#fff3cd
- style D fill:#ffeaa7
- style E fill:#fdcb6e
- style G fill:#55a3ff
-```
-
-## Service Registration Architecture
-
-```mermaid
-graph TD
- A[ServiceRegistrar] --> B[Reflection Analysis]
- B --> C{Dependencies?}
- C -->|No| D[Immediate Registration]
- C -->|Yes| E[Wait for Infrastructure]
- E --> F[Infrastructure Ready]
- F --> G[Register Service]
- G --> H[Boot Routes]
-
- style A fill:#74b9ff
- style B fill:#0984e3
- style D fill:#00b894
- style G fill:#00b894
-```
-
-## Package Organization
-
-```mermaid
-graph LR
- A[pkg/request] -->|Validates| F[Handler]
- B[pkg/response] -->|Formats| F
- F -->|Uses| G[Service Logic]
- G -->|Uses| H[Async Infrastructure]
- H -->|Worker Pools| I[Infrastructure Managers]
- G -->|Returns| F
-
- style A fill:#ffeb9c
- style B fill:#9cf09c
- style H fill:#ffeaa7
- style I fill:#fdcb6e
- style F fill:#9cccff
- style G fill:#ff9c9c
-
-## Complete CRUD Example Flow
-
-```mermaid
-graph TD
- subgraph "GET /api/v1/users (List)"
- A1[Parse Pagination] --> A2[Fetch Data]
- A2 --> A3[Calculate Meta]
- A3 --> A4[SuccessWithMeta]
- end
-
- subgraph "GET /api/v1/users/:id (Detail)"
- B1[Get ID from URL] --> B2[Find in DB]
- B2 -->|Found| B3[Success]
- B2 -->|Not Found| B4[NotFound]
- end
-
- subgraph "POST /api/v1/users (Create)"
- C1[Bind & Validate] --> C2{Valid?}
- C2 -->|No| C3[ValidationError]
- C2 -->|Yes| C4[Create in DB]
- C4 --> C5[Created 201]
- end
-
- subgraph "PUT /api/v1/users/:id (Update)"
- D1[Get ID + Bind] --> D2{Valid?}
- D2 -->|No| D3[ValidationError]
- D2 -->|Yes| D4[Update in DB]
- D4 --> D5[Success]
- end
-
- subgraph "DELETE /api/v1/users/:id (Delete)"
- E1[Get ID] --> E2[Delete from DB]
- E2 --> E3[NoContent 204]
- end
-
- style A4 fill:#d4edda
- style B3 fill:#d4edda
- style B4 fill:#f8d7da
- style C3 fill:#f8d7da
- style C5 fill:#d4edda
- style D3 fill:#f8d7da
- style D5 fill:#d4edda
- style E3 fill:#d4edda
-```
-
-## Response Helper Functions Map
-
-```mermaid
-mindmap
- root((Response Helpers))
- Success Responses
- Success 200
- SuccessWithMeta 200
- Created 201
- NoContent 204
- Client Errors
- BadRequest 400
- Unauthorized 401
- Forbidden 403
- NotFound 404
- Conflict 409
- ValidationError 422
- Server Errors
- InternalServerError 500
- ServiceUnavailable 503
- Custom
- Error custom code
-```
-
-## Validation Tags Reference
-
-```mermaid
-graph TB
- subgraph "String Validators"
- S1[required]
- S2[email]
- S3[min max]
- S4[len]
- end
-
- subgraph "Number Validators"
- N1[gte lte]
- N2[gt lt]
- N3[min max]
- end
-
- subgraph "Custom Validators"
- C1[phone]
- C2[username]
- end
-
- subgraph "Choice Validators"
- O1[oneof]
- end
-
- style S1 fill:#ffcccc
- style N1 fill:#ccffcc
- style C1 fill:#ccccff
- style O1 fill:#ffffcc
-```
diff --git a/docs_wiki/ASYNC_INFRASTRUCTURE.md b/docs_wiki/ASYNC_INFRASTRUCTURE.md
deleted file mode 100644
index 4c614e5..0000000
--- a/docs_wiki/ASYNC_INFRASTRUCTURE.md
+++ /dev/null
@@ -1,1170 +0,0 @@
-# Async Infrastructure Implementation Guide
-
-## Overview
-
-This document describes the async infrastructure implementation that ensures all database operations, caching, message queuing, and file operations run asynchronously to avoid blocking the main application thread. The implementation uses Go's goroutines, channels, and worker pools to provide non-blocking operations while maintaining thread safety.
-
-The system also includes **async infrastructure initialization** that allows the HTTP server to start immediately without waiting for database connections and other infrastructure components to initialize. This provides instant responsiveness while infrastructure components initialize in the background.
-
-## Key Components
-
-### 1. AsyncResult Types
-
-The async infrastructure uses generic `AsyncResult[T]` types to handle asynchronous operations:
-
-```go
-type AsyncResult[T any] struct {
- Value T
- Error error
- Done chan struct{}
-}
-```
-
-**Key Methods:**
-- `Wait()` - Blocks until operation completes
-- `WaitWithTimeout(timeout)` - Waits with timeout
-- `IsDone()` - Non-blocking check if operation is complete
-
-### 2. Worker Pools
-
-Each infrastructure component includes a worker pool for managing concurrent operations:
-
-```go
-type WorkerPool struct {
- workers int
- jobQueue chan func()
- stopChan chan struct{}
- stopped chan struct{}
-}
-```
-
-**Benefits:**
-- Controlled concurrency
-- Resource management
-- Panic recovery
-- Graceful shutdown
-
-### 3. Batch Operations
-
-Support for batching multiple operations:
-
-```go
-type BatchAsyncResult[T any] struct {
- Results []AsyncResult[T]
- Done chan struct{}
-}
-```
-
-## Infrastructure Components
-
-### Redis Manager
-
-**Async Operations:**
-```go
-// Set value asynchronously
-result := redisManager.SetAsync(ctx, "key", "value", time.Hour)
-
-// Get value asynchronously
-result := redisManager.GetAsync(ctx, "key")
-
-// Wait for result
-value, err := result.Wait()
-```
-
-**Batch Operations:**
-```go
-// Set multiple keys
-kvPairs := map[string]interface{}{"key1": "value1", "key2": "value2"}
-result := redisManager.SetBatchAsync(ctx, kvPairs, time.Hour)
-
-// Get multiple keys
-keys := []string{"key1", "key2"}
-result := redisManager.GetBatchAsync(ctx, keys)
-```
-
-**Worker Pool Integration:**
-```go
-// Submit background job
-redisManager.SubmitAsyncJob(func() {
- // Long-running Redis operation
-})
-```
-
-### Kafka Manager
-
-**Async Operations:**
-```go
-// Publish message asynchronously
-result := kafkaManager.PublishAsync(ctx, "topic", []byte("message"))
-
-// Publish with key
-result := kafkaManager.PublishWithKeyAsync(ctx, "topic", []byte("key"), []byte("message"))
-```
-
-**Batch Operations:**
-```go
-// Publish multiple messages
-messages := [][]byte{[]byte("msg1"), []byte("msg2")}
-result := kafkaManager.PublishBatchAsync(ctx, "topic", messages)
-```
-
-**Consumer Operations:**
-```go
-// Start consumer asynchronously (doesn't block)
-kafkaManager.ConsumeAsync(ctx, "topic", func(key, value []byte) error {
- // Handle message
- return nil
-})
-```
-
-### MinIO Manager
-
-**Async Operations:**
-```go
-// Upload file asynchronously
-file, _ := os.Open("file.txt")
-defer file.Close()
-result := minioManager.UploadFileAsync(ctx, "object.txt", file, size, "text/plain")
-```
-
-**Batch Operations:**
-```go
-// Upload multiple files
-uploads := []struct{
- ObjectName, Reader, Size, ContentType
-}{/* file data */}
-result := minioManager.UploadBatchAsync(ctx, uploads)
-```
-
-### PostgreSQL Manager
-
-**Async Operations:**
-```go
-// Execute query asynchronously
-result := postgresManager.QueryAsync(ctx, "SELECT * FROM users", args...)
-
-// Execute DML operations
-insertResult := postgresManager.InsertAsync(ctx, "INSERT INTO users...", args...)
-updateResult := postgresManager.UpdateAsync(ctx, "UPDATE users...", args...)
-deleteResult := postgresManager.DeleteAsync(ctx, "DELETE FROM users...", args...)
-```
-
-**GORM Async Operations:**
-```go
-// Async GORM operations
-createResult := postgresManager.GORMCreateAsync(ctx, &user)
-findResult := postgresManager.GORMFindAsync(ctx, &users)
-updateResult := postgresManager.GORMUpdateAsync(ctx, &user, updates, "id = ?", id)
-deleteResult := postgresManager.GORMDeleteAsync(ctx, &user, "id = ?", id)
-```
-
-### MongoDB Manager
-
-**Async Operations:**
-```go
-// CRUD operations
-insertResult := mongoManager.InsertOneAsync(ctx, "collection", document)
-findResult := mongoManager.FindAsync(ctx, "collection", filter)
-updateResult := mongoManager.UpdateOneAsync(ctx, "collection", filter, update)
-deleteResult := mongoManager.DeleteOneAsync(ctx, "collection", filter)
-```
-
-**Batch Operations:**
-```go
-// Batch insert
-inserts := []struct{Collection string; Document interface{}}{/* data */}
-result := mongoManager.InsertBatchAsync(ctx, inserts)
-```
-
-### Cron Manager
-
-**Async Job Execution:**
-```go
-// Add job that executes asynchronously in worker pool
-cronManager.AddAsyncJob("name", "schedule", func() {
- // Job logic (runs in worker pool, doesn't block main thread)
-}))
-
-// Run job immediately (asynchronously)
-cronManager.RunJobNow(jobID)
-
-// Get job status and scheduling info
-jobs := cronManager.GetJobs() // Returns scheduled jobs with next run times
-```
-
-**Configuration:**
-```yaml
-cron:
- enabled: true
- jobs:
- log_cleanup: "0 0 * * *" # Daily at midnight
- health_check: "*/10 * * * * *" # Every 10 seconds
-```
-
-**Features:**
-- **Async Execution**: Jobs run in worker pools to avoid blocking
-- **Schedule Management**: Add, remove, update job schedules dynamically
-- **Status Monitoring**: View active jobs and execution history
-- **Graceful Shutdown**: Clean termination of running jobs
-
-## Async Infrastructure Initialization
-
-The application implements **async infrastructure initialization** that allows the HTTP server to start immediately without waiting for database connections and other infrastructure components to initialize. This provides instant responsiveness while infrastructure components initialize in the background.
-
-### Infrastructure Initialization Manager
-
-The `InfraInitManager` manages asynchronous initialization of all infrastructure components:
-
-```go
-type InfraInitManager struct {
- status map[string]*InfraInitStatus
- mu sync.RWMutex
- logger *logger.Logger
- doneChan chan struct{}
-}
-```
-
-### Initialization Process
-
-1. **Immediate Server Start**: HTTP server starts immediately without waiting
-2. **Background Initialization**: Infrastructure components initialize concurrently in goroutines
-3. **Progress Tracking**: Real-time status monitoring of initialization progress
-4. **Health Endpoints**: API endpoints provide initialization status
-
-### Usage Example
-
-```go
-// In server startup
-infraInitManager := infrastructure.NewInfraInitManager(logger)
-
-// Start async initialization (doesn't block)
-redisMgr, kafkaMgr, _, postgresMgr, mongoMgr, cronMgr :=
- infraInitManager.StartAsyncInitialization(config, logger)
-
-// HTTP server starts immediately here
-// Infrastructure initializes in background
-
-// Check initialization status
-status := infraInitManager.GetStatus()
-progress := infraInitManager.GetInitializationProgress()
-```
-
-### Health Check Endpoints
-
-#### GET /health
-Enhanced health check with infrastructure status:
-```json
-{
- "status": "ok",
- "server_ready": true,
- "infrastructure": {
- "redis": {
- "name": "redis",
- "initialized": true,
- "start_time": "2025-12-19T13:13:00Z",
- "duration": "2.5s",
- "progress": 1.0
- },
- "postgres": {
- "name": "postgres",
- "initialized": true,
- "start_time": "2025-12-19T13:13:00Z",
- "duration": "3.2s",
- "progress": 1.0
- }
- },
- "initialization_progress": 0.85
-}
-```
-
-#### GET /health/infrastructure
-Detailed infrastructure initialization status:
-```json
-{
- "redis": {
- "name": "redis",
- "initialized": true,
- "error": "",
- "start_time": "2025-12-19T13:13:00Z",
- "duration": "2.5s",
- "progress": 1.0
- },
- "postgres": {
- "name": "postgres",
- "initialized": false,
- "error": "connection timeout",
- "start_time": "2025-12-19T13:13:00Z",
- "duration": "30s",
- "progress": 0.0
- }
-}
-```
-
-### Benefits of Async Initialization
-
-- **Instant Server Responsiveness**: HTTP server available immediately
-- **Graceful Degradation**: Services work with partially initialized infrastructure
-- **Better User Experience**: No waiting for database connections on startup
-- **Fault Tolerance**: Failed components don't prevent server startup
-- **Monitoring Integration**: Real-time visibility into initialization progress
-
-### Initialization Order
-
-Components initialize concurrently but with logical dependencies:
-
-1. **Redis** - Fast cache initialization
-2. **PostgreSQL/MongoDB** - Database connections (may take longer)
-3. **Kafka** - Message queue connections
-4. **MinIO** - Object storage
-5. **Cron** - Scheduled jobs
-
-### Clean Service Registration with Automatic Dependency Detection
-
-The application implements **clean service registration** with automatic dependency detection using a dedicated `ServiceRegistrar`. Services are registered **only once** in a central location, and the system automatically determines infrastructure dependencies through reflection analysis.
-
-#### Architecture Overview
-
-**Clean Separation of Concerns:**
-- **`internal/services/register.go`**: Service registration logic and dependency analysis
-- **`internal/server/server.go`**: Clean server startup without registration complexity
-
-**Key Components:**
-- **`ServiceRegistrar`**: Handles all service registration and dependency management
-- **`ServiceDefinition`**: Simple struct holding service name and constructor
-- **Reflection Analysis**: Automatic dependency detection based on struct field types
-
-#### Service Definition (One Clean Location)
-
-```go
-// internal/services/register.go
-allServices := []ServiceDefinition{
- {
- Name: "service_a",
- Constructor: func() interface{ Service } {
- return modules.NewServiceA(sr.config.Services.IsEnabled("service_a"))
- },
- },
- {
- Name: "service_d",
- Constructor: func() interface{ Service } {
- return modules.NewServiceD(sr.postgresManager, sr.config.Services.IsEnabled("service_d"), sr.logger)
- },
- },
- {
- Name: "service_f",
- Constructor: func() interface{ Service } {
- return modules.NewServiceF(sr.postgresConnMgr, sr.config.Services.IsEnabled("service_f"), sr.logger)
- },
- },
- {
- Name: "service_g",
- Constructor: func() interface{ Service } {
- return modules.NewServiceG(sr.mongoConnMgr, sr.config.Services.IsEnabled("service_g"), sr.logger)
- },
- },
-}
-```
-
-#### Server Startup (Clean and Simple)
-
-```go
-// internal/server/server.go - Clean server startup
-serviceRegistrar := services.NewServiceRegistrar(
- s.config, s.logger, s.infraInitManager,
- s.redisManager, s.kafkaManager, s.postgresManager,
- s.postgresConnectionManager, s.mongoManager,
- s.mongoConnectionManager, s.cronManager,
-)
-
-// Register ALL services with automatic dependency detection
-dependentServiceCount := serviceRegistrar.RegisterAllServices(registry, s.echo)
-
-// Wait for dependent services, then start monitoring
-if dependentServiceCount > 0 {
- time.Sleep(time.Duration(dependentServiceCount) * 500 * time.Millisecond)
-}
-// Start monitoring with complete service list
-```
-
-#### Reflection-Based Dependency Detection
-
-The `analyzeConstructorDependencies()` method automatically determines dependencies:
-
-```go
-func (sr *ServiceRegistrar) analyzeConstructorDependencies(constructor func() interface{ Service }) []string {
- service := constructor() // Create service instance
-
- // Use reflection to examine struct fields for infrastructure types
- serviceValue := reflect.ValueOf(service)
- serviceType := serviceValue.Type()
-
- var dependencies []string
- for i := 0; i < serviceType.NumField(); i++ {
- fieldType := serviceType.Field(i).Type.String()
-
- switch fieldType {
- case "infrastructure.PostgresConnectionManager":
- if sr.config.Postgres.Enabled || sr.config.PostgresMultiConfig.Enabled {
- dependencies = append(dependencies, "postgres")
- }
- case "infrastructure.MongoConnectionManager":
- if sr.config.Mongo.Enabled || sr.config.MongoMultiConfig.Enabled {
- dependencies = append(dependencies, "mongodb")
- }
- // ... other infrastructure types automatically detected
- }
- }
- return dependencies
-}
-```
-
-#### Synchronous Registration Process with Complete Synchronization
-
-**Phase 1: Infrastructure-Independent Services**
-```go
-// Start immediately - no dependencies
-for _, svc := range independentServices {
- registry.Register(svc.Constructor())
-}
-registry.Boot(echo)
-```
-
-**Phase 2: Infrastructure-Dependent Services (Synchronous)**
-```go
-// Use channel to track completion of ALL dependent services
-dependentDoneChan := make(chan struct{}, len(dependentServices))
-
-for _, svc := range dependentServices {
- go func(serviceDef DependentServiceDefinition) {
- defer func() { dependentDoneChan <- struct{}{} }()
-
- // Wait for each required infrastructure component
- for _, dep := range serviceDef.Dependencies {
- for !infraInitManager.IsInitialized(dep) {
- time.Sleep(100 * time.Millisecond)
- }
- }
- // Register service when dependencies are ready
- registry.Register(serviceDef.Constructor())
- registry.BootService(echo, serviceDef.Constructor())
- }(svc)
-}
-
-// Wait for ALL dependent services to complete registration
-for i := 0; i < len(dependentServices); i++ {
- <-dependentDoneChan // Blocks until each service completes
-}
-```
-
-**Phase 3: Monitoring Startup (After All Services)**
-```go
-// Now ALL services are registered - build complete service list
-var servicesList []monitoring.ServiceInfo
-for _, srv := range registry.GetServices() {
- servicesList = append(servicesList, monitoring.ServiceInfo{
- Name: srv.Name(),
- Active: srv.Enabled(),
- Endpoints: srv.Endpoints(),
- })
-}
-go monitoring.Start(config, servicesList) // Complete service list
-```
-
-#### Benefits of Clean Architecture
-
-- **Separation of Concerns**: Registration logic separated from server logic
-- **Single Definition**: Services defined only once, no duplication
-- **Automatic Detection**: Dependencies determined by examining actual code
-- **Maintainable**: Easy to add new services - just add to the list
-- **Clean Server Code**: Server startup logic remains simple and focused
-- **Type-Safe**: Uses Go's type system for reliable dependency detection
-
-#### Adding New Services
-
-To add a new service, simply add it to the `allServices` slice:
-
-```go
-{
- Name: "service_new",
- Constructor: func() interface{ Service } {
- return modules.NewService(s.someManager, s.config.Services.IsEnabled("service_new"))
- },
-},
-// System automatically detects dependencies based on constructor parameters
-```
-
-This approach provides a **clean, maintainable, and automatic** service registration system that scales effortlessly as new services and infrastructure components are added.
-
-### Error Handling
-
-Failed initializations are logged but don't prevent server startup:
-
-```go
-// Initialization continues even if some components fail
-if err != nil {
- status.Error = err.Error()
- logger.Error("Failed to initialize infrastructure component", err, "component", name)
-}
-```
-
-### Configuration
-
-Infrastructure components initialize based on their configuration settings:
-
-```yaml
-redis:
- enabled: true # Will initialize if enabled
-
-postgres:
- enabled: false # Will skip initialization
-
-mongo:
- enabled: true
- multi_config:
- enabled: true # Will initialize multi-connection setup
-```
-
-## Graceful Shutdown
-
-The application implements **graceful shutdown** that properly disconnects all infrastructure components when receiving SIGTERM or SIGINT signals. This ensures clean resource cleanup and prevents data corruption.
-
-### Shutdown Process
-
-1. **Signal Handling**: Application catches SIGTERM/SIGINT signals
-2. **Infrastructure Shutdown**: Components shut down in reverse order of initialization
-3. **Resource Cleanup**: Connections closed, worker pools stopped
-4. **Logging**: Detailed shutdown progress logged
-5. **Error Reporting**: Shutdown errors reported but don't prevent completion
-
-### Shutdown Order
-
-Components are shut down in reverse order to ensure dependencies are handled correctly:
-
-1. **Cron Manager** - Stop scheduled jobs first
-2. **MongoDB Connections** - Close document database connections
-3. **PostgreSQL Connections** - Close relational database connections
-4. **Kafka Manager** - Stop message producers/consumers
-5. **Redis Manager** - Close cache connections last
-
-### Usage Example
-
-```go
-// Signal handling in main.go
-sigChan := make(chan os.Signal, 1)
-signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)
-
-<-sigChan
-
-// Graceful shutdown
-err := server.Shutdown(context.Background(), logger)
-if err != nil {
- logger.Error("Shutdown completed with errors", err)
-} else {
- logger.Info("Shutdown completed successfully")
-}
-```
-
-### Shutdown Method
-
-The `Server.Shutdown()` method performs orderly shutdown:
-
-```go
-func (s *Server) Shutdown(ctx context.Context, logger *logger.Logger) error {
- // 1. Cron Manager
- if s.cronManager != nil {
- logger.Info("Shutting down Cron Manager...")
- if err := s.cronManager.Close(); err != nil {
- return fmt.Errorf("cron shutdown error: %w", err)
- }
- }
-
- // 2. MongoDB connections
- if s.mongoConnectionManager != nil {
- logger.Info("Shutting down MongoDB connections...")
- if err := s.mongoConnectionManager.CloseAll(); err != nil {
- return fmt.Errorf("mongodb shutdown error: %w", err)
- }
- }
-
- // 3. PostgreSQL connections
- if s.postgresConnectionManager != nil {
- logger.Info("Shutting down PostgreSQL connections...")
- if err := s.postgresConnectionManager.CloseAll(); err != nil {
- return fmt.Errorf("postgres shutdown error: %w", err)
- }
- }
-
- // 4. Kafka Manager
- if s.kafkaManager != nil {
- logger.Info("Shutting down Kafka Manager...")
- if err := s.kafkaManager.Close(); err != nil {
- return fmt.Errorf("kafka shutdown error: %w", err)
- }
- }
-
- // 5. Redis Manager
- if s.redisManager != nil {
- logger.Info("Shutting down Redis Manager...")
- if err := s.redisManager.Close(); err != nil {
- return fmt.Errorf("redis shutdown error: %w", err)
- }
- }
-
- return nil
-}
-```
-
-### Benefits of Graceful Shutdown
-
-- **Data Integrity**: Prevents partial writes and corruption
-- **Resource Cleanup**: Ensures all connections are properly closed
-- **Clean Termination**: No hanging processes or zombie goroutines
-- **Monitoring**: Shutdown progress is logged and monitored
-- **Kubernetes Compatibility**: Works with container orchestration systems
-
-### Error Handling During Shutdown
-
-Shutdown errors are logged but don't prevent the shutdown process:
-
-```go
-if err := component.Close(); err != nil {
- shutdownErrors = append(shutdownErrors, fmt.Errorf("component shutdown error: %w", err))
- logger.Error("Error shutting down component", err)
-} else {
- logger.Info("Component shut down successfully")
-}
-```
-
-### Timeout Handling
-
-Shutdown operations can be given timeouts to prevent hanging:
-
-```go
-shutdownCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
-defer cancel()
-
-err := server.Shutdown(shutdownCtx, logger)
-```
-
-### Integration with Orchestration
-
-Works seamlessly with container orchestration systems:
-
-```bash
-# Docker graceful shutdown
-docker stop --timeout 30 container_name
-
-# Kubernetes graceful termination
-terminationGracePeriodSeconds: 30
-```
-
-### Testing Shutdown
-
-Test graceful shutdown behavior:
-
-```bash
-# Send SIGTERM signal
-kill -TERM
-
-# Or use Ctrl+C in interactive mode
-# Application will log shutdown progress and exit cleanly
-```
-
-## Usage Patterns
-
-### Synchronous Usage (Blocking)
-
-```go
-// Execute async operation and wait for result
-result := redisManager.GetAsync(ctx, "key")
-value, err := result.Wait()
-
-if err != nil {
- // Handle error
-}
-// Use value
-```
-
-### Asynchronous Usage (Non-blocking)
-
-```go
-// Start operation without waiting
-result := redisManager.GetAsync(ctx, "key")
-
-// Continue with other work
-doOtherWork()
-
-// Check if done later
-if result.IsDone() {
- value, err := result.Wait()
- // Handle result
-}
-```
-
-### Timeout Handling
-
-```go
-// Wait with timeout
-result := redisManager.GetAsync(ctx, "key")
-value, err := result.WaitWithTimeout(5 * time.Second)
-
-if err == context.DeadlineExceeded {
- // Handle timeout
-}
-```
-
-### Batch Processing
-
-```go
-// Process multiple operations in parallel
-keys := []string{"key1", "key2", "key3"}
-result := redisManager.GetBatchAsync(ctx, keys)
-
-// Wait for all operations to complete
-values, errors := result.WaitAll()
-
-for i, value := range values {
- if errors[i] != nil {
- // Handle error for this operation
- }
- // Process value
-}
-```
-
-### Worker Pool Submission
-
-```go
-// Submit long-running tasks to worker pool
-redisManager.SubmitAsyncJob(func() {
- // Long-running operation
- heavyComputation()
- redisManager.Set(ctx, "result", computedValue, time.Hour)
-})
-```
-
-## Service Implementation Examples
-
-### Database Service with Async Operations
-
-```go
-type UserService struct {
- db *infrastructure.PostgresManager
-}
-
-func (s *UserService) CreateUser(c echo.Context) error {
- var user User
- if err := c.Bind(&user); err != nil {
- return response.BadRequest(c, "Invalid user data")
- }
-
- // Async database operation
- result := s.db.GORMCreateAsync(context.Background(), &user)
-
- // Wait for completion
- _, err := result.Wait()
- if err != nil {
- return response.InternalServerError(c, err.Error())
- }
-
- return response.Created(c, user)
-}
-
-func (s *UserService) GetUsers(c echo.Context) error {
- var users []User
-
- // Async query
- result := s.db.GORMFindAsync(context.Background(), &users)
-
- // Wait for completion
- _, err := result.Wait()
- if err != nil {
- return response.InternalServerError(c, err.Error())
- }
-
- return response.Success(c, users)
-}
-```
-
-### Cache Service with Async Operations
-
-```go
-type CacheService struct {
- redis *infrastructure.RedisManager
-}
-
-func (s *CacheService) SetMultiple(ctx context.Context, data map[string]interface{}) error {
- // Async batch set
- result := s.redis.SetBatchAsync(ctx, data, time.Hour)
-
- // Wait for all operations
- _, errors := result.WaitAll()
-
- // Check for any errors
- for _, err := range errors {
- if err != nil {
- return err
- }
- }
-
- return nil
-}
-```
-
-### Message Queue Service
-
-```go
-type QueueService struct {
- kafka *infrastructure.KafkaManager
-}
-
-func (s *QueueService) PublishEvents(ctx context.Context, events []Event) error {
- // Convert events to messages
- messages := make([][]byte, len(events))
- for i, event := range events {
- data, _ := json.Marshal(event)
- messages[i] = data
- }
-
- // Async batch publish
- result := s.kafka.PublishBatchAsync(ctx, "events", messages)
-
- // Wait for completion
- _, errors := result.WaitAll()
-
- // Check for errors
- for _, err := range errors {
- if err != nil {
- return err
- }
- }
-
- return nil
-}
-```
-
-## Performance Benefits
-
-### Non-blocking Operations
-
-- **Main Thread Availability**: HTTP handlers return immediately while operations run in background
-- **Concurrent Processing**: Multiple operations can run simultaneously
-- **Resource Efficiency**: Better utilization of system resources
-
-### Example Performance Comparison
-
-**Synchronous Approach:**
-```
-Request → DB Query (2s) → Response
-Total: 2 seconds per request
-Throughput: 0.5 requests/second
-```
-
-**Asynchronous Approach:**
-```
-Request → Start Async DB Query → Return Response (immediate)
-DB Query completes (2s) → Result stored/cached
-Total: ~0ms per request (non-blocking)
-Throughput: Limited by DB capacity, not request handling
-```
-
-### Resource Management
-
-- **Worker Pools**: Control maximum concurrent operations
-- **Connection Pooling**: Reuse database connections efficiently
-- **Timeout Handling**: Prevent hanging operations
-- **Graceful Shutdown**: Clean up resources properly
-
-## Error Handling
-
-### Operation-specific Errors
-
-```go
-result := redisManager.GetAsync(ctx, "key")
-value, err := result.Wait()
-
-if err != nil {
- switch err {
- case redis.Nil:
- // Key not found
- case context.DeadlineExceeded:
- // Operation timed out
- default:
- // Other error
- }
-}
-```
-
-### Batch Error Handling
-
-```go
-result := redisManager.GetBatchAsync(ctx, keys)
-values, errors := result.WaitAll()
-
-for i, err := range errors {
- if err != nil {
- log.Printf("Error getting key %s: %v", keys[i], err)
- // Handle specific error
- } else {
- // Process values[i]
- }
-}
-```
-
-### Panic Recovery
-
-Async operations include panic recovery:
-
-```go
-// Automatic panic handling in ExecuteAsync
-go func() {
- defer func() {
- if r := recover(); r != nil {
- result.Complete(zeroValue, fmt.Errorf("operation panicked: %v", r))
- }
- }()
- // Operation logic
-}()
-```
-
-## Monitoring and Observability
-
-### Status Monitoring
-
-Each manager provides status information including async operation stats:
-
-```go
-// Redis status includes pool information
-redisStatus := redisManager.GetStatus()
-// {"connected": true, "pool_workers": 10, "active_jobs": 3}
-
-// Cron status includes worker pool info
-cronStatus := cronManager.GetStatus()
-// {"active": true, "jobs": [...], "pool_workers": 5}
-```
-
-### Performance Metrics
-
-- **Operation Latency**: Time taken for async operations
-- **Queue Depth**: Number of pending operations
-- **Error Rates**: Failure rates for different operation types
-- **Resource Usage**: Memory and CPU usage by worker pools
-
-## Configuration
-
-### Worker Pool Configuration
-
-```yaml
-# Infrastructure worker pool sizes
-infrastructure:
- redis:
- workers: 10
- kafka:
- workers: 5
- minio:
- workers: 8
- postgres:
- workers: 15
- mongodb:
- workers: 12
- cron:
- workers: 5
-```
-
-### Timeout Configuration
-
-```yaml
-# Operation timeouts
-infrastructure:
- timeouts:
- redis: 30s
- kafka: 60s
- minio: 300s
- postgres: 30s
- mongodb: 30s
-```
-
-## Best Practices
-
-### 1. Context Usage
-
-Always use context for cancellation:
-
-```go
-ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
-defer cancel()
-
-result := dbManager.QueryAsync(ctx, "SELECT * FROM users")
-```
-
-### 2. Error Handling
-
-Handle errors appropriately:
-
-```go
-result := redisManager.GetAsync(ctx, "key")
-value, err := result.Wait()
-
-if err == context.Canceled {
- // Request was canceled
- return
-}
-if err == context.DeadlineExceeded {
- // Operation timed out
- return
-}
-// Handle other errors
-```
-
-### 3. Resource Cleanup
-
-Use defer for cleanup:
-
-```go
-result := fileManager.UploadAsync(ctx, "file.txt", reader, size, contentType)
-defer func() {
- if reader != nil {
- reader.Close()
- }
-}()
-
-// Wait for operation
-_, err := result.Wait()
-```
-
-### 4. Batch Operations
-
-Use batch operations for multiple items:
-
-```go
-// Instead of multiple individual operations
-for _, key := range keys {
- redisManager.SetAsync(ctx, key, value, ttl)
-}
-
-// Use batch operation
-kvPairs := make(map[string]interface{})
-for _, key := range keys {
- kvPairs[key] = value
-}
-redisManager.SetBatchAsync(ctx, kvPairs, ttl)
-```
-
-### 5. Worker Pool Sizing
-
-Size worker pools based on load:
-
-```go
-// High-throughput service
-redisPool := NewWorkerPool(50)
-
-// Low-throughput service
-cronPool := NewWorkerPool(5)
-```
-
-## Migration Guide
-
-### Converting Synchronous Code
-
-**Before (Synchronous):**
-```go
-func (s *Service) GetUser(id string) (*User, error) {
- var user User
- err := s.db.Where("id = ?", id).First(&user).Error
- return &user, err
-}
-```
-
-**After (Asynchronous):**
-```go
-func (s *Service) GetUser(id string) (*User, error) {
- var user User
- result := s.db.GORMFirstAsync(ctx, &user, id)
- _, err := result.Wait()
- return &user, err
-}
-```
-
-### Gradual Migration
-
-1. **Identify blocking operations**
-2. **Add async versions alongside sync versions**
-3. **Update callers gradually**
-4. **Remove sync versions after full migration**
-
-## Testing
-
-### Unit Testing Async Operations
-
-```go
-func TestAsyncRedisGet(t *testing.T) {
- // Setup
- redis := setupTestRedis()
- ctx := context.Background()
-
- // Set test data
- redis.Set(ctx, "test_key", "test_value", time.Hour)
-
- // Test async operation
- result := redis.GetAsync(ctx, "test_key")
-
- // Wait with timeout
- value, err := result.WaitWithTimeout(5 * time.Second)
-
- assert.NoError(t, err)
- assert.Equal(t, "test_value", value)
-}
-```
-
-### Integration Testing
-
-```go
-func TestAsyncDatabaseOperations(t *testing.T) {
- // Setup database
- db := setupTestDatabase()
-
- // Test batch operations
- users := []User{{Name: "Alice"}, {Name: "Bob"}}
- result := db.GORMCreateBatchAsync(ctx, users)
-
- _, errors := result.WaitAll()
- for _, err := range errors {
- assert.NoError(t, err)
- }
-}
-```
-
-## Troubleshooting
-
-### Common Issues
-
-**Operations taking too long:**
-- Check worker pool sizing
-- Monitor queue depths
-- Add timeouts to operations
-
-**Memory leaks:**
-- Ensure proper cleanup of resources
-- Monitor goroutine counts
-- Use context cancellation
-
-**Deadlocks:**
-- Avoid blocking operations in async jobs
-- Use timeouts for all operations
-- Monitor for circular dependencies
-
-**Resource exhaustion:**
-- Limit concurrent operations
-- Implement backpressure
-- Monitor system resources
-
-## Conclusion
-
-The async infrastructure implementation provides:
-
-- **Non-blocking operations** for better application responsiveness
-- **Controlled concurrency** through worker pools
-- **Resource efficiency** with proper connection pooling
-- **Fault tolerance** with timeout and error handling
-- **Scalability** through batch operations and connection management
-
-This approach ensures that your Go application can handle high loads while maintaining excellent user experience and system stability.
diff --git a/docs_wiki/BUILD_SCRIPTS.md b/docs_wiki/BUILD_SCRIPTS.md
deleted file mode 100644
index b0b4b1b..0000000
--- a/docs_wiki/BUILD_SCRIPTS.md
+++ /dev/null
@@ -1,482 +0,0 @@
-# Enhanced Build Scripts with Garble Obfuscation
-
-## What's New in This Version
-
-### Enhanced Build Scripts (Latest Update)
-The build scripts have been significantly enhanced with the following new features:
-
-- **Automatic Tool Installation**: Scripts now automatically check for and install required Go tools (`goversioninfo`, `garble`) if not present
-- **Interactive Obfuscation Choice**: Users are prompted to choose between standard Go build or garble obfuscation build
-- **Timeout Handling**: 10-second timeout with sensible default (no obfuscation) for CI/CD compatibility
-- **Updated Step Numbers**: Progress indicators now show [0/6] through [5/6] to account for tool checking phase
-- **Enhanced Error Handling**: Better error messages and recovery options for tool installation failures
-
-### TUI Improvements
-The Terminal User Interface has been significantly enhanced with advanced features:
-
-- **Scrollable Log Display**: Full scrolling support through application logs with arrow keys, page up/down, and home/end
-- **Real-time Log Filtering**: Press "/" to open a modal filter dialog for searching logs by content or level
-- **Auto-scroll Control**: Manual toggle (F1) to enable/disable automatic scrolling to bottom on new logs
-- **Log Management**: Press F2 to clear all accumulated logs and reset the view state
-- **Modal Filter Interface**: Clean, centered dialog with black background for focused log filtering
-- **Sticky Header/Footer**: App information and controls remain visible while scrolling through logs
-- **Thread-safe Operations**: All log operations are properly synchronized for concurrent access
-- **Unlimited Log Storage**: Removed 1000 log limit to allow unlimited log storage
-- **Default Auto-scroll**: Auto-scroll enabled by default on application startup
-- **Reusable Dialog System**: Template-based dialog components in `pkg/tui/template/` for easy reuse
-- **Dialog Footer Cleanup**: Removed scroll count from footer for cleaner interface
-
-### Customizable Parameter Parsing System
-The application now features a highly customizable parameter parsing system that allows easy addition and configuration of command-line flags:
-
-- **Flag Definition System**: Command-line flags are defined in `cmd/app/main.go` using a structured `FlagDefinition` type
-- **Dynamic Parsing**: The parsing logic in `pkg/utils/parameter.go` automatically handles flag validation and type conversion
-- **Easy Extension**: Add new flags by simply adding entries to the flagDefinitions slice
-- **Type Safety**: Support for string, int, and bool flag types with built-in validation
-- **Custom Validators**: Each flag can have custom validation functions for business logic rules
-- **Modular Architecture**: Parsing logic separated from configuration for maintainability
-
-**Example of Adding a New Flag:**
-```go
-// In cmd/app/main.go
-var flagDefinitions = []utils.FlagDefinition{
- {
- Name: "c",
- DefaultValue: "",
- Description: "URL to load configuration from (YAML format)",
- Validator: func(value interface{}) error {
- if str, ok := value.(string); ok && str != "" {
- if _, err := url.ParseRequestURI(str); err != nil {
- return fmt.Errorf("invalid config URL format: %w", err)
- }
- }
- return nil
- },
- },
- // Add new flags easily
- {
- Name: "port",
- DefaultValue: 8080,
- Description: "Server port to listen on",
- Validator: func(value interface{}) error {
- if port, ok := value.(int); ok && (port < 1 || port > 65535) {
- return fmt.Errorf("port must be between 1 and 65535")
- }
- return nil
- },
- },
-}
-```
-
-### Previous Features (Still Included)
-- Cross-platform support (Unix/Linux/macOS and Windows)
-- Automatic backup and archiving of previous builds
-- Process management (stops running application instances)
-- Asset copying (config files, web assets, databases)
-- Comprehensive error handling and troubleshooting
-
-## Overview
-
-The enhanced build scripts (`scripts/build.sh` for Unix/Linux/macOS and `scripts/build.bat` for Windows) now include automatic tool installation and user choice for code obfuscation using `garble`. These scripts provide a complete build pipeline with backup management, cross-platform compatibility, and production-ready binary generation.
-
-## Key Features
-
-- **Automatic Tool Installation**: Checks and installs required tools (`goversioninfo`, `garble`)
-- **User Choice for Obfuscation**: Interactive prompt to enable/disable code obfuscation
-- **Timeout Handling**: 10-second timeout with default behavior (no obfuscation)
-- **Cross-Platform**: Native implementations for Unix/Linux/macOS and Windows
-- **Backup Management**: Automatic backup and archiving of previous builds
-- **Process Management**: Stops running application instances before building
-- **Asset Copying**: Automatically copies configuration and web assets
-
-## Enhanced Build Process
-
-### Tool Installation Phase
-
-1. **Check goversioninfo**: Verifies if `goversioninfo` is installed
- - If not found: Automatically installs `github.com/josephspurrier/goversioninfo/cmd/goversioninfo@latest`
- - If found: Continues to next step
-
-2. **Check garble**: Verifies if `garble` is installed
- - If not found: Automatically installs `mvdan.cc/garble@latest`
- - If found: Continues to next step
-
-### User Choice Phase
-
-3. **Obfuscation Prompt**: Interactive prompt with timeout
- ```
- Use garble build for obfuscation? (y/N, timeout 10s):
- ```
- - **Y/y**: Enables code obfuscation using `garble build`
- - **N/n or timeout**: Uses standard `go build`
- - **Default**: No obfuscation (safer for development)
-
-### Standard Build Phase
-
-4. **Version Info Generation**: Runs `goversioninfo -platform-specific`
-5. **Binary Compilation**: Builds with appropriate command
- - With obfuscation: `garble build -ldflags="-s -w"`
- - Without obfuscation: `go build -ldflags="-s -w"`
-6. **Asset Management**: Copies configuration files, web assets, and databases
-
-## Usage Examples
-
-### Unix/Linux/macOS
-
-```bash
-# Interactive build with user choice
-./scripts/build.sh
-
-# Example output:
-# (\_/)
-/# (o.o) stackyard Builder by diameter-tscd
-# c(")(")
-# ------------------------------------------------------------------------------
-# [0/6] Checking required tools...
-# + goversioninfo found
-# + garble found
-# Use garble build for obfuscation? (y/N, timeout 10s): y
-# + Using garble build
-# [1/6] Checking for running process...
-# + App is not running.
-# [2/6] Backing up old files...
-# + Backup created at: dist/backups/20251220_235500
-# [3/6] Archiving backup...
-# + Backup archived: dist/backups/20251220_235500.zip
-# [4/6] Building Go binary...
-# + Build successful: dist/stackyard
-# [5/6] Copying assets...
-# + Copying web folder...
-# + Copying config.yaml...
-# SUCCESS! Build ready at: dist/
-```
-
-### Windows
-
-```cmd
-# Interactive build with user choice
-scripts\build.bat
-
-# Example output:
-# (\_/)
-# (o.o) stackyard Builder by diameter-tscd
-# c(")(")
-# ------------------------------------------------------------------------------
-# [0/6] Checking required tools...
-# + goversioninfo found
-# + garble found
-# Use garble build for obfuscation? (Y/N, default N, timeout 10s): Y
-# + Using garble build
-# [1/6] Checking for running process...
-# + App is not running.
-# [2/6] Backing up old files...
-# + Backup created at: dist/backups/20251220_235500
-# [3/6] Archiving backup...
-# + Backup archived: dist/backups/20251220_235500.zip
-# [4/6] Building Go binary...
-# + Build successful: dist/stackyard.exe
-# [5/6] Copying assets...
-# + Copying web folder...
-# SUCCESS! Build ready at: dist\
-```
-
-## Code Obfuscation with Garble
-
-### What is Garble?
-
-Garble is a Go code obfuscation tool that:
-- **Obfuscates identifiers**: Renames functions, variables, and types
-- **Removes debug info**: Strips file paths and source information
-- **Maintains functionality**: Preserves program behavior
-- **Increases binary size**: Obfuscated binaries are slightly larger
-
-### When to Use Obfuscation
-
-**Recommended for:**
-- Production deployments
-- Commercial applications
-- Security-sensitive code
-- Intellectual property protection
-
-**Not recommended for:**
-- Development builds
-- Debugging scenarios
-- Open source projects
-- Performance-critical applications (minor overhead)
-
-### Obfuscation Effects
-
-```go
-// Original code
-package main
-
-func calculateTotal(items []Item) int {
- total := 0
- for _, item := range items {
- total += item.price
- }
- return total
-}
-
-// Obfuscated result (example)
-package main
-
-func A(items []B) int {
- C := 0
- for _, D := range items {
- C += D.E
- }
- return C
-}
-```
-
-## Configuration Options
-
-### Build Script Variables
-
-| Variable | Unix/Linux/macOS | Windows | Description |
-|----------|------------------|---------|-------------|
-| `DIST_DIR` | `dist` | `dist` | Output directory |
-| `APP_NAME` | `stackyard` | `stackyard.exe` | Binary name |
-| `MAIN_PATH` | `./cmd/app/main.go` | `./cmd/app/main.go` | Main Go file |
-
-### Timeout Behavior
-
-- **Timeout Duration**: 10 seconds
-- **Default Choice**: N (no obfuscation)
-- **Platform Differences**:
- - Unix: Uses `read -t` with signal handling
- - Windows: Uses `choice /T` command
-
-## Error Handling
-
-### Tool Installation Failures
-
-```bash
-# If goversioninfo installation fails
-[0/6] Checking required tools...
-! goversioninfo not found. Installing...
-x Failed to install goversioninfo
-# Script exits with error code
-```
-
-### Build Failures
-
-```bash
-# If Go compilation fails
-[4/6] Building Go binary...
-x Build FAILED! Exit code: 2
-# Script exits with build error code
-```
-
-### Recovery Options
-
-**Clean Rebuild:**
-```bash
-# Remove dist directory and rebuild
-rm -rf dist/
-./scripts/build.sh
-```
-
-**Skip Obfuscation:**
-```bash
-# Choose 'N' when prompted or wait for timeout
-# Script will use standard go build
-```
-
-## Performance Comparison
-
-### Build Times
-
-| Build Type | Average Time | Binary Size | Notes |
-|------------|--------------|-------------|-------|
-| Standard Go | ~15-30s | ~15-25MB | Normal compilation |
-| Garble Build | ~45-90s | ~18-28MB | Slower, larger binary |
-| UPX Compressed | +10-20s | ~6-10MB | Additional compression |
-
-### Runtime Performance
-
-- **Standard Build**: Baseline performance
-- **Garble Build**: ~1-5% slower due to obfuscated symbols
-- **Memory Usage**: No significant difference
-
-## Security Considerations
-
-### Code Protection
-
-**Obfuscation Benefits:**
-- Makes reverse engineering more difficult
-- Protects intellectual property
-- Complicates debugging by attackers
-- Reduces information leakage
-
-**Limitations:**:
-- Not encryption (can be deobfuscated with effort)
-- Source code recovery is difficult but not impossible
-- Performance debugging becomes harder
-
-### Best Practices
-
-1. **Development**: Use standard builds for easier debugging
-2. **Staging**: Test with obfuscated builds before production
-3. **Production**: Always use obfuscated builds for security
-4. **CI/CD**: Automate obfuscation for consistent deployments
-
-## Integration with CI/CD
-
-### GitHub Actions Example
-
-```yaml
-name: Build and Deploy
-
-on:
- push:
- branches: [ main ]
-
-jobs:
- build:
- runs-on: ubuntu-latest
- steps:
- - uses: actions/checkout@v3
-
- - name: Install Go tools
- run: |
- go install github.com/josephspurrier/goversioninfo/cmd/goversioninfo@latest
- go install mvdan.cc/garble@latest
-
- - name: Build without obfuscation
- run: go build -ldflags="-s -w" -o app ./cmd/app/main.go
-
- - name: Build with obfuscation
- run: garble build -ldflags="-s -w" -o app-obfuscated ./cmd/app/main.go
-```
-
-### Docker Integration
-
-```dockerfile
-# Multi-stage build with obfuscation choice
-FROM golang:1.21-alpine AS builder
-
-# Install tools
-RUN go install github.com/josephspurrier/goversioninfo/cmd/goversioninfo@latest
-RUN go install mvdan.cc/garble@latest
-
-# Copy source
-COPY . .
-
-# Build with obfuscation (for production)
-RUN goversioninfo -platform-specific
-RUN garble build -ldflags="-s -w" -o main ./cmd/app
-
-FROM alpine:latest
-COPY --from=builder /app/main .
-CMD ["./main"]
-```
-
-## Troubleshooting
-
-### Common Issues
-
-**"garble: command not found"**
-- Ensure garble was installed successfully
-- Check Go environment: `go env`
-- Verify installation: `go list mvdan.cc/garble`
-
-**"Build takes too long"**
-- Garble builds are slower by design
-- Consider using standard builds for development
-- Use build caching in CI/CD
-
-**"Obfuscated binary crashes"**
-- Test obfuscated builds thoroughly before production
-- Some reflection-based code may need adjustments
-- Check for hardcoded function names
-
-**"Timeout reached during prompt"**
-- Script continues with default (no obfuscation)
-- For automated builds, consider removing the prompt
-- Use environment variables for CI/CD
-
-### Debug Options
-
-**Verbose Output:**
-```bash
-# Add to build script for debugging
-set -x # Unix
-@echo on # Windows
-```
-
-**Skip Prompt (Automated Builds):**
-```bash
-# Modify script to skip user interaction
-USE_GARBLE=false # Always use standard build
-# Or
-USE_GARBLE=true # Always use obfuscation
-```
-
-## Migration Guide
-
-### From Previous Version
-
-1. **Backup existing scripts** (optional but recommended)
-2. **Replace build scripts** with enhanced versions
-3. **Test installation** of required tools
-4. **Verify builds** work with both obfuscation options
-5. **Update CI/CD** pipelines if needed
-
-### Backward Compatibility
-
-- **Existing builds**: Continue to work unchanged
-- **Configuration**: Same environment variables and paths
-- **Output format**: Identical directory structure
-- **Dependencies**: Only adds optional tools
-
-## Advanced Usage
-
-### Custom Build Flags
-
-Modify the build commands for additional flags:
-
-```bash
-# Unix/Linux/macOS
-garble build -ldflags="-s -w -X main.version=1.2.3" -o "$DIST_DIR/$APP_NAME" "$MAIN_PATH"
-
-# Windows
-garble build -ldflags="-s -w -X main.version=1.2.3" -o "%DIST_DIR%\%APP_NAME%" %MAIN_PATH%
-```
-
-### Environment-Based Choice
-
-For automated environments:
-
-```bash
-# Set environment variable
-export USE_GARBLE=true
-
-# Modify script to check environment
-if [ "$USE_GARBLE" = "true" ]; then
- # Skip prompt, use garble
-else
- # Show prompt
-fi
-```
-
-### Multiple Build Targets
-
-```bash
-# Build both versions
-./scripts/build.sh # Interactive choice
-USE_GARBLE=false ./scripts/build.sh # Standard only
-USE_GARBLE=true ./scripts/build.sh # Obfuscated only
-```
-
-## Conclusion
-
-The enhanced build scripts provide a robust, user-friendly solution for Go application building with optional code obfuscation. The interactive prompts ensure developers can make informed choices about security vs. development convenience, while the automatic tool installation removes setup barriers.
-
-Key benefits:
-- **Security**: Optional code obfuscation for production deployments
-- **Usability**: Interactive prompts with sensible defaults
-- **Automation**: CI/CD friendly with timeout handling
-- **Reliability**: Comprehensive error handling and backup management
-- **Cross-platform**: Native implementations for all major platforms
diff --git a/docs_wiki/CHANGE_PACKAGE_SCRIPTS.md b/docs_wiki/CHANGE_PACKAGE_SCRIPTS.md
deleted file mode 100644
index eb07761..0000000
--- a/docs_wiki/CHANGE_PACKAGE_SCRIPTS.md
+++ /dev/null
@@ -1,340 +0,0 @@
-# Package Name Change Scripts
-
-## Overview
-
-The Package Name Change Scripts provide automated tools for renaming the Go module package name across the entire codebase. These scripts are essential when refactoring, renaming, or migrating a Go project to a new module path.
-
-## Features
-
-- **Cross-Platform Support**: Separate implementations for Unix/Linux/macOS and Windows
-- **Comprehensive Updates**: Updates both the module declaration and all import paths
-- **Safety Mechanisms**: Backup creation and error validation
-- **Pure Native Tools**: No external dependencies required
-- **Recursive Processing**: Handles all `.go` files in the project directory tree
-
-## Scripts Overview
-
-### change_package.sh (Unix/Linux/macOS)
-
-A Bash script that uses standard Unix utilities for text processing and file manipulation.
-
-**Prerequisites:**
-- Bash shell
-- `sed` (stream editor)
-- `find` (file search utility)
-- `awk` (text processing)
-- `grep` (pattern matching)
-
-### change_package.bat (Windows)
-
-A Windows batch script that uses pure CMD commands for maximum compatibility.
-
-**Prerequisites:**
-- Windows Command Prompt
-- No external tools required (uses built-in CMD features)
-
-## Usage
-
-### Basic Usage
-
-**Unix/Linux/macOS:**
-```bash
-chmod +x scripts/change_package.sh
-./scripts/change_package.sh "github.com/new-org/new-project"
-```
-
-**Windows:**
-```cmd
-scripts\change_package.bat github.com/new-org/new-project
-```
-
-### Examples
-
-```bash
-# Change to a GitHub repository
-./scripts/change_package.sh "github.com/mycompany/myproject"
-
-# Change to a local/private module
-./scripts/change_package.sh "mycompany.com/internal/project"
-
-# Change to a generic domain
-scripts\change_package.bat "example.com/my-app"
-
-# Change to a subdirectory module
-./scripts/change_package.sh "github.com/user/project/v2"
-```
-
-## How It Works
-
-The scripts perform the following operations in sequence:
-
-### 1. Validation Phase
-
-- **Input Validation**: Ensures a new module name is provided as argument
-- **Module Discovery**: Reads the current module name from `go.mod`
-- **Error Checking**: Validates that `go.mod` exists and contains a valid module declaration
-
-### 2. Update Phase
-
-- **Module Declaration**: Updates the `module` line in `go.mod`
-- **Import Path Scanning**: Recursively finds all `.go` files in the project
-- **Import Path Replacement**: Updates all import statements containing the old module name
-
-### 3. Completion Phase
-
-- **Success Reporting**: Displays completion message with statistics
-- **Backup Notification**: Reminds user about backup files (Unix/Linux/macOS)
-
-## Technical Implementation
-
-### Unix/Linux/macOS Implementation
-
-The Bash script uses:
-
-```bash
-# Read current module name
-CURRENT_MODULE=$(grep '^module ' go.mod | awk '{print $2}')
-
-# Update go.mod with backup
-sed -i.bak "s|^module $CURRENT_MODULE|module $NEW_MODULE|" go.mod
-
-# Update all .go files with backup
-find . -name "*.go" -type f -exec sed -i.bak "s|$CURRENT_MODULE|$NEW_MODULE|g" {} +
-```
-
-### Windows Implementation
-
-The batch script uses CMD's built-in string replacement:
-
-```batch
-REM Enable delayed expansion for variable manipulation
-setlocal enabledelayedexpansion
-
-REM Extract current module name
-for /f "tokens=2" %%i in ('findstr "^module " go.mod') do set CURRENT_MODULE=%%i
-
-REM Replace in go.mod using temporary file
-(for /f "delims=" %%i in (go.mod) do (
- set "line=%%i"
- set "line=!line:%search%=%replace%!"
- echo !line!
-)) > "%tempfile%"
-
-REM Replace in all .go files
-for /r %%f in (*.go) do (
- REM Similar replacement logic for each file
-)
-```
-
-## Safety Features
-
-### Backup Creation
-
-**Unix/Linux/macOS:**
-- Creates `.bak` backup files for all modified files
-- Preserves original content in case of errors
-- Easy cleanup after verification
-
-**Windows:**
-- No automatic backups (CMD limitations)
-- Recommends committing changes before running
-- Files are overwritten directly
-
-### Error Handling
-
-- **Argument Validation**: Stops if no new module name provided
-- **File Existence**: Checks for `go.mod` presence
-- **Module Detection**: Validates current module name extraction
-- **Operation Verification**: Checks success of file operations
-
-### Recovery Options
-
-If something goes wrong:
-
-1. **Restore from backups** (Unix/Linux/macOS):
- ```bash
- find . -name "*.bak" -exec bash -c 'mv "$1" "${1%.bak}"' _ {} \;
- ```
-
-2. **Git revert** (if changes were committed):
- ```bash
- git checkout HEAD~1 go.mod
- git checkout HEAD~1 -- "*.go"
- ```
-
-## Best Practices
-
-### Before Running
-
-1. **Commit Current Changes**:
- ```bash
- git add .
- git commit -m "Before package name change"
- ```
-
-2. **Test Compilation**:
- ```bash
- go build ./...
- go test ./...
- ```
-
-3. **Backup Important Files** (additional safety):
- ```bash
- cp go.mod go.mod.backup
- ```
-
-### After Running
-
-1. **Verify Changes**:
- ```bash
- grep -r "old-module-name" . --include="*.go"
- ```
-
-2. **Clean Up Backups** (Unix/Linux/macOS):
- ```bash
- find . -name "*.bak" -delete
- ```
-
-3. **Update Dependencies**:
- ```bash
- go mod tidy
- ```
-
-4. **Test Again**:
- ```bash
- go build ./...
- go test ./...
- ```
-
-5. **Update IDE/Editor**:
- - Restart your IDE
- - Clear any cached module information
-
-## Common Use Cases
-
-### Repository Rename
-
-When moving a project to a new GitHub organization:
-
-```bash
-# Old: github.com/old-org/project
-# New: github.com/new-org/project
-./scripts/change_package.sh "github.com/new-org/project"
-```
-
-### Versioning
-
-When creating a new major version:
-
-```bash
-# Old: github.com/user/project
-# New: github.com/user/project/v2
-./scripts/change_package.sh "github.com/user/project/v2"
-```
-
-### Internal Migration
-
-When moving from public to private repositories:
-
-```bash
-# Old: github.com/user/project
-# New: company.com/internal/project
-./scripts/change_package.sh "company.com/internal/project"
-```
-
-## Troubleshooting
-
-### "Module declaration not found"
-
-**Cause**: `go.mod` file missing or malformed
-**Solution**: Ensure you're in the correct project directory and `go.mod` exists
-
-### "Permission denied"
-
-**Cause**: File permissions or write access issues
-**Solution**: Check file permissions and run with appropriate privileges
-
-### "sed: command not found" (Unix/Linux/macOS)
-
-**Cause**: `sed` not installed
-**Solution**: Install sed utility (usually pre-installed on most systems)
-
-### "FINDSTR is not recognized" (Windows)
-
-**Cause**: Running in PowerShell instead of CMD
-**Solution**: Use `cmd.exe` or modify script for PowerShell compatibility
-
-### Import Paths Not Updated
-
-**Cause**: Custom import aliases or complex import statements
-**Solution**: Manually review and update any remaining imports
-
-### IDE Still Shows Old Imports
-
-**Cause**: IDE caching module information
-**Solution**: Restart IDE and clear module cache
-
-## Performance Considerations
-
-- **Large Codebases**: The script processes all `.go` files recursively
-- **File Count**: Performance scales with the number of Go files
-- **Backup Overhead**: Unix script creates backups (extra I/O)
-- **Memory Usage**: Minimal memory footprint for both implementations
-
-## Integration with CI/CD
-
-### GitHub Actions Example
-
-```yaml
-- name: Change Package Name
- run: |
- chmod +x scripts/change_package.sh
- ./scripts/change_package.sh "github.com/${{ github.repository }}"
-
-- name: Verify Changes
- run: |
- go mod tidy
- go build ./...
-```
-
-### Jenkins Pipeline Example
-
-```groovy
-stage('Package Rename') {
- steps {
- sh 'chmod +x scripts/change_package.sh'
- sh "./scripts/change_package.sh ${NEW_MODULE_NAME}"
- }
-}
-
-stage('Verification') {
- steps {
- sh 'go mod tidy'
- sh 'go build ./...'
- }
-}
-```
-
-## Security Considerations
-
-- **Input Validation**: Scripts validate module name format
-- **Path Safety**: Operations confined to project directory
-- **No Network Access**: Scripts work entirely offline
-- **No External Dependencies**: Reduces supply chain risks
-
-## Future Enhancements
-
-Potential improvements for future versions:
-
-- **Dry Run Mode**: Preview changes without applying them
-- **Selective Updates**: Update only specific directories
-- **Import Alias Handling**: Better support for custom import aliases
-- **Validation Checks**: Verify new module name format
-- **Progress Indicators**: Show progress for large codebases
-- **Undo Functionality**: Automated rollback capability
-
-## Conclusion
-
-The Package Name Change Scripts provide a reliable, cross-platform solution for Go module renaming. They handle the complexity of updating both module declarations and import paths while providing safety mechanisms to prevent data loss.
-
-The scripts follow the principle of "convention over configuration" - they work automatically with standard Go project structures without requiring additional setup or configuration files.
diff --git a/docs_wiki/CONFIGURATION_GUIDE.md b/docs_wiki/CONFIGURATION_GUIDE.md
deleted file mode 100644
index 01fc2a4..0000000
--- a/docs_wiki/CONFIGURATION_GUIDE.md
+++ /dev/null
@@ -1,460 +0,0 @@
-# Configuration Guide
-
-This document provides a comprehensive guide to configuring the application using the `config.yaml` file. It includes all available configuration options with explanations and examples.
-
-## Table of Contents
-
-- [Basic Configuration](#basic-configuration)
-- [Server Configuration](#server-configuration)
-- [Services Configuration](#services-configuration)
-- [Authentication](#authentication)
-- [Redis Configuration](#redis-configuration)
-- [Kafka Configuration](#kafka-configuration)
-- [PostgreSQL Configuration](#postgresql-configuration)
- - [Single Connection (Legacy)](#single-connection-legacy)
- - [Multiple Connections (Recommended)](#multiple-connections-recommended)
-- [Monitoring Configuration](#monitoring-configuration)
- - [MinIO Configuration](#minio-configuration)
- - [External Services](#external-services)
-- [Cron Jobs](#cron-jobs)
-- [Encryption](#encryption)
-
-## Basic Configuration
-
-```yaml
-app:
- name: "My Fancy Go App"
- version: "1.0.0"
- debug: true
- env: "development"
- banner_path: "banner.txt"
- startup_delay: 3 # seconds to display boot screen (0 to skip)
- quiet_startup: true # suppress console logs (TUI only, logs still go to monitoring)
- enable_tui: true # enable fancy TUI mode (false = traditional console logging)
-```
-
-| Option | Type | Description | Default |
-|--------|------|-------------|---------|
-| `name` | string | Application name | "My Fancy Go App" |
-| `version` | string | Application version | "1.0.0" |
-| `debug` | boolean | Enable debug mode | false |
-| `env` | string | Environment (development, production, etc.) | "development" |
-| `banner_path` | string | Path to banner text file | "banner.txt" |
-| `startup_delay` | integer | Seconds to display boot screen (0 to skip) | 0 |
-| `quiet_startup` | boolean | Suppress console logs during startup | false |
-| `enable_tui` | boolean | Enable Terminal User Interface | false |
-
-## Server Configuration
-
-```yaml
-server:
- port: "8080"
-```
-
-| Option | Type | Description | Default |
-|--------|------|-------------|---------|
-| `port` | string | Server port | "8080" |
-
-## Services Configuration
-
-```yaml
-services:
- service_a: true
- service_b: false
- service_c: true
- service_d: false
- service_encryption: false
-```
-
-Each service can be enabled or disabled individually. Set to `true` to enable, `false` to disable.
-
-## Authentication
-
-```yaml
-auth:
- type: "apikey"
- secret: "super-secret-key"
-```
-
-| Option | Type | Description | Default |
-|--------|------|-------------|---------|
-| `type` | string | Authentication type | "apikey" |
-| `secret` | string | Secret key for authentication | "" |
-
-## Redis Configuration
-
-```yaml
-redis:
- enabled: false
- address: "localhost:6379"
- password: ""
- db: 0
-```
-
-| Option | Type | Description | Default |
-|--------|------|-------------|---------|
-| `enabled` | boolean | Enable Redis | false |
-| `address` | string | Redis server address | "localhost:6379" |
-| `password` | string | Redis password | "" |
-| `db` | integer | Redis database number | 0 |
-
-## Kafka Configuration
-
-```yaml
-kafka:
- enabled: false
- brokers:
- - "localhost:9092"
- topic: "my-topic"
- group_id: "my-group"
-```
-
-| Option | Type | Description | Default |
-|--------|------|-------------|---------|
-| `enabled` | boolean | Enable Kafka | false |
-| `brokers` | array | List of Kafka broker addresses | ["localhost:9092"] |
-| `topic` | string | Default Kafka topic | "my-topic" |
-| `group_id` | string | Consumer group ID | "my-group" |
-
-## PostgreSQL Configuration
-
-The application supports both single and multiple PostgreSQL connections. Multiple connections allow you to connect to different databases and switch between them dynamically through the web monitoring interface.
-
-### Single Connection (Legacy)
-
-```yaml
-postgres:
- enabled: true
- host: "localhost"
- port: 5432
- user: "postgres"
- password: "Mypostgres01"
- dbname: "primary_db"
- sslmode: "disable"
- max_open_conns: 10
- max_idle_conns: 5
-```
-
-### Multiple Connections (Recommended)
-
-```yaml
-postgres:
- enabled: true
- connections:
- - name: "primary"
- enabled: true
- host: "localhost"
- port: 5432
- user: "postgres"
- password: "Mypostgres01"
- dbname: "primary_db"
- sslmode: "disable"
-
- - name: "secondary"
- enabled: true
- host: "localhost"
- port: 5433
- user: "postgres"
- password: "Mypostgres01"
- dbname: "secondary_db"
- sslmode: "disable"
-
- - name: "analytics"
- enabled: false # Disabled by default
- host: "analytics.example.com"
- port: 5432
- user: "analytics_user"
- password: "analytics_password"
- dbname: "analytics_db"
- sslmode: "require"
-```
-
-### Multiple Connections Features
-
-- **Dynamic Switching**: Switch between database connections through the web monitoring interface
-- **Connection Health**: Monitor the status of each database connection individually
-- **Selective Queries**: Run queries on specific databases by selecting the connection
-- **Load Distribution**: Distribute read/write operations across multiple databases
-- **Failover Support**: Automatic fallback when connections become unavailable
-
-### Web Monitoring Interface
-
-When multiple connections are configured, the PostgreSQL monitoring page (`/monitoring/postgres`) provides:
-
-- **Connection Selector**: Dropdown to choose which database to monitor/query
-- **Status Indicators**: Green/red dots showing connection health for each database
-- **Database Info**: Information about the currently selected database
-- **Query Execution**: Run SQL queries on the selected database connection
-- **Running Queries**: View active queries on the selected database
-
-### Usage Examples
-
-#### Monitoring Multiple Databases
-
-1. Configure multiple connections in `config.yaml`
-2. Start the application with monitoring enabled
-3. Access the monitoring dashboard at `http://localhost:9090`
-4. Navigate to the "Postgres" tab
-5. Use the connection dropdown to switch between databases
-6. Monitor health and run queries on each database individually
-
-#### High Availability Setup
-
-```yaml
-postgres:
- enabled: true
- connections:
- - name: "primary"
- enabled: true
- host: "db-primary.example.com"
- port: 5432
- user: "app_user"
- password: "${PRIMARY_DB_PASSWORD}"
- dbname: "app_db"
- sslmode: "require"
-
- - name: "replica"
- enabled: true
- host: "db-replica.example.com"
- port: 5432
- user: "app_user"
- password: "${REPLICA_DB_PASSWORD}"
- dbname: "app_db"
- sslmode: "require"
-```
-
-| Option | Type | Description | Default |
-|--------|------|-------------|---------|
-| `enabled` | boolean | Enable PostgreSQL | false |
-| `connections` | array | List of PostgreSQL connections | [] |
-| `connections[].name` | string | Connection name identifier | "" |
-| `connections[].enabled` | boolean | Enable this specific connection | false |
-| `connections[].host` | string | PostgreSQL host | "localhost" |
-| `connections[].port` | integer | PostgreSQL port | 5432 |
-| `connections[].user` | string | PostgreSQL username | "postgres" |
-| `connections[].password` | string | PostgreSQL password | "" |
-| `connections[].dbname` | string | Database name | "" |
-| `connections[].sslmode` | string | SSL mode (disable, require, etc.) | "disable" |
-
-## Monitoring Configuration
-
-```yaml
-monitoring:
- enabled: true
- port: "9090"
- password: "admin"
- obfuscate_api: true
- title: "Stackyard Admin"
- subtitle: "Dashboard Monitor"
- max_photo_size_mb: 2
- upload_dir: "web/monitoring/uploads"
-```
-
-| Option | Type | Description | Default |
-|--------|------|-------------|---------|
-| `enabled` | boolean | Enable monitoring | false |
-| `port` | string | Monitoring port | "9090" |
-| `password` | string | Monitoring password | "" |
-| `obfuscate_api` | boolean | Enable API obfuscation | false |
-| `title` | string | Monitoring dashboard title | "Stackyard Admin" |
-| `subtitle` | string | Monitoring dashboard subtitle | "" |
-| `max_photo_size_mb` | integer | Maximum photo upload size in MB | 2 |
-| `upload_dir` | string | Upload directory path | "web/monitoring/uploads" |
-
-### MinIO Configuration
-
-```yaml
-monitoring:
- minio:
- enabled: true
- endpoint: "localhost:9003"
- access_key: "minioadmin"
- secret_key: "minioadmin"
- use_ssl: false
- bucket: "main"
-```
-
-| Option | Type | Description | Default |
-|--------|------|-------------|---------|
-| `enabled` | boolean | Enable MinIO integration | false |
-| `endpoint` | string | MinIO endpoint | "localhost:9000" |
-| `access_key` | string | MinIO access key | "minioadmin" |
-| `secret_key` | string | MinIO secret key | "minioadmin" |
-| `use_ssl` | boolean | Use SSL for MinIO connection | false |
-| `bucket` | string | Default bucket name | "main" |
-
-### External Services
-
-```yaml
-monitoring:
- external:
- services:
- - name: "Google"
- url: "https://google.com"
- - name: "Soundcloud"
- url: "https://soundcloud.com"
- - name: "Local API"
- url: "http://localhost:8080/health"
-```
-
-## Cron Jobs
-
-```yaml
-cron:
- enabled: true
- jobs:
- log_cleanup: "0 0 * * *" # Run at midnight
- health_check: "*/10 * * * * *" # Every 10 seconds
-```
-
-| Option | Type | Description | Default |
-|--------|------|-------------|---------|
-| `enabled` | boolean | Enable cron jobs | false |
-| `jobs` | object | Cron job definitions | {} |
-
-## Encryption
-
-```yaml
-encryption:
- enabled: false
- algorithm: "aes-256-gcm"
- key: ""
- rotate_keys: false
- key_rotation_interval: "24h"
-```
-
-| Option | Type | Description | Default |
-|--------|------|-------------|---------|
-| `enabled` | boolean | Enable encryption | false |
-| `algorithm` | string | Encryption algorithm | "aes-256-gcm" |
-| `key` | string | Encryption key (32 bytes for AES-256) | "" |
-| `rotate_keys` | boolean | Enable automatic key rotation | false |
-| `key_rotation_interval` | string | Key rotation interval | "24h" |
-
-## Complete Example Configuration
-
-```yaml
-# Example configuration with multiple PostgreSQL connections
-app:
- name: "My Fancy Go App"
- version: "1.0.0"
- debug: true
- env: "development"
- banner_path: "banner.txt"
- startup_delay: 3 # seconds to display boot screen (0 to skip)
- quiet_startup: true # suppress console logs (TUI only, logs still go to monitoring)
- enable_tui: true # enable fancy TUI mode (false = traditional console logging)
-
-server:
- port: "8080"
-
-services:
- service_a: true
- service_b: false
- service_c: true
- service_d: false
- service_encryption: false
-
-auth:
- type: "apikey"
- secret: "super-secret-key"
-
-redis:
- enabled: false
- address: "localhost:6379"
- password: ""
- db: 0
-
-kafka:
- enabled: false
- brokers:
- - "localhost:9092"
- topic: "my-topic"
- group_id: "my-group"
-
-# NEW: Multiple PostgreSQL connections configuration
-postgres:
- enabled: true
- connections:
- - name: "primary"
- enabled: true
- host: "localhost"
- port: 5432
- user: "postgres"
- password: "Mypostgres01"
- dbname: "primary_db"
- sslmode: "disable"
-
- - name: "secondary"
- enabled: true
- host: "localhost"
- port: 5433
- user: "postgres"
- password: "Mypostgres01"
- dbname: "secondary_db"
- sslmode: "disable"
-
- - name: "analytics"
- enabled: false # Disabled by default
- host: "analytics.example.com"
- port: 5432
- user: "analytics_user"
- password: "analytics_password"
- dbname: "analytics_db"
- sslmode: "require"
-
-monitoring:
- enabled: true
- port: "9090"
- password: "admin"
- obfuscate_api: true
- title: "Stackyard Admin"
- subtitle: "Dashboard Monitor"
- max_photo_size_mb: 2
- upload_dir: "web/monitoring/uploads"
-
- minio:
- enabled: true
- endpoint: "localhost:9003"
- access_key: "minioadmin"
- secret_key: "minioadmin"
- use_ssl: false
- bucket: "main"
-
- external:
- services:
- - name: "Google"
- url: "https://google.com"
- - name: "Soundcloud"
- url: "https://soundcloud.com"
- - name: "Local API"
- url: "http://localhost:8080/health"
-
-cron:
- enabled: true
- jobs:
- log_cleanup: "0 0 * * *"
- health_check: "*/10 * * * * *"
-
-encryption:
- enabled: false
- algorithm: "aes-256-gcm"
- key: ""
- rotate_keys: false
- key_rotation_interval: "24h"
-```
-
-## Usage
-
-1. Copy the example configuration to `config.yaml`
-2. Modify the values according to your environment
-3. Ensure sensitive information (passwords, secrets) are properly secured
-4. For production environments, consider using environment variables for sensitive data
-
-## Best Practices
-
-- Use environment variables for sensitive configuration (passwords, API keys)
-- Disable unused services to reduce resource consumption
-- Use meaningful names for PostgreSQL connections
-- Monitor connection health through the monitoring dashboard
-- Regularly rotate encryption keys if encryption is enabled
diff --git a/docs_wiki/DEVELOPMENT.md b/docs_wiki/DEVELOPMENT.md
new file mode 100644
index 0000000..9d77405
--- /dev/null
+++ b/docs_wiki/DEVELOPMENT.md
@@ -0,0 +1,661 @@
+# Development Guide
+
+This guide covers how to extend and customize Stackyard for your specific needs. Learn to add new services, integrate databases, handle API requests, and deploy your application.
+
+## Adding New Services
+
+Services are the core building blocks of Stackyard applications. Each service encapsulates business logic and exposes API endpoints.
+
+### Basic Service Structure
+
+Create a new service in `internal/services/modules/service_yourname.go`:
+
+```go
+package modules
+
+import (
+ "stackyard/pkg/response"
+ "github.com/labstack/echo/v4"
+)
+
+type YourService struct {
+ enabled bool
+}
+
+func NewYourService(enabled bool) *YourService {
+ return &YourService{enabled: enabled}
+}
+
+func (s *YourService) Name() string { return "Your Service" }
+func (s *YourService) Enabled() bool { return s.enabled }
+func (s *YourService) Endpoints() []string { return []string{"/your-api"} }
+
+func (s *YourService) RegisterRoutes(g *echo.Group) {
+ // Register your API endpoints here
+ g.GET("/your-api", s.getData)
+ g.POST("/your-api", s.createData)
+}
+
+func (s *YourService) getData(c echo.Context) error {
+ // Your business logic here
+ data := map[string]string{"message": "Hello from your service!"}
+ return response.Success(c, data, "Data retrieved")
+}
+
+func (s *YourService) createData(c echo.Context) error {
+ // Handle POST request
+ return response.Created(c, nil, "Data created")
+}
+```
+
+### Register Your Service
+
+Add to `internal/server/server.go`:
+
+```go
+// Find the service registration section and add:
+registry.Register(modules.NewYourService(s.config.Services.IsEnabled("your_service")))
+```
+
+### Enable in Configuration
+
+Add to `config.yaml`:
+
+```yaml
+services:
+ your_service: true
+```
+
+### Test Your Service
+
+```bash
+# Restart the application
+go run cmd/app/main.go
+
+# Test the endpoint
+curl http://localhost:8080/api/v1/your-api
+```
+
+## API Development
+
+### Request Handling & Validation
+
+Stackyard provides built-in request validation and standardized responses.
+
+#### Basic Request Handling
+
+```go
+func (s *YourService) createUser(c echo.Context) error {
+ // Parse JSON request
+ var req CreateUserRequest
+ if err := c.Bind(&req); err != nil {
+ return response.BadRequest(c, "Invalid request format")
+ }
+
+ // Validate request
+ if req.Name == "" {
+ return response.BadRequest(c, "Name is required")
+ }
+
+ // Process request
+ user := User{Name: req.Name, Email: req.Email}
+ // Save to database...
+
+ return response.Created(c, user, "User created successfully")
+}
+```
+
+#### Advanced Validation with Tags
+
+```go
+type CreateUserRequest struct {
+ Name string `json:"name" validate:"required,min=2,max=50"`
+ Email string `json:"email" validate:"required,email"`
+ Age int `json:"age" validate:"required,gte=18,lte=120"`
+ Phone string `json:"phone" validate:"required,phone"`
+ Password string `json:"password" validate:"required,min=8"`
+}
+
+func (s *YourService) createUser(c echo.Context) error {
+ var req CreateUserRequest
+
+ // Bind and validate in one step
+ if err := request.Bind(c, &req); err != nil {
+ if validationErr, ok := err.(*request.ValidationError); ok {
+ return response.ValidationError(c, "Validation failed", validationErr.GetFieldErrors())
+ }
+ return response.BadRequest(c, err.Error())
+ }
+
+ // Request is valid, proceed...
+ return response.Created(c, req, "User created")
+}
+```
+
+#### Custom Validators
+
+Add custom validation rules in `pkg/request/request.go`:
+
+```go
+// Add to the validator initialization
+validate.RegisterValidation("phone", func(fl validator.FieldLevel) bool {
+ phone := fl.Field().String()
+ // Your phone validation logic
+ matched, _ := regexp.MatchString(`^\+?[1-9]\d{1,14}$`, phone)
+ return matched
+})
+
+validate.RegisterValidation("username", func(fl validator.FieldLevel) bool {
+ username := fl.Field().String()
+ // Alphanumeric, 3-20 characters
+ matched, _ := regexp.MatchString(`^[a-zA-Z0-9]{3,20}$`, username)
+ return matched
+})
+```
+
+### Response Types
+
+#### Success Responses
+
+```go
+// Simple success
+return response.Success(c, data, "Operation completed")
+
+// Success with metadata (pagination)
+meta := response.CalculateMeta(page, perPage, total)
+return response.SuccessWithMeta(c, data, meta, "Data retrieved")
+
+// Created (201)
+return response.Created(c, newResource, "Resource created")
+
+// No content (204)
+return response.NoContent(c)
+```
+
+#### Error Responses
+
+```go
+// Bad request
+return response.BadRequest(c, "Invalid input data")
+
+// Not found
+return response.NotFound(c, "Resource not found")
+
+// Unauthorized
+return response.Unauthorized(c, "Authentication required")
+
+// Forbidden
+return response.Forbidden(c, "Access denied")
+
+// Validation error with field details
+fieldErrors := map[string]string{
+ "email": "Invalid email format",
+ "password": "Must be at least 8 characters",
+}
+return response.ValidationError(c, "Validation failed", fieldErrors)
+```
+
+#### Pagination
+
+```go
+type PaginationRequest struct {
+ Page int `query:"page" json:"page"`
+ PerPage int `query:"per_page" json:"per_page"`
+}
+
+func (s *YourService) listUsers(c echo.Context) error {
+ var pagination PaginationRequest
+ if err := c.Bind(&pagination); err != nil {
+ return response.BadRequest(c, "Invalid pagination parameters")
+ }
+
+ page := pagination.GetPage() // Default: 1
+ perPage := pagination.GetPerPage() // Default: 10, Max: 100
+ offset := pagination.GetOffset() // Calculated offset
+
+ // Query with pagination
+ users, total, err := s.getUsersWithPagination(offset, perPage)
+ if err != nil {
+ return response.InternalServerError(c, "Failed to fetch users")
+ }
+
+ // Return with pagination metadata
+ meta := response.CalculateMeta(page, perPage, total)
+ return response.SuccessWithMeta(c, users, meta, "Users retrieved")
+}
+```
+
+## Database Integration
+
+### PostgreSQL with GORM
+
+#### Basic Model & Operations
+
+```go
+type User struct {
+ gorm.Model
+ Name string `json:"name" gorm:"not null"`
+ Email string `json:"email" gorm:"unique;not null"`
+ Password string `json:"-" gorm:"not null"` // Don't serialize
+}
+
+func (s *UserService) createUser(c echo.Context) error {
+ var req CreateUserRequest
+ if err := request.Bind(c, &req); err != nil {
+ return response.BadRequest(c, err.Error())
+ }
+
+ user := User{
+ Name: req.Name,
+ Email: req.Email,
+ Password: hashPassword(req.Password),
+ }
+
+ if err := s.db.Create(&user).Error; err != nil {
+ return response.InternalServerError(c, "Failed to create user")
+ }
+
+ return response.Created(c, user, "User created")
+}
+
+func (s *UserService) getUser(c echo.Context) error {
+ id := c.Param("id")
+ var user User
+
+ if err := s.db.First(&user, id).Error; err != nil {
+ if errors.Is(err, gorm.ErrRecordNotFound) {
+ return response.NotFound(c, "User not found")
+ }
+ return response.InternalServerError(c, "Database error")
+ }
+
+ return response.Success(c, user, "User retrieved")
+}
+```
+
+#### Dependency Injection
+
+Services receive database managers through constructor injection:
+
+```go
+type UserService struct {
+ enabled bool
+ db *infrastructure.PostgresManager
+}
+
+func NewUserService(db *infrastructure.PostgresManager, enabled bool) *UserService {
+ return &UserService{
+ enabled: enabled,
+ db: db,
+ }
+}
+```
+
+### Redis Caching
+
+#### Basic Caching Operations
+
+```go
+type CacheService struct {
+ redis *infrastructure.RedisManager
+}
+
+func (s *CacheService) cacheUser(userID string, user User) error {
+ ctx := context.Background()
+ data, err := json.Marshal(user)
+ if err != nil {
+ return err
+ }
+
+ return s.redis.Set(ctx, fmt.Sprintf("user:%s", userID), string(data), time.Hour)
+}
+
+func (s *CacheService) getCachedUser(userID string) (*User, error) {
+ ctx := context.Background()
+ data, err := s.redis.Get(ctx, fmt.Sprintf("user:%s", userID))
+ if err != nil {
+ return nil, err
+ }
+
+ var user User
+ if err := json.Unmarshal([]byte(data), &user); err != nil {
+ return nil, err
+ }
+
+ return &user, nil
+}
+```
+
+#### Cache-Aside Pattern
+
+```go
+func (s *UserService) getUserWithCache(c echo.Context) error {
+ userID := c.Param("id")
+
+ // Try cache first
+ if user, err := s.cache.getCachedUser(userID); err == nil {
+ return response.Success(c, user, "User retrieved from cache")
+ }
+
+ // Cache miss - get from database
+ var user User
+ if err := s.db.First(&user, userID).Error; err != nil {
+ if errors.Is(err, gorm.ErrRecordNotFound) {
+ return response.NotFound(c, "User not found")
+ }
+ return response.InternalServerError(c, "Database error")
+ }
+
+ // Cache for future requests
+ s.cache.cacheUser(userID, user)
+
+ return response.Success(c, user, "User retrieved")
+}
+```
+
+## Event Streaming
+
+### Server-Sent Events (SSE)
+
+Add real-time capabilities to your services:
+
+```go
+type NotificationService struct {
+ enabled bool
+ broadcaster *utils.EventBroadcaster
+}
+
+func NewNotificationService(enabled bool) *NotificationService {
+ return &NotificationService{
+ enabled: enabled,
+ broadcaster: utils.NewEventBroadcaster(),
+ }
+}
+
+func (s *NotificationService) RegisterRoutes(g *echo.Group) {
+ g.GET("/notifications/stream", s.streamNotifications)
+ g.POST("/notifications/send", s.sendNotification)
+}
+
+func (s *NotificationService) streamNotifications(c echo.Context) error {
+ // Subscribe to notification stream
+ client := s.broadcaster.Subscribe("notifications")
+ defer s.broadcaster.Unsubscribe(client.ID)
+
+ // Set SSE headers
+ c.Response().Header().Set(echo.HeaderContentType, "text/event-stream")
+ c.Response().Header().Set(echo.HeaderCacheControl, "no-cache")
+ c.Response().Header().Set(echo.HeaderConnection, "keep-alive")
+
+ // Listen for events
+ for {
+ select {
+ case event := <-client.Channel:
+ // Send SSE event
+ c.Response().Write([]byte(fmt.Sprintf("data: %s\n\n", event.Data)))
+ c.Response().Flush()
+ case <-c.Request().Context().Done():
+ return nil
+ }
+ }
+}
+
+func (s *NotificationService) sendNotification(c echo.Context) error {
+ var notification map[string]interface{}
+ if err := c.Bind(¬ification); err != nil {
+ return response.BadRequest(c, "Invalid notification data")
+ }
+
+ // Broadcast to all subscribers
+ s.broadcaster.Broadcast("notifications", "notification", "New notification", notification)
+
+ return response.Success(c, nil, "Notification sent")
+}
+```
+
+## File Upload Handling
+
+### Basic File Upload
+
+```go
+func (s *FileService) uploadFile(c echo.Context) error {
+ // Get file from form
+ file, err := c.FormFile("file")
+ if err != nil {
+ return response.BadRequest(c, "No file provided")
+ }
+
+ // Open uploaded file
+ src, err := file.Open()
+ if err != nil {
+ return response.InternalServerError(c, "Failed to open file")
+ }
+ defer src.Close()
+
+ // Upload to storage (MinIO, local, etc.)
+ result, err := s.storage.UploadFile(context.Background(),
+ fmt.Sprintf("uploads/%s", file.Filename),
+ src, file.Size, file.Header.Get("Content-Type"))
+
+ if err != nil {
+ return response.InternalServerError(c, "Upload failed")
+ }
+
+ return response.Created(c, map[string]interface{}{
+ "filename": file.Filename,
+ "size": file.Size,
+ "url": s.storage.GetFileUrl(result.Key),
+ }, "File uploaded successfully")
+}
+```
+
+## Configuration Management
+
+### Adding New Configuration Options
+
+Add to `config/config.go`:
+
+```go
+type YourServiceConfig struct {
+ APIKey string `yaml:"api_key"`
+ Timeout int `yaml:"timeout" default:"30"`
+ Endpoints []string `yaml:"endpoints"`
+}
+
+type Config struct {
+ // ... existing fields ...
+ YourService YourServiceConfig `yaml:"your_service"`
+}
+```
+
+Use in `config.yaml`:
+
+```yaml
+your_service:
+ api_key: "your-api-key"
+ timeout: 60
+ endpoints:
+ - "https://api.example.com"
+ - "https://backup.example.com"
+```
+
+## Testing
+
+### Unit Tests
+
+```go
+func TestUserService_GetUser(t *testing.T) {
+ // Setup
+ mockDB := &mocks.PostgresManager{}
+ service := NewUserService(mockDB, true)
+
+ // Mock expectations
+ expectedUser := User{ID: 1, Name: "John"}
+ mockDB.On("First", mock.AnythingOfType("*User"), "1").Return(nil).Run(func(args mock.Arguments) {
+ user := args.Get(0).(*User)
+ *user = expectedUser
+ })
+
+ // Test
+ c, rec := setupEchoContext()
+ c.SetParamNames("id")
+ c.SetParamValues("1")
+
+ err := service.getUser(c)
+ assert.NoError(t, err)
+
+ // Verify response
+ var response map[string]interface{}
+ json.Unmarshal(rec.Body.Bytes(), &response)
+ assert.True(t, response["success"].(bool))
+ assert.Equal(t, "User retrieved", response["message"])
+}
+```
+
+### Integration Tests
+
+```go
+func TestUserAPI(t *testing.T) {
+ // Start test server
+ e := echo.New()
+ // ... setup routes ...
+
+ // Test HTTP requests
+ req := httptest.NewRequest(http.MethodGet, "/api/v1/users/1", nil)
+ rec := httptest.NewRecorder()
+ c := e.NewContext(req, rec)
+ c.SetParamNames("id")
+ c.SetParamValues("1")
+
+ // Execute request
+ err := userHandler(c)
+ assert.NoError(t, err)
+
+ // Verify response
+ assert.Equal(t, http.StatusOK, rec.Code)
+ var response map[string]interface{}
+ json.Unmarshal(rec.Body.Bytes(), &response)
+ assert.True(t, response["success"].(bool))
+}
+```
+
+## Deployment
+
+### Building for Production
+
+```bash
+# Build optimized binary
+go build -ldflags="-w -s" -o app cmd/app/main.go
+
+# Or use the build script
+./scripts/build.sh
+```
+
+### Docker Deployment
+
+Create `Dockerfile`:
+
+```dockerfile
+FROM golang:1.21-alpine AS builder
+WORKDIR /app
+COPY go.mod go.sum ./
+RUN go mod download
+COPY . .
+RUN go build -ldflags="-w -s" -o main cmd/app/main.go
+
+FROM alpine:latest
+RUN apk --no-cache add ca-certificates
+WORKDIR /root/
+COPY --from=builder /app/main .
+EXPOSE 8080 9090
+CMD ["./main"]
+```
+
+Build and run:
+
+```bash
+docker build -t myapp .
+docker run -p 8080:8080 -p 9090:9090 myapp
+```
+
+### Environment Variables
+
+Override configuration with environment variables:
+
+```bash
+export APP_DEBUG=false
+export SERVER_PORT=3000
+export MONITORING_PASSWORD=secure-password
+export POSTGRES_PASSWORD=prod-password
+
+go run cmd/app/main.go
+```
+
+## Best Practices
+
+### Service Design
+
+1. **Single Responsibility**: Each service should do one thing well
+2. **Dependency Injection**: Inject infrastructure dependencies
+3. **Error Handling**: Use consistent error responses
+4. **Validation**: Always validate input data
+5. **Logging**: Log important operations and errors
+
+### API Design
+
+1. **RESTful URLs**: Use consistent URL patterns
+2. **HTTP Status Codes**: Use appropriate status codes
+3. **JSON Responses**: Stick to the standard response format
+4. **Versioning**: Include API versioning in URLs
+5. **Documentation**: Document all endpoints
+
+### Performance
+
+1. **Caching**: Cache frequently accessed data
+2. **Pagination**: Always paginate large datasets
+3. **Async Operations**: Use async operations for slow tasks
+4. **Connection Pooling**: Database connections are automatically pooled
+5. **Indexes**: Add database indexes for performance
+
+### Security
+
+1. **Input Validation**: Validate all user inputs
+2. **Authentication**: Implement proper authentication
+3. **Authorization**: Check permissions for operations
+4. **HTTPS**: Use HTTPS in production
+5. **Secrets**: Never commit secrets to version control
+
+## Troubleshooting
+
+### Common Development Issues
+
+**Service not registering:**
+- Check that the service is added to `internal/server/server.go`
+- Verify the config key matches in `config.yaml`
+- Check for compilation errors
+
+**Database connection errors:**
+- Verify database credentials
+- Check network connectivity
+- Ensure database server is running
+
+**API validation errors:**
+- Check request JSON structure
+- Verify validation tags on struct fields
+- Test with valid/invalid data
+
+**Performance issues:**
+- Add database indexes
+- Implement caching
+- Check for N+1 query problems
+- Monitor memory usage
+
+## Next Steps
+
+Now that you understand how to develop with Stackyard, explore:
+
+- **[Architecture Overview](ARCHITECTURE.md)** - Deep dive into the technical design
+- **[API Reference](REFERENCE.md)** - Complete technical documentation
+- **Built-in Services** - Study `service_a.go`, `service_b.go` for examples
+
+Happy developing! 🎯
diff --git a/docs_wiki/DOCKER_CONTAINERIZATION.md b/docs_wiki/DOCKER_CONTAINERIZATION.md
deleted file mode 100644
index c541fc9..0000000
--- a/docs_wiki/DOCKER_CONTAINERIZATION.md
+++ /dev/null
@@ -1,579 +0,0 @@
-# Docker Containerization Guide
-
-## Overview
-
-This project includes comprehensive Docker containerization with multi-stage builds for development, testing, and production environments. The Docker setup provides consistent deployment across different environments while optimizing for security, performance, and maintainability.
-
-## Dockerfile Architecture
-
-The `Dockerfile` implements a multi-stage build strategy with optimized stages for minimal image sizes:
-
-### 1. Builder Stage (Optimized)
-
-```dockerfile
-FROM golang:1.25.5-alpine3.23 AS builder
-
-WORKDIR /app
-
-# Install build dependencies and UPX for compression
-RUN apk add --no-cache upx
-
-# Copy go mod files
-COPY go.mod go.sum ./
-
-# Download dependencies
-RUN go mod download
-
-# Copy source code
-COPY . .
-
-# Build the binary with optimizations
-RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
- go build \
- -ldflags="-w -s" \
- -trimpath \
- -o main ./cmd/app
-
-# Compress binary with UPX (ultra-brute for maximum compression)
-RUN upx --ultra-brute main
-```
-
-**Optimizations Applied:**
-- **UPX Compression**: Reduces binary size by 50-70% using ultra-brute compression
-- **Build Flags**: `-ldflags="-w -s"` removes debugging information and symbol table
-- **Trimpath**: Removes file system paths from the compiled executable
-- **Static Linking**: `CGO_ENABLED=0` ensures fully static binaries
-
-**Purpose**: Compiles the Go application into a static binary
-- Uses Alpine Linux for smaller base image
-- Downloads dependencies separately for better layer caching
-- Produces a statically linked binary with `CGO_ENABLED=0`
-- Targets Linux platform for container compatibility
-
-### 2. Test Stage
-
-```dockerfile
-FROM builder AS test
-
-# Run tests
-RUN go test ./...
-```
-
-**Purpose**: Executes the test suite in an isolated environment
-- Inherits all source code and dependencies from builder stage
-- Runs `go test ./...` to execute all test packages
-- Can be targeted separately for CI/CD testing pipelines
-
-### 3. Development Stage
-
-```dockerfile
-FROM golang:1.25.5-alpine3.23 AS dev
-
-WORKDIR /app
-
-# Copy go mod files
-COPY go.mod go.sum ./
-
-# Download dependencies
-RUN go mod download
-
-# Copy source code
-COPY . .
-
-# Build the binary
-RUN go build -o main ./cmd/app
-
-# Configure for Docker environment
-ENV APP_QUIET_STARTUP=false
-ENV APP_ENABLE_TUI=false
-
-# Expose ports for main API server and monitoring server
-EXPOSE 8080 9090
-
-# Run the application
-CMD ["./main"]
-```
-
-**Purpose**: Provides a development environment with hot-reload capabilities
-- Includes full Go toolchain for development tools
-- Mounts source code for live development
-- Exposes both main API (8080) and monitoring (9090) ports
-- Automatically disables TUI and quiet startup for containerized environment
-
-### 4. Production Stage
-
-```dockerfile
-FROM alpine:latest AS prod
-
-# Install ca-certificates for HTTPS
-RUN apk --no-cache add ca-certificates
-
-WORKDIR /root/
-
-# Copy the binary from builder stage
-COPY --from=builder /app/main .
-
-# Configure for Docker environment
-ENV APP_QUIET_STARTUP=false
-ENV APP_ENABLE_TUI=false
-
-# Expose ports for main API server and monitoring server
-EXPOSE 8080 9090
-
-# Run the application
-CMD ["./main"]
-```
-
-**Purpose**: Creates a minimal production image
-- Uses Alpine Linux for security and small size
-- Includes only the compiled binary and runtime dependencies
-- No source code or build tools included
-- Optimized for production deployment
-- Automatically disables TUI and quiet startup for containerized environment
-
-## Building Docker Images
-
-The project includes automated build scripts for building Docker images across different environments.
-
-### Using Build Scripts
-
-#### Unix/Linux/macOS
-
-```bash
-# Make script executable (first time only)
-chmod +x scripts/docker_build.sh
-
-# Build with default settings
-./scripts/docker_build.sh
-
-# Build with custom app name and image name
-./scripts/docker_build.sh "my-app" "myregistry/myapp"
-```
-
-#### Windows
-
-```cmd
-# Build with default settings
-scripts\docker_build.bat
-
-# Build with custom app name and image name
-scripts\docker_build.bat "my-app" "myregistry/myapp"
-```
-
-**Script Parameters:**
-- `APP_NAME`: Application name (default: "stackyard")
-- `IMAGE_NAME`: Docker image name (default: "myapp")
-- `TARGET`: Build target - "all", "test", "dev", or "prod" (default: "all")
-
-**What the scripts do:**
-1. **Test Stage**: Builds and runs tests to ensure code quality (for "test" and "all" targets)
-2. **Development Stage**: Builds development image with full Go toolchain (for "dev" and "all" targets)
-3. **Production Stage**: Builds optimized production image (for "prod" and "all" targets)
-4. **Cleanup**: Removes dangling Docker images to save space
-
-**Target Options:**
-- `all`: Build all stages (test, dev, prod) - default behavior
-- `test`: Build and run tests only (2 steps)
-- `dev`: Build development image only (1 step)
-- `prod`: Build production image only (1 step) - Alpine (~50MB) with full monitoring
-- `prod-slim`: Build slim production image (1 step) - Ubuntu (~30-40MB) with full monitoring
-- `prod-minimal`: Build minimal production image (1 step) - BusyBox (~10-20MB) with full monitoring
-- `ultra-prod`: Build ultra-minimal production image only (1 step) - smallest size using Distroless (~15-30MB, no monitoring)
-- `ultra-all`: Build all ultra-minimal stages (ultra-test, ultra-dev, ultra-prod)
-- `ultra-dev`: Build ultra-minimal development image (Distroless) - runs pre-built binary only
-- `ultra-test`: Build ultra-minimal test image (Distroless) - runs pre-built binary only
-
-**Usage Examples:**
-```bash
-# Build everything (default)
-./scripts/docker_build.sh
-
-# Build only production image (fastest)
-./scripts/docker_build.sh "myapp" "myregistry/myapp" "prod"
-
-# Build slim production image (~30-40MB, more secure)
-./scripts/docker_build.sh "myapp" "myregistry/myapp" "prod-slim"
-
-# Build minimal production image (~10-20MB)
-./scripts/docker_build.sh "myapp" "myregistry/myapp" "prod-minimal"
-
-# Build ultra-minimal production image (smallest, no monitoring)
-./scripts/docker_build.sh "myapp" "myregistry/myapp" "ultra-prod"
-
-# Build everything with ultra-prod for production
-./scripts/docker_build.sh "myapp" "myregistry/myapp" "ultra-all"
-```
-
-### Manual Docker Commands
-
-If you prefer to build manually:
-
-#### Development Build
-
-```bash
-# Build development image
-docker build --target dev -t myapp:dev .
-
-# Run development container
-docker run -p 8080:8080 -p 9090:9090 myapp:dev
-```
-
-#### Testing Build
-
-```bash
-# Build and run tests
-docker build --target test -t myapp:test .
-
-# Run tests only (will exit after tests complete)
-docker run myapp:test
-```
-
-#### Production Build
-
-```bash
-# Build production image
-docker build --target prod -t myapp:latest .
-
-# Run production container
-docker run -p 8080:8080 -p 9090:9090 myapp:latest
-```
-
-## Configuration in Containers
-
-### Environment Variables
-
-The application supports configuration via environment variables that override `config.yaml`:
-
-```bash
-# Run with custom configuration
-docker run \
- -e SERVER_PORT=3000 \
- -e MONITORING_PORT=4000 \
- -e REDIS_ENABLED=true \
- -e REDIS_HOST=redis-server \
- -p 3000:3000 \
- -p 4000:4000 \
- myapp:latest
-```
-
-### Volume Mounts
-
-For development with live reloading:
-
-```bash
-# Mount config file
-docker run \
- -v $(pwd)/config.yaml:/app/config.yaml \
- -p 8080:8080 \
- -p 9090:9090 \
- myapp:dev
-```
-
-For production with external config:
-
-```bash
-# Mount external config
-docker run \
- -v /path/to/config.yaml:/root/config.yaml \
- -p 8080:8080 \
- -p 9090:9090 \
- myapp:latest
-```
-
-## Networking
-
-### Port Configuration
-
-The application exposes two main ports:
-
-- **8080**: Main API server port (configurable via `server.port`)
-- **9090**: Monitoring web interface port (configurable via `monitoring.port`)
-
-### Port Mapping Examples
-
-```bash
-# Default port mapping
-docker run -p 8080:8080 -p 9090:9090 myapp:latest
-
-# Custom host ports
-docker run -p 3000:8080 -p 4000:9090 myapp:latest
-
-# Bind to specific interface
-docker run -p 127.0.0.1:8080:8080 myapp:latest
-```
-
-## Docker Compose Integration
-
-### Basic docker-compose.yml
-
-```yaml
-version: '3.8'
-
-services:
- app:
- build:
- context: .
- target: prod
- ports:
- - "8080:8080"
- - "9090:9090"
- environment:
- - SERVER_PORT=8080
- - MONITORING_PORT=9090
- depends_on:
- - postgres
- - redis
-
- postgres:
- image: postgres:15-alpine
- environment:
- - POSTGRES_DB=myapp
- - POSTGRES_USER=myapp
- - POSTGRES_PASSWORD=password
- volumes:
- - postgres_data:/var/lib/postgresql/data
- ports:
- - "5432:5432"
-
- redis:
- image: redis:7-alpine
- ports:
- - "6379:6379"
-
-volumes:
- postgres_data:
-```
-
-### Development docker-compose.yml
-
-```yaml
-version: '3.8'
-
-services:
- app:
- build:
- context: .
- target: dev
- ports:
- - "8080:8080"
- - "9090:9090"
- volumes:
- - .:/app
- - /app/main
- environment:
- - APP_ENV=development
- - APP_DEBUG=true
- depends_on:
- - postgres
- - redis
-
- postgres:
- image: postgres:15-alpine
- environment:
- - POSTGRES_DB=myapp_dev
- - POSTGRES_USER=myapp
- - POSTGRES_PASSWORD=password
- volumes:
- - postgres_dev_data:/var/lib/postgresql/data
- ports:
- - "5432:5432"
-
- redis:
- image: redis:7-alpine
- ports:
- - "6379:6379"
-
-volumes:
- postgres_dev_data:
-```
-
-## Multi-Stage Build Benefits
-
-### Security
-- **Minimal Attack Surface**: Production images contain only the runtime binary
-- **No Source Code**: Source code is not included in production images
-- **Alpine Linux**: Uses secure, minimal base images with regular security updates
-
-### Performance
-- **Small Image Size**: Multi-stage builds reduce final image size significantly
-- **Fast Startup**: Optimized binary compilation with static linking
-- **Layer Caching**: Dependencies are cached in separate layers for faster rebuilds
-
-### Maintainability
-- **Clear Separation**: Each stage has a specific purpose and can be built independently
-- **Targeted Builds**: Can build specific stages for different environments
-- **Easy Debugging**: Development stage includes full toolchain for troubleshooting
-
-## CI/CD Integration
-
-### GitHub Actions Example
-
-```yaml
-name: Build and Deploy
-
-on:
- push:
- branches: [ main ]
-
-jobs:
- test:
- runs-on: ubuntu-latest
- steps:
- - uses: actions/checkout@v3
- - name: Build test image
- run: docker build --target test -t myapp:test .
- - name: Run tests
- run: docker run myapp:test
-
- build:
- needs: test
- runs-on: ubuntu-latest
- steps:
- - uses: actions/checkout@v3
- - name: Build production image
- run: docker build --target prod -t myapp:latest .
- - name: Push to registry
- run: |
- echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin
- docker tag myapp:latest myregistry/myapp:latest
- docker push myregistry/myapp:latest
-```
-
-### Jenkins Pipeline Example
-
-```groovy
-pipeline {
- agent any
-
- stages {
- stage('Test') {
- steps {
- sh 'docker build --target test -t myapp:test .'
- sh 'docker run myapp:test'
- }
- }
-
- stage('Build') {
- steps {
- sh 'docker build --target prod -t myapp:latest .'
- }
- }
-
- stage('Deploy') {
- steps {
- sh 'docker-compose up -d'
- }
- }
- }
-}
-```
-
-## Troubleshooting
-
-### Common Issues
-
-**"exec ./main: no such file or directory"**
-- Ensure `CGO_ENABLED=0` in builder stage for static linking
-- Check that the binary was built correctly in the builder stage
-
-**"connection refused" to database**
-- Ensure database containers are started before the app
-- Use `depends_on` in docker-compose for proper startup order
-
-**Large image size**
-- Verify that production stage only copies the binary from builder
-- Use `.dockerignore` to exclude unnecessary files
-
-**Permission denied on volume mounts**
-- Ensure proper file permissions on mounted directories
-- Check user permissions in the container
-
-### Debug Commands
-
-```bash
-# Check running containers
-docker ps
-
-# View container logs
-docker logs
-
-# Execute shell in running container (not available in Distroless)
-docker exec -it /bin/sh
-
-# Inspect image layers
-docker history myapp:latest
-
-# Check image size
-docker images myapp
-
-# Verify which base image was used
-docker inspect myapp:ultra | grep -A 5 "RepoTags"
-```
-
-### Ultra-Prod Issues
-
-**"Still shows Alpine image for ultra-prod"**
-- **Cause**: Docker may be showing cached images or intermediate layers
-- **Solution**: Check `docker images` for the specific tag, or run `docker build --no-cache --target ultra-prod -t myapp:ultra .`
-
-**"Ultra-prod image not much smaller"**
-- **Cause**: Binary size may still be large despite UPX compression
-- **Solution**: Check binary size with `ls -lh main` in builder stage, ensure UPX is working
-
-**"Cannot run ultra-prod container"**
-- **Cause**: Distroless images have no shell, debugging is limited
-- **Solution**: Use Alpine-based prod image for debugging, switch to ultra-prod for production
-
-## Best Practices
-
-### Security
-1. **Use Official Images**: Base images from trusted sources only
-2. **Regular Updates**: Keep base images and dependencies updated
-3. **Minimal Images**: Use Alpine variants for smaller attack surface
-4. **No Secrets in Images**: Use environment variables or secret management
-
-### Performance
-1. **Multi-Stage Builds**: Separate build and runtime stages
-2. **Layer Optimization**: Order commands to maximize layer caching
-3. **Minimal Base Images**: Use Alpine Linux for production
-4. **Static Binaries**: Compile with `CGO_ENABLED=0` for portability
-
-### Development Workflow
-1. **Volume Mounts**: Mount source code for live development
-2. **Hot Reload**: Use development stage with full Go toolchain
-3. **Debug Tools**: Include debugging tools in development images
-4. **Consistent Environments**: Use same base images across team
-
-## Migration Guide
-
-### From Single-Stage to Multi-Stage
-
-**Before (single-stage):**
-```dockerfile
-FROM golang:1.21-alpine
-
-WORKDIR /app
-COPY . .
-RUN go build -o main ./cmd/app
-
-CMD ["./main"]
-```
-
-**After (multi-stage):**
-```dockerfile
-FROM golang:1.25.5-alpine3.23 AS builder
-# ... build stage
-
-FROM alpine:latest AS prod
-# ... production stage
-```
-
-### Benefits of Migration
-- **50-70% smaller images**: Remove build dependencies from final image
-- **Better security**: No source code or build tools in production
-- **Faster deployments**: Smaller images transfer and start faster
-- **Flexible builds**: Can build different stages for different purposes
-
-## Conclusion
-
-The Docker containerization setup provides a robust, secure, and efficient way to deploy the Go application across different environments. The multi-stage build approach ensures optimal image sizes, security, and performance while maintaining flexibility for development and testing workflows.
diff --git a/docs_wiki/ENCRYPTION_API.md b/docs_wiki/ENCRYPTION_API.md
deleted file mode 100644
index 7e47a16..0000000
--- a/docs_wiki/ENCRYPTION_API.md
+++ /dev/null
@@ -1,435 +0,0 @@
-# API Request/Response Encryption
-
-## Overview
-
-The API Request/Response Encryption feature provides end-to-end encryption for all API communications between clients and the server. This feature enhances security by encrypting sensitive data in transit, protecting against man-in-the-middle attacks and ensuring data confidentiality.
-
-## Features
-
-- **AES-256-GCM Encryption**: Industry-standard authenticated encryption providing both confidentiality and integrity
-- **Automatic Middleware**: Transparent encryption/decryption for all API endpoints
-- **Configurable**: Enable/disable encryption via configuration
-- **Key Management**: Support for key rotation and secure key storage
-- **Selective Encryption**: Skip encryption for health checks and system endpoints
-
-## Configuration
-
-### Basic Configuration
-
-Add the following section to your `config.yaml` file:
-
-```yaml
-encryption:
- enabled: true # Enable encryption feature
- algorithm: "aes-256-gcm" # Encryption algorithm
- key: "your-32-byte-secret-key-here-12345678" # Encryption key (32 bytes for AES-256)
- rotate_keys: false # Enable automatic key rotation
- key_rotation_interval: "24h" # Key rotation interval (when enabled)
-```
-
-### Environment Variables
-
-You can also configure encryption using environment variables:
-
-```bash
-export ENCRYPTION_ENABLED=true
-export ENCRYPTION_ALGORITHM="aes-256-gcm"
-export ENCRYPTION_KEY="your-32-byte-secret-key-here-12345678"
-export ENCRYPTION_ROTATE_KEYS=false
-export ENCRYPTION_KEY_ROTATION_INTERVAL="24h"
-```
-
-## Implementation Details
-
-### Middleware Architecture
-
-The encryption middleware operates at the HTTP layer, providing transparent encryption/decryption:
-
-1. **Request Processing**:
- - Checks for `X-Encrypted-Request: true` header
- - Decrypts request body if encrypted
- - Validates content type (JSON only)
-
-2. **Response Processing**:
- - Encrypts JSON responses when encryption is enabled
- - Sets `X-Encrypted-Response: true` header
- - Sets `X-Encryption-Algorithm` header
-
-3. **Endpoint Exclusions**:
- - `/health` - Health check endpoint
- - `/restart` - Server restart endpoint
- - `/api/v1/encryption/*` - Encryption service endpoints
-
-### Encryption Service Endpoints
-
-The encryption service provides the following endpoints under `/api/v1/encryption`:
-
-#### POST `/encrypt` - Encrypt Data
-
-**Request:**
-```json
-{
- "data": "sensitive data to encrypt",
- "content_type": "application/json"
-}
-```
-
-**Response:**
-```json
-{
- "status": "success",
- "message": "Data encrypted successfully",
- "data": {
- "encrypted_data": "base64-encoded-encrypted-data",
- "algorithm": "aes-256-gcm",
- "timestamp": 1234567890,
- "content_type": "application/json"
- }
-}
-```
-
-#### POST `/decrypt` - Decrypt Data
-
-**Request:**
-```json
-{
- "encrypted_data": "base64-encoded-encrypted-data",
- "content_type": "application/json"
-}
-```
-
-**Response:**
-```json
-{
- "status": "success",
- "message": "Data decrypted successfully",
- "data": {
- "decrypted_data": "original decrypted data",
- "algorithm": "aes-256-gcm",
- "timestamp": 1234567890,
- "content_type": "application/json"
- }
-}
-```
-
-#### GET `/status` - Get Encryption Status
-
-**Response:**
-```json
-{
- "status": "success",
- "message": "Encryption service status",
- "data": {
- "enabled": true,
- "algorithm": "aes-256-gcm",
- "current_key": "abcd...",
- "key_length": 32,
- "rotate_keys": false,
- "last_rotation": 1234567890
- }
-}
-```
-
-#### POST `/key-rotate` - Rotate Encryption Key
-
-**Request:**
-```json
-{
- "new_key": "new-32-byte-secret-key-here-12345678"
-}
-```
-
-**Response:**
-```json
-{
- "status": "success",
- "message": "Key rotation successful",
- "data": {
- "message": "Encryption key rotated successfully",
- "new_key_preview": "abcd..."
- }
-}
-```
-
-## Client Implementation Guide
-
-### JavaScript Client Example
-
-```javascript
-import axios from 'axios';
-import { encrypt, decrypt } from './encryption-utils';
-
-const API_BASE_URL = 'http://localhost:8080/api/v1';
-
-// Encryption utility functions
-export async function encryptData(data) {
- const response = await axios.post(`${API_BASE_URL}/encryption/encrypt`, {
- data: JSON.stringify(data),
- content_type: 'application/json'
- });
- return response.data.data.encrypted_data;
-}
-
-export async function decryptData(encryptedData) {
- const response = await axios.post(`${API_BASE_URL}/encryption/decrypt`, {
- encrypted_data: encryptedData,
- content_type: 'application/json'
- });
- return JSON.parse(response.data.data.decrypted_data);
-}
-
-// Encrypted API request
-export async function encryptedRequest(endpoint, method = 'GET', data = null) {
- const config = {
- headers: {
- 'Content-Type': 'application/json'
- }
- };
-
- if (data) {
- // Encrypt the request data
- const encryptedData = await encryptData(data);
- config.data = encryptedData;
- config.headers['X-Encrypted-Request'] = 'true';
- }
-
- const response = await axios({
- method,
- url: `${API_BASE_URL}${endpoint}`,
- ...config
- });
-
- // Check if response is encrypted
- if (response.headers['x-encrypted-response'] === 'true') {
- return decryptData(response.data);
- }
-
- return response.data;
-}
-
-// Usage example
-async function getUsers() {
- try {
- const users = await encryptedRequest('/users', 'GET');
- console.log('Users:', users);
- } catch (error) {
- console.error('Request failed:', error);
- }
-}
-```
-
-### Python Client Example
-
-```python
-import requests
-import json
-import base64
-
-API_BASE_URL = "http://localhost:8080/api/v1"
-
-def encrypt_data(data):
- response = requests.post(
- f"{API_BASE_URL}/encryption/encrypt",
- json={
- "data": json.dumps(data),
- "content_type": "application/json"
- }
- )
- return response.json()["data"]["encrypted_data"]
-
-def decrypt_data(encrypted_data):
- response = requests.post(
- f"{API_BASE_URL}/encryption/decrypt",
- json={
- "encrypted_data": encrypted_data,
- "content_type": "application/json"
- }
- )
- return json.loads(response.json()["data"]["decrypted_data"])
-
-def encrypted_request(endpoint, method="GET", data=None):
- headers = {
- "Content-Type": "application/json"
- }
-
- if data:
- encrypted_data = encrypt_data(data)
- headers["X-Encrypted-Request"] = "true"
- data = encrypted_data
-
- response = requests.request(
- method,
- f"{API_BASE_URL}{endpoint}",
- headers=headers,
- json=data if data else None
- )
-
- if response.headers.get("X-Encrypted-Response") == "true":
- return decrypt_data(response.text)
-
- return response.json()
-
-# Usage example
-users = encrypted_request("/users", "GET")
-print("Users:", users)
-```
-
-## Security Best Practices
-
-### Key Management
-
-1. **Key Length**: Always use 32-byte keys for AES-256
-2. **Key Storage**: Store encryption keys in environment variables or secret management systems
-3. **Key Rotation**: Regularly rotate encryption keys (recommended every 24-48 hours)
-4. **Production Keys**: Never commit production keys to version control
-
-### Configuration Recommendations
-
-```yaml
-# Production configuration example
-encryption:
- enabled: true
- algorithm: "aes-256-gcm"
- key: "${ENCRYPTION_KEY}" # Load from environment variable
- rotate_keys: true
- key_rotation_interval: "24h"
-```
-
-### Deployment Checklist
-
-1. ✅ Configure encryption in `config.yaml`
-2. ✅ Set strong encryption key (32 bytes minimum)
-3. ✅ Enable encryption middleware
-4. ✅ Test encryption endpoints
-5. ✅ Update client applications to use encrypted requests
-6. ✅ Monitor encryption service status
-7. ✅ Implement key rotation schedule
-
-## Troubleshooting
-
-### Common Issues
-
-**Issue: "Failed to decrypt request body"**
-- **Cause**: Invalid encryption key or corrupted data
-- **Solution**: Verify encryption key matches between client and server
-
-**Issue: "X-Encrypted-Request header missing"**
-- **Cause**: Client not setting encryption header
-- **Solution**: Ensure client sets `X-Encrypted-Request: true` header
-
-**Issue: "Content type not supported"**
-- **Cause**: Trying to encrypt non-JSON content
-- **Solution**: Only encrypt JSON requests/responses
-
-**Issue: "Encrypted data too short"**
-- **Cause**: Invalid or truncated encrypted data
-- **Solution**: Check data integrity and encryption process
-
-### Debugging Tips
-
-1. **Check Headers**: Verify `X-Encrypted-Request` and `X-Encrypted-Response` headers
-2. **Validate Keys**: Ensure encryption keys match between client and server
-3. **Test Endpoints**: Use `/encryption/status` to verify service health
-4. **Enable Debug Logging**: Set `app.debug: true` for detailed logs
-
-## Performance Considerations
-
-- **Overhead**: AES-256-GCM adds minimal processing overhead (~1-5ms per request)
-- **Caching**: Consider caching frequently accessed encrypted responses
-- **Batch Processing**: For bulk operations, encrypt/decrypt data in batches
-- **Key Rotation**: Schedule key rotation during low-traffic periods
-
-## Migration Guide
-
-### From Unencrypted to Encrypted API
-
-1. **Phase 1: Prepare Infrastructure**
- - Configure encryption in development environment
- - Test encryption endpoints
- - Update client libraries
-
-2. **Phase 2: Dual Mode Operation**
- - Enable encryption middleware
- - Support both encrypted and unencrypted requests
- - Gradually migrate clients
-
-3. **Phase 3: Full Encryption**
- - Enforce encryption for all requests
- - Remove unencrypted fallback
- - Monitor performance and errors
-
-### Backward Compatibility
-
-The encryption feature is designed to be backward compatible:
-
-- **Disabled by Default**: Encryption is opt-in via configuration
-- **Graceful Degradation**: System continues to work if encryption fails
-- **Selective Encryption**: Critical endpoints can be encrypted while others remain unencrypted
-
-## Advanced Configuration
-
-### Custom Encryption Algorithms
-
-While AES-256-GCM is recommended, you can implement custom algorithms:
-
-```go
-// Custom encryption service implementation
-type CustomEncryptionService struct {
- // Implement custom encryption logic
-}
-
-// Register custom service
-registry.Register(modules.NewServiceEWithCustomAlgorithm(
- s.config.Encryption.Enabled,
- "custom-algorithm",
- customEncryptionLogic
-))
-```
-
-### Performance Optimization
-
-For high-throughput applications:
-
-```yaml
-# Performance-tuned configuration
-encryption:
- enabled: true
- algorithm: "aes-256-gcm"
- key: "${ENCRYPTION_KEY}"
- # Consider hardware-accelerated encryption if available
- use_hardware_acceleration: true
-```
-
-## Monitoring and Observability
-
-### Metrics
-
-The encryption service exposes the following metrics:
-
-- **Encryption Requests**: Count of encrypted requests
-- **Decryption Requests**: Count of decrypted requests
-- **Key Rotations**: Count of key rotation operations
-- **Encryption Latency**: Time taken for encryption operations
-- **Decryption Latency**: Time taken for decryption operations
-
-### Logging
-
-Encryption-related events are logged with the following structure:
-
-```json
-{
- "level": "info",
- "message": "Encrypted request processed",
- "path": "/api/v1/users",
- "method": "POST",
- "algorithm": "aes-256-gcm",
- "latency_ms": 2.4
-}
-```
-
-## Compliance and Standards
-
-- **GDPR**: Meets data protection requirements for personal data
-- **HIPAA**: Suitable for healthcare data encryption
-- **PCI DSS**: Compliant with payment card industry standards
-- **NIST**: Follows NIST recommendations for cryptographic standards
\ No newline at end of file
diff --git a/docs_wiki/ERROR_HANDLING.md b/docs_wiki/ERROR_HANDLING.md
deleted file mode 100644
index 1488bbf..0000000
--- a/docs_wiki/ERROR_HANDLING.md
+++ /dev/null
@@ -1,206 +0,0 @@
-# HTTP Error Handling
-
-## Overview
-
-This document describes the custom HTTP error handling implementation that ensures all error responses are returned in a consistent JSON format rather than the default Echo HTML responses.
-
-## Custom Error Handler
-
-The server implements a custom HTTP error handler located in `internal/server/server.go`. This handler intercepts all HTTP errors and converts them to standardized JSON responses.
-
-### Key Features
-
-- All 404 (Not Found) errors return a specific JSON response with incident tracking
-- All HTTP errors return JSON instead of HTML
-- Non-HTTP errors are caught and return a 500 Internal Server Error
-- Error details include correlation ID for debugging and tracking
-
-## 404 Not Found Response
-
-When a request is made to an endpoint that does not exist, the server returns the following JSON response:
-
-### Response Format
-
-```json
-{
- "success": false,
- "status": 404,
- "error": {
- "code": "ENDPOINT_NOT_FOUND",
- "message": "Endpoint not found. This incident will be reported.",
- "details": {
- "path": "/api/v1/unknown-path",
- "method": "GET"
- }
- },
- "timestamp": 1734235788,
- "datetime": "2024-12-15T10:09:48+07:00",
- "correlation_id": "550e8400-e29b-41d4-a716-446655440000"
-}
-```
-
-### Response Fields
-
-| Field | Type | Description |
-|-------|------|-------------|
-| `success` | boolean | Always `false` for error responses |
-| `status` | integer | HTTP status code (404) |
-| `error.code` | string | Error code identifier (`ENDPOINT_NOT_FOUND`) |
-| `error.message` | string | Human-readable error message |
-| `error.details.path` | string | The requested URL path that was not found |
-| `error.details.method` | string | The HTTP method used (GET, POST, etc.) |
-| `timestamp` | integer | Unix timestamp of the response |
-| `datetime` | string | ISO8601 formatted datetime |
-| `correlation_id` | string | Unique request ID for tracking and debugging |
-
-## Other HTTP Errors
-
-For other HTTP errors (400, 401, 403, 405, 500, etc.), the server returns a simplified JSON response:
-
-```json
-{
- "success": false,
- "status": 405,
- "error": {
- "code": "HTTP_ERROR",
- "message": "Method Not Allowed"
- },
- "timestamp": 1734235788,
- "datetime": "2024-12-15T10:09:48+07:00",
- "correlation_id": "550e8400-e29b-41d4-a716-446655440000"
-}
-```
-
-## Internal Server Errors
-
-For unexpected non-HTTP errors, the server returns a 500 Internal Server Error:
-
-```json
-{
- "success": false,
- "status": 500,
- "error": {
- "code": "INTERNAL_ERROR",
- "message": "An unexpected error occurred"
- },
- "timestamp": 1734235788,
- "datetime": "2024-12-15T10:09:48+07:00",
- "correlation_id": "550e8400-e29b-41d4-a716-446655440000"
-}
-```
-
-## Error Codes Reference
-
-| HTTP Status | Error Code | Description |
-|-------------|------------|-------------|
-| 400 | `BAD_REQUEST` | Invalid request parameters |
-| 401 | `UNAUTHORIZED` | Authentication required |
-| 403 | `FORBIDDEN` | Access denied |
-| 404 | `ENDPOINT_NOT_FOUND` | Requested endpoint does not exist |
-| 404 | `NOT_FOUND` | Requested resource not found (used in handlers) |
-| 405 | `HTTP_ERROR` | Method not allowed |
-| 409 | `CONFLICT` | Resource conflict |
-| 422 | `VALIDATION_ERROR` | Request validation failed |
-| 500 | `INTERNAL_ERROR` | Internal server error |
-| 503 | `SERVICE_UNAVAILABLE` | Service temporarily unavailable |
-
-## Implementation Details
-
-### Location
-
-The error handler is implemented in the `New()` function in `internal/server/server.go`:
-
-```go
-e.HTTPErrorHandler = func(err error, c echo.Context) {
- l.Error("HTTP Error", err)
-
- // Handle HTTP errors with JSON response
- if he, ok := err.(*echo.HTTPError); ok {
- var message string
- code := he.Code
-
- // Custom message for 404 Not Found
- if code == 404 {
- message = "Endpoint not found. This incident will be reported."
- response.Error(c, code, "ENDPOINT_NOT_FOUND", message, map[string]interface{}{
- "path": c.Request().URL.Path,
- "method": c.Request().Method,
- })
- return
- }
-
- // For other HTTP errors, use the original message if it's a string
- if msg, ok := he.Message.(string); ok {
- message = msg
- } else {
- message = "An unexpected error occurred"
- }
- response.Error(c, code, "HTTP_ERROR", message)
- return
- }
-
- // For non-HTTP errors, return internal server error
- response.InternalServerError(c, "An unexpected error occurred")
-}
-```
-
-### Logging
-
-All HTTP errors are logged with the following information:
-
-- Error message
-- Stack trace (if available)
-- Request context
-
-Logs can be viewed in the monitoring interface when monitoring is enabled.
-
-## Testing
-
-### Test 404 Response
-
-```bash
-# Request to unknown endpoint
-curl -X GET http://localhost:8080/api/v1/unknown-endpoint
-
-# Expected response
-{
- "success": false,
- "status": 404,
- "error": {
- "code": "ENDPOINT_NOT_FOUND",
- "message": "Endpoint not found. This incident will be reported.",
- "details": {
- "path": "/api/v1/unknown-endpoint",
- "method": "GET"
- }
- },
- "timestamp": 1734235788,
- "datetime": "2024-12-15T10:09:48+07:00",
- "correlation_id": "..."
-}
-```
-
-### Test Method Not Allowed
-
-```bash
-# POST request to GET-only endpoint
-curl -X POST http://localhost:8080/health
-
-# Expected response
-{
- "success": false,
- "status": 405,
- "error": {
- "code": "HTTP_ERROR",
- "message": "Method Not Allowed"
- },
- "timestamp": 1734235788,
- "datetime": "2024-12-15T10:09:48+07:00",
- "correlation_id": "..."
-}
-```
-
-## Related Documentation
-
-- [API Response Structure](./API_RESPONSE_STRUCTURE.md) - Detailed response format documentation
-- [Integration Guide](./INTEGRATION_GUIDE.md) - How to handle errors in client applications
diff --git a/docs_wiki/EVENT_STREAMING.md b/docs_wiki/EVENT_STREAMING.md
deleted file mode 100644
index b5c343c..0000000
--- a/docs_wiki/EVENT_STREAMING.md
+++ /dev/null
@@ -1,828 +0,0 @@
-# Live Event Streaming Documentation
-
-## Overview
-
-The Event Streaming System provides comprehensive real-time event streaming capabilities through a dual-implementation demonstration showcasing different architectural approaches:
-
-### Service H - Event Streaming Showcase
-**Dual-implementation demonstration** (`internal/services/modules/service_h.go`) showcasing both full implementation and utility approaches:
-- **Full Implementation**: Complete event streaming with all broadcasting logic included
-- **Utility Demo**: Clean implementation using `pkg/utils/broadcast.go`
-- Two sets of API endpoints for comparison
-- Educational example of different architectural patterns
-
-### Broadcast Utility (`pkg/utils/broadcast.go`)
-**Reusable broadcasting component** extracted for maximum reusability:
-- Clean, well-documented utility for any service to use
-- Thread-safe operations with proper synchronization
-- Enhanced methods for monitoring and management
-- Easy to integrate: just `utils.NewEventBroadcaster()`
-
-## Service Comparison
-
-| Implementation | Approach | API Prefix | Lines of Code | Benefits |
-|----------------|----------|------------|---------------|----------|
-| **Service H (Utility)** | Uses `pkg/utils/broadcast.go` | `/events/` | ~150 lines | Clean, simple, easy to understand |
-
-Service H demonstrates how easy it is to implement event streaming using the broadcast utility.
-
-## Key Features
-
-- **Multiple Event Streams**: Support for concurrent event streams with independent client subscriptions
-- **Server-Sent Events (SSE)**: Standards-compliant real-time push notifications
-- **Event Broadcasting**: Send events to specific streams or broadcast to all streams
-- **Automated Stream Generators**: Background processes generating sample events for demonstration
-- **Stream Management**: Dynamic start, stop, pause, and resume operations for streams
-- **Client Management**: Automatic subscription/unsubscription with buffered channels
-- **Thread-Safe Operations**: Concurrent-safe event broadcasting and client management
-
-## Architecture
-
-### Core Components
-
-#### 1. EventBroadcaster Utility (`pkg/utils/broadcast.go`)
-The broadcast functionality has been extracted into a reusable utility in `pkg/utils/broadcast.go`. This makes it easy for any service to implement event streaming without duplicating code.
-
-**Usage in Service G:**
-```go
-broadcaster := utils.NewEventBroadcaster()
-
-// Subscribe to a stream
-client := broadcaster.Subscribe("my-stream")
-
-// Broadcast to a stream
-broadcaster.Broadcast("my-stream", "event_type", "message", data)
-
-// Broadcast to all streams
-broadcaster.BroadcastToAll("global_event", "message", data)
-```
-
-**Core Utility Types:**
-```go
-type EventBroadcaster struct {
- streams map[string][]*StreamClient // streamID -> clients
- clients map[string]*StreamClient // clientID -> client
- mu sync.RWMutex
- nextID int
- clientTTL time.Duration
-}
-
-type EventData struct {
- ID string `json:"id"`
- Type string `json:"type"`
- Message string `json:"message"`
- Data map[string]interface{} `json:"data,omitempty"`
- Timestamp int64 `json:"timestamp"`
- StreamID string `json:"stream_id,omitempty"`
-}
-```
-
-#### 2. StreamClient
-Represents a connected client for a specific event stream.
-
-```go
-type StreamClient struct {
- ID string
- StreamID string
- Channel chan EventData // Buffered channel (100 messages)
-}
-```
-
-#### 3. EventData
-Standardized event data structure sent to clients.
-
-```go
-type EventData struct {
- ID string `json:"id"`
- Type string `json:"type"`
- Message string `json:"message"`
- Data map[string]interface{} `json:"data,omitempty"`
- Timestamp int64 `json:"timestamp"`
- StreamID string `json:"stream_id,omitempty"`
-}
-```
-
-#### 4. StreamGenerator
-Manages automated event generation for demonstration streams.
-
-```go
-type StreamGenerator struct {
- streamID string
- broadcaster *EventBroadcaster
- running bool
- paused bool
- stopChan chan struct{}
- pauseChan chan struct{}
- mu sync.RWMutex
-}
-```
-
-## API Endpoints
-
-### Stream Subscription
-
-#### GET `/api/v1/events/stream/{stream_id}`
-Subscribe to a real-time event stream using Server-Sent Events.
-
-**Parameters:**
-- `stream_id` (path): The ID of the stream to subscribe to
-
-**Response Headers:**
-```
-Content-Type: text/event-stream
-Cache-Control: no-cache
-Connection: keep-alive
-Access-Control-Allow-Origin: *
-Access-Control-Allow-Headers: Cache-Control
-```
-
-**Event Format:**
-```json
-data: {"id":"evt_123456789","type":"user_action","message":"User logged in","data":{"user_id":"12345","action":"login"},"timestamp":1642598400,"stream_id":"default"}
-
-data: {"id":"evt_123456790","type":"system_alert","message":"High CPU usage detected","data":{"cpu_percent":85.5,"threshold":80},"timestamp":1642598401,"stream_id":"system"}
-
-```
-
-**Example Usage (JavaScript):**
-```javascript
-const eventSource = new EventSource('/api/v1/events/stream/default');
-
-eventSource.onmessage = function(event) {
- const eventData = JSON.parse(event.data);
- console.log('Received event:', eventData);
-
- switch(eventData.type) {
- case 'user_action':
- handleUserAction(eventData);
- break;
- case 'system_alert':
- handleSystemAlert(eventData);
- break;
- case 'notification':
- showNotification(eventData);
- break;
- }
-};
-
-eventSource.onerror = function(error) {
- console.error('EventSource failed:', error);
- eventSource.close();
-};
-```
-
-**Example Usage (curl):**
-```bash
-curl -N -H "Accept: text/event-stream" \
- -H "Cache-Control: no-cache" \
- http://localhost:8080/api/v1/events/stream/default
-```
-
-### Event Broadcasting
-
-#### POST `/api/v1/events/broadcast`
-Broadcast an event to a specific stream or all streams.
-
-**Request Body:**
-```json
-{
- "stream_id": "notifications", // Optional: empty string broadcasts to all streams
- "type": "custom_event",
- "message": "Custom notification message",
- "data": {
- "priority": "high",
- "sender": "admin",
- "recipients": ["user1", "user2"]
- }
-}
-```
-
-**Success Response:**
-```json
-{
- "success": true,
- "message": "Event broadcasted to stream: notifications",
- "timestamp": 1642598400
-}
-```
-
-**Broadcast to All Streams:**
-```json
-{
- "type": "system_maintenance",
- "message": "System maintenance scheduled",
- "data": {
- "maintenance_window": "2024-01-15T02:00:00Z",
- "duration_minutes": 30
- }
-}
-```
-
-### Stream Information
-
-#### GET `/api/v1/events/streams`
-Retrieve information about active streams and their connected clients.
-
-**Response:**
-```json
-{
- "success": true,
- "data": {
- "default": {
- "clients": 5,
- "active": true,
- "generator": {
- "running": true,
- "paused": false
- }
- },
- "system": {
- "clients": 2,
- "active": true,
- "generator": {
- "running": true,
- "paused": false
- }
- },
- "notifications": {
- "clients": 0,
- "active": true,
- "generator": {
- "running": false,
- "paused": false
- }
- }
- },
- "timestamp": 1642598400
-}
-```
-
-### Stream Management
-
-#### POST `/api/v1/events/stream/{stream_id}/start`
-Start or restart a stream generator.
-
-**Response:**
-```json
-{
- "success": true,
- "message": "Stream 'system' started",
- "timestamp": 1642598400
-}
-```
-
-#### POST `/api/v1/events/stream/{stream_id}/stop`
-Stop a stream generator and remove it.
-
-**Response:**
-```json
-{
- "success": true,
- "message": "Stream 'system' stopped and removed",
- "timestamp": 1642598400
-}
-```
-
-#### POST `/api/v1/events/stream/{stream_id}/pause`
-Pause a running stream generator.
-
-**Response:**
-```json
-{
- "success": true,
- "message": "Stream 'system' paused",
- "timestamp": 1642598400
-}
-```
-
-#### POST `/api/v1/events/stream/{stream_id}/resume`
-Resume a paused stream generator.
-
-**Response:**
-```json
-{
- "success": true,
- "message": "Stream 'system' resumed",
- "timestamp": 1642598400
-}
-```
-
-## Default Streams
-
-The service automatically starts four default streams with sample event generators:
-
-### 1. `default` Stream
-**Purpose:** General events and notifications
-**Sample Events:**
-- User actions (login, logout, profile updates)
-- General system notifications
-- Application-specific events
-
-### 2. `system` Stream
-**Purpose:** System-level alerts and metrics
-**Sample Events:**
-- High CPU/memory usage alerts
-- Disk space warnings
-- Service health status updates
-- System performance metrics
-
-### 3. `user-activity` Stream
-**Purpose:** User action events
-**Sample Events:**
-- User authentication events
-- Profile modifications
-- Permission changes
-- User-generated content updates
-
-### 4. `notifications` Stream
-**Purpose:** Application notifications
-**Sample Events:**
-- Push notifications
-- Alert messages
-- Scheduled reminders
-- System announcements
-
-## Event Types
-
-### Predefined Event Types
-
-- **`user_action`**: User-initiated actions (login, update profile, etc.)
-- **`system_alert`**: System warnings and alerts (high CPU, low disk space)
-- **`data_update`**: Database or data changes
-- **`notification`**: General notifications and messages
-- **`metric_update`**: System or application metrics
-- **`stream_started`**: Stream initialization events
-- **`connection`**: Client connection events
-
-### Custom Event Types
-
-You can define and use any custom event types in your applications:
-
-```json
-{
- "type": "order_created",
- "message": "New order received",
- "data": {
- "order_id": "ORD-12345",
- "customer_id": "CUST-67890",
- "total_amount": 299.99,
- "items": ["widget-a", "widget-b"]
- }
-}
-```
-
-## Usage Examples
-
-### Frontend Integration (React)
-
-```javascript
-import { useEffect, useState } from 'react';
-
-function EventStreamComponent({ streamId }) {
- const [events, setEvents] = useState([]);
- const [isConnected, setIsConnected] = useState(false);
-
- useEffect(() => {
- const eventSource = new EventSource(`/api/v1/events/stream/${streamId}`);
-
- eventSource.onopen = () => {
- setIsConnected(true);
- };
-
- eventSource.onmessage = (event) => {
- const eventData = JSON.parse(event.data);
- setEvents(prev => [...prev.slice(-9), eventData]); // Keep last 10 events
- };
-
- eventSource.onerror = (error) => {
- console.error('EventSource error:', error);
- setIsConnected(false);
- };
-
- return () => {
- eventSource.close();
- };
- }, [streamId]);
-
- return (
-
-
Connection Status: {isConnected ? '🟢 Connected' : '🔴 Disconnected'}
-
- {events.map((event, index) => (
-
-
{event.type}: {event.message}
- {event.data &&
{JSON.stringify(event.data, null, 2)}}
-
- ))}
-
-
+