backer/llms.txt

# llms.txt for Backer

## Overview
Backer is a Go-based backup utility that creates compressed archives on-the-fly and serves them via HTTP/HTTPS with basic authentication. Supports gzip, pgzip, bzip2, zstd, lz4, xz compression.

## Quick Start
```bash
make                                    # Build binary
make build VERSION=0.1.0                # Build with version
backer                                  # Run with default config (/etc/backer.json)
backer -c /path/to/config.json          # Run with custom config
backer --version                        # Show version
backer -h                               # Show help
curl -u user:pass https://server:8086/archive -o backup.tar.gz
```

## Config File Locations
- Linux: `/etc/backer.json`
- FreeBSD/NetBSD/OpenBSD/DragonFly: `/usr/local/etc/backer.json`
- Override with `-c` flag

## Directory Structure
```
/
├── cmd/backer/main.go       # Entry point, CLI flags, server startup
├── internal/
│   ├── backer/              # Core logic
│   │   ├── types.go         # Config struct definition
│   │   ├── consts.go        # Named constants for magic numbers
│   │   ├── globals.go       # Global C Config variable
│   │   ├── archive.go       # Common archive logic (createArchiveStream, writeFilesToTar)
│   │   ├── gzip.go          # CreateTarGzStream, CreateTarPgzipStream
│   │   ├── bzip2.go         # CreateTarBzip2Stream, mapCompressionLevelToBzip2
│   │   ├── zstd.go          # CreateTarZstdStream, zstdLevel
│   │   ├── lz4.go           # CreateTarLz4Stream, mapCompressionLevelToLz4
│   │   ├── xz.go            # CreateTarXzStream
│   │   ├── loadConfig.go    # HJSON/JSON config parsing with validation
│   │   ├── newServer.go     # HTTP server, auth handler, context helpers
│   │   ├── getFilesFromDirectories.go  # Directory walking, exclusion filtering, isExcluded
│   │   └── *_test.go        # Test files
│   └── log/main.go          # Logging wrapper around stdlib slog
├── contrib/                 # Init scripts
│   ├── backer               # OpenRC (Alpine/Gentoo)
│   ├── backer.service       # systemd
│   └── backer.freebsd       # FreeBSD rc.d
├── test_data/               # Test fixtures
├── data/config_example.json  # Example config
├── Makefile                 # Build targets
├── README.md                # User documentation
└── LICENSE                  # MIT License
```

## Config Options
| Option | Default | Description |
|--------|---------|-------------|
| address | 0.0.0.0 | Listen address |
| port | 8086 | Listen port |
| cert/key | - | TLS cert/key paths (required when nohttps=false) |
| nohttps | false | Disable HTTPS |
| location | /archive | API endpoint path |
| user | - | Basic auth username (required) |
| password | - | Basic auth password (required) |
| log | stderr | Log output |
| loglevel | info | Log verbosity (debug/info/warn/error) |
| directories | - | Directories to backup (required) |
| backup_timeout | 60 | Stream timeout in minutes (1-1440) |
| dir_scan_timeout | 60 | Directory scan timeout in minutes (1-1440) |
| compression_level | 9 | Compression level (1-9) |
| exclude_patterns | [] | Regex patterns to exclude from backup |
| filename_prefix | backup | Prefix for backup filename in Content-Disposition header |
| default_compression | gzip | Default compression algorithm (gzip, pgzip, bzip2, zstd, lz4, xz) |
| compression_algorithm | gzip | Alias for default_compression (deprecated) |

## Compression Level Mapping

The `compression_level` config option (1-9) is mapped differently for each algorithm:

| Algorithm | Level Mapping |
|-----------|---------------|
| gzip/pgzip/bzip2 | 1-9 directly maps to compression levels |
| lz4 | 1-3 → Fastest (0), 4-6 → Default (1), 7-9 → Best (2) |
| zstd | 1-3 → Fastest, 4-6 → Default, 7-9 → BetterCompression |
| xz | Uses default compression (no configurable level) |

## Key Patterns

### Global Config
Intentional design - uses global `C` variable:
```go
var C Config  // internal/backer/globals.go
```

### Config Parsing
HJSON requires double conversion (hjson → map → json → struct):
```go
hjson.Unmarshal(buf, &tmp)  // tmp is map[string]any
json.Marshal(tmp)
json.Unmarshal(buf, &C)
```

### Context-Aware I/O
Two helper functions in newServer.go handle context cancellation during streaming:
- `writeWithContext(ctx, fn)` — checks context before calling fn
- `copyWithContext(ctx, dst, src)` — buffered copy (32KB) with cancellation checks, computes MD5 hash

```go
func writeWithContext(ctx context.Context, fn func() error) error
func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) (int64, string, error)
```

### HTTP Server Middleware
Two middleware functions wrap the handler:
- `withServerHeader` — adds "Server: backer" header to all responses
- `withStatusLogging` — logs 404 and 405 responses

```go
Handler: withServerHeader(withStatusLogging(mux))
```

### Unmatched Route Logging
The server wraps http.ResponseWriter to capture status codes:
- **404 Not Found**: `"Not Found from {clientIP} to {path}"`
- **405 Method Not Allowed**: `"Method Not Allowed from {clientIP} for {method} {path}"`

Uses `responseWriterWrapper` that tracks status code via WriteHeader() method.

### Pipe Error Handling
Uses `pipeErr` tracking to avoid noisy errors when reader closes early.
Skips files that can't be read (logs warning, continues backup).
- `isPipeClosedError()` checks for "io: read/write on closed pipe" errors

### Content-Disposition Filename
Backup filenames include timestamp and configured prefix: `backup-20060102-150405.tar.gz` (extension depends on compression_algorithm).

### Special File Types
- **Device files** (character/block) are included with correct major/minor numbers.
- **Sockets** are skipped (no entry created).
- **Symlinks** are stored as symlinks (target path stored) - uses os.Lstat, no cycles possible.
- **Named pipes** and other special files are included (tar header created).
- **Hard links** are deduplicated - first occurrence stored with content, subsequent as link entries.
- **Regular files** are compressed; other file types have no data section.

### Error Recovery
- `writeWithContext` checks context before calling fn.
- `copyWithContext` does buffered copy with cancellation checks.

### Client IP Detection
Checks `X-Forwarded-For` header first (for proxied requests), then falls back to `RemoteAddr`.

### TLS Error Handling
Uses `ServerWrapper` to intercept and classify errors from Serve() and ServeTLS():
- TLS-specific errors logged at debug level to reduce log clutter
- Uses errors.As to detect: tls.AlertError, tls.CertificateVerificationError, tls.ECHRejectionError, tls.RecordHeaderError

### HTTP Routing & Compression Selection
The server uses http.NewServeMux() for routing. Multiple routes support different compression formats:
```go
mux.Handle(C.Location, http.HandlerFunc(backupHandler))           // /archive
mux.Handle(C.Location+".tar.gz", http.HandlerFunc(backupHandler))  // archive.tar.gz
mux.Handle(C.Location+".tar.xz", http.HandlerFunc(backupHandler))   // archive.tar.xz
mux.Handle(C.Location+".tar.bz2", http.HandlerFunc(backupHandler)) // archive.tar.bz2
// ... etc (.tar.lz4, .tar.zst, .tar.zstd)
```

Compression algorithm determined by path extension using `path.Ext()`:
- Full path: `/archive.tar.xz` → `.xz` → xz
- Bare extension: `tar.xz` → `.xz` → xz
- Default when no extension matches

### Performance Notes
- Exclude patterns are precompiled after config load for faster matching.
- Compression level is configurable (1-9); default 9 uses more CPU.
- Large directory walks use `filepath.WalkDir` (efficient).
- Context cancellation checked on each file during directory scan.

## Coding Style & Conventions

### Function Naming
```go
// Exported (public): VerbNoun pattern
NewServer()
LoadConfig()
CreateTarGzStream()
GetFilesFromDirectories()

// Unexported (private): verbNoun or isNoun pattern
writeWithContext()
copyWithContext()
isPipeClosedError()
isExcluded()
defaultConfigPath()
```

### JSON Tags
Struct fields use PascalCase, json tags use snake_case:
```go
BackupTimeout     int `json:"backup_timeout"`
CompressionLevel  int `json:"compression_level"`
DirScanTimeout    int `json:"dir_scan_timeout"`
```

### Error Messages
- Start with capital letter (ST1005 disabled)
- Use past tense: `"Failed to..."`, `"Unable to..."`
- Include context: `fmt.Errorf("Failed to close %s: %w", name, err)`
- Use `%v` for errors (not `%s`)

### Logging
- Past tense: `"Failed to close gzip writer: %v", err`
- Context included: `"Skipping %s: %v", filename, err`
- Appropriate level: Warn for recoverable, Error for issues, Fatal for startup failures
- Log levels: error, warn, info, debug

### Config Validation Pattern
For each config option: set default first, then immediately validate:
```go
// Port: set default, then validate range.
if C.Port == 0 {
    C.Port = 8086
    log.Warnf("Config option port is not set, fallback to %d", C.Port)
}

if C.Port < 1 || C.Port > 65535 {
    return fmt.Errorf("Config option port must be between 1 and 65535, got %d", C.Port)
}
```

### Go Formatting
- Tabs for indentation
- Vim modeline at EOF: `/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

### Linter Suppressions (intentional)
| Rule | Reason |
|------|--------|
| ST1005 | Capitalized error messages for readability |
| QF1001 | De Morgan's law disabled for auth condition clarity |
| revive (error strings) | Same reason as ST1005 |

### Comment Style
- Profanity allowed (pet project)
- Use `//nolint:` only when absolutely necessary

## Testing
```bash
go test ./...                    # Run tests
go test -v -run TestName ./...   # Run specific test
golangci-lint run ./...          # Lint
```

Coverage: ~69% on internal/backer (~100% on internal/log)

### Test Conventions
- Reset global config: `C = Config{}`
- Reset exclude patterns: `excludePatterns = nil`
- Use `t.TempDir()` for temporary files
- Helper function `writeTempConfig()` for config tests

## Build
```bash
make           # Build binary (version from git or "dev")
make build VERSION=0.1.0  # Build with specific version
make clean     # Remove binary
make test      # Run tests
make upgrade   # Update dependencies
make help      # Show targets
```

### Versioning
Version is injected at build time via ldflags:
```go
var version = "dev"  // overridden by: -X main.version=$(VERSION)
```

## Request Flow (Good Path)
```
Request → Method Check (GET) → Auth Check → Get Files (with timeout) → Create Archive → Stream → Complete
```

## Request Flow (Error Paths)
| Request | Response | Logged |
|---------|----------|--------|
| Non-GET method | 405 Method Not Allowed | Yes (info) |
| No auth | 401 Unauthorized | Yes (info) |
| Wrong credentials | 401 Unauthorized | Yes (info) |
| Unknown path | 404 Not Found | Yes (info) |
| Dir scan timeout | 504 Request Timeout | Yes (error) |
| Failed to get files | 500 Internal Server Error | Yes (error) |
| Client disconnect | - | Yes (info) |
| Backup completed | 200 OK | Yes (info) |