backer/llms.txt

# llms.txt for Backer

## Overview
Backer is a Go-based backup utility that creates compressed archives on-the-fly and serves them via HTTP/HTTPS with basic authentication. Supports gzip, pgzip, bzip2, zstd, lz4, xz compression.

## Quick Start
```bash
make                                    # Build binary
make build VERSION=0.1.0                # Build with version
backer                                  # Run with default config (/etc/backer.json)
backer -c /path/to/config.json          # Run with custom config
backer --version                        # Show version
backer -h                               # Show help
curl -u user:pass https://server:8086/archive -o backup.tar.gz
```

## Config File Locations
- Linux: `/etc/backer.json`
- FreeBSD/NetBSD/OpenBSD/DragonFly: `/usr/local/etc/backer.json`
- Override with `-c` flag

## Directory Structure
```
/
├── cmd/backer/main.go       # Entry point, CLI flags, server startup
├── internal/
│   ├── backer/              # Core logic
│   │   ├── types.go         # Config struct definition
│   │   ├── consts.go        # Named constants for magic numbers
│   │   ├── globals.go       # Global C Config variable
│   │   ├── archive.go       # Common archive logic (createArchiveStream, writeFilesToTar)
│   │   ├── gzip.go          # CreateTarGzStream, CreateTarPgzipStream
│   │   ├── bzip2.go         # CreateTarBzip2Stream, mapCompressionLevelToBzip2
│   │   ├── zstd.go          # CreateTarZstdStream, zstdLevel
│   │   ├── lz4.go           # CreateTarLz4Stream, mapCompressionLevelToLz4
│   │   ├── xz.go            # CreateTarXzStream
│   │   ├── loadConfig.go    # HJSON/JSON config parsing with validation
│   │   ├── newServer.go     # HTTP server, auth handler, context helpers
│   │   ├── getFilesFromDirectories.go  # Directory walking, exclusion filtering, isExcluded
│   │   └── *_test.go        # Test files
│   └── log/main.go          # Logging wrapper around stdlib slog
├── contrib/                 # Init scripts
│   ├── backer               # OpenRC (Alpine/Gentoo)
│   ├── backer.service       # systemd
│   └── backer.freebsd       # FreeBSD rc.d
├── test_data/               # Test fixtures
├── data/config_example.json  # Example config
├── Makefile                 # Build targets
├── README.md                # User documentation
└── LICENSE                  # MIT License
```

## Config Options
| Option | Default | Description |
|--------|---------|-------------|
| address | 0.0.0.0 | Listen address |
| port | 8086 | Listen port |
| cert/key | - | TLS cert/key paths (required when nohttps=false) |
| nohttps | false | Disable HTTPS |
| location | /archive | API endpoint path |
| user | - | Basic auth username (required) |
| password | - | Basic auth password (required) |
| log | stderr | Log output |
| loglevel | info | Log verbosity (debug/info/warn/error) |
| directories | - | Directories to backup (required) |
| backup_timeout | 60 | Stream timeout in minutes (1-1440) |
| dir_scan_timeout | 60 | Directory scan timeout in minutes (1-1440) |
| compression_level | 9 | Compression level (1-9) |
| exclude_patterns | [] | Regex patterns to exclude from backup |
| filename_prefix | backup | Prefix for backup filename in Content-Disposition header |
| default_compression | gzip | Default compression algorithm (gzip, pgzip, bzip2, zstd, lz4, xz) |
| compression_algorithm | gzip | Alias for default_compression (deprecated) |

## Compression Level Mapping

The `compression_level` config option (1-9) is mapped differently for each algorithm:

| Algorithm | Level Mapping |
|-----------|---------------|
| gzip/pgzip/bzip2 | 1-9 directly maps to compression levels |
| lz4 | 1-3 → Fastest (0), 4-6 → Default (1), 7-9 → Best (2) |
| zstd | 1-3 → Fastest, 4-6 → Default, 7-9 → BetterCompression |
| xz | Uses default compression (no configurable level) |

## Key Patterns

### Global Config
Intentional design - uses global `C` variable:
```go
var C Config  // internal/backer/globals.go
```

### Config Parsing
HJSON requires double conversion (hjson → map → json → struct):
```go
hjson.Unmarshal(buf, &tmp)  // tmp is map[string]any
json.Marshal(tmp)
json.Unmarshal(buf, &C)
```

### Context-Aware I/O
Two helper functions in newServer.go handle context cancellation during streaming:
- `writeWithContext(ctx, fn)` — checks context before calling fn
- `copyWithContext(ctx, dst, src)` — buffered copy (32KB) with cancellation checks, computes MD5 hash

```go
func writeWithContext(ctx context.Context, fn func() error) error
func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) (int64, string, error)
```

### HTTP Server Middleware
Two middleware functions wrap the handler:
- `withServerHeader` — adds "Server: backer" header to all responses
- `withStatusLogging` — logs 404 and 405 responses

```go
Handler: withServerHeader(withStatusLogging(mux))
```

### Unmatched Route Logging
The server wraps http.ResponseWriter to capture status codes:
- **404 Not Found**: `"Not Found from {clientIP} to {path}"`
- **405 Method Not Allowed**: `"Method Not Allowed from {clientIP} for {method} {path}"`

Uses `responseWriterWrapper` that tracks status code via WriteHeader() method.

### Pipe Error Handling
Uses `pipeErr` tracking to avoid noisy errors when reader closes early.
Skips files that can't be read (logs warning, continues backup).
- `isPipeClosedError()` checks for "io: read/write on closed pipe" errors

### Content-Disposition Filename
Backup filenames include timestamp and configured prefix: `backup-20060102-150405.tar.gz` (extension depends on compression_algorithm).

### Special File Types
- **Regular files**: streamed into tar archive with content
- **Directories**: included as tar directory entries
- **Symlinks**: stored as symlink entries (target path preserved) - uses os.Lstat, no cycles possible
- **Broken symlinks**: archived with their target path (even if target doesn't exist)
- **Device files** (char/block): included with correct major/minor numbers
- **Named pipes (FIFOs)**: included (header only, no data)
- **Sockets**: skipped (not archivable)
- **Hard links**: deduplicated - first occurrence stored with content, subsequent as link entries
- **GNU tar format**: uses `header.Format = tar.FormatGNU` for GNU tar compatibility

### Test Coverage
All entity types are tested: regular files, empty files, directories, symlinks, broken symlinks, symlinks to directories, hard links, device files, named pipes (FIFOs), special characters in filenames.

### Error Recovery
- `writeWithContext` checks context before calling fn.
- `copyWithContext` does buffered copy with cancellation checks.

### Client IP Detection
Checks `X-Forwarded-For` header first (for proxied requests), then falls back to `RemoteAddr`.

### TLS Error Handling
Uses `ServerWrapper` to intercept and classify errors from Serve() and ServeTLS():
- TLS-specific errors logged at debug level to reduce log clutter
- Uses errors.As to detect: tls.AlertError, tls.CertificateVerificationError, tls.ECHRejectionError, tls.RecordHeaderError

### HTTP Routing & Compression Selection
The server uses http.NewServeMux() for routing. Multiple routes support different compression formats:
```go
mux.Handle(C.Location, http.HandlerFunc(backupHandler))           // /archive
mux.Handle(C.Location+".tar.gz", http.HandlerFunc(backupHandler))  // archive.tar.gz
mux.Handle(C.Location+".tar.xz", http.HandlerFunc(backupHandler))   // archive.tar.xz
mux.Handle(C.Location+".tar.bz2", http.HandlerFunc(backupHandler)) // archive.tar.bz2
// ... etc (.tar.lz4, .tar.zst, .tar.zstd)
```

Compression algorithm determined by path extension using `path.Ext()`:
- Full path: `/archive.tar.xz` → `.xz` → xz
- Bare extension: `tar.xz` → `.xz` → xz
- Default when no extension matches

### Performance Notes
- Exclude patterns are precompiled after config load for faster matching.
- Compression level is configurable (1-9); default 9 uses more CPU.
- Large directory walks use `filepath.WalkDir` (efficient).
- Context cancellation checked on each file during directory scan.

## Coding Style & Conventions

### Function Naming
```go
// Exported (public): VerbNoun pattern
NewServer()
LoadConfig()
CreateTarGzStream()
GetFilesFromDirectories()

// Unexported (private): verbNoun or isNoun pattern
writeWithContext()
copyWithContext()
isPipeClosedError()
isExcluded()
defaultConfigPath()
```

### JSON Tags
Struct fields use PascalCase, json tags use snake_case:
```go
BackupTimeout     int `json:"backup_timeout"`
CompressionLevel  int `json:"compression_level"`
DirScanTimeout    int `json:"dir_scan_timeout"`
```

### Error Messages
- Start with capital letter (ST1005 disabled)
- Use past tense: `"Failed to..."`, `"Unable to..."`
- Include context: `fmt.Errorf("Failed to close %s: %w", name, err)`
- Use `%v` for errors (not `%s`)

### Logging
- Past tense: `"Failed to close gzip writer: %v", err`
- Context included: `"Skipping %s: %v", filename, err`
- Appropriate level: Warn for recoverable, Error for issues, Fatal for startup failures
- Log levels: error, warn, info, debug

### Config Validation Pattern
For each config option: set default first, then immediately validate:
```go
// Port: set default, then validate range.
if C.Port == 0 {
    C.Port = 8086
    log.Warnf("Config option port is not set, fallback to %d", C.Port)
}

if C.Port < 1 || C.Port > 65535 {
    return fmt.Errorf("Config option port must be between 1 and 65535, got %d", C.Port)
}
```

### Go Formatting
- Tabs for indentation
- Vim modeline at EOF: `/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

### Linter Suppressions (intentional)
| Rule | Reason |
|------|--------|
| ST1005 | Capitalized error messages for readability |
| QF1001 | De Morgan's law disabled for auth condition clarity |
| revive (error strings) | Same reason as ST1005 |

### Comment Style
- Profanity allowed (pet project)
- Use `//nolint:` only when absolutely necessary

## Testing
```bash
go test ./...                    # Run tests (runs make clean first)
go test -v -run TestName ./...   # Run specific test
golangci-lint run ./...          # Lint
make test                        # Clean + test (deterministic)
```

Coverage: ~64% on internal/backer (~100% on internal/log)

### Test Data
All test data is created by `TestMain` in `internal/backer/main_test.go` before tests run and cleaned up afterward. This ensures tests are independent of pre-existing files.

- `test_data/test1/foo/` — test directory with files, subdirs, empty_dir
- `test_data/test1/bar/` — test directory with files and a directory named `goodbye.txt/` (edge case: directory with file extension)
- `test_data/test1/hardlinks/` — test directory with hard-linked files (original.txt, hardlink1.txt, hardlink2.txt)
- `test_data/test1/symlinks/` — test directory with symbolic links
- `test_data/test1/empty_dir/` — empty directory for testing
- `test_data/tmp/` — reserved for test temporary files (use instead of system `/tmp`)

### Test Conventions
- Reset global config: `C = Config{}`
- Reset exclude patterns: `excludePatterns = nil`
- Use `t.TempDir()` for temporary files
- Helper function `writeTempConfig()` for config tests
- Test data created by TestMain setup, cleaned after tests
- Always check errors with meaningful diagnostics (use `t.Fatalf` for setup errors)

## Build
```bash
make           # Clean + build binary (version from git or "dev")
make build VERSION=0.1.0  # Build with specific version
make clean     # Remove binary, test_data/tmp, test_data/test1
make test      # Clean + test (deterministic)
make upgrade   # Update dependencies
make help      # Show targets
```

### Versioning
Version is injected at build time via ldflags:
```go
var version = "dev"  // overridden by: -X main.version=$(VERSION)
```

## Request Flow (Good Path)
```
Request → Method Check (GET) → Auth Check → Get Files (with timeout) → Create Archive → Stream → Complete
```

## Request Flow (Error Paths)
| Request | Response | Logged |
|---------|----------|--------|
| Non-GET method | 405 Method Not Allowed | Yes (info) |
| No auth | 401 Unauthorized | Yes (info) |
| Wrong credentials | 401 Unauthorized | Yes (info) |
| Unknown path | 404 Not Found | Yes (info) |
| Dir scan timeout | 504 Request Timeout | Yes (error) |
| Failed to get files | 500 Internal Server Error | Yes (error) |
| Client disconnect | - | Yes (info) |
| Backup completed | 200 OK | Yes (info) |