backer/AGENTS.md

# AGENTS.md

## Project Overview

Backer is a Go-based HTTP/HTTPS backup server that creates compressed archives on-the-fly from configured directories. Supports multiple compression algorithms: gzip, pgzip, bzip2, zstd, lz4, xz. It serves a single endpoint that clients (e.g., `curl`) can query to download a timestamped backup archive. Licensed under MIT.

## Fundamental Rules

- **Do not apply changes without explicit user consent.** When asked to analyze, review, or suggest any changes, present your proposal first and wait for confirmation before editing any files. Always ask "Should I implement this?" or "Should I apply these changes?" before modifying anything.
- **Keep codebase simple, small, maintainable, safe and human-readable.** Favour minimal, clear solutions over clever or abstract ones. Prefer clarity over cleverness. Prioritize explicitness. Minimize magic. Do not introduce patterns, layers, or indirection unless they directly solve a concrete problem. Every line of code must earn its place.
- **Create small focused modules and separate functionality over files.** Do not make universal implementation; try to move functions to separate files and name files after function names.
- **Use a consistent structure.** Subsequent searches will be faster and consume less context.
- **Avoid duplicate names.** This is not strict rule. If codebase contain multiple files with the same name (e.g., index.ts), we have to read them all to discover the one exactly we need. This consumes time and context.
- **When stuck, ask clarification questions, or present a short plan, or both.** Do not try to guess intent or make assumptions - ask.
- **Always make TODO plan before any job and follow it closely.**
- **Report any found defects, issues, bugs.** And keep track of them.
- **Comment the code.** Add code comments to explain corner cases, complex logic, business rules, and design decisions. Focus on **what** problem the code is solving rather than **how** it is solving it. Write comments in easy to read and understand language, for humans to comprehend. Every public API and structure field must have a docstring.
- **Use domain names.** Use naming conventions that reflect their purpose and domain concepts, not their technical implementation (e.g., OrderProcessor instead of OrderServiceFactory).
- **Test corner cases.** Write comprehensive test suites that cover various scenarios and edge cases.
- **Use tags and labels.** In comments, add tags or labels to files, classes, or functions to categorize them based on functionality, domain concepts, or design patterns.
- **Avoid obvious comments that do not add value.**
- **Remove dead code.** A half-finished refactoring can actually harm the contributions.
- **Update AGENTS.md and TODO.md automatically.** Keep AGENTS.md dense, detailed and concise.

## Workflow Rules

- **Don't use `context` package unless absolutely necessary.** Try to find another solution first.
- **Create functional and unit tests if possible/necessary** to verify changes that software code works at is should.

## Build & Test Commands

```bash
make build                    # Build binary (version from git or "dev")
make build VERSION=0.1.0     # Build with specific version tag
make test                     # Run all tests (go test ./...)
make clean                    # Remove built binary
make                          # Clean + build
make upgrade                  # Update dependencies (go get -u, mod tidy, vendor)
make help                     # Show all targets
```

Version is injected at build time via ldflags: `-X main.version=$(VERSION)`.

## Linting

```bash
golangci-lint run             # Run linter (configured via .golangci.yml)
```

The project uses golangci-lint v2 with a strict set of linters. See `.golangci.yml` for the full configuration.

## Code Style & Conventions

- **Language**: Go 1.25
- **Formatting**: `gofmt` + `goimports` (enforced by linter)
- **Imports**: Grouped imports, single `import` block required (`grouper` linter)
- **Declaration order**: Type, const, var, func (`decorder` linter)
- **Named returns**: Avoid named return values, especially in `defer` blocks (`nonamedreturns` linter)
- **Naming**: Follow Go initialisms (HTTP, TLS, IP, etc.) as defined in `staticcheck` config
- **File footer**: Each source file ends with `/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */`

### Godoc Comments

All exported types, functions, and methods must have godoc comments (`godoclint`, `godot`). Comments must end with a period.

### Error Handling

- **Error messages must be capitalized** (e.g., `"Failed to read config"`, not `"failed to read config"`). The `ST1005` linter rule is disabled to allow this.
- Use `fmt.Errorf` with `%w` for error wrapping.
- Use `%v` for errors in format strings (not `%s`).
- Past tense style: `"Failed to..."`, `"Unable to..."`.
- Always handle errors explicitly; never discard with `_` except in tests where intentional.

### Logging Conventions

- Use `backer/internal/log` package (wrapper around Go's `slog`).
- Log levels: `error`, `warn`, `info`, `debug`.
- Past tense with context: `"Skipping %s: %v", filename, err`.
- Appropriate severity: Warn for recoverable issues, Error for problems, Fatal for startup failures.

### Config Validation Pattern

For each config option: set default first, then immediately validate range/required value:
```go
if C.Port == 0 {
    C.Port = 8086
    log.Warnf("Config option port is not set, fallback to %d", C.Port)
}

if C.Port < portMin || C.Port > portMax {
    return fmt.Errorf("Config option port must be between %d and %d, got %d", portMin, portMax, C.Port)
}
```

### Function Naming

```go
// Exported (public): VerbNoun pattern
NewServer()
LoadConfig()
CreateTarGzStream()
GetFilesFromDirectories()

// Unexported (private): verbNoun or isNoun pattern
writeWithContext()
copyWithContext()
isPipeClosedError()
defaultConfigPath()
```

### JSON Tags

Struct fields use PascalCase, json tags use snake_case:
```go
BackupTimeout     int `json:"backup_timeout"`
CompressionLevel  int `json:"compression_level"`
```

### Intentional Linter Suppressions

| Rule | Reason |
|------|--------|
| ST1005 | Disable — allows capitalized error messages (e.g., `"Failed to..."`) |
| QF1001 | De Morgan's law disabled for auth condition clarity |
| revive (error strings) | Same reason as ST1005 |

### Design Decisions (Not Issues)

The following are intentional design decisions, not bugs or defects:

- **No rate limiting**: Backups are assumed to be infrequent enough that rate limiting is unnecessary.
- **Basic Auth only**: HTTP Basic Auth is sufficient for a backup server behind a firewall.
- **Global config**: The `C Config` global variable is intentional — config is loaded once at startup.
- **No incremental backup**: Each backup is a full snapshot, not incremental.
- **Credentials in config file**: Config file should have restricted permissions (documented).

## Project Structure

```
cmd/
  backer/
    main.go                         # CLI entrypoint, flag parsing, TLS setup, server start

internal/
  backer/
    types.go                        # Config struct definition
    consts.go                       # Named constants for magic numbers
    globals.go                      # Global C Config variable, compiled excludePatterns
    archive.go                      # Common archive logic (pipe, tar, file iteration)
    gzip.go                         # CreateTarGzStream, CreateTarPgzipStream
    bzip2.go                        # CreateTarBzip2Stream
    zstd.go                         # CreateTarZstdStream, zstdLevel helper
    lz4.go                          # CreateTarLz4Stream
    xz.go                           # CreateTarXzStream
    loadConfig.go                   # HJSON/JSON config parsing with validation
    loadConfig_test.go              # Config validation tests
    newServer.go                    # HTTP server creation, auth handler, context helpers
    newServer_test.go               # Server, auth, and context helper tests
    getFilesFromDirectories.go      # Directory walking, exclusion filtering (isExcluded)
    getFilesFromDirectories_test.go # File traversal and exclude pattern tests
    utils_test.go                   # Archive stream tests
  log/
    main.go                         # Logging wrapper around stdlib slog
    main_test.go                    # Log package tests

data/
  config_example.json               # Annotated example config (HJSON with comments)

contrib/
  backer                            # OpenRC init script (Alpine/Gentoo)
  backer.service                    # systemd unit file
  backer.freebsd                    # FreeBSD rc.d script

test_data/
  test_config.json                  # Test config fixture
  example.crt / example.key         # TLS test certificates
  test1/                            # File/directory fixtures for tests

Makefile                            # Build targets
README.md                           # User documentation
LICENSE                             # MIT License
llms.txt                            # LLM-friendly project reference
```

## Configuration

Config is JSON (HJSON with comments supported). Default path depends on OS:
- Linux/macOS: `/etc/backer.json`
- FreeBSD/NetBSD/OpenBSD/DragonFly: `/usr/local/etc/backer.json`
- Override with `-c /path/to/config.json`

### Config Options

| Option | Default | Description |
|--------|---------|-------------|
| address | `0.0.0.0` | Listen address |
| port | `8086` | Listen port (1-65535) |
| cert | — | TLS certificate file path |
| key | — | TLS key file path |
| nohttps | `false` | Disable HTTPS (for dev/lab) |
| location | `/archive` | API endpoint path |
| user | — | Basic auth username (required) |
| password | — | Basic auth password (required) |
| log | stderr | Log output file path |
| loglevel | `info` | Log verbosity: error/warn/info/debug |
| directories | — | Directories to backup (required, validated for existence) |
| backup_timeout | `60` | Stream timeout in minutes (1-1440) |
| compression_level | `9` | Gzip level (1-9) |
| exclude_patterns | `[]` | Regex patterns to exclude from backup |
| filename_prefix | `backup` | Prefix for backup filename in Content-Disposition header |
| default_compression | `gzip` | Default compression algorithm: gzip, pgzip, bzip2, zstd, lz4 or xz |
| compression_algorithm | `gzip` | Alias for default_compression (deprecated) |

## Architecture & Key Patterns

### Global Config

Uses a global `C Config` variable (`internal/backer/globals.go`). This is intentional — config is loaded once at startup and never modified concurrently.

### HJSON Config Parsing

Requires double conversion (hjson → map → json → struct):
```go
hjson.Unmarshal(buf, &tmp)  // parse HJSON into map[string]any
json.Marshal(tmp)            // re-encode as strict JSON
json.Unmarshal(buf, &C)     // decode into Config struct
```

### Streaming Archive Creation

No temp files on disk. Uses `io.Pipe` with a goroutine to stream archives on-the-fly:
- Common logic in `archive.go`: `createArchiveStream()` handles pipe, goroutine, tar writing, file iteration
- Each compression algorithm has its own file: `gzip.go`, `bzip2.go`, `zstd.go`, `lz4.go`, `xz.go`
- All return an `io.ReadCloser`
- HTTP handler copies from pipe reader directly to HTTP response

### Context-Aware I/O

Two helper functions in `newServer.go` handle context cancellation during streaming:
- `writeWithContext(ctx, fn)` — checks context before calling fn
- `copyWithContext(ctx, dst, src)` — buffered copy (32KB) with cancellation checks

### Special File Type Handling

- **Regular files**: streamed into tar archive with content
- **Directories**: included as tar directory entries
- **Symlinks**: stored as symlink entries (target path preserved)
- **Device files** (char/block): included with correct major/minor
- **Named pipes**: included (header only, no data)
- **Sockets**: skipped (not archivable)
- **Hard links**: deduplicated — if multiple files share the same inode, only the first is stored with content, subsequent hard links are stored as link entries pointing to the original

### Pipe Error Recovery

`isPipeClosedError()` checks for `"io: read/write on closed pipe"` errors to avoid noisy logging when a client disconnects mid-stream.

### Auth Handler

HTTP Basic Auth. Both username AND password must match exactly. Returns `401 Unauthorized` with `WWW-Authenticate` header on failure. The condition is intentionally written without De Morgan's law simplification for clarity (linter suppression `//nolint:staticcheck`).

### Client IP Detection

Checks `X-Forwarded-For` header first (for proxied requests), then falls back to `RemoteAddr`.

### Server Configuration

- `ReadHeaderTimeout`: 5 seconds
- `WriteTimeout`: configurable via `backup_timeout` (converted to `time.Duration`)
- TLS minimum version: TLS 1.3 (when HTTPS enabled)

### TLS Error Handling

The server uses a `serverWrapper` type to intercept and classify errors from `Serve()` and `ServeTLS()`. TLS-specific errors are logged at debug level to reduce log clutter:

- Uses `errors.As` to detect TLS error types: `tls.AlertError`, `tls.CertificateVerificationError`, `tls.ECHRejectionError`, `tls.RecordHeaderError`
- Also checks for "tls:" prefix in error messages
- Additionally uses `http.Server.ErrorLog` with a custom logger that writes at debug level for net/http's internal TLS-related messages

This ensures client-side TLS errors (handshake failures, certificate issues) don't clutter production logs.

## Dependencies

- `github.com/hjson/hjson-go` v3.3.0 — HJSON config parsing
- `github.com/dsnet/compress` v0.0.1 — bzip2 compression
- `github.com/klauspost/compress` v1.18.5 — zstd compression
- `github.com/klauspost/pgzip` v1.2.6 — parallel gzip compression
- `github.com/pierrec/lz4` v2.6.1 — lz4 compression
- `github.com/ulikunitz/xz` v0.5.15 — xz compression

## Testing

```bash
go test ./...                    # Run all tests
go test -v -run TestName ./...   # Run specific test
make test                        # Via Makefile
```

Tests live alongside source files as `*_test.go` (package name, not `_test` suffix). Coverage is approximately 69% on `internal/backer` and ~100% on `internal/log`.

### Test Conventions

- Reset global config before tests: `C = Config{}`
- Reset exclude patterns: `excludePatterns = nil`
- Use `t.TempDir()` for temporary files
- Helper `writeTempConfig(t, content)` creates temp config files for validation tests
- Symlink tests skip gracefully with `t.Skip()` when not supported
- Context cancellation tests verify correct behavior

### Test Data

- `test_data/test_config.json` — valid test config (user: JohnnyGoode, password: SharpShooter)
- `test_data/test1/foo/` — test directory with files, subdirs, empty dirs
- `test_data/test1/bar/` — test directory with files and a directory named `goodbye.txt/` (edge case: directory with file extension)
- `test_data/example.crt` / `example.key` — TLS test certificates

## Service Deployment

Init scripts provided in `contrib/`:
- **systemd** (`backer.service`): `ExecStart=/usr/local/sbin/backer`, restart on failure
- **OpenRC** (`backer`): uses `supervise-daemon`, creates log/pid dirs in `start_pre`
- **FreeBSD rc.d** (`backer.freebsd`): daemonizes via `/usr/sbin/daemon`