Add AGENTS.md, simplify code, 2 fixes in log package, 3 bugs fixed.
- Add AGENTS.md for vibecoding ai data. - Remove github.com/pkg/errors dependency - writeWithContext — remove goroutine overhead, just check context before fn - copyWithContext — remove unnecessary panic recovery - Remove panic recovery tests (TestWriteWithContextPanic, TestCopyWithContextPanic, panicReader) - Error, Warn, Info, Debug, Fatal — accept ...any instead of string (enables log.Error(err)) - Close — don't panic if Log is nil, don't close stderr - Add unit tests (internal/log/main_test.go) - getClientIP — trim spaces from X-Forwarded-For, use net.SplitHostPort for IPv6 - isExcluded — normalize paths to forward slashes for cross-platform regex matching - writeWithContext godoc — fix stale comment about panic recovery - internal/log/main_test.go — log package tests (19 tests)
This commit is contained in:
eleksir
281
AGENTS.md
Normal file
281
AGENTS.md
Normal file
@@ -0,0 +1,281 @@
|
||||
# AGENTS.md
|
||||
|
||||
## Project Overview
|
||||
|
||||
Backer is a Go-based HTTP/HTTPS backup server that creates `tar.gz` archives on-the-fly from configured directories. It serves a single endpoint that clients (e.g., `curl`) can query to download a timestamped backup archive. Licensed under MIT.
|
||||
|
||||
## Workflow Rules
|
||||
|
||||
- **Do not apply changes without explicit user consent.** When asked to analyze, review, or suggest improvements, present your proposal first and wait for confirmation before editing any files. Always ask "Should I implement this?" or "Should I apply these changes?" before writing code.
|
||||
- **Keep the codebase simple, small, maintainable, safe, and human-readable.** Favor minimal, clear solutions over clever or abstract ones. Do not introduce patterns, layers, or indirection unless they directly solve a concrete problem. Every line of code must earn its place.
|
||||
- **When stuck, ask a clarification question, or propose a short plan, or both.** Do not guess intent or make assumptions — ask.
|
||||
- **Always make a TODO plan before acting. And try to follow this plan.**
|
||||
- **If you notice defects, bugs, or issues, always report it.**
|
||||
- **Track new features, known issues, bugs, and other defects in `TODO.org`.** Update it as you implement features, fix issues, fix bugs, or discover new ones — check off completed items, add new findings.
|
||||
- **Update `AGENTS.md` as the project evolves.** Keep it comprehensive but dense.
|
||||
- **Don't use `context` package unless absolutely necessary.** Try to find another solution first.
|
||||
- **Create unit tests if possible/necessary** to verify changes that are not covered by unit tests.
|
||||
|
||||
## Build & Test Commands
|
||||
|
||||
```bash
|
||||
make build # Build binary (version from git or "dev")
|
||||
make build VERSION=0.1.0 # Build with specific version tag
|
||||
make test # Run all tests (go test ./...)
|
||||
make clean # Remove built binary
|
||||
make # Clean + build
|
||||
make upgrade # Update dependencies (go get -u, mod tidy, vendor)
|
||||
make help # Show all targets
|
||||
```
|
||||
|
||||
Version is injected at build time via ldflags: `-X main.version=$(VERSION)`.
|
||||
|
||||
## Linting
|
||||
|
||||
```bash
|
||||
golangci-lint run # Run linter (configured via .golangci.yml)
|
||||
```
|
||||
|
||||
The project uses golangci-lint v2 with a strict set of linters. See `.golangci.yml` for the full configuration.
|
||||
|
||||
## Code Style & Conventions
|
||||
|
||||
- **Language**: Go 1.25
|
||||
- **Formatting**: `gofmt` + `goimports` (enforced by linter)
|
||||
- **Imports**: Grouped imports, single `import` block required (`grouper` linter)
|
||||
- **Declaration order**: Type, const, var, func (`decorder` linter)
|
||||
- **Named returns**: Avoid named return values, especially in `defer` blocks (`nonamedreturns` linter)
|
||||
- **Naming**: Follow Go initialisms (HTTP, TLS, IP, etc.) as defined in `staticcheck` config
|
||||
- **File footer**: Each source file ends with `/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */`
|
||||
|
||||
### Godoc Comments
|
||||
|
||||
All exported types, functions, and methods must have godoc comments (`godoclint`, `godot`). Comments must end with a period.
|
||||
|
||||
### Error Handling
|
||||
|
||||
- **Error messages must be capitalized** (e.g., `"Failed to read config"`, not `"failed to read config"`). The `ST1005` linter rule is disabled to allow this.
|
||||
- Use `fmt.Errorf` with `%w` for error wrapping.
|
||||
- Use `%v` for errors in format strings (not `%s`).
|
||||
- Past tense style: `"Failed to..."`, `"Unable to..."`.
|
||||
- Always handle errors explicitly; never discard with `_` except in tests where intentional.
|
||||
|
||||
### Logging Conventions
|
||||
|
||||
- Use `backer/internal/log` package (wrapper around Go's `slog`).
|
||||
- Log levels: `error`, `warn`, `info`, `debug`.
|
||||
- Past tense with context: `"Skipping %s: %v", filename, err`.
|
||||
- Appropriate severity: Warn for recoverable issues, Error for problems, Fatal for startup failures.
|
||||
|
||||
### Config Validation Pattern
|
||||
|
||||
For each config option: set default first, then immediately validate range/required value:
|
||||
```go
|
||||
if C.Port == 0 {
|
||||
C.Port = 8086
|
||||
log.Warnf("Config option port is not set, fallback to %d", C.Port)
|
||||
}
|
||||
|
||||
if C.Port < 1 || C.Port > 65535 {
|
||||
return fmt.Errorf("Port must be between 1 and 65535, got %d", C.Port)
|
||||
}
|
||||
```
|
||||
|
||||
### Function Naming
|
||||
|
||||
```go
|
||||
// Exported (public): VerbNoun pattern
|
||||
NewServer()
|
||||
LoadConfig()
|
||||
CreateTarGzStream()
|
||||
GetFilesFromDirectories()
|
||||
|
||||
// Unexported (private): verbNoun or isNoun pattern
|
||||
writeWithContext()
|
||||
copyWithContext()
|
||||
isPipeClosedError()
|
||||
defaultConfigPath()
|
||||
```
|
||||
|
||||
### JSON Tags
|
||||
|
||||
Struct fields use PascalCase, json tags use snake_case:
|
||||
```go
|
||||
BackupTimeout int `json:"backup_timeout"`
|
||||
CompressionLevel int `json:"compression_level"`
|
||||
```
|
||||
|
||||
### Intentional Linter Suppressions
|
||||
|
||||
| Rule | Reason |
|
||||
|------|--------|
|
||||
| ST1005 | Disable — allows capitalized error messages (e.g., `"Failed to..."`) |
|
||||
| QF1001 | De Morgan's law disabled for auth condition clarity |
|
||||
| revive (error strings) | Same reason as ST1005 |
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
cmd/
|
||||
backer/
|
||||
main.go # CLI entrypoint, flag parsing, TLS setup, server start
|
||||
|
||||
internal/
|
||||
backer/
|
||||
types.go # Config struct definition
|
||||
globals.go # Global C Config variable, compiled excludePatterns
|
||||
loadConfig.go # HJSON/JSON config parsing with validation
|
||||
loadConfig_test.go # Config validation tests
|
||||
newServer.go # HTTP server creation, auth handler, context helpers
|
||||
newServer_test.go # Server, auth, and context helper tests
|
||||
utils.go # CreateTarGzStream (tar.gz streaming via io.Pipe)
|
||||
utils_test.go # Tar.gz stream tests
|
||||
getFilesFromDirectories.go # Directory walking, exclusion filtering (isExcluded)
|
||||
getFilesFromDirectories_test.go # File traversal and exclude pattern tests
|
||||
log/
|
||||
main.go # Logging wrapper around stdlib slog
|
||||
main_test.go # Log package tests
|
||||
|
||||
data/
|
||||
config_example.json # Annotated example config (HJSON with comments)
|
||||
|
||||
contrib/
|
||||
backer # OpenRC init script (Alpine/Gentoo)
|
||||
backer.service # systemd unit file
|
||||
backer.freebsd # FreeBSD rc.d script
|
||||
|
||||
test_data/
|
||||
test_config.json # Test config fixture
|
||||
example.crt / example.key # TLS test certificates
|
||||
test1/ # File/directory fixtures for tests
|
||||
|
||||
Makefile # Build targets
|
||||
TODO.org # Project backlog (Org mode)
|
||||
README.md # User documentation
|
||||
LICENSE # MIT License
|
||||
llms.txt # LLM-friendly project reference
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Config is JSON (HJSON with comments supported). Default path depends on OS:
|
||||
- Linux/macOS: `/etc/backer.json`
|
||||
- FreeBSD/NetBSD/OpenBSD/DragonFly: `/usr/local/etc/backer.json`
|
||||
- Override with `-c /path/to/config.json`
|
||||
|
||||
### Config Options
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| address | `0.0.0.0` | Listen address |
|
||||
| port | `8086` | Listen port (1-65535) |
|
||||
| cert | — | TLS certificate file path |
|
||||
| key | — | TLS key file path |
|
||||
| nohttps | `false` | Disable HTTPS (for dev/lab) |
|
||||
| location | `/archive` | API endpoint path |
|
||||
| user | — | Basic auth username (required) |
|
||||
| password | — | Basic auth password (required) |
|
||||
| log | stderr | Log output file path |
|
||||
| loglevel | `info` | Log verbosity: error/warn/info/debug |
|
||||
| directories | — | Directories to backup (required, validated for existence) |
|
||||
| backup_timeout | `60` | Stream timeout in minutes (1-1440) |
|
||||
| compression_level | `9` | Gzip level (1-9) |
|
||||
| exclude_patterns | `[]` | Regex patterns to exclude from backup |
|
||||
|
||||
## Architecture & Key Patterns
|
||||
|
||||
### Global Config
|
||||
|
||||
Uses a global `C Config` variable (`internal/backer/globals.go`). This is intentional — config is loaded once at startup and never modified concurrently.
|
||||
|
||||
### HJSON Config Parsing
|
||||
|
||||
Requires double conversion (hjson → map → json → struct):
|
||||
```go
|
||||
hjson.Unmarshal(buf, &tmp) // parse HJSON into map[string]any
|
||||
json.Marshal(tmp) // re-encode as strict JSON
|
||||
json.Unmarshal(buf, &C) // decode into Config struct
|
||||
```
|
||||
|
||||
### Streaming Archive Creation
|
||||
|
||||
No temp files on disk. Uses `io.Pipe` with a goroutine to stream tar.gz on-the-fly:
|
||||
- `CreateTarGzStream()` returns an `io.ReadCloser`
|
||||
- Goroutine writes tar entries → gzip writer → pipe writer
|
||||
- HTTP handler copies from pipe reader directly to HTTP response
|
||||
|
||||
### Context-Aware I/O
|
||||
|
||||
Two helper functions in `newServer.go` handle context cancellation during streaming:
|
||||
- `writeWithContext(ctx, fn)` — checks context before calling fn
|
||||
- `copyWithContext(ctx, dst, src)` — buffered copy (32KB) with cancellation checks
|
||||
|
||||
### Special File Type Handling
|
||||
|
||||
- **Regular files**: streamed into tar archive with content
|
||||
- **Directories**: included as tar directory entries
|
||||
- **Symlinks**: stored as symlink entries (target path preserved)
|
||||
- **Device files** (char/block): included with correct major/minor
|
||||
- **Named pipes**: included (header only, no data)
|
||||
- **Sockets**: skipped (not archivable)
|
||||
|
||||
### Pipe Error Recovery
|
||||
|
||||
`isPipeClosedError()` checks for `"io: read/write on closed pipe"` errors to avoid noisy logging when a client disconnects mid-stream.
|
||||
|
||||
### Auth Handler
|
||||
|
||||
HTTP Basic Auth. Both username AND password must match exactly. Returns `401 Unauthorized` with `WWW-Authenticate` header on failure. The condition is intentionally written without De Morgan's law simplification for clarity (linter suppression `//nolint:staticcheck`).
|
||||
|
||||
### Client IP Detection
|
||||
|
||||
Checks `X-Forwarded-For` header first (for proxied requests), then falls back to `RemoteAddr`.
|
||||
|
||||
### Server Configuration
|
||||
|
||||
- `ReadHeaderTimeout`: 5 seconds
|
||||
- `WriteTimeout`: configurable via `backup_timeout` (converted to `time.Duration`)
|
||||
- TLS minimum version: TLS 1.3 (when HTTPS enabled)
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `github.com/hjson/hjson-go` v3.3.0 — HJSON config parsing
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
go test ./... # Run all tests
|
||||
go test -v -run TestName ./... # Run specific test
|
||||
make test # Via Makefile
|
||||
```
|
||||
|
||||
Tests live alongside source files as `*_test.go` (package name, not `_test` suffix). Coverage is approximately 69% on `internal/backer` and ~100% on `internal/log`.
|
||||
|
||||
### Test Conventions
|
||||
|
||||
- Reset global config before tests: `C = Config{}`
|
||||
- Reset exclude patterns: `excludePatterns = nil`
|
||||
- Use `t.TempDir()` for temporary files
|
||||
- Helper `writeTempConfig(t, content)` creates temp config files for validation tests
|
||||
- Symlink tests skip gracefully with `t.Skip()` when not supported
|
||||
- Context cancellation tests verify correct behavior
|
||||
|
||||
### Test Data
|
||||
|
||||
- `test_data/test_config.json` — valid test config (user: JohnnyGoode, password: SharpShooter)
|
||||
- `test_data/test1/foo/` — test directory with files, subdirs, empty dirs
|
||||
- `test_data/test1/bar/` — test directory with files and a directory named `goodbye.txt/` (edge case: directory with file extension)
|
||||
- `test_data/example.crt` / `example.key` — TLS test certificates
|
||||
|
||||
## Service Deployment
|
||||
|
||||
Init scripts provided in `contrib/`:
|
||||
- **systemd** (`backer.service`): `ExecStart=/usr/local/sbin/backer`, restart on failure
|
||||
- **OpenRC** (`backer`): uses `supervise-daemon`, creates log/pid dirs in `start_pre`
|
||||
- **FreeBSD rc.d** (`backer.freebsd`): daemonizes via `/usr/sbin/daemon`
|
||||
|
||||
## TODO / Roadmap
|
||||
|
||||
See `TODO.org` for current backlog:
|
||||
- Consider alternative compression algorithms (bz2, xz, lzo, lz4)
|
||||
- Configurable filename prefix for `Content-Disposition` header
|
||||
Reference in New Issue
Block a user