Compare commits
4 Commits
v0.3
...
06ee9e081f
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
06ee9e081f | ||
|
|
8beba095d0 | ||
|
|
afffc99a8a | ||
|
|
6201b03230 |
30
AGENTS.md
30
AGENTS.md
@@ -4,17 +4,28 @@
|
||||
|
||||
Backer is a Go-based HTTP/HTTPS backup server that creates compressed archives on-the-fly from configured directories. Supports multiple compression algorithms: gzip, pgzip, bzip2, zstd, lz4, xz. It serves a single endpoint that clients (e.g., `curl`) can query to download a timestamped backup archive. Licensed under MIT.
|
||||
|
||||
## Fundamental Rules
|
||||
|
||||
- **Do not apply changes without explicit user consent.** When asked to analyze, review, or suggest any changes, present your proposal first and wait for confirmation before editing any files. Always ask "Should I implement this?" or "Should I apply these changes?" before modifying anything.
|
||||
- **Keep codebase simple, small, maintainable, safe and human-readable.** Favour minimal, clear solutions over clever or abstract ones. Prefer clarity over cleverness. Prioritize explicitness. Minimize magic. Do not introduce patterns, layers, or indirection unless they directly solve a concrete problem. Every line of code must earn its place.
|
||||
- **Create small focused modules and separate functionality over files.** Do not make universal implementation; try to move functions to separate files and name files after function names.
|
||||
- **Use a consistent structure.** Subsequent searches will be faster and consume less context.
|
||||
- **Avoid duplicate names.** This is not strict rule. If codebase contain multiple files with the same name (e.g., index.ts), we have to read them all to discover the one exactly we need. This consumes time and context.
|
||||
- **When stuck, ask clarification questions, or present a short plan, or both.** Do not try to guess intent or make assumptions - ask.
|
||||
- **Always make TODO plan before any job and follow it closely.**
|
||||
- **Report any found defects, issues, bugs.** And keep track of them.
|
||||
- **Comment the code.** Add code comments to explain corner cases, complex logic, business rules, and design decisions. Focus on **what** problem the code is solving rather than **how** it is solving it. Write comments in easy to read and understand language, for humans to comprehend. Every public API and structure field must have a docstring.
|
||||
- **Use domain names.** Use naming conventions that reflect their purpose and domain concepts, not their technical implementation (e.g., OrderProcessor instead of OrderServiceFactory).
|
||||
- **Test corner cases.** Write comprehensive test suites that cover various scenarios and edge cases.
|
||||
- **Use tags and labels.** In comments, add tags or labels to files, classes, or functions to categorize them based on functionality, domain concepts, or design patterns.
|
||||
- **Avoid obvious comments that do not add value.**
|
||||
- **Remove dead code.** A half-finished refactoring can actually harm the contributions.
|
||||
- **Update AGENTS.md and TODO.md automatically.** Keep AGENTS.md dense, detailed and concise.
|
||||
|
||||
## Workflow Rules
|
||||
|
||||
- **Do not apply changes without explicit user consent.** When asked to analyze, review, or suggest improvements, present your proposal first and wait for confirmation before editing any files. Always ask "Should I implement this?" or "Should I apply these changes?" before writing code.
|
||||
- **Keep the codebase simple, small, maintainable, safe, and human-readable.** Favor minimal, clear solutions over clever or abstract ones. Do not introduce patterns, layers, or indirection unless they directly solve a concrete problem. Every line of code must earn its place.
|
||||
- **When stuck, ask a clarification question, or propose a short plan, or both.** Do not guess intent or make assumptions — ask.
|
||||
- **Always make a TODO plan before acting. And try to follow this plan.**
|
||||
- **If you notice defects, bugs, or issues, always report it.**
|
||||
- **Track new features, known issues, bugs, and other defects.** Update documentation as you implement features, fix issues, fix bugs, or discover new ones.
|
||||
- **Update `AGENTS.md` as the project evolves.** Keep it comprehensive but dense.
|
||||
- **Don't use `context` package unless absolutely necessary.** Try to find another solution first.
|
||||
- **Create unit tests if possible/necessary** to verify changes that are not covered by unit tests.
|
||||
- **Create functional and unit tests if possible/necessary** to verify changes that software code works at is should.
|
||||
|
||||
## Build & Test Commands
|
||||
|
||||
@@ -187,7 +198,8 @@ Config is JSON (HJSON with comments supported). Default path depends on OS:
|
||||
| compression_level | `9` | Gzip level (1-9) |
|
||||
| exclude_patterns | `[]` | Regex patterns to exclude from backup |
|
||||
| filename_prefix | `backup` | Prefix for backup filename in Content-Disposition header |
|
||||
| compression_algorithm | `gzip` | Compression algorithm: gzip, pgzip, bzip2, zstd, lz4 or xz |
|
||||
| default_compression | `gzip` | Default compression algorithm: gzip, pgzip, bzip2, zstd, lz4 or xz |
|
||||
| compression_algorithm | `gzip` | Alias for default_compression (deprecated) |
|
||||
|
||||
## Architecture & Key Patterns
|
||||
|
||||
|
||||
19
README.md
19
README.md
@@ -28,10 +28,22 @@ If you running FreeBSD, use the rc.d script `contrib/backer.freebsd` and place i
|
||||
Build, install and configure backer. On your backup storage server you can do
|
||||
|
||||
```bash
|
||||
curl -u username:password https://your_server:8086/archive -o "backup-$(date +%Y%m%d-%H%M%S).tar.gz"
|
||||
wget --quiet --no-check-certificate --content-disposition --http-user=username --http-password=password https://your_server:8086/archive
|
||||
```
|
||||
|
||||
The filename and extension depend on your `filename_prefix` and `compression_algorithm` settings.
|
||||
The filename and extension depend on your `filename_prefix` and `default_compression` settings.
|
||||
|
||||
You can also override compression by specifying the archive extension in the URL:
|
||||
|
||||
```bash
|
||||
wget --quiet --no-check-certificate --content-disposition --http-user=username --http-password=password https://your_server:8086/archive.xz
|
||||
wget --quiet --no-check-certificate --content-disposition --http-user=username --http-password=password https://your_server:8086/archive.gz # Uses pgzip
|
||||
wget --quiet --no-check-certificate --content-disposition --http-user=username --http-password=password https://your_server:8086/archive.bz2
|
||||
wget --quiet --no-check-certificate --content-disposition --http-user=username --http-password=password https://your_server:8086/archive.zst
|
||||
wget --quiet --no-check-certificate --content-disposition --http-user=username --http-password=password https://your_server:8086/archive.lz4
|
||||
```
|
||||
|
||||
Note: `/archive.tar.gz` always uses pgzip (parallel gzip) for better performance.
|
||||
|
||||
Of course, something can go wrong, so you must check http status code after downloading backup.
|
||||
|
||||
@@ -56,7 +68,8 @@ All options described in example config.
|
||||
| compression_level | 9 | Compression level. | 1 (fastest) to 9 (best compression). |
|
||||
| exclude_patterns | [] | Regex patterns to exclude from backup. | E.g., `[".*\\.tmp$", "/node_modules/"]`. |
|
||||
| filename_prefix | "backup" | Prefix for backup filename. | E.g., "mybackup" produces `mybackup-20260325-092341.tar.gz`. |
|
||||
| compression_algorithm | "gzip" | Compression algorithm. | Options: gzip, pgzip, bzip2, zstd, lz4, xz. |
|
||||
| default_compression | "gzip" | Default compression algorithm. | Options: gzip, pgzip, bzip2, zstd, lz4, xz. |
|
||||
| compression_algorithm | "gzip" | Alias for default_compression. | You should use default_compression instead. |
|
||||
|
||||
## Compression algorithms
|
||||
|
||||
|
||||
@@ -7,10 +7,10 @@ pidfile="/run/$SVCNAME/$SVCNAME.pid"
|
||||
|
||||
depend() {
|
||||
need net
|
||||
use dns
|
||||
}
|
||||
|
||||
start_pre() {
|
||||
mkdir -p /var/log/$SVCNAME
|
||||
mkdir -p /run/$SVCNAME
|
||||
}
|
||||
|
||||
@@ -26,27 +26,30 @@
|
||||
/* Where to store logs. */
|
||||
"log": "/var/log/backer.log",
|
||||
|
||||
/* Verbosity of logs */
|
||||
/* Verbosity of logs. */
|
||||
"loglevel": "info",
|
||||
|
||||
/* Diretories to backup */
|
||||
"directories": [
|
||||
"/path/to/directory1",
|
||||
"/path/to/directory2"
|
||||
],
|
||||
|
||||
/* Backup streaming timeout in minutes. */
|
||||
"backup_timeout": 60,
|
||||
|
||||
/* Gzip compression level (1-9, where 9 is best compression). */
|
||||
/* Gzip compression level (1-9, where 9 is best compression). */
|
||||
"compression_level": 9,
|
||||
|
||||
/* Regex patterns to exclude from backup. */
|
||||
"exclude_patterns": [],
|
||||
|
||||
/* Prefix for backup filename in Content-Disposition header. */
|
||||
"filename_prefix": "backup",
|
||||
|
||||
/* Compression algorithm: gzip, pgzip, bzip2, zstd, lz4 or xz. */
|
||||
"compression_algorithm": "gzip"
|
||||
/* Compression algorithm alias (deprecated: use default_compression). */
|
||||
"compression_algorithm": "gzip",
|
||||
|
||||
/* Default compression algorithm: gzip, pgzip, bzip2, zstd, lz4 or xz. */
|
||||
"default_compression": "gzip",
|
||||
|
||||
/* Regex patterns to exclude from backup. */
|
||||
"exclude_patterns": [],
|
||||
|
||||
/* Directories to backup. */
|
||||
"directories": [
|
||||
"/path/to/directory1",
|
||||
"/path/to/directory2"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -50,8 +50,8 @@ const (
|
||||
// defaultFilenamePrefix is the default prefix for backup filenames.
|
||||
defaultFilenamePrefix = "backup"
|
||||
|
||||
// defaultCompressionAlgorithm is the default compression algorithm.
|
||||
defaultCompressionAlgorithm = "gzip"
|
||||
// defaultDefaultCompression is the default compression algorithm.
|
||||
defaultDefaultCompression = "gzip"
|
||||
)
|
||||
|
||||
/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */
|
||||
|
||||
@@ -135,15 +135,20 @@ func LoadConfig(path string) error {
|
||||
log.Warnf("Config option filename_prefix is not set, fallback to %s", C.FilenamePrefix)
|
||||
}
|
||||
|
||||
// CompressionAlgorithm: default and validate.
|
||||
if C.CompressionAlgorithm == "" {
|
||||
C.CompressionAlgorithm = defaultCompressionAlgorithm
|
||||
|
||||
log.Warnf("Config option compression_algorithm is not set, fallback to %s", C.CompressionAlgorithm)
|
||||
// DefaultCompression: default and validate. If compression_algorithm is set (deprecated), use it as fallback.
|
||||
if C.DefaultCompression == "" {
|
||||
if C.CompressionAlgorithm != "" {
|
||||
C.DefaultCompression = C.CompressionAlgorithm
|
||||
} else {
|
||||
C.DefaultCompression = defaultDefaultCompression
|
||||
}
|
||||
}
|
||||
|
||||
if C.CompressionAlgorithm != "gzip" && C.CompressionAlgorithm != "bzip2" && C.CompressionAlgorithm != "zstd" && C.CompressionAlgorithm != "lz4" && C.CompressionAlgorithm != "xz" && C.CompressionAlgorithm != "pgzip" {
|
||||
return fmt.Errorf("Config option compression_algorithm must be gzip, bzip2, zstd, lz4, xz or pgzip, got %s", C.CompressionAlgorithm)
|
||||
switch C.DefaultCompression {
|
||||
case "gzip", "pgzip", "bzip2", "xz", "lz4", "zstd":
|
||||
log.Infof("Using %s as defailt compression algorithm", C.DefaultCompression)
|
||||
default:
|
||||
return fmt.Errorf("Config option default_compression must be gzip, bzip2, zstd, lz4, xz or pgzip, got %s", C.DefaultCompression)
|
||||
}
|
||||
|
||||
// Compile exclude patterns.
|
||||
|
||||
@@ -227,8 +227,8 @@ func TestLoadConfigDefaultValues(t *testing.T) {
|
||||
t.Errorf("C.FilenamePrefix should default to backup, got %s", C.FilenamePrefix)
|
||||
}
|
||||
|
||||
if C.CompressionAlgorithm != "gzip" {
|
||||
t.Errorf("C.CompressionAlgorithm should default to gzip, got %s", C.CompressionAlgorithm)
|
||||
if C.DefaultCompression != "gzip" {
|
||||
t.Errorf("C.DefaultCompression should default to gzip, got %s", C.DefaultCompression)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -248,8 +248,8 @@ func TestLoadConfigCompressionAlgorithmBzip2(t *testing.T) {
|
||||
t.Errorf("Expected no error, got %v", err)
|
||||
}
|
||||
|
||||
if C.CompressionAlgorithm != "bzip2" {
|
||||
t.Errorf("C.CompressionAlgorithm should be bzip2, got %s", C.CompressionAlgorithm)
|
||||
if C.DefaultCompression != "bzip2" {
|
||||
t.Errorf("C.DefaultCompression should be bzip2, got %s", C.DefaultCompression)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -269,8 +269,8 @@ func TestLoadConfigCompressionAlgorithmZstd(t *testing.T) {
|
||||
t.Errorf("Expected no error, got %v", err)
|
||||
}
|
||||
|
||||
if C.CompressionAlgorithm != "zstd" {
|
||||
t.Errorf("C.CompressionAlgorithm should be zstd, got %s", C.CompressionAlgorithm)
|
||||
if C.DefaultCompression != "zstd" {
|
||||
t.Errorf("C.DefaultCompression should be zstd, got %s", C.DefaultCompression)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -290,8 +290,8 @@ func TestLoadConfigCompressionAlgorithmLz4(t *testing.T) {
|
||||
t.Errorf("Expected no error, got %v", err)
|
||||
}
|
||||
|
||||
if C.CompressionAlgorithm != "lz4" {
|
||||
t.Errorf("C.CompressionAlgorithm should be lz4, got %s", C.CompressionAlgorithm)
|
||||
if C.DefaultCompression != "lz4" {
|
||||
t.Errorf("C.DefaultCompression should be lz4, got %s", C.DefaultCompression)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -311,8 +311,8 @@ func TestLoadConfigCompressionAlgorithmXz(t *testing.T) {
|
||||
t.Errorf("Expected no error, got %v", err)
|
||||
}
|
||||
|
||||
if C.CompressionAlgorithm != "xz" {
|
||||
t.Errorf("C.CompressionAlgorithm should be xz, got %s", C.CompressionAlgorithm)
|
||||
if C.DefaultCompression != "xz" {
|
||||
t.Errorf("C.DefaultCompression should be xz, got %s", C.DefaultCompression)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -332,8 +332,8 @@ func TestLoadConfigCompressionAlgorithmPgzip(t *testing.T) {
|
||||
t.Errorf("Expected no error, got %v", err)
|
||||
}
|
||||
|
||||
if C.CompressionAlgorithm != "pgzip" {
|
||||
t.Errorf("C.CompressionAlgorithm should be pgzip, got %s", C.CompressionAlgorithm)
|
||||
if C.DefaultCompression != "pgzip" {
|
||||
t.Errorf("C.DefaultCompression should be pgzip, got %s", C.DefaultCompression)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"path"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -46,8 +47,8 @@ func NewServer(configPath string) (*http.Server, error) {
|
||||
return
|
||||
}
|
||||
|
||||
// Both username AND password must match. Fuck De Morgan and his bullshit laws - we explicitly require
|
||||
// that the whole condition fails. That *is* the point.
|
||||
// Both username AND password must match. Explicitly require that the whole condition
|
||||
// fails when either is incorrect, per the authentication design.
|
||||
//nolint:staticcheck // QF1001: De Morgan's law is irrelevant here.
|
||||
if !(username == C.User && password == C.Password) {
|
||||
w.Header().Set("WWW-Authenticate", `Basic realm="restricted", charset="UTF-8"`)
|
||||
@@ -76,9 +77,14 @@ func NewServer(configPath string) (*http.Server, error) {
|
||||
var (
|
||||
archive io.ReadCloser
|
||||
extension string
|
||||
algorithm string
|
||||
)
|
||||
|
||||
switch C.CompressionAlgorithm {
|
||||
requestedExt := strings.TrimPrefix(path.Ext(r.URL.Path), ".")
|
||||
|
||||
algorithm = getCompressionAlgorithm(requestedExt)
|
||||
|
||||
switch algorithm {
|
||||
case "bzip2":
|
||||
archive = CreateTarBzip2Stream(ctx, files)
|
||||
extension = "tar.bz2"
|
||||
@@ -128,19 +134,8 @@ func NewServer(configPath string) (*http.Server, error) {
|
||||
log.Infof("Backup completed: client=%s user=%s files=%d bytes=%d duration=%s", clientIP, username, len(files), bytesWritten, duration)
|
||||
})
|
||||
|
||||
// Decision about which mode (http or https) to run made in cmd/backer/main.go, where we run server.ListenAndServe()
|
||||
// or server.ListenAndServeTLS() depending on value of C.NoHTTPS.
|
||||
if C.NoHTTPS {
|
||||
return &http.Server{
|
||||
Addr: fmt.Sprintf("%s:%d", C.Address, C.Port),
|
||||
Handler: withServerHeader(mux),
|
||||
ReadHeaderTimeout: readHeaderTimeout,
|
||||
WriteTimeout: time.Duration(C.BackupTimeout) * time.Minute,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Decision about which mode (http or https) to run made in cmd/backer/main.go, where we run server.ListenAndServe()
|
||||
// or server.ListenAndServeTLS() depending on value of C.NoHTTPS.
|
||||
// HTTP mode if nohttps is enabled in config.
|
||||
// HTTPS mode if nohttps is disabled in config.
|
||||
if C.NoHTTPS {
|
||||
return &http.Server{
|
||||
Addr: fmt.Sprintf("%s:%d", C.Address, C.Port),
|
||||
@@ -153,7 +148,7 @@ func NewServer(configPath string) (*http.Server, error) {
|
||||
return &http.Server{
|
||||
Addr: fmt.Sprintf("%s:%d", C.Address, C.Port),
|
||||
Handler: withServerHeader(mux),
|
||||
ReadHeaderTimeout: 5 * time.Second,
|
||||
ReadHeaderTimeout: readHeaderTimeout,
|
||||
WriteTimeout: time.Duration(C.BackupTimeout) * time.Minute,
|
||||
TLSConfig: &tls.Config{
|
||||
MinVersion: tls.VersionTLS13,
|
||||
@@ -218,4 +213,24 @@ func getClientIP(r *http.Request) string {
|
||||
return host
|
||||
}
|
||||
|
||||
// getCompressionAlgorithm determines which compression algorithm to use based on the requested extension.
|
||||
// If the extension is recognized, it returns the corresponding algorithm; otherwise, it returns the default.
|
||||
// For .tar.gz extension, pgzip is always used for parallel compression.
|
||||
func getCompressionAlgorithm(requestedExt string) string {
|
||||
switch requestedExt {
|
||||
case "tar.bz2":
|
||||
return "bzip2"
|
||||
case "tar.zst":
|
||||
return "zstd"
|
||||
case "tar.lz4":
|
||||
return "lz4"
|
||||
case "tar.xz":
|
||||
return "xz"
|
||||
case "tar.gz":
|
||||
return "pgzip"
|
||||
default:
|
||||
return C.DefaultCompression
|
||||
}
|
||||
}
|
||||
|
||||
/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */
|
||||
|
||||
@@ -18,7 +18,8 @@ type (
|
||||
CompressionLevel int `json:"compression_level"` // Compression level (1-9, default 9).
|
||||
ExcludePatterns []string `json:"exclude_patterns"` // Regex patterns to exclude from backup.
|
||||
FilenamePrefix string `json:"filename_prefix"` // Prefix for backup filename (default: "backup").
|
||||
CompressionAlgorithm string `json:"compression_algorithm"` // Compression algorithm: gzip, pgzip, bzip2, zstd, lz4 or xz.
|
||||
CompressionAlgorithm string `json:"compression_algorithm"` // Compression algorithm alias (deprecated: use default_compression).
|
||||
DefaultCompression string `json:"default_compression"` // Default compression algorithm: gzip, pgzip, bzip2, zstd, lz4 or xz.
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user