Update README.md

- Fix 404 on downloading archive with extension
- Add downloaded archive length to log
- Add md5 hash to log

.gitignore

@@ -1,6 +1,8 @@
 # Binary
 /backer
 /backer.exe
+/cmd/backer/backer
+/cmd/backer/backer.exe
 
 # Local config (may contain secrets)
 /config.json
@@ -27,3 +29,5 @@ Thumbs.db
 *.exe
 *.test
 *.out
+
+/test_data/tmp

.golangci.yml

@@ -74,6 +74,7 @@ linters:
         - paramTypeCombine
         - whyNoLint
         - hugeParam
+        - unnamedResult
     godoclint:
       default: all
       ignore-unexported: false

AGENTS.md

@@ -4,17 +4,28 @@
 
 Backer is a Go-based HTTP/HTTPS backup server that creates compressed archives on-the-fly from configured directories. Supports multiple compression algorithms: gzip, pgzip, bzip2, zstd, lz4, xz. It serves a single endpoint that clients (e.g., `curl`) can query to download a timestamped backup archive. Licensed under MIT.
 
+## Fundamental Rules
+
+- **Do not apply changes without explicit user consent.** When asked to analyze, review, or suggest any changes, present your proposal first and wait for confirmation before editing any files. Always ask "Should I implement this?" or "Should I apply these changes?" before modifying anything.
+- **Keep codebase simple, small, maintainable, safe and human-readable.** Favour minimal, clear solutions over clever or abstract ones. Prefer clarity over cleverness. Prioritize explicitness. Minimize magic. Do not introduce patterns, layers, or indirection unless they directly solve a concrete problem. Every line of code must earn its place.
+- **Create small focused modules and separate functionality over files.** Do not make universal implementation; try to move functions to separate files and name files after function names.
+- **Use a consistent structure.** Subsequent searches will be faster and consume less context.
+- **Avoid duplicate names.** This is not strict rule. If codebase contain multiple files with the same name (e.g., index.ts), we have to read them all to discover the one exactly we need. This consumes time and context.
+- **When stuck, ask clarification questions, or present a short plan, or both.** Do not try to guess intent or make assumptions - ask.
+- **Always make TODO plan before any job and follow it closely.**
+- **Report any found defects, issues, bugs.** And keep track of them.
+- **Comment the code.** Add code comments to explain corner cases, complex logic, business rules, and design decisions. Focus on **what** problem the code is solving rather than **how** it is solving it. Write comments in easy to read and understand language, for humans to comprehend. Every public API and structure field must have a docstring.
+- **Use domain names.** Use naming conventions that reflect their purpose and domain concepts, not their technical implementation (e.g., OrderProcessor instead of OrderServiceFactory).
+- **Test corner cases.** Write comprehensive test suites that cover various scenarios and edge cases.
+- **Use tags and labels.** In comments, add tags or labels to files, classes, or functions to categorize them based on functionality, domain concepts, or design patterns.
+- **Avoid obvious comments that do not add value.**
+- **Remove dead code.** A half-finished refactoring can actually harm the contributions.
+- **Update AGENTS.md automatically.** Keep AGENTS.md dense, detailed and concise.
+
 ## Workflow Rules
 
-- **Do not apply changes without explicit user consent.** When asked to analyze, review, or suggest improvements, present your proposal first and wait for confirmation before editing any files. Always ask "Should I implement this?" or "Should I apply these changes?" before writing code.
-- **Keep the codebase simple, small, maintainable, safe, and human-readable.** Favor minimal, clear solutions over clever or abstract ones. Do not introduce patterns, layers, or indirection unless they directly solve a concrete problem. Every line of code must earn its place.
-- **When stuck, ask a clarification question, or propose a short plan, or both.** Do not guess intent or make assumptions — ask.
-- **Always make a TODO plan before acting. And try to follow this plan.**
-- **If you notice defects, bugs, or issues, always report it.**
-- **Track new features, known issues, bugs, and other defects.** Update documentation as you implement features, fix issues, fix bugs, or discover new ones.
-- **Update `AGENTS.md` as the project evolves.** Keep it comprehensive but dense.
 - **Don't use `context` package unless absolutely necessary.** Try to find another solution first.
-- **Create unit tests if possible/necessary** to verify changes that are not covered by unit tests.
+- **Create functional and unit tests if possible/necessary** to verify changes that software code works at is should.
 
 ## Build & Test Commands
 
@@ -113,6 +124,18 @@ CompressionLevel  int `json:"compression_level"`
 | QF1001 | De Morgan's law disabled for auth condition clarity |
 | revive (error strings) | Same reason as ST1005 |
 
+### Design Decisions (Not Issues)
+
+The following are intentional design decisions, not bugs or defects:
+
+- **No rate limiting**: Backups are assumed to be infrequent enough that rate limiting is unnecessary.
+- **Backup scheduling**: Client-side decision — server serves backups on-request only.
+- **Basic Auth only**: HTTP Basic Auth is sufficient for a backup server behind a firewall.
+- **Global config**: The `C Config` global variable is intentional — config is loaded once at startup.
+- **No incremental backup**: Each backup is a full snapshot, not incremental.
+- **Credentials in config file**: Config file should have restricted permissions (documented).
+- **Thread-safe logging**: The global `Log` variable uses mutex protection to enable parallel test execution; this is not needed for single-server production use but allows tests to run in parallel without race conditions.
+
 ## Project Structure
 
 ```
@@ -184,10 +207,12 @@ Config is JSON (HJSON with comments supported). Default path depends on OS:
 | loglevel | `info` | Log verbosity: error/warn/info/debug |
 | directories | — | Directories to backup (required, validated for existence) |
 | backup_timeout | `60` | Stream timeout in minutes (1-1440) |
+| dir_scan_timeout | `60` | Directory scan timeout in minutes (1-1440) |
 | compression_level | `9` | Gzip level (1-9) |
 | exclude_patterns | `[]` | Regex patterns to exclude from backup |
 | filename_prefix | `backup` | Prefix for backup filename in Content-Disposition header |
-| compression_algorithm | `gzip` | Compression algorithm: gzip, pgzip, bzip2, zstd, lz4 or xz |
+| default_compression | `gzip` | Default compression algorithm: gzip, pgzip, bzip2, zstd, lz4 or xz |
+| compression_algorithm | `gzip` | Alias for default_compression (deprecated) |
 
 ## Architecture & Key Patterns
 
@@ -216,16 +241,17 @@ No temp files on disk. Uses `io.Pipe` with a goroutine to stream archives on-the
 
 Two helper functions in `newServer.go` handle context cancellation during streaming:
 - `writeWithContext(ctx, fn)` — checks context before calling fn
-- `copyWithContext(ctx, dst, src)` — buffered copy (32KB) with cancellation checks
+- `copyWithContext(ctx, dst, src)` — buffered copy (32KB) with cancellation checks, computes MD5 hash of copied data
 
 ### Special File Type Handling
 
 - **Regular files**: streamed into tar archive with content
 - **Directories**: included as tar directory entries
-- **Symlinks**: stored as symlink entries (target path preserved)
+- **Symlinks**: stored as symlink entries (target path preserved). Backer never resolves or follows symlinks during directory traversal — it uses `os.Lstat` and `filepath.WalkDir` which do not follow symlinks. This means symlink cycles are impossible to occur.
 - **Device files** (char/block): included with correct major/minor
 - **Named pipes**: included (header only, no data)
 - **Sockets**: skipped (not archivable)
+- **Hard links**: deduplicated — if multiple files share the same inode, only the first is stored with content, subsequent hard links are stored as link entries pointing to the original
 
 ### Pipe Error Recovery
 
@@ -233,7 +259,7 @@ Two helper functions in `newServer.go` handle context cancellation during stream
 
 ### Auth Handler
 
-HTTP Basic Auth. Both username AND password must match exactly. Returns `401 Unauthorized` with `WWW-Authenticate` header on failure. The condition is intentionally written without De Morgan's law simplification for clarity (linter suppression `//nolint:staticcheck`).
+HTTP Basic Auth. Both username AND password must match exactly. Returns `401 Unauthorized` with `WWW-Authenticate` header on failure. Uses timing-safe comparison to prevent timing attacks. The condition is intentionally written without De Morgan's law simplification for clarity (linter suppression `//nolint:staticcheck`).
 
 ### Client IP Detection
 
@@ -243,8 +269,63 @@ Checks `X-Forwarded-For` header first (for proxied requests), then falls back to
 
 - `ReadHeaderTimeout`: 5 seconds
 - `WriteTimeout`: configurable via `backup_timeout` (converted to `time.Duration`)
+- `IdleTimeout`: 10 seconds (for keep-alive connections; backer disables keep-alives explicitly)
+- `MaxHeaderBytes`: 1KB (prevents header-based DoS attacks)
+- `DisableKeepAlives`: true (each connection closed after request, no connection reuse)
 - TLS minimum version: TLS 1.3 (when HTTPS enabled)
 
+### TLS Error Handling
+
+The server uses a `ServerWrapper` type to intercept and classify errors from `Serve()` and `ServeTLS()`. TLS-specific errors are logged at debug level to reduce log clutter:
+
+- Uses `errors.As` to detect TLS error types: `tls.AlertError`, `tls.CertificateVerificationError`, `tls.ECHRejectionError`, `tls.RecordHeaderError`
+- Also checks for "tls:" prefix in error messages
+- Additionally uses `http.Server.ErrorLog` with a custom logger that writes at debug level for net/http's internal TLS-related messages
+
+This ensures client-side TLS errors (handshake failures, certificate issues) don't clutter production logs.
+
+### Compression Level Mapping
+
+The `compression_level` config option (1-9) is mapped differently for each compression algorithm:
+
+| Algorithm | Level Mapping |
+|-----------|----------------|
+| gzip | 1-9 directly maps to gzip compression levels |
+| pgzip | 1-9 directly maps to pgzip compression levels |
+| bzip2 | 1-9 directly maps to bzip2 compression levels |
+| zstd | 1-3: Fastest, 4-6: Default, 7-9: BetterCompression |
+| lz4 | 1-3: Fastest (0), 4-6: Default (1), 7-9: Best (2) |
+| xz | Uses default compression (no configurable level) |
+
+### HTTP Routing & Compression Selection
+
+The server uses `http.NewServeMux()` for routing. Multiple routes are registered to support different compression formats:
+
+```go
+mux.Handle(C.Location, http.HandlerFunc(backupHandler))           // /archive
+mux.Handle(C.Location+".tar.gz", http.HandlerFunc(backupHandler))  // /archive.tar.gz
+mux.Handle(C.Location+".tar.xz", http.HandlerFunc(backupHandler))   // /archive.tar.xz
+mux.Handle(C.Location+".tar.bz2", http.HandlerFunc(backupHandler)) // /archive.tar.bz2
+// ... etc
+```
+
+The compression algorithm is determined by extracting the extension from the request path using `path.Ext()`:
+- For `/archive.tar.xz`, `path.Ext()` returns `.tar.xz` → maps to xz
+- For `/archive`, `path.Ext()` returns `""` → uses default compression
+
+The `getCompressionAlgorithm()` function handles both full paths and bare extensions:
+- Full path: `/archive.tar.xz` → `.tar.xz` → xz
+- Bare extension: `tar.xz` → `.tar.xz` → xz
+- Single ext: `xz` → `.xz` → xz
+
+### Unmatched Route Logging
+
+The server wraps the `http.ResponseWriter` to capture status codes and log unmatched routes:
+- **404 Not Found**: Logged as `"Not Found from {clientIP} to {path}"`
+- **405 Method Not Allowed**: Logged as `"Method Not Allowed from {clientIP} for {method} {path}"`
+
+This uses a `responseWriterWrapper` that tracks the status code via the `WriteHeader()` method.
+
 ## Dependencies
 
 - `github.com/hjson/hjson-go` v3.3.0 — HJSON config parsing

README.md

@@ -28,10 +28,22 @@ If you running FreeBSD, use the rc.d script `contrib/backer.freebsd` and place i
 Build, install and configure backer. On your backup storage server you can do
 
 ```bash
-curl -u username:password https://your_server:8086/archive -o "backup-$(date +%Y%m%d-%H%M%S).tar.gz"
+wget --quiet --no-check-certificate --content-disposition --http-user=username --http-password=password https://your_server:8086/archive
 ```
 
-The filename and extension depend on your `filename_prefix` and `compression_algorithm` settings.
+The filename and extension depend on your `filename_prefix` and `default_compression` settings.
+
+You can also override compression by specifying the archive extension in the URL:
+
+```bash
+wget --quiet --no-check-certificate --content-disposition --http-user=username --http-password=password https://your_server:8086/archive.xz
+wget --quiet --no-check-certificate --content-disposition --http-user=username --http-password=password https://your_server:8086/archive.gz    # Uses pgzip
+wget --quiet --no-check-certificate --content-disposition --http-user=username --http-password=password https://your_server:8086/archive.bz2
+wget --quiet --no-check-certificate --content-disposition --http-user=username --http-password=password https://your_server:8086/archive.zst
+wget --quiet --no-check-certificate --content-disposition --http-user=username --http-password=password https://your_server:8086/archive.lz4
+```
+
+Note: `/archive.tar.gz` always uses pgzip (parallel gzip) for better performance.
 
 Of course, something can go wrong, so you must check http status code after downloading backup.
 
@@ -53,10 +65,12 @@ All options described in example config.
 | loglevel | "info" | Verbosity of logs. | Options: error, warn, info, debug. |
 | directories | — | Directories to backup. | Required. Array of paths. |
 | backup_timeout | 60 | Timeout in minutes for backup streaming. | Range: 1-1440 minutes. |
+| dir_scan_timeout | 60 | Timeout in minutes for getting file list from all backupped directories. | Range: 1-1440 minutes. |
 | compression_level | 9 | Compression level. | 1 (fastest) to 9 (best compression). |
 | exclude_patterns | [] | Regex patterns to exclude from backup. | E.g., `[".*\\.tmp$", "/node_modules/"]`. |
 | filename_prefix | "backup" | Prefix for backup filename. | E.g., "mybackup" produces `mybackup-20260325-092341.tar.gz`. |
-| compression_algorithm | "gzip" | Compression algorithm. | Options: gzip, pgzip, bzip2, zstd, lz4, xz. |
+| default_compression | "gzip" | Default compression algorithm. | Options: gzip, pgzip, bzip2, zstd, lz4, xz. |
+| compression_algorithm | "gzip" | Alias for default_compression. | You should use default_compression instead. |
 
 ## Compression algorithms
 
@@ -64,7 +78,7 @@ AI suggests this table as expected performance.
 
 | Algorithm | Speed   | Compression | Notes |
 | :---      | :---    | :---        | :--- |
-| gzip | Fast | Good | Default. Standard, widely compatible. |
+| gzip      | Fast    | Good        | Standard, widely compatible. |
 | pgzip     | Fast    | Good        | Parallel gzip. Uses multiple CPU cores, slightly larger files. |
 | bzip2     | Slow    | Better      | Pure Go. Good compression, slower than gzip. |
 | zstd      | Fast    | Very good   | Good balance of speed and compression. |
@@ -77,12 +91,21 @@ network bandwidth, located in near by country I've got these results
 | Algorithm | Size       | Time    | CPU Load | RAM RSS+Shm, Mb | Notes |
 | :---      | ---:       | ---:    | ---:     | ---:            | :---  |
 | None      | 1461250551 |         |     0-2% |             0+0 | original dataset |
+| gzip      |  823381547 | 16m:22s |      96% |            18+6 | stdlib compression/gzip |
+| pgzip     |  823783924 | 13m:50s |   93-97% |            28+5 | multithreaded (parallel gzip), fully gzip compatible, klauspost/pgzip |
+| bzip2     |  797053471 |  9m:47s |   92-94% |            56+6 | dsnet/compress/bzip2 |
+| xz        |  515106988 | 16m:03s |      96% |           116+5 | ulikunitz/xz |
+| zstd      |  617330529 |  6m:39s |   40-99% |            64+6 | klauspost/compress/zstd |
 | lz4       | 1097745915 |  6m:12s |     4-6% |            30+6 | multithreaded lz4, pierrec/lz4 |
-| zstd      |  614015773 |  4m:52s |   40-60% |            60+6 | klauspost/compress/zstd |
-| pgzip     |  817163857 | 11m:25s |   96-97% |            37+6 | multithreaded (parallel gzip), fully gzip compatible, klauspost/pgzip |
-| gzip      |  816734188 | 14m:30s |      96% |            18+6 | stdlib compression/gzip |
-| bzip2     |  788061284 |  7m:16s |   92-94% |            43+6 | dsnet/compress/bzip2 |
-| xz        |  512004612 | 13m:53s |      96% |           116+6 | ulikunitz/xz |
+
+The `compression_level` option (1-9) maps differently for each algorithm:
+
+| Algorithm        | Level Mapping |
+| :---             | :---          |
+| gzip/pgzip/bzip2 | 1-9 directly maps to compression levels |
+| lz4              | 1-3 → Fastest (0), 4-6 → Default (1), 7-9 → Best (2) |
+| zstd             | 1-3 → Fastest, 4-6 → Default, 7-9 → BetterCompression |
+| xz               | Uses default compression (no configurable level) |
 
 Obvously in case of lz4 and zstd the bottleneck was network. And I have to admit, that network definitely was not
 100Mbit, but it was pretty hard to measure real bandwidth. Anyway let's pretend that during experiment it was roughly
@@ -98,6 +121,52 @@ good enough cypher.
 
 And one more thing user and password are mandatory, we should have at least some minimal barriers, right?
 
+## Supported File Types
+
+Backer handles various file types when creating archives:
+
+- **Regular files**: Included with full content
+- **Directories**: Included as directory entries
+- **Symlinks**: Stored as symlink entries (target path preserved)
+- **Device files**: Character and block devices included with correct major/minor
+- **Named pipes**: Included (header only, no data)
+- **Hard links**: Deduplicated — multiple hard links to the same file are stored as link entries to avoid duplication
+
+## Logging
+
+Backer logs events via Go's `slog`. Log levels: error, warn, info, debug.
+### Backup Completion Log
+
+When a backup completes, the following information is logged:
+- **client**: Client IP address
+- **user**: Authenticated username
+- **files**: Number of files backed up
+- **bytes**: Size of the created archive in bytes
+- **md5**: MD5 hash of the created archive (for verification)
+- **duration**: Time taken to create the archive
+
+Example:
+```
+Backup completed: client=127.0.0.1 user=admin files=42 bytes=1234567 md5=abc123def456 duration=2.5s
+```
+### TLS Error Handling
+
+TLS handshake errors and other client-side SSL/TLS errors are logged at **debug** level to reduce log clutter.
+This includes:
+- Certificate verification failures
+- Handshake failures
+- Protocol errors
+- TLS-specific connection issues
+
+### Graceful Shutdown
+
+When backer receives a termination signal (SIGTERM or SIGINT), it will:
+1. Log a warning message: "Shutdown signal received, shutting down immediately..."
+2. Stop accepting new connections and abort any in-progress downloads immediately
+3. Clean up resources and exit
+
+This ensures no partial/corrupted backups are left. Clients will need to restart their backup. For a forced immediate stop, use `SIGKILL`.
+
 ## Special Thanks
 
 This project was developed with the assistance of **opencode** (AI co-programmer powered by MIMO). The AI helped with:

cmd/backer/main.go

@@ -2,6 +2,7 @@
 package main
 
 import (
+	"errors"
 	"flag"
 	"fmt"
 	"net/http"
@@ -40,10 +41,14 @@ func main() {
 		log.Fatalf("Failed to create server: %v", err)
 	}
 
-	log.Infof("Starting server at %s", server.Addr)
+	// Setup signal handling for immediate shutdown.
+	log.Infof("Installing signal handler")
+	setupSignalHandler(server)
+
+	log.Infof("Starting backer version %s at %s", version, server.Addr)
 
 	if backer.C.NoHTTPS {
-		if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
+		if err := server.Serve(); err != nil && !errors.Is(err, http.ErrServerClosed) {
 			log.Fatalf("Failed to start server: %v", err)
 		}
 	} else {
@@ -55,10 +60,12 @@ func main() {
 			log.Fatalf("Certificate key file error: %v", err)
 		}
 
-		if err := server.ListenAndServeTLS(backer.C.Cert, backer.C.Key); err != nil && err != http.ErrServerClosed {
+		if err := server.ServeTLS(backer.C.Cert, backer.C.Key); err != nil && !errors.Is(err, http.ErrServerClosed) {
 			log.Fatalf("Failed to start server: %v", err)
 		}
 	}
+
+	log.Infof("Server stopped")
 }
 
 // defaultConfigPath returns the default config file path based on the OS.

cmd/backer/sigHandler.go

@@ -0,0 +1,29 @@
+package main
+
+import (
+	"os"
+	"os/signal"
+	"syscall"
+
+	"backer/internal/backer"
+	"backer/internal/log"
+)
+
+// setupSignalHandler sets up signal handling for immediate shutdown.
+// SIGTERM is commonly sent by systemd, k8s, and other container orchestrators.
+// SIGINT is sent on Ctrl+C.
+func setupSignalHandler(server *backer.ServerWrapper) {
+	signalCh := make(chan os.Signal, 1)
+	signal.Notify(signalCh, os.Interrupt, syscall.SIGTERM)
+
+	go func() {
+		<-signalCh
+		log.Warnf("Shutdown signal received, shutting down immediately...")
+		// Close the embedded http.Server directly.
+		if closeErr := server.Close(); closeErr != nil {
+			log.Errorf("Error during server shutdown: %v", closeErr)
+		}
+	}()
+}
+
+/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

contrib/backer.openrc

@@ -7,10 +7,10 @@ pidfile="/run/$SVCNAME/$SVCNAME.pid"
 
 depend() {
         need net
+        use dns
 }
 
 start_pre() {
-        mkdir -p /var/log/$SVCNAME
         mkdir -p /run/$SVCNAME
 }
 

data/config_example.json

@@ -26,27 +26,33 @@
     /* Where to store logs.                                                                                           */
     "log": "/var/log/backer.log",
 
-    /* Verbosity of logs                                                                                              */
+    /* Verbosity of logs.                                                                                             */
     "loglevel": "info",
 
-    /* Diretories to backup                                                                                           */
-    "directories": [
-        "/path/to/directory1",
-        "/path/to/directory2"
-    ],
-
     /* Backup streaming timeout in minutes.                                                                           */
     "backup_timeout": 60,
 
     /* Gzip compression level (1-9, where 9 is best compression).                                                     */
     "compression_level": 9,
 
-    /* Regex patterns to exclude from backup.                                                                         */
-    "exclude_patterns": [],
-
     /* Prefix for backup filename in Content-Disposition header.                                                      */
     "filename_prefix": "backup",
 
-    /* Compression algorithm: gzip, pgzip, bzip2, zstd, lz4 or xz.                                                  */
-    "compression_algorithm": "gzip"
+    /* Compression algorithm alias (deprecated: use default_compression).                                             */
+    "compression_algorithm": "gzip",
+
+    /* Default compression algorithm: gzip, pgzip, bzip2, zstd, lz4 or xz.                                            */
+    "default_compression": "gzip",
+
+    /* Regex patterns to exclude from backup.                                                                         */
+    "exclude_patterns": [],
+
+    /* Maximum allowed time, spent for getting file list from backupped directories.                                  */
+    "dir_scan_timeout": 60,
+
+    /* Directories to backup.                                                                                         */
+    "directories": [
+        "/path/to/directory1",
+        "/path/to/directory2"
+    ]
 }

internal/backer/archive.go

@@ -73,7 +73,14 @@ func createArchiveStream(ctx context.Context, filepaths []string, writerFactory
 }
 
 // writeFilesToTar iterates over filepaths and writes each file to the tar archive.
+// It tracks inodes to handle hard links - if a file has already been archived,
+// subsequent hard links to the same inode are stored as link entries.
+// This is the Unix implementation (syscall not available on Windows).
 func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, closePipeWithError func(error)) {
+	// Track inodes to handle hard links: inode -> first path added.
+	// Uses simple file path tracking on Windows, inode tracking on Unix.
+	seenInodes := make(map[uint64]string)
+
 	for _, fpath := range filepaths {
 		var (
 			err        error
@@ -116,6 +123,21 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 			continue
 		}
 
+		// Check for hard link - if inode was already seen, store as link instead of content.
+		if st.Mode().IsRegular() {
+			inode := getInode(st)
+			if inode != 0 {
+				if existingPath, seen := seenInodes[inode]; seen {
+					// This is a hard link to already-archived file.
+					linkTarget = existingPath
+					log.Debugf("Hard link detected: %s -> %s", fpath, existingPath)
+				} else {
+					// First time seeing this inode - track it.
+					seenInodes[inode] = fpath
+				}
+			}
+		}
+
 		header, err := tar.FileInfoHeader(st, linkTarget)
 
 		if err != nil {
@@ -124,6 +146,11 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 			continue
 		}
 
+		// If linkTarget is set, ensure the header is marked as a link.
+		if linkTarget != "" {
+			header.Typeflag = tar.TypeLink
+		}
+
 		header.Format = tar.FormatGNU
 		header.Name = strings.TrimPrefix(filepath.ToSlash(filepath.Clean(fpath)), "/")
 
@@ -146,7 +173,7 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 				continue
 			}
 
-			err = copyWithContext(ctx, tw, f)
+			_, _, err = copyWithContext(ctx, tw, f)
 
 			if e := f.Close(); e != nil {
 				log.Warnf("Failed to close file %s: %v", fpath, e)
@@ -161,6 +188,27 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 	}
 }
 
+// getInode extracts the inode number from [os.FileInfo].
+// Returns 0 if unable to determine (e.g., on Windows or with certain file systems).
+func getInode(fi os.FileInfo) uint64 {
+	if fi.Sys() == nil {
+		return 0
+	}
+
+	// Use type assertion to get inode - works on Unix, returns 0 on Windows.
+	type inodeGetter interface {
+		Dev() uint64
+		Ino() uint64
+	}
+
+	if ig, ok := fi.Sys().(inodeGetter); ok {
+		// Combine device and inode for uniqueness across file systems.
+		return (ig.Dev() << 32) | ig.Ino()
+	}
+
+	return 0
+}
+
 // isPipeClosedError checks if the error is due to a closed pipe.
 func isPipeClosedError(err error) bool {
 	if err == nil {

internal/backer/bzip2.go

@@ -10,8 +10,22 @@ import (
 // CreateTarBzip2Stream takes a list of files and returns a reader that reads from a bzip2-compressed tar archive.
 func CreateTarBzip2Stream(ctx context.Context, filepaths []string) io.ReadCloser {
 	return createArchiveStream(ctx, filepaths, func(w io.Writer) (io.WriteCloser, error) {
-		return bzip2.NewWriter(w, nil)
+		level := mapCompressionLevelToBzip2(C.CompressionLevel)
+
+		return bzip2.NewWriter(w, &bzip2.WriterConfig{Level: level})
 	})
 }
 
+// mapCompressionLevelToBzip2 maps 1-9 compression level to bzip2 level.
+func mapCompressionLevelToBzip2(level int) int {
+	if level < 1 {
+		return bzip2.DefaultCompression
+	}
+	if level > 9 {
+		return bzip2.BestCompression
+	}
+
+	return level
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/consts.go

@@ -11,6 +11,12 @@ const (
 	// copyBufferSize is the buffer size used for streaming file contents.
 	copyBufferSize = 32 * 1024
 
+	// maxHeaderBytes is the maximum size of request headers (1KB).
+	maxHeaderBytes = 1024
+
+	// idleTimeout is the maximum time to wait for the next request on keep-alive connections.
+	idleTimeout = 10 * time.Second
+
 	// portMin is the minimum valid port number.
 	portMin = 1
 
@@ -23,6 +29,12 @@ const (
 	// backupTimeoutMax is the maximum backup timeout in minutes (24 hours).
 	backupTimeoutMax = 1440
 
+	// dirScanTimeoutMin is the minimum directory scan timeout in minutes.
+	dirScanTimeoutMin = 1
+
+	// dirScanTimeoutMax is the maximum directory scan timeout in minutes (24 hours).
+	dirScanTimeoutMax = 1440
+
 	// compressionLevelMin is the minimum compression level.
 	compressionLevelMin = 1
 
@@ -35,6 +47,9 @@ const (
 	// defaultBackupTimeout is the default backup timeout in minutes.
 	defaultBackupTimeout = 60
 
+	// defaultDirScanTimeout is the default directory scan timeout in minutes (1 hour).
+	defaultDirScanTimeout = 60
+
 	// defaultCompressionLevel is the default compression level.
 	defaultCompressionLevel = 9
 
@@ -50,8 +65,8 @@ const (
 	// defaultFilenamePrefix is the default prefix for backup filenames.
 	defaultFilenamePrefix = "backup"
 
-	// defaultCompressionAlgorithm is the default compression algorithm.
-	defaultCompressionAlgorithm = "gzip"
+	// defaultDefaultCompression is the default compression algorithm.
+	defaultDefaultCompression = "gzip"
 )
 
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/getFilesFromDirectories.go

@@ -1,10 +1,13 @@
 package backer
 
 import (
+	"context"
+	"errors"
 	"io/fs"
+	"os"
 	"path/filepath"
 
-	"backer/internal/log"
+	backerlog "backer/internal/log"
 )
 
 // isExcluded checks if path matches any of the configured exclude patterns.
@@ -22,19 +25,38 @@ func isExcluded(path string) bool {
 }
 
 // GetFilesFromDirectories makes a file list of given directories.
-func GetFilesFromDirectories(directories []string) ([]string, error) {
+// Logs errors for directories that don't exist or can't be accessed but continues with valid directories.
+// Takes context with timeout from config dir_scan_timeout setting.
+func GetFilesFromDirectories(ctx context.Context, directories []string) ([]string, error) {
 	var files []string
 
 	for _, dir := range directories {
+		// Check directory exists before walking.
+		if _, err := os.Stat(dir); err != nil {
+			if errors.Is(err, os.ErrNotExist) {
+				backerlog.Errorf("Configured backup directory not found: %s", dir)
+			} else {
+				backerlog.Errorf("Failed to access configured directory %s: %v", dir, err)
+			}
+
+			continue
+		}
+
 		err := filepath.WalkDir(dir, func(path string, de fs.DirEntry, err error) error { //nolint: revive
+			select {
+			case <-ctx.Done():
+				return ctx.Err()
+			default:
+			}
+
 			if err != nil {
-				log.Warnf("Skipping %s: %v", path, err)
+				backerlog.Warnf("Skipping %s: %v", path, err)
 
 				return nil
 			}
 
 			if isExcluded(path) {
-				log.Debugf("Excluding: %s", path)
+				backerlog.Debugf("Excluding: %s", path)
 
 				return nil
 			}
@@ -45,7 +67,13 @@ func GetFilesFromDirectories(directories []string) ([]string, error) {
 		})
 
 		if err != nil {
-			log.Warnf("Failed to walk directory %s: %v", dir, err)
+			if errors.Is(err, context.DeadlineExceeded) {
+				backerlog.Errorf("Directory scan timed out after scanning %d files", len(files))
+
+				return files, ErrDirectoryScanTimeout{ScannedFiles: len(files)}
+			}
+
+			backerlog.Warnf("Failed to walk directory %s: %v", dir, err)
 		}
 	}
 

internal/backer/getFilesFromDirectories_test.go

@@ -1,7 +1,9 @@
 package backer
 
 import (
+	"context"
 	"os"
+	"path/filepath"
 	"regexp"
 	"slices"
 	"strings"
@@ -13,30 +15,47 @@ func normalizePath(p string) string {
 	return strings.ReplaceAll(p, "\\", "/")
 }
 
+// normalizePathSlice normalizes all paths in a slice.
+func normalizePathSlice(paths []string) []string {
+	result := make([]string, len(paths))
+	for i, p := range paths {
+		result[i] = normalizePath(p)
+	}
+	return result
+}
+
 // TestGetFilesFromDirectories checks if GetFilesFromDirectories returns the proper amount of strings, correct strings and no error.
 func TestGetFilesFromDirectories(t *testing.T) {
-	// Ensure empty_dir exists for the test.
-	os.MkdirAll("../../test_data/test1/foo/empty_dir", 0755)
+	testDataPath := "../../test_data"
+
+	// Log the paths we're checking for debugging
+	t.Logf("Checking directories: %v", []string{
+		filepath.Join(testDataPath, "test1", "foo"),
+		filepath.Join(testDataPath, "test1", "bar"),
+	})
 
 	var (
 		input = Config{
-			Directories: []string{"../../test_data/test1/foo", "../../test_data/test1/bar"},
+			Directories: []string{
+				filepath.Join(testDataPath, "test1", "foo"),
+				filepath.Join(testDataPath, "test1", "bar"),
+			},
 		}
 		expectedData = []string{
-			"../../test_data/test1/foo",
-			"../../test_data/test1/foo/empty_dir",
-			"../../test_data/test1/foo/hello_breakfast.txt",
-			"../../test_data/test1/foo/mydir",
-			"../../test_data/test1/foo/mydir/myfile.txt",
-			"../../test_data/test1/foo/some_text.txt",
-			"../../test_data/test1/bar",
-			"../../test_data/test1/bar/goodbye.txt",
-			"../../test_data/test1/bar/goodbye.txt/text.txt",
-			"../../test_data/test1/bar/hello.txt",
+			filepath.Join(testDataPath, "test1", "foo"),
+			filepath.Join(testDataPath, "test1", "foo", "empty_dir"),
+			filepath.Join(testDataPath, "test1", "foo", "hello_breakfast.txt"),
+			filepath.Join(testDataPath, "test1", "foo", "mydir"),
+			filepath.Join(testDataPath, "test1", "foo", "mydir", "myfile.txt"),
+			filepath.Join(testDataPath, "test1", "foo", "some_text.txt"),
+			filepath.Join(testDataPath, "test1", "bar"),
+			filepath.Join(testDataPath, "test1", "bar", "goodbye.txt"),
+			filepath.Join(testDataPath, "test1", "bar", "goodbye.txt", "text.txt"),
+			filepath.Join(testDataPath, "test1", "bar", "hello.txt"),
 		}
 	)
 
-	output, err := GetFilesFromDirectories(input.Directories)
+	output, err := GetFilesFromDirectories(context.Background(), input.Directories)
 
 	if err != nil {
 		t.Errorf("Expected no error, got %v", err)
@@ -51,7 +70,8 @@ func TestGetFilesFromDirectories(t *testing.T) {
 
 	for _, filepath := range output {
 		normalized := normalizePath(filepath)
-		if !slices.Contains(expectedData, normalized) {
+		normalizedExpected := normalizePathSlice(expectedData)
+		if !slices.Contains(normalizedExpected, normalized) {
 			t.Errorf("expectedData data does not contain string from output: %s", filepath)
 		}
 	}
@@ -59,7 +79,8 @@ func TestGetFilesFromDirectories(t *testing.T) {
 
 // TestGetFilesFromDirectoriesSingleDir tests with a single directory.
 func TestGetFilesFromDirectoriesSingleDir(t *testing.T) {
-	output, err := GetFilesFromDirectories([]string{"../../test_data/test1/foo"})
+	testDataPath := filepath.Join("../../test_data")
+	output, err := GetFilesFromDirectories(context.Background(), []string{filepath.Join(testDataPath, "test1", "foo")})
 
 	if err != nil {
 		t.Errorf("Expected no error, got %v", err)
@@ -70,7 +91,7 @@ func TestGetFilesFromDirectoriesSingleDir(t *testing.T) {
 	}
 
 	// Should contain the directory itself
-	if !slices.Contains(output, "../../test_data/test1/foo") {
+	if !slices.Contains(output, filepath.Join(testDataPath, "test1", "foo")) {
 		t.Error("Output should contain the root directory")
 	}
 }
@@ -80,7 +101,7 @@ func TestGetFilesFromDirectoriesEmptyDir(t *testing.T) {
 	// Create temp empty dir
 	tmpDir := t.TempDir()
 
-	output, err := GetFilesFromDirectories([]string{tmpDir})
+	output, err := GetFilesFromDirectories(context.Background(), []string{tmpDir})
 
 	if err != nil {
 		t.Errorf("Expected no error, got %v", err)
@@ -93,23 +114,48 @@ func TestGetFilesFromDirectoriesEmptyDir(t *testing.T) {
 }
 
 // TestGetFilesFromDirectoriesNonExistent tests with non-existent directory.
+// Should log error but not fail - returns empty list.
 func TestGetFilesFromDirectoriesNonExistent(t *testing.T) {
-	output, err := GetFilesFromDirectories([]string{"/nonexistent/path/that/does/not/exist"})
+	testDataPath := filepath.Join("../../test_data")
+	nonExistentPath := filepath.Join(testDataPath, "nonexistent", "path", "that", "does", "not", "exist")
+	output, err := GetFilesFromDirectories(context.Background(), []string{nonExistentPath})
 
+	// Should not return error, but logs error
 	if err != nil {
-		t.Errorf("Expected no error (logs warning instead), got %v", err)
+		t.Errorf("Expected no error (logs error instead), got %v", err)
 	}
 
+	// Should return empty list
 	if len(output) != 0 {
 		t.Errorf("Expected empty output for non-existent directory, got %d files", len(output))
 	}
 }
 
+// TestGetFilesFromDirectoriesPartialFailure tests with mix of valid and invalid directories.
+func TestGetFilesFromDirectoriesPartialFailure(t *testing.T) {
+	testDataPath := filepath.Join("../../test_data")
+	// Create a temp directory that exists
+	tmpDir := t.TempDir()
+
+	output, err := GetFilesFromDirectories(context.Background(), []string{tmpDir, filepath.Join(testDataPath, "nonexistent", "path", "that", "does", "not", "exist")})
+
+	// Should not return error, but logs error
+	if err != nil {
+		t.Errorf("Expected no error (logs error instead), got %v", err)
+	}
+
+	// Should return files from valid directory
+	if len(output) == 0 {
+		t.Error("Expected files from valid directory")
+	}
+}
+
 // TestGetFilesFromDirectoriesMultipleDirs tests multiple directories.
 func TestGetFilesFromDirectoriesMultipleDirs(t *testing.T) {
-	output, err := GetFilesFromDirectories([]string{
-		"../../test_data/test1/foo",
-		"../../test_data/test1/bar",
+	testDataPath := filepath.Join("../../test_data")
+	output, err := GetFilesFromDirectories(context.Background(), []string{
+		filepath.Join(testDataPath, "test1", "foo"),
+		filepath.Join(testDataPath, "test1", "bar"),
 	})
 
 	if err != nil {
@@ -118,10 +164,10 @@ func TestGetFilesFromDirectoriesMultipleDirs(t *testing.T) {
 
 	// Should contain entries from both directories
 	hasFoo := slices.ContainsFunc(output, func(s string) bool {
-		return s == "../../test_data/test1/foo" || s == "../../test_data/test1/foo/hello_breakfast.txt"
+		return s == filepath.Join(testDataPath, "test1", "foo") || s == filepath.Join(testDataPath, "test1", "foo", "hello_breakfast.txt")
 	})
 	hasBar := slices.ContainsFunc(output, func(s string) bool {
-		return s == "../../test_data/test1/bar" || s == "../../test_data/test1/bar/hello.txt"
+		return s == filepath.Join(testDataPath, "test1", "bar") || s == filepath.Join(testDataPath, "test1", "bar", "hello.txt")
 	})
 
 	if !hasFoo {
@@ -149,7 +195,7 @@ func TestGetFilesFromDirectoriesSymlink(t *testing.T) {
 		t.Skip("Symlinks not supported, skipping test")
 	}
 
-	output, err := GetFilesFromDirectories([]string{tmpDir})
+	output, err := GetFilesFromDirectories(context.Background(), []string{tmpDir})
 
 	if err != nil {
 		t.Errorf("Expected no error, got %v", err)
@@ -228,7 +274,7 @@ func TestGetFilesFromDirectoriesWithExcludes(t *testing.T) {
 		regexp.MustCompile(`\.tmp$`),
 	}
 
-	output, err := GetFilesFromDirectories([]string{tmpDir})
+	output, err := GetFilesFromDirectories(context.Background(), []string{tmpDir})
 	if err != nil {
 		t.Fatalf("Unexpected error: %v", err)
 	}

internal/backer/loadConfig.go

@@ -95,6 +95,17 @@ func LoadConfig(path string) error {
 		return fmt.Errorf("Config option backup_timeout must be between %d and %d minutes, got %d", backupTimeoutMin, backupTimeoutMax, C.BackupTimeout)
 	}
 
+	// DirScanTimeout: set default, then validate range.
+	if C.DirScanTimeout == 0 {
+		C.DirScanTimeout = defaultDirScanTimeout
+
+		log.Warnf("Config option dir_scan_timeout is not set, fallback to %d minutes", C.DirScanTimeout)
+	}
+
+	if C.DirScanTimeout < dirScanTimeoutMin || C.DirScanTimeout > dirScanTimeoutMax {
+		return fmt.Errorf("Config option dir_scan_timeout must be between %d and %d minutes, got %d", dirScanTimeoutMin, dirScanTimeoutMax, C.DirScanTimeout)
+	}
+
 	// CompressionLevel: set default, then validate range.
 	if C.CompressionLevel == 0 {
 		C.CompressionLevel = defaultCompressionLevel
@@ -135,15 +146,20 @@ func LoadConfig(path string) error {
 		log.Warnf("Config option filename_prefix is not set, fallback to %s", C.FilenamePrefix)
 	}
 
-	// CompressionAlgorithm: default and validate.
-	if C.CompressionAlgorithm == "" {
-		C.CompressionAlgorithm = defaultCompressionAlgorithm
-
-		log.Warnf("Config option compression_algorithm is not set, fallback to %s", C.CompressionAlgorithm)
+	// DefaultCompression: default and validate. If compression_algorithm is set (deprecated), use it as fallback.
+	if C.DefaultCompression == "" {
+		if C.CompressionAlgorithm != "" {
+			C.DefaultCompression = C.CompressionAlgorithm
+		} else {
+			C.DefaultCompression = defaultDefaultCompression
+		}
 	}
 
-	if C.CompressionAlgorithm != "gzip" && C.CompressionAlgorithm != "bzip2" && C.CompressionAlgorithm != "zstd" && C.CompressionAlgorithm != "lz4" && C.CompressionAlgorithm != "xz" && C.CompressionAlgorithm != "pgzip" {
-		return fmt.Errorf("Config option compression_algorithm must be gzip, bzip2, zstd, lz4, xz or pgzip, got %s", C.CompressionAlgorithm)
+	switch C.DefaultCompression {
+	case "gzip", "pgzip", "bzip2", "xz", "lz4", "zstd":
+		// No-op: compression algorithm validated.
+	default:
+		return fmt.Errorf("Config option default_compression must be gzip, bzip2, zstd, lz4, xz or pgzip, got %s", C.DefaultCompression)
 	}
 
 	// Compile exclude patterns.

internal/backer/loadConfig_test.go

@@ -1,7 +1,9 @@
 package backer
 
 import (
+	"encoding/json"
 	"os"
+	"path/filepath"
 	"slices"
 	"testing"
 )
@@ -10,18 +12,46 @@ func TestLoadConfig(t *testing.T) {
 	// Reset global config
 	C = Config{}
 
-	var (
-		testConfigPath = "./../../test_data/test_config.json"
-		expectedData   = Config{
+	testDataPath := filepath.Join(t.TempDir(), "test_data")
+	os.MkdirAll(filepath.Join(testDataPath, "test1", "foo"), 0755)
+	os.MkdirAll(filepath.Join(testDataPath, "test1", "bar"), 0755)
+
+	// Create test config file
+	testConfigPath := filepath.Join(testDataPath, "test_config.json")
+	testStruct := struct {
+		Address     string   `json:"address"`
+		Port        int      `json:"port"`
+		Directories []string `json:"directories"`
+		Location    string   `json:"location"`
+		User        string   `json:"user"`
+		Password    string   `json:"password"`
+		NoHTTPS     bool     `json:"nohttps"`
+	}{
+		Address:     "127.0.0.1",
+		Port:        8086,
+		Directories: []string{filepath.Join(testDataPath, "test1", "foo"), filepath.Join(testDataPath, "test1", "bar")},
+		Location:    "/archive",
+		User:        "test",
+		Password:    "test",
+		NoHTTPS:     true,
+	}
+	testConfigBytes, _ := json.MarshalIndent(testStruct, "", "  ")
+	if err := os.WriteFile(testConfigPath, testConfigBytes, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	expectedData := Config{
 		Address: "127.0.0.1",
 		Port:    8086,
 		Directories: []string{
-				"../../test_data/test1/foo",
-				"../../test_data/test1/bar",
+			filepath.Join(testDataPath, "test1", "foo"),
+			filepath.Join(testDataPath, "test1", "bar"),
 		},
 		Location: "/archive",
+		User:     "test",
+		Password: "test",
+		NoHTTPS:  true,
 	}
-	)
 
 	err := LoadConfig(testConfigPath)
 
@@ -57,6 +87,8 @@ func TestLoadConfig(t *testing.T) {
 
 // TestLoadConfigMissingUser tests that config fails when user is missing.
 func TestLoadConfigMissingUser(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"port": 8086,
 		"password": "test",
@@ -72,6 +104,8 @@ func TestLoadConfigMissingUser(t *testing.T) {
 
 // TestLoadConfigMissingPassword tests that config fails when password is missing.
 func TestLoadConfigMissingPassword(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"port": 8086,
 		"user": "test",
@@ -87,6 +121,8 @@ func TestLoadConfigMissingPassword(t *testing.T) {
 
 // TestLoadConfigMissingDirectories tests that config fails when directories is missing.
 func TestLoadConfigMissingDirectories(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"port": 8086,
 		"user": "test",
@@ -102,6 +138,8 @@ func TestLoadConfigMissingDirectories(t *testing.T) {
 
 // TestLoadConfigInvalidPort tests that config fails with invalid port.
 func TestLoadConfigInvalidPort(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"port": 99999,
 		"user": "test",
@@ -118,6 +156,8 @@ func TestLoadConfigInvalidPort(t *testing.T) {
 
 // TestLoadConfigNonExistentDirectory tests that config fails with non-existent directory.
 func TestLoadConfigNonExistentDirectory(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"port": 8086,
 		"user": "test",
@@ -134,6 +174,8 @@ func TestLoadConfigNonExistentDirectory(t *testing.T) {
 
 // TestLoadConfigHTTPSWithoutCert tests that config fails when HTTPS enabled but no cert.
 func TestLoadConfigHTTPSWithoutCert(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"port": 8086,
 		"user": "test",
@@ -151,6 +193,8 @@ func TestLoadConfigHTTPSWithoutCert(t *testing.T) {
 
 // TestLoadConfigNoHTTPS tests that config succeeds with nohttps=true and no cert.
 func TestLoadConfigNoHTTPS(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"port": 8086,
 		"user": "test",
@@ -172,6 +216,8 @@ func TestLoadConfigNoHTTPS(t *testing.T) {
 
 // TestLoadConfigFilenamePrefixCustom tests custom filename prefix.
 func TestLoadConfigFilenamePrefixCustom(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -193,6 +239,8 @@ func TestLoadConfigFilenamePrefixCustom(t *testing.T) {
 
 // TestLoadConfigDefaultValues tests default values are set correctly.
 func TestLoadConfigDefaultValues(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -227,13 +275,15 @@ func TestLoadConfigDefaultValues(t *testing.T) {
 		t.Errorf("C.FilenamePrefix should default to backup, got %s", C.FilenamePrefix)
 	}
 
-	if C.CompressionAlgorithm != "gzip" {
-		t.Errorf("C.CompressionAlgorithm should default to gzip, got %s", C.CompressionAlgorithm)
+	if C.DefaultCompression != "gzip" {
+		t.Errorf("C.DefaultCompression should default to gzip, got %s", C.DefaultCompression)
 	}
 }
 
 // TestLoadConfigCompressionAlgorithmBzip2 tests bzip2 compression algorithm.
 func TestLoadConfigCompressionAlgorithmBzip2(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -248,13 +298,15 @@ func TestLoadConfigCompressionAlgorithmBzip2(t *testing.T) {
 		t.Errorf("Expected no error, got %v", err)
 	}
 
-	if C.CompressionAlgorithm != "bzip2" {
-		t.Errorf("C.CompressionAlgorithm should be bzip2, got %s", C.CompressionAlgorithm)
+	if C.DefaultCompression != "bzip2" {
+		t.Errorf("C.DefaultCompression should be bzip2, got %s", C.DefaultCompression)
 	}
 }
 
 // TestLoadConfigCompressionAlgorithmZstd tests zstd compression algorithm.
 func TestLoadConfigCompressionAlgorithmZstd(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -269,13 +321,15 @@ func TestLoadConfigCompressionAlgorithmZstd(t *testing.T) {
 		t.Errorf("Expected no error, got %v", err)
 	}
 
-	if C.CompressionAlgorithm != "zstd" {
-		t.Errorf("C.CompressionAlgorithm should be zstd, got %s", C.CompressionAlgorithm)
+	if C.DefaultCompression != "zstd" {
+		t.Errorf("C.DefaultCompression should be zstd, got %s", C.DefaultCompression)
 	}
 }
 
 // TestLoadConfigCompressionAlgorithmLz4 tests lz4 compression algorithm.
 func TestLoadConfigCompressionAlgorithmLz4(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -290,13 +344,15 @@ func TestLoadConfigCompressionAlgorithmLz4(t *testing.T) {
 		t.Errorf("Expected no error, got %v", err)
 	}
 
-	if C.CompressionAlgorithm != "lz4" {
-		t.Errorf("C.CompressionAlgorithm should be lz4, got %s", C.CompressionAlgorithm)
+	if C.DefaultCompression != "lz4" {
+		t.Errorf("C.DefaultCompression should be lz4, got %s", C.DefaultCompression)
 	}
 }
 
 // TestLoadConfigCompressionAlgorithmXz tests xz compression algorithm.
 func TestLoadConfigCompressionAlgorithmXz(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -311,13 +367,15 @@ func TestLoadConfigCompressionAlgorithmXz(t *testing.T) {
 		t.Errorf("Expected no error, got %v", err)
 	}
 
-	if C.CompressionAlgorithm != "xz" {
-		t.Errorf("C.CompressionAlgorithm should be xz, got %s", C.CompressionAlgorithm)
+	if C.DefaultCompression != "xz" {
+		t.Errorf("C.DefaultCompression should be xz, got %s", C.DefaultCompression)
 	}
 }
 
 // TestLoadConfigCompressionAlgorithmPgzip tests pgzip compression algorithm.
 func TestLoadConfigCompressionAlgorithmPgzip(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -332,13 +390,15 @@ func TestLoadConfigCompressionAlgorithmPgzip(t *testing.T) {
 		t.Errorf("Expected no error, got %v", err)
 	}
 
-	if C.CompressionAlgorithm != "pgzip" {
-		t.Errorf("C.CompressionAlgorithm should be pgzip, got %s", C.CompressionAlgorithm)
+	if C.DefaultCompression != "pgzip" {
+		t.Errorf("C.DefaultCompression should be pgzip, got %s", C.DefaultCompression)
 	}
 }
 
 // TestLoadConfigCompressionAlgorithmInvalid tests invalid compression algorithm.
 func TestLoadConfigCompressionAlgorithmInvalid(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -367,7 +427,6 @@ func writeTempConfig(t *testing.T, content string) string {
 	tmpFile.Close()
 	// Reset global config before loading
 	C = Config{}
-	excludePatterns = nil
 	return tmpFile.Name()
 }
 

internal/backer/lz4.go

@@ -10,8 +10,26 @@ import (
 // CreateTarLz4Stream takes a list of files and returns a reader that reads from an lz4-compressed tar archive.
 func CreateTarLz4Stream(ctx context.Context, filepaths []string) io.ReadCloser {
 	return createArchiveStream(ctx, filepaths, func(w io.Writer) (io.WriteCloser, error) {
-		return lz4.NewWriter(w), nil
+		lz4Writer := lz4.NewWriter(w)
+		// Map compression_level (1-9) to lz4 compression level.
+		// 1-3: fast, 4-6: balanced, 7-9: better compression.
+		lz4Writer.Header.CompressionLevel = mapCompressionLevelToLz4(C.CompressionLevel) //nolint: staticcheck
+
+		return lz4Writer, nil
 	})
 }
 
+// mapCompressionLevelToLz4 maps 1-9 compression level to lz4 compression level.
+// 1-3: fast, 4-6: balanced, 7-9: better compression.
+func mapCompressionLevelToLz4(level int) int {
+	switch {
+	case level <= 3:
+		return 0 // Fastest.
+	case level <= 6:
+		return 1 // Default.
+	default:
+		return 2 // Best compression.
+	}
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/newServer.go

@@ -2,17 +2,69 @@ package backer
 
 import (
 	"context"
+	"crypto/md5"
+	"crypto/subtle"
 	"crypto/tls"
+	"encoding/hex"
+	"errors"
 	"fmt"
 	"io"
 	"net"
 	"net/http"
+	"path"
 	"strings"
 	"time"
 
 	"backer/internal/log"
 )
 
+// isTLSError checks if the given error is TLS-related.
+// It checks for both TLS-specific error types and error messages containing "tls:".
+func isTLSError(err error) bool {
+	if err == nil {
+		return false
+	}
+
+	var alertErr *tls.AlertError
+	if errors.As(err, &alertErr) && alertErr != nil {
+		return true
+	}
+
+	var certErr *tls.CertificateVerificationError
+	if errors.As(err, &certErr) && certErr != nil {
+		return true
+	}
+
+	var echErr *tls.ECHRejectionError
+	if errors.As(err, &echErr) && echErr != nil {
+		return true
+	}
+
+	var recordErr *tls.RecordHeaderError
+	if errors.As(err, &recordErr) && recordErr != nil {
+		return true
+	}
+
+	return strings.HasPrefix(err.Error(), "tls:")
+}
+
+// withStatusLogging wraps a handler to log 404 and 405 responses.
+func withStatusLogging(next http.Handler) http.Handler {
+	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		rw := &responseWriterWrapper{ResponseWriter: w, statusCode: http.StatusOK}
+		next.ServeHTTP(rw, r)
+
+		clientIP := getClientIP(r)
+
+		switch rw.statusCode {
+		case http.StatusNotFound:
+			log.Infof("Not Found from %s to %s", clientIP, r.RequestURI)
+		case http.StatusMethodNotAllowed:
+			log.Infof("Method Not Allowed from %s for %s %s", clientIP, r.Method, r.RequestURI)
+		}
+	})
+}
+
 // withServerHeader wraps a handler to add Server header to all responses.
 func withServerHeader(next http.Handler) http.Handler {
 	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -22,7 +74,8 @@ func withServerHeader(next http.Handler) http.Handler {
 }
 
 // NewServer starts new server instance with given config file.
-func NewServer(configPath string) (*http.Server, error) {
+// It returns a ServerWrapper that intercepts and classifies server errors.
+func NewServer(configPath string) (*ServerWrapper, error) {
 	_ = log.Init("info", "") // Initialize logger with defaults for early logging.
 
 	if err := LoadConfig(configPath); err != nil {
@@ -33,9 +86,18 @@ func NewServer(configPath string) (*http.Server, error) {
 		log.Errorf("Failed to initialize logger: %v", err)
 	}
 
+	log.Infof("Using %s as default compression algorithm", C.DefaultCompression)
+
 	mux := http.NewServeMux()
 
-	mux.HandleFunc(C.Location, func(w http.ResponseWriter, r *http.Request) {
+	backupHandler := func(w http.ResponseWriter, r *http.Request) {
+		// Only GET method is allowed.
+		if r.Method != http.MethodGet {
+			http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
+
+			return
+		}
+
 		username, password, ok := r.BasicAuth()
 
 		// No auth? Good bye.
@@ -43,16 +105,18 @@ func NewServer(configPath string) (*http.Server, error) {
 			w.Header().Set("WWW-Authenticate", `Basic realm="restricted", charset="UTF-8"`)
 			http.Error(w, "Unauthorized", http.StatusUnauthorized)
 
+			log.Infof("Access denied from %s to %s (no username or password given)", r.RemoteAddr, r.RequestURI)
+
 			return
 		}
 
-		// Both username AND password must match. Fuck De Morgan and his bullshit laws - we explicitly require
-		// that the whole condition fails. That *is* the point.
-		//nolint:staticcheck // QF1001: De Morgan's law is irrelevant here.
-		if !(username == C.User && password == C.Password) {
+		// Both username AND password must match. Use timing-safe comparison to prevent timing attacks.
+		if !timingSafeMatch(username, C.User) || !timingSafeMatch(password, C.Password) {
 			w.Header().Set("WWW-Authenticate", `Basic realm="restricted", charset="UTF-8"`)
 			http.Error(w, "Unauthorized", http.StatusUnauthorized)
 
+			log.Infof("Access denied from %s to %s (username and/or password incorrect)", r.RemoteAddr, r.RequestURI)
+
 			return
 		}
 
@@ -61,31 +125,56 @@ func NewServer(configPath string) (*http.Server, error) {
 
 		log.Infof("Backup started: client=%s user=%s", clientIP, username)
 
-		ctx := r.Context()
-		files, err := GetFilesFromDirectories(C.Directories)
+		// Create context with dir_scan_timeout from request context.
+		ctx, cancel := context.WithTimeout(r.Context(), time.Duration(C.DirScanTimeout)*time.Minute)
+		defer cancel()
+
+		files, err := GetFilesFromDirectories(ctx, C.Directories)
 
 		if err != nil {
+			switch {
+			case errors.Is(err, context.DeadlineExceeded):
+				w.Header().Set("Server", "backer")
+				http.Error(w, "Directory scan timed out", http.StatusRequestTimeout)
+
+				log.Errorf("Directory scan timed out: %v", err)
+
+			default:
 				log.Errorf("Failed to get files: %v", err)
 
 				w.Header().Set("Server", "backer")
 				http.Error(w, "Failed to get files", http.StatusInternalServerError)
+			}
 
 			return
 		}
 
+		if len(files) == 0 {
+			log.Warnf("No files found to backup in configured directories")
+		}
+
 		var (
 			archive   io.ReadCloser
 			extension string
+			algorithm string
 		)
 
-		switch C.CompressionAlgorithm {
+		requestedExt := r.URL.Path
+
+		algorithm = getCompressionAlgorithm(requestedExt)
+
+		switch algorithm {
 		case "bzip2":
 			archive = CreateTarBzip2Stream(ctx, files)
 			extension = "tar.bz2"
 
 		case "zstd":
 			archive = CreateTarZstdStream(ctx, files)
+			if strings.HasSuffix(requestedExt, ".zst") {
 				extension = "tar.zst"
+			} else {
+				extension = "tar.zstd"
+			}
 
 		case "lz4":
 			archive = CreateTarLz4Stream(ctx, files)
@@ -110,55 +199,80 @@ func NewServer(configPath string) (*http.Server, error) {
 			}
 		}()
 
-		timestamp := time.Now().Format("20060102-150405")
+		timestamp := backupStart.Format("20060102-150405")
 		filename := fmt.Sprintf("%s-%s.%s", C.FilenamePrefix, timestamp, extension)
 
 		w.Header().Set("Content-Disposition", "attachment; filename="+filename)
 		w.Header().Set("Content-Type", "application/octet-stream")
 
-		bytesWritten, err := io.Copy(w, archive)
+		bytesWritten, md5Hash, err := copyWithContext(ctx, w, archive)
+		if err != nil {
 			duration := time.Since(backupStart)
 
-		if err != nil {
+			if errors.Is(err, context.Canceled) {
+				log.Infof("Backup canceled: client=%s user=%s duration=%s", clientIP, username, duration)
+
+				return
+			}
+
+			if isPipeClosedError(err) {
+				log.Infof("Client disconnected during backup: client=%s user=%s", clientIP, username)
+
+				return
+			}
+
 			log.Errorf("Backup failed: client=%s user=%s duration=%s error=%v", clientIP, username, duration, err)
 
 			return
 		}
 
-		log.Infof("Backup completed: client=%s user=%s files=%d bytes=%d duration=%s", clientIP, username, len(files), bytesWritten, duration)
-	})
-
-	// Decision about which mode (http or https) to run made in cmd/backer/main.go, where we run server.ListenAndServe()
-	// or server.ListenAndServeTLS() depending on value of C.NoHTTPS.
-	if C.NoHTTPS {
-		return &http.Server{
-			Addr:              fmt.Sprintf("%s:%d", C.Address, C.Port),
-			Handler:           withServerHeader(mux),
-			ReadHeaderTimeout: readHeaderTimeout,
-			WriteTimeout:      time.Duration(C.BackupTimeout) * time.Minute,
-		}, nil
+		log.Infof("Backup completed: client=%s user=%s files=%d bytes=%d md5=%s duration=%s", clientIP, username, len(files), bytesWritten, md5Hash, time.Since(backupStart))
 	}
 
-	// Decision about which mode (http or https) to run made in cmd/backer/main.go, where we run server.ListenAndServe()
-	// or server.ListenAndServeTLS() depending on value of C.NoHTTPS.
-	if C.NoHTTPS {
-		return &http.Server{
-			Addr:              fmt.Sprintf("%s:%d", C.Address, C.Port),
-			Handler:           withServerHeader(mux),
-			ReadHeaderTimeout: readHeaderTimeout,
-			WriteTimeout:      time.Duration(C.BackupTimeout) * time.Minute,
-		}, nil
+	mux.Handle(C.Location, http.HandlerFunc(backupHandler))
+
+	extensions := []string{".tar.gz", ".tar.xz", ".tar.bz2", ".tar.lz4", ".tar.zst", ".tar.zstd"}
+	for _, ext := range extensions {
+		muxPath := C.Location + ext
+		mux.Handle(muxPath, http.HandlerFunc(backupHandler))
 	}
 
-	return &http.Server{
+	// HTTP mode if nohttps is enabled in config.
+	// HTTPS mode if nohttps is disabled in config.
+	if C.NoHTTPS {
+		srv := &ServerWrapper{
+			Server: &http.Server{
 				Addr:              fmt.Sprintf("%s:%d", C.Address, C.Port),
-		Handler:           withServerHeader(mux),
-		ReadHeaderTimeout: 5 * time.Second,
+				Handler:           withServerHeader(withStatusLogging(mux)),
+				ReadHeaderTimeout: readHeaderTimeout,
 				WriteTimeout:      time.Duration(C.BackupTimeout) * time.Minute,
+				IdleTimeout:       idleTimeout,
+				MaxHeaderBytes:    maxHeaderBytes,
+				ErrorLog:          log.DebugLogger(),
+			},
+		}
+		srv.SetKeepAlivesEnabled(false)
+
+		return srv, nil
+	}
+
+	srv := &ServerWrapper{
+		Server: &http.Server{
+			Addr:              fmt.Sprintf("%s:%d", C.Address, C.Port),
+			Handler:           withServerHeader(withStatusLogging(mux)),
+			ReadHeaderTimeout: readHeaderTimeout,
+			WriteTimeout:      time.Duration(C.BackupTimeout) * time.Minute,
+			IdleTimeout:       idleTimeout,
+			MaxHeaderBytes:    maxHeaderBytes,
 			TLSConfig: &tls.Config{
 				MinVersion: tls.VersionTLS13,
 			},
-	}, nil
+			ErrorLog: log.DebugLogger(),
+		},
+	}
+	srv.SetKeepAlivesEnabled(false)
+
+	return srv, nil
 }
 
 // writeWithContext checks if context is canceled before executing fn.
@@ -174,31 +288,50 @@ func writeWithContext(ctx context.Context, fn func() error) error {
 }
 
 // copyWithContext copies data from src to dst using a 32KB buffer, respecting
-// context cancellation. If the context is canceled, the copy stops and returns
-// ctx.Err().
-func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) error {
+// context cancellation. Returns bytes written, MD5 hash (hex string), and any error encountered.
+//
+//nolint:gosec
+func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) (int64, string, error) {
 	buf := make([]byte, copyBufferSize)
 
+	var bytesWritten int64
+
+	hasher := md5.New()
+
 	for {
 		select {
 		case <-ctx.Done():
-			return ctx.Err()
+			return bytesWritten, "", ctx.Err()
 		default:
 		}
 
 		n, readErr := src.Read(buf)
 		if n > 0 {
-			if _, werr := dst.Write(buf[:n]); werr != nil {
-				return werr
+			select {
+			case <-ctx.Done():
+				return bytesWritten, "", ctx.Err()
+			default:
 			}
+
+			if _, werr := hasher.Write(buf[:n]); werr != nil {
+				return bytesWritten, "", werr
+			}
+
+			if _, werr := dst.Write(buf[:n]); werr != nil {
+				return bytesWritten, "", werr
+			}
+
+			bytesWritten += int64(n)
 		}
 
 		if readErr != nil {
 			if readErr == io.EOF {
-				return nil
+				hash := hex.EncodeToString(hasher.Sum(nil))
+
+				return bytesWritten, hash, nil
 			}
 
-			return readErr
+			return bytesWritten, "", readErr
 		}
 	}
 }
@@ -218,4 +351,102 @@ func getClientIP(r *http.Request) string {
 	return host
 }
 
+// getCompressionAlgorithm determines which compression algorithm to use based on the requested extension.
+// If the extension is recognized, it returns the corresponding algorithm; otherwise, it returns the default.
+// For .tar.gz extension, pgzip is always used for parallel compression.
+// The input can be either a full path like "/archive.tar.xz" or just the extension like "tar.xz" or "xz".
+func getCompressionAlgorithm(input string) string {
+	// Extract extension from full path (e.g., "/archive.tar.xz" -> ".xz").
+	// or use the input directly if it's just an extension (e.g., "tar.xz" -> ".xz").
+	ext := path.Ext(input)
+	if ext == "" {
+		ext = "." + input
+	}
+
+	switch ext {
+	case ".tar.bz2":
+		return "bzip2"
+	case ".tar.zstd":
+		return "zstd"
+	case ".tar.zst":
+		return "zstd"
+	case ".tar.lz4":
+		return "lz4"
+	case ".tar.xz":
+		return "xz"
+	case ".tar.gz":
+		return "pgzip"
+	case ".xz":
+		return "xz"
+	case ".bz2":
+		return "bzip2"
+	case ".zst":
+		return "zstd"
+	case ".zstd":
+		return "zstd"
+	case ".lz4":
+		return "lz4"
+	case ".gz":
+		return "pgzip"
+	default:
+		return C.DefaultCompression
+	}
+}
+
+type (
+	// ServerWrapper wraps [http.Server] to intercept and classify errors.
+	ServerWrapper struct {
+		*http.Server
+	}
+
+	// responseWriterWrapper wraps http.ResponseWriter to capture status code.
+	responseWriterWrapper struct {
+		http.ResponseWriter
+		statusCode int
+	}
+)
+
+// WriteHeader captures the status code for logging.
+func (rw *responseWriterWrapper) WriteHeader(code int) {
+	rw.statusCode = code
+	rw.ResponseWriter.WriteHeader(code)
+}
+
+// Serve starts the server and classifies any errors returned.
+// TLS-related errors are logged at debug level, other errors at warn level.
+func (sw *ServerWrapper) Serve() error {
+	err := sw.ListenAndServe()
+
+	if err != nil {
+		if isTLSError(err) {
+			log.Debugf("Server error: %v", err)
+		} else {
+			log.Warnf("Server error: %v", err)
+		}
+	}
+
+	return err
+}
+
+// ServeTLS starts the TLS server and classifies any errors returned.
+func (sw *ServerWrapper) ServeTLS(certFile, keyFile string) error {
+	err := sw.ListenAndServeTLS(certFile, keyFile)
+
+	if err != nil {
+		if isTLSError(err) {
+			log.Debugf("Server TLS error: %v", err)
+		} else {
+			log.Warnf("Server TLS error: %v", err)
+		}
+	}
+
+	return err
+}
+
+// timingSafeMatch performs constant-time comparison to prevent timing attacks.
+// Returns 1 if the strings are equal, 0 otherwise.
+func timingSafeMatch(a, b string) bool {
+	return subtle.ConstantTimeCompare([]byte(a), []byte(b)) == 1
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/newServer_test.go

@@ -4,6 +4,8 @@ import (
 	"archive/tar"
 	"compress/gzip"
 	"context"
+	"crypto/tls"
+	"errors"
 	"io"
 	"net/http"
 	"net/http/httptest"
@@ -11,6 +13,166 @@ import (
 	"testing"
 )
 
+// TestIsTLSError tests the isTLSError function.
+func TestIsTLSError(t *testing.T) {
+	tests := []struct {
+		name     string
+		err      error
+		expected bool
+	}{
+		{
+			name:     "nil error",
+			err:      nil,
+			expected: false,
+		},
+		{
+			name:     "regular error",
+			err:      errors.New("some random error"),
+			expected: false,
+		},
+		{
+			name:     "TLS prefix error",
+			err:      errors.New("tls: handshake failure"),
+			expected: true,
+		},
+		{
+			name:     "TLS AlertError",
+			err:      tls.AlertError(80),
+			expected: true,
+		},
+		{
+			name:     "TLS CertificateVerificationError",
+			err:      &tls.CertificateVerificationError{},
+			expected: true,
+		},
+		{
+			name:     "TLS ECHRejectionError",
+			err:      &tls.ECHRejectionError{RetryConfigList: []byte("rejected")},
+			expected: true,
+		},
+		{
+			name:     "TLS RecordHeaderError",
+			err:      &tls.RecordHeaderError{},
+			expected: true,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			result := isTLSError(tt.err)
+			if result != tt.expected {
+				t.Errorf("isTLSError(%v) = %v, expected %v", tt.err, result, tt.expected)
+			}
+		})
+	}
+}
+
+// TestGetClientIP tests the getClientIP function.
+func TestGetClientIP(t *testing.T) {
+	tests := []struct {
+		name       string
+		xffHeader  string
+		remoteAddr string
+		expectedIP string
+	}{
+		{
+			name:       "no X-Forwarded-For",
+			xffHeader:  "",
+			remoteAddr: "192.168.1.100:12345",
+			expectedIP: "192.168.1.100",
+		},
+		{
+			name:       "single IP in X-Forwarded-For",
+			xffHeader:  "10.0.0.1",
+			remoteAddr: "192.168.1.100:12345",
+			expectedIP: "10.0.0.1",
+		},
+		{
+			name:       "multiple IPs in X-Forwarded-For",
+			xffHeader:  "10.0.0.1, 192.168.1.100, 172.16.0.1",
+			remoteAddr: "192.168.1.100:12345",
+			expectedIP: "10.0.0.1",
+		},
+		{
+			name:       "X-Forwarded-For with spaces",
+			xffHeader:  "  10.0.0.1  ,  192.168.1.100  ",
+			remoteAddr: "192.168.1.100:12345",
+			expectedIP: "10.0.0.1",
+		},
+		{
+			name:       "invalid RemoteAddr",
+			xffHeader:  "",
+			remoteAddr: "invalid",
+			expectedIP: "invalid",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			req := httptest.NewRequest("GET", "/archive", nil)
+			if tt.xffHeader != "" {
+				req.Header.Set("X-Forwarded-For", tt.xffHeader)
+			}
+			req.RemoteAddr = tt.remoteAddr
+
+			result := getClientIP(req)
+			if result != tt.expectedIP {
+				t.Errorf("getClientIP() = %q, expected %q", result, tt.expectedIP)
+			}
+		})
+	}
+}
+
+// TestGetCompressionAlgorithm tests the getCompressionAlgorithm function.
+func TestGetCompressionAlgorithm(t *testing.T) {
+	tests := []struct {
+		name     string
+		ext      string
+		expected string
+	}{
+		{
+			name:     "tar.bz2",
+			ext:      "tar.bz2",
+			expected: "bzip2",
+		},
+		{
+			name:     "tar.zst",
+			ext:      "tar.zst",
+			expected: "zstd",
+		},
+		{
+			name:     "tar.lz4",
+			ext:      "tar.lz4",
+			expected: "lz4",
+		},
+		{
+			name:     "tar.xz",
+			ext:      "tar.xz",
+			expected: "xz",
+		},
+		{
+			name:     "tar.gz",
+			ext:      "tar.gz",
+			expected: "pgzip",
+		},
+		{
+			name:     "default with gzip config",
+			ext:      "",
+			expected: "gzip",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			C = Config{DefaultCompression: "gzip"}
+			result := getCompressionAlgorithm(tt.ext)
+			if result != tt.expected {
+				t.Errorf("getCompressionAlgorithm(%q) = %q, expected %q", tt.ext, result, tt.expected)
+			}
+		})
+	}
+}
+
 // TestNewServerSuccess tests successful server creation.
 func TestNewServerSuccess(t *testing.T) {
 	C = Config{} // Reset global config.
@@ -189,7 +351,7 @@ func TestHandlerArchiveStream(t *testing.T) {
 		}
 
 		ctx := r.Context()
-		files, err := GetFilesFromDirectories(C.Directories)
+		files, err := GetFilesFromDirectories(ctx, C.Directories)
 		if err != nil {
 			http.Error(w, "Failed to get files", http.StatusInternalServerError)
 			return
@@ -301,7 +463,7 @@ func TestCopyWithContextSuccess(t *testing.T) {
 	src := strings.NewReader("hello world")
 	dst := &strings.Builder{}
 
-	err := copyWithContext(ctx, dst, src)
+	_, _, err := copyWithContext(ctx, dst, src)
 	if err != nil {
 		t.Errorf("Expected no error, got %v", err)
 	}
@@ -311,6 +473,25 @@ func TestCopyWithContextSuccess(t *testing.T) {
 	}
 }
 
+// TestCopyWithContextMD5Hash tests that MD5 hash is correctly computed.
+func TestCopyWithContextMD5Hash(t *testing.T) {
+	ctx := context.Background()
+	data := "hello world"
+	src := strings.NewReader(data)
+	dst := &strings.Builder{}
+
+	_, md5Hash, err := copyWithContext(ctx, dst, src)
+	if err != nil {
+		t.Errorf("Expected no error, got %v", err)
+	}
+
+	// MD5 of "hello world" is 5eb63bbbe01eeed093cb22bb8f5acdc3
+	expectedMD5 := "5eb63bbbe01eeed093cb22bb8f5acdc3"
+	if md5Hash != expectedMD5 {
+		t.Errorf("Expected MD5 %s, got %s", expectedMD5, md5Hash)
+	}
+}
+
 // TestCopyWithContextCancellation tests copy with cancelled context.
 func TestCopyWithContextCancellation(t *testing.T) {
 	ctx, cancel := context.WithCancel(context.Background())
@@ -325,17 +506,116 @@ func TestCopyWithContextCancellation(t *testing.T) {
 
 	// Start copy in goroutine
 	errCh := make(chan error, 1)
+	bytesCh := make(chan int64, 1)
 	go func() {
-		errCh <- copyWithContext(ctx, dst, src)
+		n, _, err := copyWithContext(ctx, dst, src)
+		bytesCh <- n
+		errCh <- err
 	}()
 
 	// Cancel context immediately
 	cancel()
 
 	err := <-errCh
+	<-bytesCh
 	if err != context.Canceled {
 		t.Errorf("Expected context.Canceled, got %v", err)
 	}
 }
 
+// TestStatusLogging404 tests that 404 responses are logged.
+func TestStatusLogging404(t *testing.T) {
+	C = Config{} // Reset global config.
+	excludePatterns = nil
+
+	// Create a minimal mux that returns 404 for all paths
+	testMux := http.NewServeMux()
+	testMux.HandleFunc("/archive", func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusOK)
+	})
+
+	wrappedMux := withStatusLogging(testMux)
+
+	// Use a ResponseRecorder to capture the response
+	rr := httptest.NewRecorder()
+	req, _ := http.NewRequest("GET", "/nonexistent", nil)
+	req.RemoteAddr = "127.0.0.1:12345"
+
+	wrappedMux.ServeHTTP(rr, req)
+
+	if rr.Code != http.StatusNotFound {
+		t.Errorf("Expected status 404, got %d", rr.Code)
+	}
+}
+
+// TestStatusLogging405 tests that 405 responses are logged.
+func TestStatusLogging405(t *testing.T) {
+	C = Config{} // Reset global config.
+	excludePatterns = nil
+
+	// Create a minimal mux that returns 405 for wrong method
+	testMux := http.NewServeMux()
+	testMux.HandleFunc("/archive", func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusMethodNotAllowed)
+	})
+
+	wrappedMux := withStatusLogging(testMux)
+
+	rr := httptest.NewRecorder()
+	req, _ := http.NewRequest("GET", "/archive", nil)
+	req.RemoteAddr = "127.0.0.1:12345"
+
+	wrappedMux.ServeHTTP(rr, req)
+
+	if rr.Code != http.StatusMethodNotAllowed {
+		t.Errorf("Expected status 405, got %d", rr.Code)
+	}
+}
+
+// TestMethodNotAllowedHandler tests that non-GET methods return 405.
+func TestMethodNotAllowedHandler(t *testing.T) {
+	C = Config{
+		Address:     "0.0.0.0",
+		Port:        8086,
+		Location:    "/archive",
+		User:        "testuser",
+		Password:    "testpass",
+		Directories: []string{"../../test_data/test1/foo"},
+	}
+	excludePatterns = nil
+
+	// Create test server
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		username, password, ok := r.BasicAuth()
+		if !ok || username != C.User || password != C.Password {
+			http.Error(w, "Unauthorized", http.StatusUnauthorized)
+
+			return
+		}
+
+		if r.Method != http.MethodGet {
+			w.Header().Set("Server", "backer")
+			http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
+
+			return
+		}
+
+		w.WriteHeader(http.StatusOK)
+	}))
+	defer server.Close()
+
+	// Test POST method
+	req, _ := http.NewRequest("POST", server.URL+"/archive", nil)
+	req.SetBasicAuth("testuser", "testpass")
+	resp, err := http.DefaultClient.Do(req)
+	if err != nil {
+		t.Fatalf("Request failed: %v", err)
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != http.StatusMethodNotAllowed {
+		t.Errorf("Expected status 405, got %d", resp.StatusCode)
+	}
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/types.go

@@ -15,11 +15,23 @@ type (
 		LogLevel             string   `json:"loglevel"`
 		Directories          []string `json:"directories"`
 		BackupTimeout        int      `json:"backup_timeout"`        // Timeout in minutes for backup streaming.
+		DirScanTimeout       int      `json:"dir_scan_timeout"`      // Timeout in minutes for directory scanning.
 		CompressionLevel     int      `json:"compression_level"`     // Compression level (1-9, default 9).
 		ExcludePatterns      []string `json:"exclude_patterns"`      // Regex patterns to exclude from backup.
 		FilenamePrefix       string   `json:"filename_prefix"`       // Prefix for backup filename (default: "backup").
-		CompressionAlgorithm string   `json:"compression_algorithm"` // Compression algorithm: gzip, pgzip, bzip2, zstd, lz4 or xz.
+		CompressionAlgorithm string   `json:"compression_algorithm"` // Compression algorithm alias (deprecated: use default_compression).
+		DefaultCompression   string   `json:"default_compression"`   // Default compression algorithm: gzip, pgzip, bzip2, zstd, lz4 or xz.
+	}
+
+	// ErrDirectoryScanTimeout is returned when directory scan times out.
+	ErrDirectoryScanTimeout struct {
+		ScannedFiles int
 	}
 )
 
+// Error returns the error message for ErrDirectoryScanTimeout.
+func (e ErrDirectoryScanTimeout) Error() string {
+	return "Directory scan timed out"
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/utils_test.go

@@ -7,6 +7,7 @@ import (
 	"io"
 	"os"
 	"path/filepath"
+	"runtime"
 	"testing"
 )
 
@@ -179,6 +180,73 @@ func TestCreateTarGzStreamSymlink(t *testing.T) {
 	}
 }
 
+// TestCreateTarGzStreamHardLink tests hard link handling - should deduplicate content.
+func TestCreateTarGzStreamHardLink(t *testing.T) {
+	if runtime.GOOS == "windows" {
+		t.Skip("Hard link detection not supported on Windows (inodes unavailable)")
+	}
+
+	tmpDir := t.TempDir()
+
+	// Create original file
+	originalFile := filepath.Join(tmpDir, "original.txt")
+	if err := os.WriteFile(originalFile, []byte("shared content"), 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	// Create hard link
+	hardLinkFile := filepath.Join(tmpDir, "hardlink.txt")
+	if err := os.Link(originalFile, hardLinkFile); err != nil {
+		t.Skip("Hard links not supported, skipping test")
+	}
+
+	ctx := context.Background()
+	reader := CreateTarGzStream(ctx, []string{originalFile, hardLinkFile})
+	defer reader.Close()
+
+	gzReader, err := gzip.NewReader(reader)
+	if err != nil {
+		t.Fatalf("Failed to create gzip reader: %v", err)
+	}
+	defer gzReader.Close()
+
+	tarReader := tar.NewReader(gzReader)
+
+	entries := 0
+	linkCount := 0
+
+	for {
+		header, err := tarReader.Next()
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			t.Fatalf("Failed to read tar entry: %v", err)
+		}
+
+		entries++
+		t.Logf("Entry %d: name=%s typeflag=%c linkname=%s", entries, header.Name, header.Typeflag, header.Linkname)
+
+		if header.Typeflag == tar.TypeLink {
+			linkCount++
+			// Verify it's a link to the original
+			if header.Linkname != originalFile && header.Linkname != filepath.ToSlash(originalFile) {
+				t.Errorf("Expected linkname '%s', got '%s'", originalFile, header.Linkname)
+			}
+		}
+	}
+
+	// Should have 2 entries: one file content, one as hard link
+	if entries != 2 {
+		t.Errorf("Expected 2 entries, got %d", entries)
+	}
+
+	// One should be a hard link (TypeLink)
+	if linkCount != 1 {
+		t.Errorf("Expected 1 hard link entry, got %d", linkCount)
+	}
+}
+
 // TestCreateTarGzStreamCancellation tests context cancellation.
 func TestCreateTarGzStreamCancellation(t *testing.T) {
 	tmpDir := t.TempDir()
@@ -356,4 +424,144 @@ func TestCreateTarGzStreamDeviceFile(t *testing.T) {
 	}
 }
 
+// TestCompressionLevelMappingLz4 tests that lz4 compression levels are correctly mapped.
+func TestCompressionLevelMappingLz4(t *testing.T) {
+	tests := []struct {
+		input    int
+		expected int
+	}{
+		{1, 0},
+		{2, 0},
+		{3, 0},
+		{4, 1},
+		{5, 1},
+		{6, 1},
+		{7, 2},
+		{8, 2},
+		{9, 2},
+	}
+
+	for _, tt := range tests {
+		C = Config{CompressionLevel: tt.input}
+		result := mapCompressionLevelToLz4(tt.input)
+		if result != tt.expected {
+			t.Errorf("mapCompressionLevelToLz4(%d) = %d, expected %d", tt.input, result, tt.expected)
+		}
+	}
+}
+
+// TestCompressionLevelMappingXz tests that xz compression is not configurable via level.
+// xz uses default compression - this test verifies the stream works.
+func TestCompressionLevelMappingXz(t *testing.T) {
+	// xz doesn't support configurable compression levels in the same way.
+	// It always uses default compression. This test verifies the stream is created successfully.
+	C = Config{CompressionLevel: 6}
+
+	tmpDir := t.TempDir()
+	testFile := filepath.Join(tmpDir, "test.txt")
+	content := []byte("test content")
+	if err := os.WriteFile(testFile, content, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarXzStream(ctx, []string{testFile})
+	defer reader.Close()
+
+	data, err := io.ReadAll(reader)
+	if err != nil {
+		t.Fatalf("Failed to read xz archive: %v", err)
+	}
+
+	if len(data) == 0 {
+		t.Error("Expected non-empty archive")
+	}
+}
+
+// TestCompressionLevelMappingBzip2 tests that bzip2 compression levels are correctly mapped.
+func TestCompressionLevelMappingBzip2(t *testing.T) {
+	tests := []struct {
+		input    int
+		expected int
+	}{
+		{1, 1},
+		{5, 5},
+		{9, 9},
+	}
+
+	for _, tt := range tests {
+		C = Config{CompressionLevel: tt.input}
+		result := mapCompressionLevelToBzip2(tt.input)
+		if result != tt.expected {
+			t.Errorf("mapCompressionLevelToBzip2(%d) = %d, expected %d", tt.input, result, tt.expected)
+		}
+	}
+}
+
+// TestCreateTarBzip2Stream tests bzip2 compression with configured level.
+func TestCreateTarBzip2Stream(t *testing.T) {
+	C = Config{CompressionLevel: 6}
+
+	tmpDir := t.TempDir()
+	testFile := filepath.Join(tmpDir, "test.txt")
+	content := []byte("test content for bzip2")
+	if err := os.WriteFile(testFile, content, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarBzip2Stream(ctx, []string{testFile})
+	defer reader.Close()
+
+	// Verify we can read the archive
+	_, err := io.ReadAll(reader)
+	if err != nil {
+		t.Fatalf("Failed to read bzip2 archive: %v", err)
+	}
+}
+
+// TestCreateTarLz4Stream tests lz4 compression with configured level.
+func TestCreateTarLz4Stream(t *testing.T) {
+	C = Config{CompressionLevel: 6}
+
+	tmpDir := t.TempDir()
+	testFile := filepath.Join(tmpDir, "test.txt")
+	content := []byte("test content for lz4")
+	if err := os.WriteFile(testFile, content, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarLz4Stream(ctx, []string{testFile})
+	defer reader.Close()
+
+	// Verify we can read the archive
+	_, err := io.ReadAll(reader)
+	if err != nil {
+		t.Fatalf("Failed to read lz4 archive: %v", err)
+	}
+}
+
+// TestCreateTarXzStream tests xz compression with configured level.
+func TestCreateTarXzStream(t *testing.T) {
+	C = Config{CompressionLevel: 6}
+
+	tmpDir := t.TempDir()
+	testFile := filepath.Join(tmpDir, "test.txt")
+	content := []byte("test content for xz")
+	if err := os.WriteFile(testFile, content, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarXzStream(ctx, []string{testFile})
+	defer reader.Close()
+
+	// Verify we can read the archive
+	_, err := io.ReadAll(reader)
+	if err != nil {
+		t.Fatalf("Failed to read xz archive: %v", err)
+	}
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/xz.go

@@ -10,6 +10,8 @@ import (
 // CreateTarXzStream takes a list of files and returns a reader that reads from an xz-compressed tar archive.
 func CreateTarXzStream(ctx context.Context, filepaths []string) io.ReadCloser {
 	return createArchiveStream(ctx, filepaths, func(w io.Writer) (io.WriteCloser, error) {
+		// xz doesn't have a simple level parameter; use default compression.
+		// The compression level affects block size indirectly through the algorithm.
 		return xz.NewWriter(w)
 	})
 }

internal/log/main.go

@@ -2,26 +2,56 @@
 package log
 
 import (
+	"context"
 	"fmt"
+	"log"
 	"log/slog"
 	"os"
+	"strings"
+	"sync"
 	"time"
 )
 
 var (
 	// Log contains *[os.File] pointing to log file descriptor.
+	// Protected by mu for thread-safe access, primarily to enable parallel tests.
+	// In production, this is set once at startup and never modified thereafter.
 	Log *os.File
+
+	// mu protects Log during initialization and when DebugLogger is called.
+	// This is necessary to allow parallel test execution where each test
+	// may initialize the logger with a different log file.
+	mu sync.RWMutex
+
+	// logLevel stores the configured log level for use by DebugLogger.
+	logLevel slog.Level
+)
+
+// slogWriter implements [io.Writer] to redirect standard log to slog.
+type (
+	slogWriter struct {
+		handler slog.Handler
+	}
 )
 
 // Init sets up logger stuff.
 // level can be error, warn, info, debug; if something other is supplied, info level is selected.
+// Thread-safe: uses mutex to allow parallel test execution.
 func Init(level, filename string) error {
+	mu.Lock()
+	defer mu.Unlock()
+
 	var (
 		err      error
 		loglevel slog.Level
 	)
 
 	if filename != "" {
+		// Close existing log file if re-initializing (e.g., in tests).
+		if Log != nil && Log != os.Stderr {
+			Log.Close()
+		}
+
 		Log, err = os.OpenFile(filename, os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0644)
 
 		if err != nil {
@@ -49,6 +79,8 @@ func Init(level, filename string) error {
 		loglevel = slog.LevelInfo
 	}
 
+	logLevel = loglevel
+
 	opts := &slog.HandlerOptions{
 		// Use the ReplaceAttr function on the handler options
 		// to be able to replace any single attribute in the log output.
@@ -122,6 +154,47 @@ func Debugf(format string, a ...any) {
 	slog.Debug(fmt.Sprintf(format, a...))
 }
 
+// DebugLogger returns a standard library logger that writes at the configured log level.
+// This is useful for assigning to http.Server.ErrorLog to route net/http errors
+// through our logger while respecting the configured log level.
+func DebugLogger() *log.Logger {
+	mu.RLock()
+	defer mu.RUnlock()
+
+	opts := &slog.HandlerOptions{
+		Level: logLevel,
+	}
+
+	handler := slog.NewTextHandler(Log, opts)
+
+	return log.New(&slogWriter{handler: handler}, "", 0)
+}
+
+// Write implements [io.Writer]. It differentiates TLS/SSL errors from other errors.
+// TLS-related errors are logged at debug level, while other errors (handler issues,
+// unexpected behavior) are logged at warn level.
+func (w *slogWriter) Write(p []byte) (int, error) {
+	msg := string(p)
+
+	isTLSError := strings.Contains(msg, "tls") ||
+		strings.Contains(msg, "ssl") ||
+		strings.Contains(msg, "TLS") ||
+		strings.Contains(msg, "SSL") ||
+		strings.Contains(msg, "handshake") ||
+		strings.Contains(msg, "certificate")
+
+	level := slog.LevelWarn
+	if isTLSError {
+		level = slog.LevelDebug
+	}
+
+	record := slog.NewRecord(time.Now(), level, msg, 0)
+
+	_ = w.handler.Handle(context.TODO(), record)
+
+	return len(p), nil
+}
+
 // Fatal logs args at error level and quits with status code 1.
 func Fatal(args ...any) {
 	slog.Error(fmt.Sprint(args...))
@@ -138,7 +211,11 @@ func Fatalf(format string, a ...any) {
 }
 
 // Close closes log file descriptor.
+// Thread-safe: uses mutex to coordinate with Init and DebugLogger.
 func Close() {
+	mu.Lock()
+	defer mu.Unlock()
+
 	if Log == nil || Log == os.Stderr {
 		return
 	}
@@ -147,6 +224,8 @@ func Close() {
 		msg := fmt.Sprintln("Failed to close log file:", err)
 		slog.Error(msg)
 	}
+
+	Log = nil
 }
 
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/log/main_test.go

@@ -354,4 +354,35 @@ func TestCloseStderr(t *testing.T) {
 	}
 }
 
+// TestDebugLogger tests that DebugLogger returns a working *log.Logger.
+func TestDebugLogger(t *testing.T) {
+	tmpFile, err := os.CreateTemp(t.TempDir(), "test_log_*.log")
+	if err != nil {
+		t.Fatal(err)
+	}
+	logFilePath := tmpFile.Name()
+	tmpFile.Close()
+
+	if err := Init("info", logFilePath); err != nil {
+		t.Fatal(err)
+	}
+	defer Close()
+
+	logger := DebugLogger()
+	if logger == nil {
+		t.Error("DebugLogger returned nil")
+	}
+
+	logger.Print("test debug message")
+
+	content, err := os.ReadFile(logFilePath)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if !strings.Contains(string(content), "test debug message") {
+		t.Error("Expected debug message in log")
+	}
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

llms.txt

@@ -30,13 +30,13 @@ curl -u user:pass https://server:8086/archive -o backup.tar.gz
 │   │   ├── globals.go       # Global C Config variable
 │   │   ├── archive.go       # Common archive logic (createArchiveStream, writeFilesToTar)
 │   │   ├── gzip.go          # CreateTarGzStream, CreateTarPgzipStream
-│   │   ├── bzip2.go         # CreateTarBzip2Stream
-│   │   ├── zstd.go          # CreateTarZstdStream
-│   │   ├── lz4.go           # CreateTarLz4Stream
+│   │   ├── bzip2.go         # CreateTarBzip2Stream, mapCompressionLevelToBzip2
+│   │   ├── zstd.go          # CreateTarZstdStream, zstdLevel
+│   │   ├── lz4.go           # CreateTarLz4Stream, mapCompressionLevelToLz4
 │   │   ├── xz.go            # CreateTarXzStream
 │   │   ├── loadConfig.go    # HJSON/JSON config parsing with validation
 │   │   ├── newServer.go     # HTTP server, auth handler, context helpers
-│   │   ├── getFilesFromDirectories.go  # Directory walking, exclusion filtering
+│   │   ├── getFilesFromDirectories.go  # Directory walking, exclusion filtering, isExcluded
 │   │   └── *_test.go        # Test files
 │   └── log/main.go          # Logging wrapper around stdlib slog
 ├── contrib/                 # Init scripts
@@ -55,7 +55,7 @@ curl -u user:pass https://server:8086/archive -o backup.tar.gz
 |--------|---------|-------------|
 | address | 0.0.0.0 | Listen address |
 | port | 8086 | Listen port |
-| cert/key | - | TLS cert/key paths |
+| cert/key | - | TLS cert/key paths (required when nohttps=false) |
 | nohttps | false | Disable HTTPS |
 | location | /archive | API endpoint path |
 | user | - | Basic auth username (required) |
@@ -64,10 +64,23 @@ curl -u user:pass https://server:8086/archive -o backup.tar.gz
 | loglevel | info | Log verbosity (debug/info/warn/error) |
 | directories | - | Directories to backup (required) |
 | backup_timeout | 60 | Stream timeout in minutes (1-1440) |
-| compression_level | 9 | Gzip level (1-9) |
+| dir_scan_timeout | 60 | Directory scan timeout in minutes (1-1440) |
+| compression_level | 9 | Compression level (1-9) |
 | exclude_patterns | [] | Regex patterns to exclude from backup |
 | filename_prefix | backup | Prefix for backup filename in Content-Disposition header |
-| compression_algorithm | gzip | Compression algorithm (gzip, pgzip, bzip2, zstd, lz4 or xz) |
+| default_compression | gzip | Default compression algorithm (gzip, pgzip, bzip2, zstd, lz4, xz) |
+| compression_algorithm | gzip | Alias for default_compression (deprecated) |
+
+## Compression Level Mapping
+
+The `compression_level` config option (1-9) is mapped differently for each algorithm:
+
+| Algorithm | Level Mapping |
+|-----------|---------------|
+| gzip/pgzip/bzip2 | 1-9 directly maps to compression levels |
+| lz4 | 1-3 → Fastest (0), 4-6 → Default (1), 7-9 → Best (2) |
+| zstd | 1-3 → Fastest, 4-6 → Default, 7-9 → BetterCompression |
+| xz | Uses default compression (no configurable level) |
 
 ## Key Patterns
 
@@ -85,16 +98,36 @@ json.Marshal(tmp)
 json.Unmarshal(buf, &C)
 ```
 
-### Stream Cancellation
-Context-aware write and copy functions:
+### Context-Aware I/O
+Two helper functions in newServer.go handle context cancellation during streaming:
+- `writeWithContext(ctx, fn)` — checks context before calling fn
+- `copyWithContext(ctx, dst, src)` — buffered copy (32KB) with cancellation checks, computes MD5 hash
+
 ```go
 func writeWithContext(ctx context.Context, fn func() error) error
-func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) error
+func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) (int64, string, error)
 ```
 
+### HTTP Server Middleware
+Two middleware functions wrap the handler:
+- `withServerHeader` — adds "Server: backer" header to all responses
+- `withStatusLogging` — logs 404 and 405 responses
+
+```go
+Handler: withServerHeader(withStatusLogging(mux))
+```
+
+### Unmatched Route Logging
+The server wraps http.ResponseWriter to capture status codes:
+- **404 Not Found**: `"Not Found from {clientIP} to {path}"`
+- **405 Method Not Allowed**: `"Method Not Allowed from {clientIP} for {method} {path}"`
+
+Uses `responseWriterWrapper` that tracks status code via WriteHeader() method.
+
 ### Pipe Error Handling
 Uses `pipeErr` tracking to avoid noisy errors when reader closes early.
 Skips files that can't be read (logs warning, continues backup).
+- `isPipeClosedError()` checks for "io: read/write on closed pipe" errors
 
 ### Content-Disposition Filename
 Backup filenames include timestamp and configured prefix: `backup-20060102-150405.tar.gz` (extension depends on compression_algorithm).
@@ -102,18 +135,43 @@ Backup filenames include timestamp and configured prefix: `backup-20060102-15040
 ### Special File Types
 - **Device files** (character/block) are included with correct major/minor numbers.
 - **Sockets** are skipped (no entry created).
-- **Symlinks** are stored as symlinks (target path stored).
+- **Symlinks** are stored as symlinks (target path stored) - uses os.Lstat, no cycles possible.
 - **Named pipes** and other special files are included (tar header created).
+- **Hard links** are deduplicated - first occurrence stored with content, subsequent as link entries.
 - **Regular files** are compressed; other file types have no data section.
 
 ### Error Recovery
 - `writeWithContext` checks context before calling fn.
 - `copyWithContext` does buffered copy with cancellation checks.
 
+### Client IP Detection
+Checks `X-Forwarded-For` header first (for proxied requests), then falls back to `RemoteAddr`.
+
+### TLS Error Handling
+Uses `ServerWrapper` to intercept and classify errors from Serve() and ServeTLS():
+- TLS-specific errors logged at debug level to reduce log clutter
+- Uses errors.As to detect: tls.AlertError, tls.CertificateVerificationError, tls.ECHRejectionError, tls.RecordHeaderError
+
+### HTTP Routing & Compression Selection
+The server uses http.NewServeMux() for routing. Multiple routes support different compression formats:
+```go
+mux.Handle(C.Location, http.HandlerFunc(backupHandler))           // /archive
+mux.Handle(C.Location+".tar.gz", http.HandlerFunc(backupHandler))  // archive.tar.gz
+mux.Handle(C.Location+".tar.xz", http.HandlerFunc(backupHandler))   // archive.tar.xz
+mux.Handle(C.Location+".tar.bz2", http.HandlerFunc(backupHandler)) // archive.tar.bz2
+// ... etc (.tar.lz4, .tar.zst, .tar.zstd)
+```
+
+Compression algorithm determined by path extension using `path.Ext()`:
+- Full path: `/archive.tar.xz` → `.xz` → xz
+- Bare extension: `tar.xz` → `.xz` → xz
+- Default when no extension matches
+
 ### Performance Notes
 - Exclude patterns are precompiled after config load for faster matching.
 - Compression level is configurable (1-9); default 9 uses more CPU.
 - Large directory walks use `filepath.WalkDir` (efficient).
+- Context cancellation checked on each file during directory scan.
 
 ## Coding Style & Conventions
 
@@ -128,7 +186,8 @@ GetFilesFromDirectories()
 // Unexported (private): verbNoun or isNoun pattern
 writeWithContext()
 copyWithContext()
-isPipeClosedError()  // predicate helper
+isPipeClosedError()
+isExcluded()
 defaultConfigPath()
 ```
 
@@ -137,6 +196,7 @@ Struct fields use PascalCase, json tags use snake_case:
 ```go
 BackupTimeout     int `json:"backup_timeout"`
 CompressionLevel  int `json:"compression_level"`
+DirScanTimeout    int `json:"dir_scan_timeout"`
 ```
 
 ### Error Messages
@@ -149,6 +209,7 @@ CompressionLevel  int `json:"compression_level"`
 - Past tense: `"Failed to close gzip writer: %v", err`
 - Context included: `"Skipping %s: %v", filename, err`
 - Appropriate level: Warn for recoverable, Error for issues, Fatal for startup failures
+- Log levels: error, warn, info, debug
 
 ### Config Validation Pattern
 For each config option: set default first, then immediately validate:
@@ -166,7 +227,7 @@ if C.Port < 1 || C.Port > 65535 {
 
 ### Go Formatting
 - Tabs for indentation
-- Vim modeline at EOF: `/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */`
+- Vim modeline at EOF: `/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */
 
 ### Linter Suppressions (intentional)
 | Rule | Reason |
@@ -186,10 +247,11 @@ go test -v -run TestName ./...   # Run specific test
 golangci-lint run ./...          # Lint
 ```
 
-Coverage: ~69% on internal/backer (main and log packages not covered by design)
+Coverage: ~69% on internal/backer (~100% on internal/log)
 
 ### Test Conventions
 - Reset global config: `C = Config{}`
+- Reset exclude patterns: `excludePatterns = nil`
 - Use `t.TempDir()` for temporary files
 - Helper function `writeTempConfig()` for config tests
 
@@ -208,3 +270,20 @@ Version is injected at build time via ldflags:
 ```go
 var version = "dev"  // overridden by: -X main.version=$(VERSION)
 ```
+
+## Request Flow (Good Path)
+```
+Request → Method Check (GET) → Auth Check → Get Files (with timeout) → Create Archive → Stream → Complete
+```
+
+## Request Flow (Error Paths)
+| Request | Response | Logged |
+|---------|----------|--------|
+| Non-GET method | 405 Method Not Allowed | Yes (info) |
+| No auth | 401 Unauthorized | Yes (info) |
+| Wrong credentials | 401 Unauthorized | Yes (info) |
+| Unknown path | 404 Not Found | Yes (info) |
+| Dir scan timeout | 504 Request Timeout | Yes (error) |
+| Failed to get files | 500 Internal Server Error | Yes (error) |
+| Client disconnect | - | Yes (info) |
+| Backup completed | 200 OK | Yes (info) |

test_data/test_config.json

@@ -5,10 +5,10 @@
     /* By default listen on port 8086, like first intel commercially successful cpu model.                            */
     "port": 8086,
 
-    /* Path location of certificate file                                                                              */
+    /* Path location of test certificate file                                                                       */
     "cert": "../../test_data/example.crt",
 
-    /* Path location of certificate key                                                                               */
+    /* Path location of test certificate key                                                                          */
     "key": "../../test_data/example.key",
 
     /* We really want to use https for backup archive tranferring.                                                    */
@@ -24,7 +24,7 @@
     "password": "SharpShooter",
 
     /* Where to store logs.                                                                                           */
-    "log": "/var/log/backer.log",
+    "log": "../../test_data/tmp/backer.log",
 
     /* Verbosity of logs                                                                                              */
     "loglevel": "info",
@@ -39,5 +39,5 @@
     "backup_timeout": 60,
 
     /* Gzip compression level (1-9, where 9 is best compression).                                                    */
-    "compression_level": 6
+    "compression_level": 3
 }