Update README.md

- Fix 404 on downloading archive with extension
- Add downloaded archive length to log
- Add md5 hash to log

.gitignore

@@ -1,6 +1,8 @@
 # Binary
 /backer
 /backer.exe
+/cmd/backer/backer
+/cmd/backer/backer.exe
 
 # Local config (may contain secrets)
 /config.json
@@ -27,3 +29,5 @@ Thumbs.db
 *.exe
 *.test
 *.out
+
+/test_data/tmp

.golangci.yml

@@ -74,6 +74,7 @@ linters:
         - paramTypeCombine
         - whyNoLint
         - hugeParam
+        - unnamedResult
     godoclint:
       default: all
       ignore-unexported: false

AGENTS.md

@@ -20,7 +20,7 @@ Backer is a Go-based HTTP/HTTPS backup server that creates compressed archives o
 - **Use tags and labels.** In comments, add tags or labels to files, classes, or functions to categorize them based on functionality, domain concepts, or design patterns.
 - **Avoid obvious comments that do not add value.**
 - **Remove dead code.** A half-finished refactoring can actually harm the contributions.
-- **Update AGENTS.md and TODO.md automatically.** Keep AGENTS.md dense, detailed and concise.
+- **Update AGENTS.md automatically.** Keep AGENTS.md dense, detailed and concise.
 
 ## Workflow Rules
 
@@ -124,6 +124,18 @@ CompressionLevel  int `json:"compression_level"`
 | QF1001 | De Morgan's law disabled for auth condition clarity |
 | revive (error strings) | Same reason as ST1005 |
 
+### Design Decisions (Not Issues)
+
+The following are intentional design decisions, not bugs or defects:
+
+- **No rate limiting**: Backups are assumed to be infrequent enough that rate limiting is unnecessary.
+- **Backup scheduling**: Client-side decision — server serves backups on-request only.
+- **Basic Auth only**: HTTP Basic Auth is sufficient for a backup server behind a firewall.
+- **Global config**: The `C Config` global variable is intentional — config is loaded once at startup.
+- **No incremental backup**: Each backup is a full snapshot, not incremental.
+- **Credentials in config file**: Config file should have restricted permissions (documented).
+- **Thread-safe logging**: The global `Log` variable uses mutex protection to enable parallel test execution; this is not needed for single-server production use but allows tests to run in parallel without race conditions.
+
 ## Project Structure
 
 ```
@@ -195,6 +207,7 @@ Config is JSON (HJSON with comments supported). Default path depends on OS:
 | loglevel | `info` | Log verbosity: error/warn/info/debug |
 | directories | — | Directories to backup (required, validated for existence) |
 | backup_timeout | `60` | Stream timeout in minutes (1-1440) |
+| dir_scan_timeout | `60` | Directory scan timeout in minutes (1-1440) |
 | compression_level | `9` | Gzip level (1-9) |
 | exclude_patterns | `[]` | Regex patterns to exclude from backup |
 | filename_prefix | `backup` | Prefix for backup filename in Content-Disposition header |
@@ -228,16 +241,17 @@ No temp files on disk. Uses `io.Pipe` with a goroutine to stream archives on-the
 
 Two helper functions in `newServer.go` handle context cancellation during streaming:
 - `writeWithContext(ctx, fn)` — checks context before calling fn
-- `copyWithContext(ctx, dst, src)` — buffered copy (32KB) with cancellation checks
+- `copyWithContext(ctx, dst, src)` — buffered copy (32KB) with cancellation checks, computes MD5 hash of copied data
 
 ### Special File Type Handling
 
 - **Regular files**: streamed into tar archive with content
 - **Directories**: included as tar directory entries
-- **Symlinks**: stored as symlink entries (target path preserved)
+- **Symlinks**: stored as symlink entries (target path preserved). Backer never resolves or follows symlinks during directory traversal — it uses `os.Lstat` and `filepath.WalkDir` which do not follow symlinks. This means symlink cycles are impossible to occur.
 - **Device files** (char/block): included with correct major/minor
 - **Named pipes**: included (header only, no data)
 - **Sockets**: skipped (not archivable)
+- **Hard links**: deduplicated — if multiple files share the same inode, only the first is stored with content, subsequent hard links are stored as link entries pointing to the original
 
 ### Pipe Error Recovery
 
@@ -245,7 +259,7 @@ Two helper functions in `newServer.go` handle context cancellation during stream
 
 ### Auth Handler
 
-HTTP Basic Auth. Both username AND password must match exactly. Returns `401 Unauthorized` with `WWW-Authenticate` header on failure. The condition is intentionally written without De Morgan's law simplification for clarity (linter suppression `//nolint:staticcheck`).
+HTTP Basic Auth. Both username AND password must match exactly. Returns `401 Unauthorized` with `WWW-Authenticate` header on failure. Uses timing-safe comparison to prevent timing attacks. The condition is intentionally written without De Morgan's law simplification for clarity (linter suppression `//nolint:staticcheck`).
 
 ### Client IP Detection
 
@@ -255,8 +269,63 @@ Checks `X-Forwarded-For` header first (for proxied requests), then falls back to
 
 - `ReadHeaderTimeout`: 5 seconds
 - `WriteTimeout`: configurable via `backup_timeout` (converted to `time.Duration`)
+- `IdleTimeout`: 10 seconds (for keep-alive connections; backer disables keep-alives explicitly)
+- `MaxHeaderBytes`: 1KB (prevents header-based DoS attacks)
+- `DisableKeepAlives`: true (each connection closed after request, no connection reuse)
 - TLS minimum version: TLS 1.3 (when HTTPS enabled)
 
+### TLS Error Handling
+
+The server uses a `ServerWrapper` type to intercept and classify errors from `Serve()` and `ServeTLS()`. TLS-specific errors are logged at debug level to reduce log clutter:
+
+- Uses `errors.As` to detect TLS error types: `tls.AlertError`, `tls.CertificateVerificationError`, `tls.ECHRejectionError`, `tls.RecordHeaderError`
+- Also checks for "tls:" prefix in error messages
+- Additionally uses `http.Server.ErrorLog` with a custom logger that writes at debug level for net/http's internal TLS-related messages
+
+This ensures client-side TLS errors (handshake failures, certificate issues) don't clutter production logs.
+
+### Compression Level Mapping
+
+The `compression_level` config option (1-9) is mapped differently for each compression algorithm:
+
+| Algorithm | Level Mapping |
+|-----------|----------------|
+| gzip | 1-9 directly maps to gzip compression levels |
+| pgzip | 1-9 directly maps to pgzip compression levels |
+| bzip2 | 1-9 directly maps to bzip2 compression levels |
+| zstd | 1-3: Fastest, 4-6: Default, 7-9: BetterCompression |
+| lz4 | 1-3: Fastest (0), 4-6: Default (1), 7-9: Best (2) |
+| xz | Uses default compression (no configurable level) |
+
+### HTTP Routing & Compression Selection
+
+The server uses `http.NewServeMux()` for routing. Multiple routes are registered to support different compression formats:
+
+```go
+mux.Handle(C.Location, http.HandlerFunc(backupHandler))           // /archive
+mux.Handle(C.Location+".tar.gz", http.HandlerFunc(backupHandler))  // /archive.tar.gz
+mux.Handle(C.Location+".tar.xz", http.HandlerFunc(backupHandler))   // /archive.tar.xz
+mux.Handle(C.Location+".tar.bz2", http.HandlerFunc(backupHandler)) // /archive.tar.bz2
+// ... etc
+```
+
+The compression algorithm is determined by extracting the extension from the request path using `path.Ext()`:
+- For `/archive.tar.xz`, `path.Ext()` returns `.tar.xz` → maps to xz
+- For `/archive`, `path.Ext()` returns `""` → uses default compression
+
+The `getCompressionAlgorithm()` function handles both full paths and bare extensions:
+- Full path: `/archive.tar.xz` → `.tar.xz` → xz
+- Bare extension: `tar.xz` → `.tar.xz` → xz
+- Single ext: `xz` → `.xz` → xz
+
+### Unmatched Route Logging
+
+The server wraps the `http.ResponseWriter` to capture status codes and log unmatched routes:
+- **404 Not Found**: Logged as `"Not Found from {clientIP} to {path}"`
+- **405 Method Not Allowed**: Logged as `"Method Not Allowed from {clientIP} for {method} {path}"`
+
+This uses a `responseWriterWrapper` that tracks the status code via the `WriteHeader()` method.
+
 ## Dependencies
 
 - `github.com/hjson/hjson-go` v3.3.0 — HJSON config parsing

README.md

@@ -65,6 +65,7 @@ All options described in example config.
 | loglevel | "info" | Verbosity of logs. | Options: error, warn, info, debug. |
 | directories | — | Directories to backup. | Required. Array of paths. |
 | backup_timeout | 60 | Timeout in minutes for backup streaming. | Range: 1-1440 minutes. |
+| dir_scan_timeout | 60 | Timeout in minutes for getting file list from all backupped directories. | Range: 1-1440 minutes. |
 | compression_level | 9 | Compression level. | 1 (fastest) to 9 (best compression). |
 | exclude_patterns | [] | Regex patterns to exclude from backup. | E.g., `[".*\\.tmp$", "/node_modules/"]`. |
 | filename_prefix | "backup" | Prefix for backup filename. | E.g., "mybackup" produces `mybackup-20260325-092341.tar.gz`. |
@@ -77,7 +78,7 @@ AI suggests this table as expected performance.
 
 | Algorithm | Speed   | Compression | Notes |
 | :---      | :---    | :---        | :--- |
-| gzip | Fast | Good | Default. Standard, widely compatible. |
+| gzip      | Fast    | Good        | Standard, widely compatible. |
 | pgzip     | Fast    | Good        | Parallel gzip. Uses multiple CPU cores, slightly larger files. |
 | bzip2     | Slow    | Better      | Pure Go. Good compression, slower than gzip. |
 | zstd      | Fast    | Very good   | Good balance of speed and compression. |
@@ -90,12 +91,21 @@ network bandwidth, located in near by country I've got these results
 | Algorithm | Size       | Time    | CPU Load | RAM RSS+Shm, Mb | Notes |
 | :---      | ---:       | ---:    | ---:     | ---:            | :---  |
 | None      | 1461250551 |         |     0-2% |             0+0 | original dataset |
+| gzip      |  823381547 | 16m:22s |      96% |            18+6 | stdlib compression/gzip |
+| pgzip     |  823783924 | 13m:50s |   93-97% |            28+5 | multithreaded (parallel gzip), fully gzip compatible, klauspost/pgzip |
+| bzip2     |  797053471 |  9m:47s |   92-94% |            56+6 | dsnet/compress/bzip2 |
+| xz        |  515106988 | 16m:03s |      96% |           116+5 | ulikunitz/xz |
+| zstd      |  617330529 |  6m:39s |   40-99% |            64+6 | klauspost/compress/zstd |
 | lz4       | 1097745915 |  6m:12s |     4-6% |            30+6 | multithreaded lz4, pierrec/lz4 |
-| zstd      |  614015773 |  4m:52s |   40-60% |            60+6 | klauspost/compress/zstd |
+
-| pgzip     |  817163857 | 11m:25s |   96-97% |            37+6 | multithreaded (parallel gzip), fully gzip compatible, klauspost/pgzip |
+The `compression_level` option (1-9) maps differently for each algorithm:
-| gzip      |  816734188 | 14m:30s |      96% |            18+6 | stdlib compression/gzip |
+
-| bzip2     |  788061284 |  7m:16s |   92-94% |            43+6 | dsnet/compress/bzip2 |
+| Algorithm        | Level Mapping |
-| xz        |  512004612 | 13m:53s |      96% |           116+6 | ulikunitz/xz |
+| :---             | :---          |
+| gzip/pgzip/bzip2 | 1-9 directly maps to compression levels |
+| lz4              | 1-3 → Fastest (0), 4-6 → Default (1), 7-9 → Best (2) |
+| zstd             | 1-3 → Fastest, 4-6 → Default, 7-9 → BetterCompression |
+| xz               | Uses default compression (no configurable level) |
 
 Obvously in case of lz4 and zstd the bottleneck was network. And I have to admit, that network definitely was not
 100Mbit, but it was pretty hard to measure real bandwidth. Anyway let's pretend that during experiment it was roughly
@@ -111,6 +121,52 @@ good enough cypher.
 
 And one more thing user and password are mandatory, we should have at least some minimal barriers, right?
 
+## Supported File Types
+
+Backer handles various file types when creating archives:
+
+- **Regular files**: Included with full content
+- **Directories**: Included as directory entries
+- **Symlinks**: Stored as symlink entries (target path preserved)
+- **Device files**: Character and block devices included with correct major/minor
+- **Named pipes**: Included (header only, no data)
+- **Hard links**: Deduplicated — multiple hard links to the same file are stored as link entries to avoid duplication
+
+## Logging
+
+Backer logs events via Go's `slog`. Log levels: error, warn, info, debug.
+### Backup Completion Log
+
+When a backup completes, the following information is logged:
+- **client**: Client IP address
+- **user**: Authenticated username
+- **files**: Number of files backed up
+- **bytes**: Size of the created archive in bytes
+- **md5**: MD5 hash of the created archive (for verification)
+- **duration**: Time taken to create the archive
+
+Example:
+```
+Backup completed: client=127.0.0.1 user=admin files=42 bytes=1234567 md5=abc123def456 duration=2.5s
+```
+### TLS Error Handling
+
+TLS handshake errors and other client-side SSL/TLS errors are logged at **debug** level to reduce log clutter.
+This includes:
+- Certificate verification failures
+- Handshake failures
+- Protocol errors
+- TLS-specific connection issues
+
+### Graceful Shutdown
+
+When backer receives a termination signal (SIGTERM or SIGINT), it will:
+1. Log a warning message: "Shutdown signal received, shutting down immediately..."
+2. Stop accepting new connections and abort any in-progress downloads immediately
+3. Clean up resources and exit
+
+This ensures no partial/corrupted backups are left. Clients will need to restart their backup. For a forced immediate stop, use `SIGKILL`.
+
 ## Special Thanks
 
 This project was developed with the assistance of **opencode** (AI co-programmer powered by MIMO). The AI helped with:

cmd/backer/main.go

@@ -2,6 +2,7 @@
 package main
 
 import (
+	"errors"
 	"flag"
 	"fmt"
 	"net/http"
@@ -40,10 +41,14 @@ func main() {
 		log.Fatalf("Failed to create server: %v", err)
 	}
 
-	log.Infof("Starting server at %s", server.Addr)
+	// Setup signal handling for immediate shutdown.
+	log.Infof("Installing signal handler")
+	setupSignalHandler(server)
+
+	log.Infof("Starting backer version %s at %s", version, server.Addr)
 
 	if backer.C.NoHTTPS {
-		if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
+		if err := server.Serve(); err != nil && !errors.Is(err, http.ErrServerClosed) {
 			log.Fatalf("Failed to start server: %v", err)
 		}
 	} else {
@@ -55,10 +60,12 @@ func main() {
 			log.Fatalf("Certificate key file error: %v", err)
 		}
 
-		if err := server.ListenAndServeTLS(backer.C.Cert, backer.C.Key); err != nil && err != http.ErrServerClosed {
+		if err := server.ServeTLS(backer.C.Cert, backer.C.Key); err != nil && !errors.Is(err, http.ErrServerClosed) {
 			log.Fatalf("Failed to start server: %v", err)
 		}
 	}
+
+	log.Infof("Server stopped")
 }
 
 // defaultConfigPath returns the default config file path based on the OS.

cmd/backer/sigHandler.go

@@ -0,0 +1,29 @@
+package main
+
+import (
+	"os"
+	"os/signal"
+	"syscall"
+
+	"backer/internal/backer"
+	"backer/internal/log"
+)
+
+// setupSignalHandler sets up signal handling for immediate shutdown.
+// SIGTERM is commonly sent by systemd, k8s, and other container orchestrators.
+// SIGINT is sent on Ctrl+C.
+func setupSignalHandler(server *backer.ServerWrapper) {
+	signalCh := make(chan os.Signal, 1)
+	signal.Notify(signalCh, os.Interrupt, syscall.SIGTERM)
+
+	go func() {
+		<-signalCh
+		log.Warnf("Shutdown signal received, shutting down immediately...")
+		// Close the embedded http.Server directly.
+		if closeErr := server.Close(); closeErr != nil {
+			log.Errorf("Error during server shutdown: %v", closeErr)
+		}
+	}()
+}
+
+/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

contrib/backer.openrc

@@ -11,7 +11,6 @@ depend() {
 }
 
 start_pre() {
-        mkdir -p /var/log/$SVCNAME
         mkdir -p /run/$SVCNAME
 }
 

data/config_example.json

@@ -47,6 +47,9 @@
     /* Regex patterns to exclude from backup.                                                                         */
     "exclude_patterns": [],
 
+    /* Maximum allowed time, spent for getting file list from backupped directories.                                  */
+    "dir_scan_timeout": 60,
+
     /* Directories to backup.                                                                                         */
     "directories": [
         "/path/to/directory1",

internal/backer/archive.go

@@ -73,7 +73,14 @@ func createArchiveStream(ctx context.Context, filepaths []string, writerFactory
 }
 
 // writeFilesToTar iterates over filepaths and writes each file to the tar archive.
+// It tracks inodes to handle hard links - if a file has already been archived,
+// subsequent hard links to the same inode are stored as link entries.
+// This is the Unix implementation (syscall not available on Windows).
 func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, closePipeWithError func(error)) {
+	// Track inodes to handle hard links: inode -> first path added.
+	// Uses simple file path tracking on Windows, inode tracking on Unix.
+	seenInodes := make(map[uint64]string)
+
 	for _, fpath := range filepaths {
 		var (
 			err        error
@@ -116,6 +123,21 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 			continue
 		}
 
+		// Check for hard link - if inode was already seen, store as link instead of content.
+		if st.Mode().IsRegular() {
+			inode := getInode(st)
+			if inode != 0 {
+				if existingPath, seen := seenInodes[inode]; seen {
+					// This is a hard link to already-archived file.
+					linkTarget = existingPath
+					log.Debugf("Hard link detected: %s -> %s", fpath, existingPath)
+				} else {
+					// First time seeing this inode - track it.
+					seenInodes[inode] = fpath
+				}
+			}
+		}
+
 		header, err := tar.FileInfoHeader(st, linkTarget)
 
 		if err != nil {
@@ -124,6 +146,11 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 			continue
 		}
 
+		// If linkTarget is set, ensure the header is marked as a link.
+		if linkTarget != "" {
+			header.Typeflag = tar.TypeLink
+		}
+
 		header.Format = tar.FormatGNU
 		header.Name = strings.TrimPrefix(filepath.ToSlash(filepath.Clean(fpath)), "/")
 
@@ -146,7 +173,7 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 				continue
 			}
 
-			err = copyWithContext(ctx, tw, f)
+			_, _, err = copyWithContext(ctx, tw, f)
 
 			if e := f.Close(); e != nil {
 				log.Warnf("Failed to close file %s: %v", fpath, e)
@@ -161,6 +188,27 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 	}
 }
 
+// getInode extracts the inode number from [os.FileInfo].
+// Returns 0 if unable to determine (e.g., on Windows or with certain file systems).
+func getInode(fi os.FileInfo) uint64 {
+	if fi.Sys() == nil {
+		return 0
+	}
+
+	// Use type assertion to get inode - works on Unix, returns 0 on Windows.
+	type inodeGetter interface {
+		Dev() uint64
+		Ino() uint64
+	}
+
+	if ig, ok := fi.Sys().(inodeGetter); ok {
+		// Combine device and inode for uniqueness across file systems.
+		return (ig.Dev() << 32) | ig.Ino()
+	}
+
+	return 0
+}
+
 // isPipeClosedError checks if the error is due to a closed pipe.
 func isPipeClosedError(err error) bool {
 	if err == nil {

internal/backer/bzip2.go

@@ -10,8 +10,22 @@ import (
 // CreateTarBzip2Stream takes a list of files and returns a reader that reads from a bzip2-compressed tar archive.
 func CreateTarBzip2Stream(ctx context.Context, filepaths []string) io.ReadCloser {
 	return createArchiveStream(ctx, filepaths, func(w io.Writer) (io.WriteCloser, error) {
-		return bzip2.NewWriter(w, nil)
+		level := mapCompressionLevelToBzip2(C.CompressionLevel)
+
+		return bzip2.NewWriter(w, &bzip2.WriterConfig{Level: level})
 	})
 }
 
+// mapCompressionLevelToBzip2 maps 1-9 compression level to bzip2 level.
+func mapCompressionLevelToBzip2(level int) int {
+	if level < 1 {
+		return bzip2.DefaultCompression
+	}
+	if level > 9 {
+		return bzip2.BestCompression
+	}
+
+	return level
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/consts.go

@@ -11,6 +11,12 @@ const (
 	// copyBufferSize is the buffer size used for streaming file contents.
 	copyBufferSize = 32 * 1024
 
+	// maxHeaderBytes is the maximum size of request headers (1KB).
+	maxHeaderBytes = 1024
+
+	// idleTimeout is the maximum time to wait for the next request on keep-alive connections.
+	idleTimeout = 10 * time.Second
+
 	// portMin is the minimum valid port number.
 	portMin = 1
 
@@ -23,6 +29,12 @@ const (
 	// backupTimeoutMax is the maximum backup timeout in minutes (24 hours).
 	backupTimeoutMax = 1440
 
+	// dirScanTimeoutMin is the minimum directory scan timeout in minutes.
+	dirScanTimeoutMin = 1
+
+	// dirScanTimeoutMax is the maximum directory scan timeout in minutes (24 hours).
+	dirScanTimeoutMax = 1440
+
 	// compressionLevelMin is the minimum compression level.
 	compressionLevelMin = 1
 
@@ -35,6 +47,9 @@ const (
 	// defaultBackupTimeout is the default backup timeout in minutes.
 	defaultBackupTimeout = 60
 
+	// defaultDirScanTimeout is the default directory scan timeout in minutes (1 hour).
+	defaultDirScanTimeout = 60
+
 	// defaultCompressionLevel is the default compression level.
 	defaultCompressionLevel = 9
 

internal/backer/getFilesFromDirectories.go

@@ -1,10 +1,13 @@
 package backer
 
 import (
+	"context"
+	"errors"
 	"io/fs"
+	"os"
 	"path/filepath"
 
-	"backer/internal/log"
+	backerlog "backer/internal/log"
 )
 
 // isExcluded checks if path matches any of the configured exclude patterns.
@@ -22,19 +25,38 @@ func isExcluded(path string) bool {
 }
 
 // GetFilesFromDirectories makes a file list of given directories.
-func GetFilesFromDirectories(directories []string) ([]string, error) {
+// Logs errors for directories that don't exist or can't be accessed but continues with valid directories.
+// Takes context with timeout from config dir_scan_timeout setting.
+func GetFilesFromDirectories(ctx context.Context, directories []string) ([]string, error) {
 	var files []string
 
 	for _, dir := range directories {
+		// Check directory exists before walking.
+		if _, err := os.Stat(dir); err != nil {
+			if errors.Is(err, os.ErrNotExist) {
+				backerlog.Errorf("Configured backup directory not found: %s", dir)
+			} else {
+				backerlog.Errorf("Failed to access configured directory %s: %v", dir, err)
+			}
+
+			continue
+		}
+
 		err := filepath.WalkDir(dir, func(path string, de fs.DirEntry, err error) error { //nolint: revive
+			select {
+			case <-ctx.Done():
+				return ctx.Err()
+			default:
+			}
+
 			if err != nil {
-				log.Warnf("Skipping %s: %v", path, err)
+				backerlog.Warnf("Skipping %s: %v", path, err)
 
 				return nil
 			}
 
 			if isExcluded(path) {
-				log.Debugf("Excluding: %s", path)
+				backerlog.Debugf("Excluding: %s", path)
 
 				return nil
 			}
@@ -45,7 +67,13 @@ func GetFilesFromDirectories(directories []string) ([]string, error) {
 		})
 
 		if err != nil {
-			log.Warnf("Failed to walk directory %s: %v", dir, err)
+			if errors.Is(err, context.DeadlineExceeded) {
+				backerlog.Errorf("Directory scan timed out after scanning %d files", len(files))
+
+				return files, ErrDirectoryScanTimeout{ScannedFiles: len(files)}
+			}
+
+			backerlog.Warnf("Failed to walk directory %s: %v", dir, err)
 		}
 	}
 

internal/backer/getFilesFromDirectories_test.go

@@ -1,7 +1,9 @@
 package backer
 
 import (
+	"context"
 	"os"
+	"path/filepath"
 	"regexp"
 	"slices"
 	"strings"
@@ -13,30 +15,47 @@ func normalizePath(p string) string {
 	return strings.ReplaceAll(p, "\\", "/")
 }
 
+// normalizePathSlice normalizes all paths in a slice.
+func normalizePathSlice(paths []string) []string {
+	result := make([]string, len(paths))
+	for i, p := range paths {
+		result[i] = normalizePath(p)
+	}
+	return result
+}
+
 // TestGetFilesFromDirectories checks if GetFilesFromDirectories returns the proper amount of strings, correct strings and no error.
 func TestGetFilesFromDirectories(t *testing.T) {
-	// Ensure empty_dir exists for the test.
+	testDataPath := "../../test_data"
-	os.MkdirAll("../../test_data/test1/foo/empty_dir", 0755)
+
+	// Log the paths we're checking for debugging
+	t.Logf("Checking directories: %v", []string{
+		filepath.Join(testDataPath, "test1", "foo"),
+		filepath.Join(testDataPath, "test1", "bar"),
+	})
 
 	var (
 		input = Config{
-			Directories: []string{"../../test_data/test1/foo", "../../test_data/test1/bar"},
+			Directories: []string{
+				filepath.Join(testDataPath, "test1", "foo"),
+				filepath.Join(testDataPath, "test1", "bar"),
+			},
 		}
 		expectedData = []string{
-			"../../test_data/test1/foo",
+			filepath.Join(testDataPath, "test1", "foo"),
-			"../../test_data/test1/foo/empty_dir",
+			filepath.Join(testDataPath, "test1", "foo", "empty_dir"),
-			"../../test_data/test1/foo/hello_breakfast.txt",
+			filepath.Join(testDataPath, "test1", "foo", "hello_breakfast.txt"),
-			"../../test_data/test1/foo/mydir",
+			filepath.Join(testDataPath, "test1", "foo", "mydir"),
-			"../../test_data/test1/foo/mydir/myfile.txt",
+			filepath.Join(testDataPath, "test1", "foo", "mydir", "myfile.txt"),
-			"../../test_data/test1/foo/some_text.txt",
+			filepath.Join(testDataPath, "test1", "foo", "some_text.txt"),
-			"../../test_data/test1/bar",
+			filepath.Join(testDataPath, "test1", "bar"),
-			"../../test_data/test1/bar/goodbye.txt",
+			filepath.Join(testDataPath, "test1", "bar", "goodbye.txt"),
-			"../../test_data/test1/bar/goodbye.txt/text.txt",
+			filepath.Join(testDataPath, "test1", "bar", "goodbye.txt", "text.txt"),
-			"../../test_data/test1/bar/hello.txt",
+			filepath.Join(testDataPath, "test1", "bar", "hello.txt"),
 		}
 	)
 
-	output, err := GetFilesFromDirectories(input.Directories)
+	output, err := GetFilesFromDirectories(context.Background(), input.Directories)
 
 	if err != nil {
 		t.Errorf("Expected no error, got %v", err)
@@ -51,7 +70,8 @@ func TestGetFilesFromDirectories(t *testing.T) {
 
 	for _, filepath := range output {
 		normalized := normalizePath(filepath)
-		if !slices.Contains(expectedData, normalized) {
+		normalizedExpected := normalizePathSlice(expectedData)
+		if !slices.Contains(normalizedExpected, normalized) {
 			t.Errorf("expectedData data does not contain string from output: %s", filepath)
 		}
 	}
@@ -59,7 +79,8 @@ func TestGetFilesFromDirectories(t *testing.T) {
 
 // TestGetFilesFromDirectoriesSingleDir tests with a single directory.
 func TestGetFilesFromDirectoriesSingleDir(t *testing.T) {
-	output, err := GetFilesFromDirectories([]string{"../../test_data/test1/foo"})
+	testDataPath := filepath.Join("../../test_data")
+	output, err := GetFilesFromDirectories(context.Background(), []string{filepath.Join(testDataPath, "test1", "foo")})
 
 	if err != nil {
 		t.Errorf("Expected no error, got %v", err)
@@ -70,7 +91,7 @@ func TestGetFilesFromDirectoriesSingleDir(t *testing.T) {
 	}
 
 	// Should contain the directory itself
-	if !slices.Contains(output, "../../test_data/test1/foo") {
+	if !slices.Contains(output, filepath.Join(testDataPath, "test1", "foo")) {
 		t.Error("Output should contain the root directory")
 	}
 }
@@ -80,7 +101,7 @@ func TestGetFilesFromDirectoriesEmptyDir(t *testing.T) {
 	// Create temp empty dir
 	tmpDir := t.TempDir()
 
-	output, err := GetFilesFromDirectories([]string{tmpDir})
+	output, err := GetFilesFromDirectories(context.Background(), []string{tmpDir})
 
 	if err != nil {
 		t.Errorf("Expected no error, got %v", err)
@@ -93,23 +114,48 @@ func TestGetFilesFromDirectoriesEmptyDir(t *testing.T) {
 }
 
 // TestGetFilesFromDirectoriesNonExistent tests with non-existent directory.
+// Should log error but not fail - returns empty list.
 func TestGetFilesFromDirectoriesNonExistent(t *testing.T) {
-	output, err := GetFilesFromDirectories([]string{"/nonexistent/path/that/does/not/exist"})
+	testDataPath := filepath.Join("../../test_data")
+	nonExistentPath := filepath.Join(testDataPath, "nonexistent", "path", "that", "does", "not", "exist")
+	output, err := GetFilesFromDirectories(context.Background(), []string{nonExistentPath})
 
+	// Should not return error, but logs error
 	if err != nil {
-		t.Errorf("Expected no error (logs warning instead), got %v", err)
+		t.Errorf("Expected no error (logs error instead), got %v", err)
 	}
 
+	// Should return empty list
 	if len(output) != 0 {
 		t.Errorf("Expected empty output for non-existent directory, got %d files", len(output))
 	}
 }
 
+// TestGetFilesFromDirectoriesPartialFailure tests with mix of valid and invalid directories.
+func TestGetFilesFromDirectoriesPartialFailure(t *testing.T) {
+	testDataPath := filepath.Join("../../test_data")
+	// Create a temp directory that exists
+	tmpDir := t.TempDir()
+
+	output, err := GetFilesFromDirectories(context.Background(), []string{tmpDir, filepath.Join(testDataPath, "nonexistent", "path", "that", "does", "not", "exist")})
+
+	// Should not return error, but logs error
+	if err != nil {
+		t.Errorf("Expected no error (logs error instead), got %v", err)
+	}
+
+	// Should return files from valid directory
+	if len(output) == 0 {
+		t.Error("Expected files from valid directory")
+	}
+}
+
 // TestGetFilesFromDirectoriesMultipleDirs tests multiple directories.
 func TestGetFilesFromDirectoriesMultipleDirs(t *testing.T) {
-	output, err := GetFilesFromDirectories([]string{
+	testDataPath := filepath.Join("../../test_data")
-		"../../test_data/test1/foo",
+	output, err := GetFilesFromDirectories(context.Background(), []string{
-		"../../test_data/test1/bar",
+		filepath.Join(testDataPath, "test1", "foo"),
+		filepath.Join(testDataPath, "test1", "bar"),
 	})
 
 	if err != nil {
@@ -118,10 +164,10 @@ func TestGetFilesFromDirectoriesMultipleDirs(t *testing.T) {
 
 	// Should contain entries from both directories
 	hasFoo := slices.ContainsFunc(output, func(s string) bool {
-		return s == "../../test_data/test1/foo" || s == "../../test_data/test1/foo/hello_breakfast.txt"
+		return s == filepath.Join(testDataPath, "test1", "foo") || s == filepath.Join(testDataPath, "test1", "foo", "hello_breakfast.txt")
 	})
 	hasBar := slices.ContainsFunc(output, func(s string) bool {
-		return s == "../../test_data/test1/bar" || s == "../../test_data/test1/bar/hello.txt"
+		return s == filepath.Join(testDataPath, "test1", "bar") || s == filepath.Join(testDataPath, "test1", "bar", "hello.txt")
 	})
 
 	if !hasFoo {
@@ -149,7 +195,7 @@ func TestGetFilesFromDirectoriesSymlink(t *testing.T) {
 		t.Skip("Symlinks not supported, skipping test")
 	}
 
-	output, err := GetFilesFromDirectories([]string{tmpDir})
+	output, err := GetFilesFromDirectories(context.Background(), []string{tmpDir})
 
 	if err != nil {
 		t.Errorf("Expected no error, got %v", err)
@@ -228,7 +274,7 @@ func TestGetFilesFromDirectoriesWithExcludes(t *testing.T) {
 		regexp.MustCompile(`\.tmp$`),
 	}
 
-	output, err := GetFilesFromDirectories([]string{tmpDir})
+	output, err := GetFilesFromDirectories(context.Background(), []string{tmpDir})
 	if err != nil {
 		t.Fatalf("Unexpected error: %v", err)
 	}

internal/backer/loadConfig.go

@@ -95,6 +95,17 @@ func LoadConfig(path string) error {
 		return fmt.Errorf("Config option backup_timeout must be between %d and %d minutes, got %d", backupTimeoutMin, backupTimeoutMax, C.BackupTimeout)
 	}
 
+	// DirScanTimeout: set default, then validate range.
+	if C.DirScanTimeout == 0 {
+		C.DirScanTimeout = defaultDirScanTimeout
+
+		log.Warnf("Config option dir_scan_timeout is not set, fallback to %d minutes", C.DirScanTimeout)
+	}
+
+	if C.DirScanTimeout < dirScanTimeoutMin || C.DirScanTimeout > dirScanTimeoutMax {
+		return fmt.Errorf("Config option dir_scan_timeout must be between %d and %d minutes, got %d", dirScanTimeoutMin, dirScanTimeoutMax, C.DirScanTimeout)
+	}
+
 	// CompressionLevel: set default, then validate range.
 	if C.CompressionLevel == 0 {
 		C.CompressionLevel = defaultCompressionLevel
@@ -146,7 +157,7 @@ func LoadConfig(path string) error {
 
 	switch C.DefaultCompression {
 	case "gzip", "pgzip", "bzip2", "xz", "lz4", "zstd":
-		log.Infof("Using %s as defailt compression algorithm", C.DefaultCompression)
+		// No-op: compression algorithm validated.
 	default:
 		return fmt.Errorf("Config option default_compression must be gzip, bzip2, zstd, lz4, xz or pgzip, got %s", C.DefaultCompression)
 	}

internal/backer/loadConfig_test.go

@@ -1,7 +1,9 @@
 package backer
 
 import (
+	"encoding/json"
 	"os"
+	"path/filepath"
 	"slices"
 	"testing"
 )
@@ -10,18 +12,46 @@ func TestLoadConfig(t *testing.T) {
 	// Reset global config
 	C = Config{}
 
-	var (
+	testDataPath := filepath.Join(t.TempDir(), "test_data")
-		testConfigPath = "./../../test_data/test_config.json"
+	os.MkdirAll(filepath.Join(testDataPath, "test1", "foo"), 0755)
-		expectedData   = Config{
+	os.MkdirAll(filepath.Join(testDataPath, "test1", "bar"), 0755)
+
+	// Create test config file
+	testConfigPath := filepath.Join(testDataPath, "test_config.json")
+	testStruct := struct {
+		Address     string   `json:"address"`
+		Port        int      `json:"port"`
+		Directories []string `json:"directories"`
+		Location    string   `json:"location"`
+		User        string   `json:"user"`
+		Password    string   `json:"password"`
+		NoHTTPS     bool     `json:"nohttps"`
+	}{
+		Address:     "127.0.0.1",
+		Port:        8086,
+		Directories: []string{filepath.Join(testDataPath, "test1", "foo"), filepath.Join(testDataPath, "test1", "bar")},
+		Location:    "/archive",
+		User:        "test",
+		Password:    "test",
+		NoHTTPS:     true,
+	}
+	testConfigBytes, _ := json.MarshalIndent(testStruct, "", "  ")
+	if err := os.WriteFile(testConfigPath, testConfigBytes, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	expectedData := Config{
 		Address: "127.0.0.1",
 		Port:    8086,
 		Directories: []string{
-				"../../test_data/test1/foo",
+			filepath.Join(testDataPath, "test1", "foo"),
-				"../../test_data/test1/bar",
+			filepath.Join(testDataPath, "test1", "bar"),
 		},
 		Location: "/archive",
+		User:     "test",
+		Password: "test",
+		NoHTTPS:  true,
 	}
-	)
 
 	err := LoadConfig(testConfigPath)
 
@@ -57,6 +87,8 @@ func TestLoadConfig(t *testing.T) {
 
 // TestLoadConfigMissingUser tests that config fails when user is missing.
 func TestLoadConfigMissingUser(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"port": 8086,
 		"password": "test",
@@ -72,6 +104,8 @@ func TestLoadConfigMissingUser(t *testing.T) {
 
 // TestLoadConfigMissingPassword tests that config fails when password is missing.
 func TestLoadConfigMissingPassword(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"port": 8086,
 		"user": "test",
@@ -87,6 +121,8 @@ func TestLoadConfigMissingPassword(t *testing.T) {
 
 // TestLoadConfigMissingDirectories tests that config fails when directories is missing.
 func TestLoadConfigMissingDirectories(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"port": 8086,
 		"user": "test",
@@ -102,6 +138,8 @@ func TestLoadConfigMissingDirectories(t *testing.T) {
 
 // TestLoadConfigInvalidPort tests that config fails with invalid port.
 func TestLoadConfigInvalidPort(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"port": 99999,
 		"user": "test",
@@ -118,6 +156,8 @@ func TestLoadConfigInvalidPort(t *testing.T) {
 
 // TestLoadConfigNonExistentDirectory tests that config fails with non-existent directory.
 func TestLoadConfigNonExistentDirectory(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"port": 8086,
 		"user": "test",
@@ -134,6 +174,8 @@ func TestLoadConfigNonExistentDirectory(t *testing.T) {
 
 // TestLoadConfigHTTPSWithoutCert tests that config fails when HTTPS enabled but no cert.
 func TestLoadConfigHTTPSWithoutCert(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"port": 8086,
 		"user": "test",
@@ -151,6 +193,8 @@ func TestLoadConfigHTTPSWithoutCert(t *testing.T) {
 
 // TestLoadConfigNoHTTPS tests that config succeeds with nohttps=true and no cert.
 func TestLoadConfigNoHTTPS(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"port": 8086,
 		"user": "test",
@@ -172,6 +216,8 @@ func TestLoadConfigNoHTTPS(t *testing.T) {
 
 // TestLoadConfigFilenamePrefixCustom tests custom filename prefix.
 func TestLoadConfigFilenamePrefixCustom(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -193,6 +239,8 @@ func TestLoadConfigFilenamePrefixCustom(t *testing.T) {
 
 // TestLoadConfigDefaultValues tests default values are set correctly.
 func TestLoadConfigDefaultValues(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -234,6 +282,8 @@ func TestLoadConfigDefaultValues(t *testing.T) {
 
 // TestLoadConfigCompressionAlgorithmBzip2 tests bzip2 compression algorithm.
 func TestLoadConfigCompressionAlgorithmBzip2(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -255,6 +305,8 @@ func TestLoadConfigCompressionAlgorithmBzip2(t *testing.T) {
 
 // TestLoadConfigCompressionAlgorithmZstd tests zstd compression algorithm.
 func TestLoadConfigCompressionAlgorithmZstd(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -276,6 +328,8 @@ func TestLoadConfigCompressionAlgorithmZstd(t *testing.T) {
 
 // TestLoadConfigCompressionAlgorithmLz4 tests lz4 compression algorithm.
 func TestLoadConfigCompressionAlgorithmLz4(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -297,6 +351,8 @@ func TestLoadConfigCompressionAlgorithmLz4(t *testing.T) {
 
 // TestLoadConfigCompressionAlgorithmXz tests xz compression algorithm.
 func TestLoadConfigCompressionAlgorithmXz(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -318,6 +374,8 @@ func TestLoadConfigCompressionAlgorithmXz(t *testing.T) {
 
 // TestLoadConfigCompressionAlgorithmPgzip tests pgzip compression algorithm.
 func TestLoadConfigCompressionAlgorithmPgzip(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -339,6 +397,8 @@ func TestLoadConfigCompressionAlgorithmPgzip(t *testing.T) {
 
 // TestLoadConfigCompressionAlgorithmInvalid tests invalid compression algorithm.
 func TestLoadConfigCompressionAlgorithmInvalid(t *testing.T) {
+	// Reset global config
+	C = Config{}
 	config := `{
 		"user": "test",
 		"password": "test",
@@ -367,7 +427,6 @@ func writeTempConfig(t *testing.T, content string) string {
 	tmpFile.Close()
 	// Reset global config before loading
 	C = Config{}
-	excludePatterns = nil
 	return tmpFile.Name()
 }
 

internal/backer/lz4.go

@@ -10,8 +10,26 @@ import (
 // CreateTarLz4Stream takes a list of files and returns a reader that reads from an lz4-compressed tar archive.
 func CreateTarLz4Stream(ctx context.Context, filepaths []string) io.ReadCloser {
 	return createArchiveStream(ctx, filepaths, func(w io.Writer) (io.WriteCloser, error) {
-		return lz4.NewWriter(w), nil
+		lz4Writer := lz4.NewWriter(w)
+		// Map compression_level (1-9) to lz4 compression level.
+		// 1-3: fast, 4-6: balanced, 7-9: better compression.
+		lz4Writer.Header.CompressionLevel = mapCompressionLevelToLz4(C.CompressionLevel) //nolint: staticcheck
+
+		return lz4Writer, nil
 	})
 }
 
+// mapCompressionLevelToLz4 maps 1-9 compression level to lz4 compression level.
+// 1-3: fast, 4-6: balanced, 7-9: better compression.
+func mapCompressionLevelToLz4(level int) int {
+	switch {
+	case level <= 3:
+		return 0 // Fastest.
+	case level <= 6:
+		return 1 // Default.
+	default:
+		return 2 // Best compression.
+	}
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/newServer.go

@@ -2,7 +2,11 @@ package backer
 
 import (
 	"context"
+	"crypto/md5"
+	"crypto/subtle"
 	"crypto/tls"
+	"encoding/hex"
+	"errors"
 	"fmt"
 	"io"
 	"net"
@@ -14,6 +18,53 @@ import (
 	"backer/internal/log"
 )
 
+// isTLSError checks if the given error is TLS-related.
+// It checks for both TLS-specific error types and error messages containing "tls:".
+func isTLSError(err error) bool {
+	if err == nil {
+		return false
+	}
+
+	var alertErr *tls.AlertError
+	if errors.As(err, &alertErr) && alertErr != nil {
+		return true
+	}
+
+	var certErr *tls.CertificateVerificationError
+	if errors.As(err, &certErr) && certErr != nil {
+		return true
+	}
+
+	var echErr *tls.ECHRejectionError
+	if errors.As(err, &echErr) && echErr != nil {
+		return true
+	}
+
+	var recordErr *tls.RecordHeaderError
+	if errors.As(err, &recordErr) && recordErr != nil {
+		return true
+	}
+
+	return strings.HasPrefix(err.Error(), "tls:")
+}
+
+// withStatusLogging wraps a handler to log 404 and 405 responses.
+func withStatusLogging(next http.Handler) http.Handler {
+	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		rw := &responseWriterWrapper{ResponseWriter: w, statusCode: http.StatusOK}
+		next.ServeHTTP(rw, r)
+
+		clientIP := getClientIP(r)
+
+		switch rw.statusCode {
+		case http.StatusNotFound:
+			log.Infof("Not Found from %s to %s", clientIP, r.RequestURI)
+		case http.StatusMethodNotAllowed:
+			log.Infof("Method Not Allowed from %s for %s %s", clientIP, r.Method, r.RequestURI)
+		}
+	})
+}
+
 // withServerHeader wraps a handler to add Server header to all responses.
 func withServerHeader(next http.Handler) http.Handler {
 	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -23,7 +74,8 @@ func withServerHeader(next http.Handler) http.Handler {
 }
 
 // NewServer starts new server instance with given config file.
-func NewServer(configPath string) (*http.Server, error) {
+// It returns a ServerWrapper that intercepts and classifies server errors.
+func NewServer(configPath string) (*ServerWrapper, error) {
 	_ = log.Init("info", "") // Initialize logger with defaults for early logging.
 
 	if err := LoadConfig(configPath); err != nil {
@@ -34,9 +86,18 @@ func NewServer(configPath string) (*http.Server, error) {
 		log.Errorf("Failed to initialize logger: %v", err)
 	}
 
+	log.Infof("Using %s as default compression algorithm", C.DefaultCompression)
+
 	mux := http.NewServeMux()
 
-	mux.HandleFunc(C.Location, func(w http.ResponseWriter, r *http.Request) {
+	backupHandler := func(w http.ResponseWriter, r *http.Request) {
+		// Only GET method is allowed.
+		if r.Method != http.MethodGet {
+			http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
+
+			return
+		}
+
 		username, password, ok := r.BasicAuth()
 
 		// No auth? Good bye.
@@ -44,16 +105,18 @@ func NewServer(configPath string) (*http.Server, error) {
 			w.Header().Set("WWW-Authenticate", `Basic realm="restricted", charset="UTF-8"`)
 			http.Error(w, "Unauthorized", http.StatusUnauthorized)
 
+			log.Infof("Access denied from %s to %s (no username or password given)", r.RemoteAddr, r.RequestURI)
+
 			return
 		}
 
-		// Both username AND password must match. Explicitly require that the whole condition
+		// Both username AND password must match. Use timing-safe comparison to prevent timing attacks.
-		// fails when either is incorrect, per the authentication design.
+		if !timingSafeMatch(username, C.User) || !timingSafeMatch(password, C.Password) {
-		//nolint:staticcheck // QF1001: De Morgan's law is irrelevant here.
-		if !(username == C.User && password == C.Password) {
 			w.Header().Set("WWW-Authenticate", `Basic realm="restricted", charset="UTF-8"`)
 			http.Error(w, "Unauthorized", http.StatusUnauthorized)
 
+			log.Infof("Access denied from %s to %s (username and/or password incorrect)", r.RemoteAddr, r.RequestURI)
+
 			return
 		}
 
@@ -62,25 +125,41 @@ func NewServer(configPath string) (*http.Server, error) {
 
 		log.Infof("Backup started: client=%s user=%s", clientIP, username)
 
-		ctx := r.Context()
+		// Create context with dir_scan_timeout from request context.
-		files, err := GetFilesFromDirectories(C.Directories)
+		ctx, cancel := context.WithTimeout(r.Context(), time.Duration(C.DirScanTimeout)*time.Minute)
+		defer cancel()
+
+		files, err := GetFilesFromDirectories(ctx, C.Directories)
 
 		if err != nil {
+			switch {
+			case errors.Is(err, context.DeadlineExceeded):
+				w.Header().Set("Server", "backer")
+				http.Error(w, "Directory scan timed out", http.StatusRequestTimeout)
+
+				log.Errorf("Directory scan timed out: %v", err)
+
+			default:
 				log.Errorf("Failed to get files: %v", err)
 
 				w.Header().Set("Server", "backer")
 				http.Error(w, "Failed to get files", http.StatusInternalServerError)
+			}
 
 			return
 		}
 
+		if len(files) == 0 {
+			log.Warnf("No files found to backup in configured directories")
+		}
+
 		var (
 			archive   io.ReadCloser
 			extension string
 			algorithm string
 		)
 
-		requestedExt := strings.TrimPrefix(path.Ext(r.URL.Path), ".")
+		requestedExt := r.URL.Path
 
 		algorithm = getCompressionAlgorithm(requestedExt)
 
@@ -91,7 +170,11 @@ func NewServer(configPath string) (*http.Server, error) {
 
 		case "zstd":
 			archive = CreateTarZstdStream(ctx, files)
+			if strings.HasSuffix(requestedExt, ".zst") {
 				extension = "tar.zst"
+			} else {
+				extension = "tar.zstd"
+			}
 
 		case "lz4":
 			archive = CreateTarLz4Stream(ctx, files)
@@ -116,44 +199,80 @@ func NewServer(configPath string) (*http.Server, error) {
 			}
 		}()
 
-		timestamp := time.Now().Format("20060102-150405")
+		timestamp := backupStart.Format("20060102-150405")
 		filename := fmt.Sprintf("%s-%s.%s", C.FilenamePrefix, timestamp, extension)
 
 		w.Header().Set("Content-Disposition", "attachment; filename="+filename)
 		w.Header().Set("Content-Type", "application/octet-stream")
 
-		bytesWritten, err := io.Copy(w, archive)
+		bytesWritten, md5Hash, err := copyWithContext(ctx, w, archive)
+		if err != nil {
 			duration := time.Since(backupStart)
 
-		if err != nil {
+			if errors.Is(err, context.Canceled) {
+				log.Infof("Backup canceled: client=%s user=%s duration=%s", clientIP, username, duration)
+
+				return
+			}
+
+			if isPipeClosedError(err) {
+				log.Infof("Client disconnected during backup: client=%s user=%s", clientIP, username)
+
+				return
+			}
+
 			log.Errorf("Backup failed: client=%s user=%s duration=%s error=%v", clientIP, username, duration, err)
 
 			return
 		}
 
-		log.Infof("Backup completed: client=%s user=%s files=%d bytes=%d duration=%s", clientIP, username, len(files), bytesWritten, duration)
+		log.Infof("Backup completed: client=%s user=%s files=%d bytes=%d md5=%s duration=%s", clientIP, username, len(files), bytesWritten, md5Hash, time.Since(backupStart))
-	})
+	}
+
+	mux.Handle(C.Location, http.HandlerFunc(backupHandler))
+
+	extensions := []string{".tar.gz", ".tar.xz", ".tar.bz2", ".tar.lz4", ".tar.zst", ".tar.zstd"}
+	for _, ext := range extensions {
+		muxPath := C.Location + ext
+		mux.Handle(muxPath, http.HandlerFunc(backupHandler))
+	}
 
 	// HTTP mode if nohttps is enabled in config.
 	// HTTPS mode if nohttps is disabled in config.
 	if C.NoHTTPS {
-		return &http.Server{
+		srv := &ServerWrapper{
+			Server: &http.Server{
 				Addr:              fmt.Sprintf("%s:%d", C.Address, C.Port),
-			Handler:           withServerHeader(mux),
+				Handler:           withServerHeader(withStatusLogging(mux)),
 				ReadHeaderTimeout: readHeaderTimeout,
 				WriteTimeout:      time.Duration(C.BackupTimeout) * time.Minute,
-		}, nil
+				IdleTimeout:       idleTimeout,
+				MaxHeaderBytes:    maxHeaderBytes,
+				ErrorLog:          log.DebugLogger(),
+			},
+		}
+		srv.SetKeepAlivesEnabled(false)
+
+		return srv, nil
 	}
 
-	return &http.Server{
+	srv := &ServerWrapper{
+		Server: &http.Server{
 			Addr:              fmt.Sprintf("%s:%d", C.Address, C.Port),
-		Handler:           withServerHeader(mux),
+			Handler:           withServerHeader(withStatusLogging(mux)),
 			ReadHeaderTimeout: readHeaderTimeout,
 			WriteTimeout:      time.Duration(C.BackupTimeout) * time.Minute,
+			IdleTimeout:       idleTimeout,
+			MaxHeaderBytes:    maxHeaderBytes,
 			TLSConfig: &tls.Config{
 				MinVersion: tls.VersionTLS13,
 			},
-	}, nil
+			ErrorLog: log.DebugLogger(),
+		},
+	}
+	srv.SetKeepAlivesEnabled(false)
+
+	return srv, nil
 }
 
 // writeWithContext checks if context is canceled before executing fn.
@@ -169,31 +288,50 @@ func writeWithContext(ctx context.Context, fn func() error) error {
 }
 
 // copyWithContext copies data from src to dst using a 32KB buffer, respecting
-// context cancellation. If the context is canceled, the copy stops and returns
+// context cancellation. Returns bytes written, MD5 hash (hex string), and any error encountered.
-// ctx.Err().
+//
-func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) error {
+//nolint:gosec
+func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) (int64, string, error) {
 	buf := make([]byte, copyBufferSize)
 
+	var bytesWritten int64
+
+	hasher := md5.New()
+
 	for {
 		select {
 		case <-ctx.Done():
-			return ctx.Err()
+			return bytesWritten, "", ctx.Err()
 		default:
 		}
 
 		n, readErr := src.Read(buf)
 		if n > 0 {
-			if _, werr := dst.Write(buf[:n]); werr != nil {
+			select {
-				return werr
+			case <-ctx.Done():
+				return bytesWritten, "", ctx.Err()
+			default:
 			}
+
+			if _, werr := hasher.Write(buf[:n]); werr != nil {
+				return bytesWritten, "", werr
+			}
+
+			if _, werr := dst.Write(buf[:n]); werr != nil {
+				return bytesWritten, "", werr
+			}
+
+			bytesWritten += int64(n)
 		}
 
 		if readErr != nil {
 			if readErr == io.EOF {
-				return nil
+				hash := hex.EncodeToString(hasher.Sum(nil))
+
+				return bytesWritten, hash, nil
 			}
 
-			return readErr
+			return bytesWritten, "", readErr
 		}
 	}
 }
@@ -216,21 +354,99 @@ func getClientIP(r *http.Request) string {
 // getCompressionAlgorithm determines which compression algorithm to use based on the requested extension.
 // If the extension is recognized, it returns the corresponding algorithm; otherwise, it returns the default.
 // For .tar.gz extension, pgzip is always used for parallel compression.
-func getCompressionAlgorithm(requestedExt string) string {
+// The input can be either a full path like "/archive.tar.xz" or just the extension like "tar.xz" or "xz".
-	switch requestedExt {
+func getCompressionAlgorithm(input string) string {
-	case "tar.bz2":
+	// Extract extension from full path (e.g., "/archive.tar.xz" -> ".xz").
+	// or use the input directly if it's just an extension (e.g., "tar.xz" -> ".xz").
+	ext := path.Ext(input)
+	if ext == "" {
+		ext = "." + input
+	}
+
+	switch ext {
+	case ".tar.bz2":
 		return "bzip2"
-	case "tar.zst":
+	case ".tar.zstd":
 		return "zstd"
-	case "tar.lz4":
+	case ".tar.zst":
+		return "zstd"
+	case ".tar.lz4":
 		return "lz4"
-	case "tar.xz":
+	case ".tar.xz":
 		return "xz"
-	case "tar.gz":
+	case ".tar.gz":
+		return "pgzip"
+	case ".xz":
+		return "xz"
+	case ".bz2":
+		return "bzip2"
+	case ".zst":
+		return "zstd"
+	case ".zstd":
+		return "zstd"
+	case ".lz4":
+		return "lz4"
+	case ".gz":
 		return "pgzip"
 	default:
 		return C.DefaultCompression
 	}
 }
 
+type (
+	// ServerWrapper wraps [http.Server] to intercept and classify errors.
+	ServerWrapper struct {
+		*http.Server
+	}
+
+	// responseWriterWrapper wraps http.ResponseWriter to capture status code.
+	responseWriterWrapper struct {
+		http.ResponseWriter
+		statusCode int
+	}
+)
+
+// WriteHeader captures the status code for logging.
+func (rw *responseWriterWrapper) WriteHeader(code int) {
+	rw.statusCode = code
+	rw.ResponseWriter.WriteHeader(code)
+}
+
+// Serve starts the server and classifies any errors returned.
+// TLS-related errors are logged at debug level, other errors at warn level.
+func (sw *ServerWrapper) Serve() error {
+	err := sw.ListenAndServe()
+
+	if err != nil {
+		if isTLSError(err) {
+			log.Debugf("Server error: %v", err)
+		} else {
+			log.Warnf("Server error: %v", err)
+		}
+	}
+
+	return err
+}
+
+// ServeTLS starts the TLS server and classifies any errors returned.
+func (sw *ServerWrapper) ServeTLS(certFile, keyFile string) error {
+	err := sw.ListenAndServeTLS(certFile, keyFile)
+
+	if err != nil {
+		if isTLSError(err) {
+			log.Debugf("Server TLS error: %v", err)
+		} else {
+			log.Warnf("Server TLS error: %v", err)
+		}
+	}
+
+	return err
+}
+
+// timingSafeMatch performs constant-time comparison to prevent timing attacks.
+// Returns 1 if the strings are equal, 0 otherwise.
+func timingSafeMatch(a, b string) bool {
+	return subtle.ConstantTimeCompare([]byte(a), []byte(b)) == 1
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/newServer_test.go

@@ -4,6 +4,8 @@ import (
 	"archive/tar"
 	"compress/gzip"
 	"context"
+	"crypto/tls"
+	"errors"
 	"io"
 	"net/http"
 	"net/http/httptest"
@@ -11,6 +13,166 @@ import (
 	"testing"
 )
 
+// TestIsTLSError tests the isTLSError function.
+func TestIsTLSError(t *testing.T) {
+	tests := []struct {
+		name     string
+		err      error
+		expected bool
+	}{
+		{
+			name:     "nil error",
+			err:      nil,
+			expected: false,
+		},
+		{
+			name:     "regular error",
+			err:      errors.New("some random error"),
+			expected: false,
+		},
+		{
+			name:     "TLS prefix error",
+			err:      errors.New("tls: handshake failure"),
+			expected: true,
+		},
+		{
+			name:     "TLS AlertError",
+			err:      tls.AlertError(80),
+			expected: true,
+		},
+		{
+			name:     "TLS CertificateVerificationError",
+			err:      &tls.CertificateVerificationError{},
+			expected: true,
+		},
+		{
+			name:     "TLS ECHRejectionError",
+			err:      &tls.ECHRejectionError{RetryConfigList: []byte("rejected")},
+			expected: true,
+		},
+		{
+			name:     "TLS RecordHeaderError",
+			err:      &tls.RecordHeaderError{},
+			expected: true,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			result := isTLSError(tt.err)
+			if result != tt.expected {
+				t.Errorf("isTLSError(%v) = %v, expected %v", tt.err, result, tt.expected)
+			}
+		})
+	}
+}
+
+// TestGetClientIP tests the getClientIP function.
+func TestGetClientIP(t *testing.T) {
+	tests := []struct {
+		name       string
+		xffHeader  string
+		remoteAddr string
+		expectedIP string
+	}{
+		{
+			name:       "no X-Forwarded-For",
+			xffHeader:  "",
+			remoteAddr: "192.168.1.100:12345",
+			expectedIP: "192.168.1.100",
+		},
+		{
+			name:       "single IP in X-Forwarded-For",
+			xffHeader:  "10.0.0.1",
+			remoteAddr: "192.168.1.100:12345",
+			expectedIP: "10.0.0.1",
+		},
+		{
+			name:       "multiple IPs in X-Forwarded-For",
+			xffHeader:  "10.0.0.1, 192.168.1.100, 172.16.0.1",
+			remoteAddr: "192.168.1.100:12345",
+			expectedIP: "10.0.0.1",
+		},
+		{
+			name:       "X-Forwarded-For with spaces",
+			xffHeader:  "  10.0.0.1  ,  192.168.1.100  ",
+			remoteAddr: "192.168.1.100:12345",
+			expectedIP: "10.0.0.1",
+		},
+		{
+			name:       "invalid RemoteAddr",
+			xffHeader:  "",
+			remoteAddr: "invalid",
+			expectedIP: "invalid",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			req := httptest.NewRequest("GET", "/archive", nil)
+			if tt.xffHeader != "" {
+				req.Header.Set("X-Forwarded-For", tt.xffHeader)
+			}
+			req.RemoteAddr = tt.remoteAddr
+
+			result := getClientIP(req)
+			if result != tt.expectedIP {
+				t.Errorf("getClientIP() = %q, expected %q", result, tt.expectedIP)
+			}
+		})
+	}
+}
+
+// TestGetCompressionAlgorithm tests the getCompressionAlgorithm function.
+func TestGetCompressionAlgorithm(t *testing.T) {
+	tests := []struct {
+		name     string
+		ext      string
+		expected string
+	}{
+		{
+			name:     "tar.bz2",
+			ext:      "tar.bz2",
+			expected: "bzip2",
+		},
+		{
+			name:     "tar.zst",
+			ext:      "tar.zst",
+			expected: "zstd",
+		},
+		{
+			name:     "tar.lz4",
+			ext:      "tar.lz4",
+			expected: "lz4",
+		},
+		{
+			name:     "tar.xz",
+			ext:      "tar.xz",
+			expected: "xz",
+		},
+		{
+			name:     "tar.gz",
+			ext:      "tar.gz",
+			expected: "pgzip",
+		},
+		{
+			name:     "default with gzip config",
+			ext:      "",
+			expected: "gzip",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			C = Config{DefaultCompression: "gzip"}
+			result := getCompressionAlgorithm(tt.ext)
+			if result != tt.expected {
+				t.Errorf("getCompressionAlgorithm(%q) = %q, expected %q", tt.ext, result, tt.expected)
+			}
+		})
+	}
+}
+
 // TestNewServerSuccess tests successful server creation.
 func TestNewServerSuccess(t *testing.T) {
 	C = Config{} // Reset global config.
@@ -189,7 +351,7 @@ func TestHandlerArchiveStream(t *testing.T) {
 		}
 
 		ctx := r.Context()
-		files, err := GetFilesFromDirectories(C.Directories)
+		files, err := GetFilesFromDirectories(ctx, C.Directories)
 		if err != nil {
 			http.Error(w, "Failed to get files", http.StatusInternalServerError)
 			return
@@ -301,7 +463,7 @@ func TestCopyWithContextSuccess(t *testing.T) {
 	src := strings.NewReader("hello world")
 	dst := &strings.Builder{}
 
-	err := copyWithContext(ctx, dst, src)
+	_, _, err := copyWithContext(ctx, dst, src)
 	if err != nil {
 		t.Errorf("Expected no error, got %v", err)
 	}
@@ -311,6 +473,25 @@ func TestCopyWithContextSuccess(t *testing.T) {
 	}
 }
 
+// TestCopyWithContextMD5Hash tests that MD5 hash is correctly computed.
+func TestCopyWithContextMD5Hash(t *testing.T) {
+	ctx := context.Background()
+	data := "hello world"
+	src := strings.NewReader(data)
+	dst := &strings.Builder{}
+
+	_, md5Hash, err := copyWithContext(ctx, dst, src)
+	if err != nil {
+		t.Errorf("Expected no error, got %v", err)
+	}
+
+	// MD5 of "hello world" is 5eb63bbbe01eeed093cb22bb8f5acdc3
+	expectedMD5 := "5eb63bbbe01eeed093cb22bb8f5acdc3"
+	if md5Hash != expectedMD5 {
+		t.Errorf("Expected MD5 %s, got %s", expectedMD5, md5Hash)
+	}
+}
+
 // TestCopyWithContextCancellation tests copy with cancelled context.
 func TestCopyWithContextCancellation(t *testing.T) {
 	ctx, cancel := context.WithCancel(context.Background())
@@ -325,17 +506,116 @@ func TestCopyWithContextCancellation(t *testing.T) {
 
 	// Start copy in goroutine
 	errCh := make(chan error, 1)
+	bytesCh := make(chan int64, 1)
 	go func() {
-		errCh <- copyWithContext(ctx, dst, src)
+		n, _, err := copyWithContext(ctx, dst, src)
+		bytesCh <- n
+		errCh <- err
 	}()
 
 	// Cancel context immediately
 	cancel()
 
 	err := <-errCh
+	<-bytesCh
 	if err != context.Canceled {
 		t.Errorf("Expected context.Canceled, got %v", err)
 	}
 }
 
+// TestStatusLogging404 tests that 404 responses are logged.
+func TestStatusLogging404(t *testing.T) {
+	C = Config{} // Reset global config.
+	excludePatterns = nil
+
+	// Create a minimal mux that returns 404 for all paths
+	testMux := http.NewServeMux()
+	testMux.HandleFunc("/archive", func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusOK)
+	})
+
+	wrappedMux := withStatusLogging(testMux)
+
+	// Use a ResponseRecorder to capture the response
+	rr := httptest.NewRecorder()
+	req, _ := http.NewRequest("GET", "/nonexistent", nil)
+	req.RemoteAddr = "127.0.0.1:12345"
+
+	wrappedMux.ServeHTTP(rr, req)
+
+	if rr.Code != http.StatusNotFound {
+		t.Errorf("Expected status 404, got %d", rr.Code)
+	}
+}
+
+// TestStatusLogging405 tests that 405 responses are logged.
+func TestStatusLogging405(t *testing.T) {
+	C = Config{} // Reset global config.
+	excludePatterns = nil
+
+	// Create a minimal mux that returns 405 for wrong method
+	testMux := http.NewServeMux()
+	testMux.HandleFunc("/archive", func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusMethodNotAllowed)
+	})
+
+	wrappedMux := withStatusLogging(testMux)
+
+	rr := httptest.NewRecorder()
+	req, _ := http.NewRequest("GET", "/archive", nil)
+	req.RemoteAddr = "127.0.0.1:12345"
+
+	wrappedMux.ServeHTTP(rr, req)
+
+	if rr.Code != http.StatusMethodNotAllowed {
+		t.Errorf("Expected status 405, got %d", rr.Code)
+	}
+}
+
+// TestMethodNotAllowedHandler tests that non-GET methods return 405.
+func TestMethodNotAllowedHandler(t *testing.T) {
+	C = Config{
+		Address:     "0.0.0.0",
+		Port:        8086,
+		Location:    "/archive",
+		User:        "testuser",
+		Password:    "testpass",
+		Directories: []string{"../../test_data/test1/foo"},
+	}
+	excludePatterns = nil
+
+	// Create test server
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		username, password, ok := r.BasicAuth()
+		if !ok || username != C.User || password != C.Password {
+			http.Error(w, "Unauthorized", http.StatusUnauthorized)
+
+			return
+		}
+
+		if r.Method != http.MethodGet {
+			w.Header().Set("Server", "backer")
+			http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
+
+			return
+		}
+
+		w.WriteHeader(http.StatusOK)
+	}))
+	defer server.Close()
+
+	// Test POST method
+	req, _ := http.NewRequest("POST", server.URL+"/archive", nil)
+	req.SetBasicAuth("testuser", "testpass")
+	resp, err := http.DefaultClient.Do(req)
+	if err != nil {
+		t.Fatalf("Request failed: %v", err)
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != http.StatusMethodNotAllowed {
+		t.Errorf("Expected status 405, got %d", resp.StatusCode)
+	}
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/types.go

@@ -15,12 +15,23 @@ type (
 		LogLevel             string   `json:"loglevel"`
 		Directories          []string `json:"directories"`
 		BackupTimeout        int      `json:"backup_timeout"`        // Timeout in minutes for backup streaming.
+		DirScanTimeout       int      `json:"dir_scan_timeout"`      // Timeout in minutes for directory scanning.
 		CompressionLevel     int      `json:"compression_level"`     // Compression level (1-9, default 9).
 		ExcludePatterns      []string `json:"exclude_patterns"`      // Regex patterns to exclude from backup.
 		FilenamePrefix       string   `json:"filename_prefix"`       // Prefix for backup filename (default: "backup").
 		CompressionAlgorithm string   `json:"compression_algorithm"` // Compression algorithm alias (deprecated: use default_compression).
 		DefaultCompression   string   `json:"default_compression"`   // Default compression algorithm: gzip, pgzip, bzip2, zstd, lz4 or xz.
 	}
+
+	// ErrDirectoryScanTimeout is returned when directory scan times out.
+	ErrDirectoryScanTimeout struct {
+		ScannedFiles int
+	}
 )
 
+// Error returns the error message for ErrDirectoryScanTimeout.
+func (e ErrDirectoryScanTimeout) Error() string {
+	return "Directory scan timed out"
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/utils_test.go

@@ -7,6 +7,7 @@ import (
 	"io"
 	"os"
 	"path/filepath"
+	"runtime"
 	"testing"
 )
 
@@ -179,6 +180,73 @@ func TestCreateTarGzStreamSymlink(t *testing.T) {
 	}
 }
 
+// TestCreateTarGzStreamHardLink tests hard link handling - should deduplicate content.
+func TestCreateTarGzStreamHardLink(t *testing.T) {
+	if runtime.GOOS == "windows" {
+		t.Skip("Hard link detection not supported on Windows (inodes unavailable)")
+	}
+
+	tmpDir := t.TempDir()
+
+	// Create original file
+	originalFile := filepath.Join(tmpDir, "original.txt")
+	if err := os.WriteFile(originalFile, []byte("shared content"), 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	// Create hard link
+	hardLinkFile := filepath.Join(tmpDir, "hardlink.txt")
+	if err := os.Link(originalFile, hardLinkFile); err != nil {
+		t.Skip("Hard links not supported, skipping test")
+	}
+
+	ctx := context.Background()
+	reader := CreateTarGzStream(ctx, []string{originalFile, hardLinkFile})
+	defer reader.Close()
+
+	gzReader, err := gzip.NewReader(reader)
+	if err != nil {
+		t.Fatalf("Failed to create gzip reader: %v", err)
+	}
+	defer gzReader.Close()
+
+	tarReader := tar.NewReader(gzReader)
+
+	entries := 0
+	linkCount := 0
+
+	for {
+		header, err := tarReader.Next()
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			t.Fatalf("Failed to read tar entry: %v", err)
+		}
+
+		entries++
+		t.Logf("Entry %d: name=%s typeflag=%c linkname=%s", entries, header.Name, header.Typeflag, header.Linkname)
+
+		if header.Typeflag == tar.TypeLink {
+			linkCount++
+			// Verify it's a link to the original
+			if header.Linkname != originalFile && header.Linkname != filepath.ToSlash(originalFile) {
+				t.Errorf("Expected linkname '%s', got '%s'", originalFile, header.Linkname)
+			}
+		}
+	}
+
+	// Should have 2 entries: one file content, one as hard link
+	if entries != 2 {
+		t.Errorf("Expected 2 entries, got %d", entries)
+	}
+
+	// One should be a hard link (TypeLink)
+	if linkCount != 1 {
+		t.Errorf("Expected 1 hard link entry, got %d", linkCount)
+	}
+}
+
 // TestCreateTarGzStreamCancellation tests context cancellation.
 func TestCreateTarGzStreamCancellation(t *testing.T) {
 	tmpDir := t.TempDir()
@@ -356,4 +424,144 @@ func TestCreateTarGzStreamDeviceFile(t *testing.T) {
 	}
 }
 
+// TestCompressionLevelMappingLz4 tests that lz4 compression levels are correctly mapped.
+func TestCompressionLevelMappingLz4(t *testing.T) {
+	tests := []struct {
+		input    int
+		expected int
+	}{
+		{1, 0},
+		{2, 0},
+		{3, 0},
+		{4, 1},
+		{5, 1},
+		{6, 1},
+		{7, 2},
+		{8, 2},
+		{9, 2},
+	}
+
+	for _, tt := range tests {
+		C = Config{CompressionLevel: tt.input}
+		result := mapCompressionLevelToLz4(tt.input)
+		if result != tt.expected {
+			t.Errorf("mapCompressionLevelToLz4(%d) = %d, expected %d", tt.input, result, tt.expected)
+		}
+	}
+}
+
+// TestCompressionLevelMappingXz tests that xz compression is not configurable via level.
+// xz uses default compression - this test verifies the stream works.
+func TestCompressionLevelMappingXz(t *testing.T) {
+	// xz doesn't support configurable compression levels in the same way.
+	// It always uses default compression. This test verifies the stream is created successfully.
+	C = Config{CompressionLevel: 6}
+
+	tmpDir := t.TempDir()
+	testFile := filepath.Join(tmpDir, "test.txt")
+	content := []byte("test content")
+	if err := os.WriteFile(testFile, content, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarXzStream(ctx, []string{testFile})
+	defer reader.Close()
+
+	data, err := io.ReadAll(reader)
+	if err != nil {
+		t.Fatalf("Failed to read xz archive: %v", err)
+	}
+
+	if len(data) == 0 {
+		t.Error("Expected non-empty archive")
+	}
+}
+
+// TestCompressionLevelMappingBzip2 tests that bzip2 compression levels are correctly mapped.
+func TestCompressionLevelMappingBzip2(t *testing.T) {
+	tests := []struct {
+		input    int
+		expected int
+	}{
+		{1, 1},
+		{5, 5},
+		{9, 9},
+	}
+
+	for _, tt := range tests {
+		C = Config{CompressionLevel: tt.input}
+		result := mapCompressionLevelToBzip2(tt.input)
+		if result != tt.expected {
+			t.Errorf("mapCompressionLevelToBzip2(%d) = %d, expected %d", tt.input, result, tt.expected)
+		}
+	}
+}
+
+// TestCreateTarBzip2Stream tests bzip2 compression with configured level.
+func TestCreateTarBzip2Stream(t *testing.T) {
+	C = Config{CompressionLevel: 6}
+
+	tmpDir := t.TempDir()
+	testFile := filepath.Join(tmpDir, "test.txt")
+	content := []byte("test content for bzip2")
+	if err := os.WriteFile(testFile, content, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarBzip2Stream(ctx, []string{testFile})
+	defer reader.Close()
+
+	// Verify we can read the archive
+	_, err := io.ReadAll(reader)
+	if err != nil {
+		t.Fatalf("Failed to read bzip2 archive: %v", err)
+	}
+}
+
+// TestCreateTarLz4Stream tests lz4 compression with configured level.
+func TestCreateTarLz4Stream(t *testing.T) {
+	C = Config{CompressionLevel: 6}
+
+	tmpDir := t.TempDir()
+	testFile := filepath.Join(tmpDir, "test.txt")
+	content := []byte("test content for lz4")
+	if err := os.WriteFile(testFile, content, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarLz4Stream(ctx, []string{testFile})
+	defer reader.Close()
+
+	// Verify we can read the archive
+	_, err := io.ReadAll(reader)
+	if err != nil {
+		t.Fatalf("Failed to read lz4 archive: %v", err)
+	}
+}
+
+// TestCreateTarXzStream tests xz compression with configured level.
+func TestCreateTarXzStream(t *testing.T) {
+	C = Config{CompressionLevel: 6}
+
+	tmpDir := t.TempDir()
+	testFile := filepath.Join(tmpDir, "test.txt")
+	content := []byte("test content for xz")
+	if err := os.WriteFile(testFile, content, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarXzStream(ctx, []string{testFile})
+	defer reader.Close()
+
+	// Verify we can read the archive
+	_, err := io.ReadAll(reader)
+	if err != nil {
+		t.Fatalf("Failed to read xz archive: %v", err)
+	}
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/xz.go

@@ -10,6 +10,8 @@ import (
 // CreateTarXzStream takes a list of files and returns a reader that reads from an xz-compressed tar archive.
 func CreateTarXzStream(ctx context.Context, filepaths []string) io.ReadCloser {
 	return createArchiveStream(ctx, filepaths, func(w io.Writer) (io.WriteCloser, error) {
+		// xz doesn't have a simple level parameter; use default compression.
+		// The compression level affects block size indirectly through the algorithm.
 		return xz.NewWriter(w)
 	})
 }

internal/log/main.go

@@ -2,26 +2,56 @@
 package log
 
 import (
+	"context"
 	"fmt"
+	"log"
 	"log/slog"
 	"os"
+	"strings"
+	"sync"
 	"time"
 )
 
 var (
 	// Log contains *[os.File] pointing to log file descriptor.
+	// Protected by mu for thread-safe access, primarily to enable parallel tests.
+	// In production, this is set once at startup and never modified thereafter.
 	Log *os.File
+
+	// mu protects Log during initialization and when DebugLogger is called.
+	// This is necessary to allow parallel test execution where each test
+	// may initialize the logger with a different log file.
+	mu sync.RWMutex
+
+	// logLevel stores the configured log level for use by DebugLogger.
+	logLevel slog.Level
+)
+
+// slogWriter implements [io.Writer] to redirect standard log to slog.
+type (
+	slogWriter struct {
+		handler slog.Handler
+	}
 )
 
 // Init sets up logger stuff.
 // level can be error, warn, info, debug; if something other is supplied, info level is selected.
+// Thread-safe: uses mutex to allow parallel test execution.
 func Init(level, filename string) error {
+	mu.Lock()
+	defer mu.Unlock()
+
 	var (
 		err      error
 		loglevel slog.Level
 	)
 
 	if filename != "" {
+		// Close existing log file if re-initializing (e.g., in tests).
+		if Log != nil && Log != os.Stderr {
+			Log.Close()
+		}
+
 		Log, err = os.OpenFile(filename, os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0644)
 
 		if err != nil {
@@ -49,6 +79,8 @@ func Init(level, filename string) error {
 		loglevel = slog.LevelInfo
 	}
 
+	logLevel = loglevel
+
 	opts := &slog.HandlerOptions{
 		// Use the ReplaceAttr function on the handler options
 		// to be able to replace any single attribute in the log output.
@@ -122,6 +154,47 @@ func Debugf(format string, a ...any) {
 	slog.Debug(fmt.Sprintf(format, a...))
 }
 
+// DebugLogger returns a standard library logger that writes at the configured log level.
+// This is useful for assigning to http.Server.ErrorLog to route net/http errors
+// through our logger while respecting the configured log level.
+func DebugLogger() *log.Logger {
+	mu.RLock()
+	defer mu.RUnlock()
+
+	opts := &slog.HandlerOptions{
+		Level: logLevel,
+	}
+
+	handler := slog.NewTextHandler(Log, opts)
+
+	return log.New(&slogWriter{handler: handler}, "", 0)
+}
+
+// Write implements [io.Writer]. It differentiates TLS/SSL errors from other errors.
+// TLS-related errors are logged at debug level, while other errors (handler issues,
+// unexpected behavior) are logged at warn level.
+func (w *slogWriter) Write(p []byte) (int, error) {
+	msg := string(p)
+
+	isTLSError := strings.Contains(msg, "tls") ||
+		strings.Contains(msg, "ssl") ||
+		strings.Contains(msg, "TLS") ||
+		strings.Contains(msg, "SSL") ||
+		strings.Contains(msg, "handshake") ||
+		strings.Contains(msg, "certificate")
+
+	level := slog.LevelWarn
+	if isTLSError {
+		level = slog.LevelDebug
+	}
+
+	record := slog.NewRecord(time.Now(), level, msg, 0)
+
+	_ = w.handler.Handle(context.TODO(), record)
+
+	return len(p), nil
+}
+
 // Fatal logs args at error level and quits with status code 1.
 func Fatal(args ...any) {
 	slog.Error(fmt.Sprint(args...))
@@ -138,7 +211,11 @@ func Fatalf(format string, a ...any) {
 }
 
 // Close closes log file descriptor.
+// Thread-safe: uses mutex to coordinate with Init and DebugLogger.
 func Close() {
+	mu.Lock()
+	defer mu.Unlock()
+
 	if Log == nil || Log == os.Stderr {
 		return
 	}
@@ -147,6 +224,8 @@ func Close() {
 		msg := fmt.Sprintln("Failed to close log file:", err)
 		slog.Error(msg)
 	}
+
+	Log = nil
 }
 
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/log/main_test.go

@@ -354,4 +354,35 @@ func TestCloseStderr(t *testing.T) {
 	}
 }
 
+// TestDebugLogger tests that DebugLogger returns a working *log.Logger.
+func TestDebugLogger(t *testing.T) {
+	tmpFile, err := os.CreateTemp(t.TempDir(), "test_log_*.log")
+	if err != nil {
+		t.Fatal(err)
+	}
+	logFilePath := tmpFile.Name()
+	tmpFile.Close()
+
+	if err := Init("info", logFilePath); err != nil {
+		t.Fatal(err)
+	}
+	defer Close()
+
+	logger := DebugLogger()
+	if logger == nil {
+		t.Error("DebugLogger returned nil")
+	}
+
+	logger.Print("test debug message")
+
+	content, err := os.ReadFile(logFilePath)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if !strings.Contains(string(content), "test debug message") {
+		t.Error("Expected debug message in log")
+	}
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

llms.txt

@@ -30,13 +30,13 @@ curl -u user:pass https://server:8086/archive -o backup.tar.gz
 │   │   ├── globals.go       # Global C Config variable
 │   │   ├── archive.go       # Common archive logic (createArchiveStream, writeFilesToTar)
 │   │   ├── gzip.go          # CreateTarGzStream, CreateTarPgzipStream
-│   │   ├── bzip2.go         # CreateTarBzip2Stream
+│   │   ├── bzip2.go         # CreateTarBzip2Stream, mapCompressionLevelToBzip2
-│   │   ├── zstd.go          # CreateTarZstdStream
+│   │   ├── zstd.go          # CreateTarZstdStream, zstdLevel
-│   │   ├── lz4.go           # CreateTarLz4Stream
+│   │   ├── lz4.go           # CreateTarLz4Stream, mapCompressionLevelToLz4
 │   │   ├── xz.go            # CreateTarXzStream
 │   │   ├── loadConfig.go    # HJSON/JSON config parsing with validation
 │   │   ├── newServer.go     # HTTP server, auth handler, context helpers
-│   │   ├── getFilesFromDirectories.go  # Directory walking, exclusion filtering
+│   │   ├── getFilesFromDirectories.go  # Directory walking, exclusion filtering, isExcluded
 │   │   └── *_test.go        # Test files
 │   └── log/main.go          # Logging wrapper around stdlib slog
 ├── contrib/                 # Init scripts
@@ -55,7 +55,7 @@ curl -u user:pass https://server:8086/archive -o backup.tar.gz
 |--------|---------|-------------|
 | address | 0.0.0.0 | Listen address |
 | port | 8086 | Listen port |
-| cert/key | - | TLS cert/key paths |
+| cert/key | - | TLS cert/key paths (required when nohttps=false) |
 | nohttps | false | Disable HTTPS |
 | location | /archive | API endpoint path |
 | user | - | Basic auth username (required) |
@@ -64,10 +64,23 @@ curl -u user:pass https://server:8086/archive -o backup.tar.gz
 | loglevel | info | Log verbosity (debug/info/warn/error) |
 | directories | - | Directories to backup (required) |
 | backup_timeout | 60 | Stream timeout in minutes (1-1440) |
-| compression_level | 9 | Gzip level (1-9) |
+| dir_scan_timeout | 60 | Directory scan timeout in minutes (1-1440) |
+| compression_level | 9 | Compression level (1-9) |
 | exclude_patterns | [] | Regex patterns to exclude from backup |
 | filename_prefix | backup | Prefix for backup filename in Content-Disposition header |
-| compression_algorithm | gzip | Compression algorithm (gzip, pgzip, bzip2, zstd, lz4 or xz) |
+| default_compression | gzip | Default compression algorithm (gzip, pgzip, bzip2, zstd, lz4, xz) |
+| compression_algorithm | gzip | Alias for default_compression (deprecated) |
+
+## Compression Level Mapping
+
+The `compression_level` config option (1-9) is mapped differently for each algorithm:
+
+| Algorithm | Level Mapping |
+|-----------|---------------|
+| gzip/pgzip/bzip2 | 1-9 directly maps to compression levels |
+| lz4 | 1-3 → Fastest (0), 4-6 → Default (1), 7-9 → Best (2) |
+| zstd | 1-3 → Fastest, 4-6 → Default, 7-9 → BetterCompression |
+| xz | Uses default compression (no configurable level) |
 
 ## Key Patterns
 
@@ -85,16 +98,36 @@ json.Marshal(tmp)
 json.Unmarshal(buf, &C)
 ```
 
-### Stream Cancellation
+### Context-Aware I/O
-Context-aware write and copy functions:
+Two helper functions in newServer.go handle context cancellation during streaming:
+- `writeWithContext(ctx, fn)` — checks context before calling fn
+- `copyWithContext(ctx, dst, src)` — buffered copy (32KB) with cancellation checks, computes MD5 hash
+
 ```go
 func writeWithContext(ctx context.Context, fn func() error) error
-func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) error
+func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) (int64, string, error)
 ```
 
+### HTTP Server Middleware
+Two middleware functions wrap the handler:
+- `withServerHeader` — adds "Server: backer" header to all responses
+- `withStatusLogging` — logs 404 and 405 responses
+
+```go
+Handler: withServerHeader(withStatusLogging(mux))
+```
+
+### Unmatched Route Logging
+The server wraps http.ResponseWriter to capture status codes:
+- **404 Not Found**: `"Not Found from {clientIP} to {path}"`
+- **405 Method Not Allowed**: `"Method Not Allowed from {clientIP} for {method} {path}"`
+
+Uses `responseWriterWrapper` that tracks status code via WriteHeader() method.
+
 ### Pipe Error Handling
 Uses `pipeErr` tracking to avoid noisy errors when reader closes early.
 Skips files that can't be read (logs warning, continues backup).
+- `isPipeClosedError()` checks for "io: read/write on closed pipe" errors
 
 ### Content-Disposition Filename
 Backup filenames include timestamp and configured prefix: `backup-20060102-150405.tar.gz` (extension depends on compression_algorithm).
@@ -102,18 +135,43 @@ Backup filenames include timestamp and configured prefix: `backup-20060102-15040
 ### Special File Types
 - **Device files** (character/block) are included with correct major/minor numbers.
 - **Sockets** are skipped (no entry created).
-- **Symlinks** are stored as symlinks (target path stored).
+- **Symlinks** are stored as symlinks (target path stored) - uses os.Lstat, no cycles possible.
 - **Named pipes** and other special files are included (tar header created).
+- **Hard links** are deduplicated - first occurrence stored with content, subsequent as link entries.
 - **Regular files** are compressed; other file types have no data section.
 
 ### Error Recovery
 - `writeWithContext` checks context before calling fn.
 - `copyWithContext` does buffered copy with cancellation checks.
 
+### Client IP Detection
+Checks `X-Forwarded-For` header first (for proxied requests), then falls back to `RemoteAddr`.
+
+### TLS Error Handling
+Uses `ServerWrapper` to intercept and classify errors from Serve() and ServeTLS():
+- TLS-specific errors logged at debug level to reduce log clutter
+- Uses errors.As to detect: tls.AlertError, tls.CertificateVerificationError, tls.ECHRejectionError, tls.RecordHeaderError
+
+### HTTP Routing & Compression Selection
+The server uses http.NewServeMux() for routing. Multiple routes support different compression formats:
+```go
+mux.Handle(C.Location, http.HandlerFunc(backupHandler))           // /archive
+mux.Handle(C.Location+".tar.gz", http.HandlerFunc(backupHandler))  // archive.tar.gz
+mux.Handle(C.Location+".tar.xz", http.HandlerFunc(backupHandler))   // archive.tar.xz
+mux.Handle(C.Location+".tar.bz2", http.HandlerFunc(backupHandler)) // archive.tar.bz2
+// ... etc (.tar.lz4, .tar.zst, .tar.zstd)
+```
+
+Compression algorithm determined by path extension using `path.Ext()`:
+- Full path: `/archive.tar.xz` → `.xz` → xz
+- Bare extension: `tar.xz` → `.xz` → xz
+- Default when no extension matches
+
 ### Performance Notes
 - Exclude patterns are precompiled after config load for faster matching.
 - Compression level is configurable (1-9); default 9 uses more CPU.
 - Large directory walks use `filepath.WalkDir` (efficient).
+- Context cancellation checked on each file during directory scan.
 
 ## Coding Style & Conventions
 
@@ -128,7 +186,8 @@ GetFilesFromDirectories()
 // Unexported (private): verbNoun or isNoun pattern
 writeWithContext()
 copyWithContext()
-isPipeClosedError()  // predicate helper
+isPipeClosedError()
+isExcluded()
 defaultConfigPath()
 ```
 
@@ -137,6 +196,7 @@ Struct fields use PascalCase, json tags use snake_case:
 ```go
 BackupTimeout     int `json:"backup_timeout"`
 CompressionLevel  int `json:"compression_level"`
+DirScanTimeout    int `json:"dir_scan_timeout"`
 ```
 
 ### Error Messages
@@ -149,6 +209,7 @@ CompressionLevel  int `json:"compression_level"`
 - Past tense: `"Failed to close gzip writer: %v", err`
 - Context included: `"Skipping %s: %v", filename, err`
 - Appropriate level: Warn for recoverable, Error for issues, Fatal for startup failures
+- Log levels: error, warn, info, debug
 
 ### Config Validation Pattern
 For each config option: set default first, then immediately validate:
@@ -166,7 +227,7 @@ if C.Port < 1 || C.Port > 65535 {
 
 ### Go Formatting
 - Tabs for indentation
-- Vim modeline at EOF: `/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */`
+- Vim modeline at EOF: `/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */
 
 ### Linter Suppressions (intentional)
 | Rule | Reason |
@@ -186,10 +247,11 @@ go test -v -run TestName ./...   # Run specific test
 golangci-lint run ./...          # Lint
 ```
 
-Coverage: ~69% on internal/backer (main and log packages not covered by design)
+Coverage: ~69% on internal/backer (~100% on internal/log)
 
 ### Test Conventions
 - Reset global config: `C = Config{}`
+- Reset exclude patterns: `excludePatterns = nil`
 - Use `t.TempDir()` for temporary files
 - Helper function `writeTempConfig()` for config tests
 
@@ -208,3 +270,20 @@ Version is injected at build time via ldflags:
 ```go
 var version = "dev"  // overridden by: -X main.version=$(VERSION)
 ```
+
+## Request Flow (Good Path)
+```
+Request → Method Check (GET) → Auth Check → Get Files (with timeout) → Create Archive → Stream → Complete
+```
+
+## Request Flow (Error Paths)
+| Request | Response | Logged |
+|---------|----------|--------|
+| Non-GET method | 405 Method Not Allowed | Yes (info) |
+| No auth | 401 Unauthorized | Yes (info) |
+| Wrong credentials | 401 Unauthorized | Yes (info) |
+| Unknown path | 404 Not Found | Yes (info) |
+| Dir scan timeout | 504 Request Timeout | Yes (error) |
+| Failed to get files | 500 Internal Server Error | Yes (error) |
+| Client disconnect | - | Yes (info) |
+| Backup completed | 200 OK | Yes (info) |

test_data/test_config.json

@@ -5,10 +5,10 @@
     /* By default listen on port 8086, like first intel commercially successful cpu model.                            */
     "port": 8086,
 
-    /* Path location of certificate file                                                                              */
+    /* Path location of test certificate file                                                                       */
     "cert": "../../test_data/example.crt",
 
-    /* Path location of certificate key                                                                               */
+    /* Path location of test certificate key                                                                          */
     "key": "../../test_data/example.key",
 
     /* We really want to use https for backup archive tranferring.                                                    */
@@ -24,7 +24,7 @@
     "password": "SharpShooter",
 
     /* Where to store logs.                                                                                           */
-    "log": "/var/log/backer.log",
+    "log": "../../test_data/tmp/backer.log",
 
     /* Verbosity of logs                                                                                              */
     "loglevel": "info",
@@ -39,5 +39,5 @@
     "backup_timeout": 60,
 
     /* Gzip compression level (1-9, where 9 is best compression).                                                    */
-    "compression_level": 6
+    "compression_level": 3
 }