Do not include test_data/{tmp,test1} in git commits

.gitignore

@@ -30,4 +30,6 @@ Thumbs.db
 *.test
 *.out
 
-/test_data/tmp
+# Test data (generated by tests)
+test_data/tmp
+test_data/test1

AGENTS.md

@@ -27,6 +27,36 @@ Backer is a Go-based HTTP/HTTPS backup server that creates compressed archives o
 - **Don't use `context` package unless absolutely necessary.** Try to find another solution first.
 - **Create functional and unit tests if possible/necessary** to verify changes that software code works at is should.
 
+### Test Rules
+
+- **Test data must be self-contained and created at test setup.** All test data should be created in `TestMain` in `internal/backer/main_test.go` before any tests run, not rely on pre-existing files in the repository. Use `test_data/test1/` as the test data directory.
+
+- **Use `test_data/tmp/` for temporary files.** When tests need temporary storage, use this directory (not system `/tmp`) to keep test data isolated and predictable.
+
+- **Always check errors with meaningful diagnostics.** Never discard errors with `_ =`. Use patterns like:
+  ```go
+  if err := os.MkdirAll(dir, 0755); err != nil {
+      t.Fatalf("Failed to create directory %s: %v", dir, err)
+  }
+  ```
+
+- **Tests must not depend on external state.** Test data should be created fresh for each test run and cleaned up afterward. Use `TestMain` with setup/cleanup functions.
+
+- **Use `t.Fatal` or `t.Fatalf` for setup errors.** When test setup fails, fail fast with clear diagnostic messages showing what failed and why.
+
+- **Reset global state between tests.** For tests that modify global variables (like `C` config), save the original value and restore it in a `defer`:
+  ```go
+  original := C
+  defer func() { C = original }()
+  ```
+
+- **Skip tests gracefully when features are unavailable.** For platform-specific features (hardlinks, symlinks), use `t.Skip()` when the feature is not supported:
+  ```go
+  if err := os.Link(orig, link); err != nil {
+      t.Skip("Hard links not supported, skipping test")
+  }
+  ```
+
 ## Build & Test Commands
 
 ```bash
@@ -207,6 +237,7 @@ Config is JSON (HJSON with comments supported). Default path depends on OS:
 | loglevel | `info` | Log verbosity: error/warn/info/debug |
 | directories | — | Directories to backup (required, validated for existence) |
 | backup_timeout | `60` | Stream timeout in minutes (1-1440) |
+| dir_scan_timeout | `60` | Directory scan timeout in minutes (1-1440) |
 | compression_level | `9` | Gzip level (1-9) |
 | exclude_patterns | `[]` | Regex patterns to exclude from backup |
 | filename_prefix | `backup` | Prefix for backup filename in Content-Disposition header |
@@ -275,13 +306,16 @@ Checks `X-Forwarded-For` header first (for proxied requests), then falls back to
 
 ### TLS Error Handling
 
-The server uses a `ServerWrapper` type to intercept and classify errors from `Serve()` and `ServeTLS()`. TLS-specific errors are logged at debug level to reduce log clutter:
+The server uses a `ServerWrapper` type to intercept and classify errors from `Serve()` and `ServeTLS()`. Two mechanisms:
 
-- Uses `errors.As` to detect TLS error types: `tls.AlertError`, `tls.CertificateVerificationError`, `tls.ECHRejectionError`, `tls.RecordHeaderError`
-- Also checks for "tls:" prefix in error messages
-- Additionally uses `http.Server.ErrorLog` with a custom logger that writes at debug level for net/http's internal TLS-related messages
+1. **Server errors** (`Serve()` / `ServeTLS()`): Uses `errors.As` to detect TLS error types (`tls.AlertError`, `tls.CertificateVerificationError`, `tls.ECHRejectionError`, `tls.RecordHeaderError`) and checks for "tls:" prefix. TLS-specific errors logged at debug level, others at warn level.
 
-This ensures client-side TLS errors (handshake failures, certificate issues) don't clutter production logs.
+2. **net/http internal errors** (`http.Server.ErrorLog`): Uses `DebugLogger()` which routes messages through `slogWriter.Write()`:
+   - TLS-related errors (keywords: "tls", "ssl", "handshake", "certificate") → logged at debug level
+   - Other errors → logged at warn level
+   - Both are filtered by configured `loglevel`. Set `loglevel: debug` to see all messages.
+
+This ensures client-side TLS errors (handshake failures, certificate issues) don't clutter production logs by default.
 
 ### Compression Level Mapping
 
@@ -302,7 +336,6 @@ The server uses `http.NewServeMux()` for routing. Multiple routes are registered
 
 ```go
 mux.Handle(C.Location, http.HandlerFunc(backupHandler))           // /archive
-mux.Handle(C.Location+"/", http.HandlerFunc(backupHandler))       // /archive/
 mux.Handle(C.Location+".tar.gz", http.HandlerFunc(backupHandler))  // /archive.tar.gz
 mux.Handle(C.Location+".tar.xz", http.HandlerFunc(backupHandler))   // /archive.tar.xz
 mux.Handle(C.Location+".tar.bz2", http.HandlerFunc(backupHandler)) // /archive.tar.bz2
@@ -318,6 +351,14 @@ The `getCompressionAlgorithm()` function handles both full paths and bare extens
 - Bare extension: `tar.xz` → `.tar.xz` → xz
 - Single ext: `xz` → `.xz` → xz
 
+### Unmatched Route Logging
+
+The server wraps the `http.ResponseWriter` to capture status codes and log unmatched routes:
+- **404 Not Found**: Logged as `"Not Found from {clientIP} to {path}"`
+- **405 Method Not Allowed**: Logged as `"Method Not Allowed from {clientIP} for {method} {path}"`
+
+This uses a `responseWriterWrapper` that tracks the status code via the `WriteHeader()` method.
+
 ## Dependencies
 
 - `github.com/hjson/hjson-go` v3.3.0 — HJSON config parsing
@@ -348,10 +389,14 @@ Tests live alongside source files as `*_test.go` (package name, not `_test` suff
 
 ### Test Data
 
-- `test_data/test_config.json` — valid test config (user: JohnnyGoode, password: SharpShooter)
-- `test_data/test1/foo/` — test directory with files, subdirs, empty dirs
+All test data is created by `TestMain` in `internal/backer/main_test.go` before tests run and cleaned up afterward. This ensures tests are independent of pre-existing files.
+
+- `test_data/test1/foo/` — test directory with files, subdirs, empty_dir
 - `test_data/test1/bar/` — test directory with files and a directory named `goodbye.txt/` (edge case: directory with file extension)
-- `test_data/example.crt` / `example.key` — TLS test certificates
+- `test_data/test1/hardlinks/` — test directory with hard-linked files (original.txt, hardlink1.txt, hardlink2.txt)
+- `test_data/test1/symlinks/` — test directory with symbolic links
+- `test_data/test1/empty_dir/` — empty directory for testing
+- `test_data/tmp/` — reserved for test temporary files (use instead of system `/tmp`)
 
 ## Service Deployment
 

Makefile

@@ -23,13 +23,15 @@ build:
 
 clean:
 	rm -f $(BINARY)
+	rm -rf test_data/tmp
+	rm -rf test_data/test1
 
 upgrade:
 	go get -u ./...
 	go mod tidy
 	go mod vendor
 
-test:
+test: clean
 	go test ./...
 
 help:

README.md

@@ -65,6 +65,7 @@ All options described in example config.
 | loglevel | "info" | Verbosity of logs. | Options: error, warn, info, debug. |
 | directories | — | Directories to backup. | Required. Array of paths. |
 | backup_timeout | 60 | Timeout in minutes for backup streaming. | Range: 1-1440 minutes. |
+| dir_scan_timeout | 60 | Timeout in minutes for getting file list from all backupped directories. | Range: 1-1440 minutes. |
 | compression_level | 9 | Compression level. | 1 (fastest) to 9 (best compression). |
 | exclude_patterns | [] | Regex patterns to exclude from backup. | E.g., `[".*\\.tmp$", "/node_modules/"]`. |
 | filename_prefix | "backup" | Prefix for backup filename. | E.g., "mybackup" produces `mybackup-20260325-092341.tar.gz`. |

data/config_example.json

@@ -47,6 +47,9 @@
     /* Regex patterns to exclude from backup.                                                                         */
     "exclude_patterns": [],
 
+    /* Maximum allowed time, spent for getting file list from backupped directories.                                  */
+    "dir_scan_timeout": 60,
+
     /* Directories to backup.                                                                                         */
     "directories": [
         "/path/to/directory1",

internal/backer/archive.go

@@ -9,6 +9,7 @@ import (
 	"os"
 	"path/filepath"
 	"strings"
+	"syscall"
 
 	"backer/internal/log"
 )
@@ -85,6 +86,8 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 		var (
 			err        error
 			linkTarget string
+			isSymlink  bool
+			isHardLink bool
 		)
 
 		select {
@@ -106,6 +109,7 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 			continue
 		}
 
+		// Handle symlinks - read target path and mark as symbolic link.
 		if st.Mode()&os.ModeSymlink != 0 {
 			linkTarget, err = os.Readlink(fpath)
 
@@ -114,6 +118,8 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 
 				continue
 			}
+
+			isSymlink = true
 		}
 
 		// Skip sockets explicitly (they cannot be backed up).
@@ -124,12 +130,15 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 		}
 
 		// Check for hard link - if inode was already seen, store as link instead of content.
-		if st.Mode().IsRegular() {
+		// Only check for regular files (not symlinks).
+		if st.Mode().IsRegular() && !isSymlink {
 			inode := getInode(st)
 			if inode != 0 {
 				if existingPath, seen := seenInodes[inode]; seen {
 					// This is a hard link to already-archived file.
 					linkTarget = existingPath
+					isHardLink = true
+
 					log.Debugf("Hard link detected: %s -> %s", fpath, existingPath)
 				} else {
 					// First time seeing this inode - track it.
@@ -146,9 +155,13 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 			continue
 		}
 
-		// If linkTarget is set, ensure the header is marked as a link.
-		if linkTarget != "" {
+		// Set correct type flag for symbolic links or hard links.
+		if isSymlink {
+			header.Typeflag = tar.TypeSymlink
+			header.Linkname = linkTarget
+		} else if isHardLink {
 			header.Typeflag = tar.TypeLink
+			header.Linkname = linkTarget
 		}
 
 		header.Format = tar.FormatGNU
@@ -164,8 +177,8 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 			return
 		}
 
-		// Only stream file contents for regular files.
-		if mode.IsRegular() {
+		// Only stream file contents for regular files that are not hard links.
+		if mode.IsRegular() && linkTarget == "" {
 			f, err := os.Open(fpath)
 			if err != nil {
 				log.Warnf("Skipping %s: open failed: %v", fpath, err)
@@ -195,15 +208,12 @@ func getInode(fi os.FileInfo) uint64 {
 		return 0
 	}
 
-	// Use type assertion to get inode - works on Unix, returns 0 on Windows.
-	type inodeGetter interface {
-		Dev() uint64
-		Ino() uint64
-	}
-
-	if ig, ok := fi.Sys().(inodeGetter); ok {
-		// Combine device and inode for uniqueness across file systems.
-		return (ig.Dev() << 32) | ig.Ino()
+	// Use reflection to get Dev and Ino fields from the syscall.Stat_t struct.
+	st := fi.Sys()
+	// Try type assertion for the struct with fields.
+	switch v := st.(type) {
+	case *syscall.Stat_t:
+		return (v.Dev << 32) | v.Ino
 	}
 
 	return 0

internal/backer/getFilesFromDirectories.go

@@ -6,7 +6,6 @@ import (
 	"io/fs"
 	"os"
 	"path/filepath"
-	"time"
 
 	backerlog "backer/internal/log"
 )
@@ -81,13 +80,4 @@ func GetFilesFromDirectories(ctx context.Context, directories []string) ([]strin
 	return files, nil
 }
 
-// GetFilesFromDirectoriesWithTimeout makes a file list of given directories with timeout.
-// Convenience wrapper that creates context from timeout in minutes.
-func GetFilesFromDirectoriesWithTimeout(timeoutMinutes int, directories []string) ([]string, error) {
-	ctx, cancel := context.WithTimeout(context.Background(), time.Duration(timeoutMinutes)*time.Minute)
-	defer cancel()
-
-	return GetFilesFromDirectories(ctx, directories)
-}
-
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/getFilesFromDirectories_test.go

@@ -15,9 +15,18 @@ func normalizePath(p string) string {
 	return strings.ReplaceAll(p, "\\", "/")
 }
 
+// normalizePathSlice normalizes all paths in a slice.
+func normalizePathSlice(paths []string) []string {
+	result := make([]string, len(paths))
+	for i, p := range paths {
+		result[i] = normalizePath(p)
+	}
+	return result
+}
+
 // TestGetFilesFromDirectories checks if GetFilesFromDirectories returns the proper amount of strings, correct strings and no error.
 func TestGetFilesFromDirectories(t *testing.T) {
-	testDataPath := filepath.Join("..", "test_data")
+	testDataPath := "../../test_data"
 
 	// Log the paths we're checking for debugging
 	t.Logf("Checking directories: %v", []string{
@@ -61,7 +70,8 @@ func TestGetFilesFromDirectories(t *testing.T) {
 
 	for _, filepath := range output {
 		normalized := normalizePath(filepath)
-		if !slices.Contains(expectedData, normalized) {
+		normalizedExpected := normalizePathSlice(expectedData)
+		if !slices.Contains(normalizedExpected, normalized) {
 			t.Errorf("expectedData data does not contain string from output: %s", filepath)
 		}
 	}
@@ -69,7 +79,7 @@ func TestGetFilesFromDirectories(t *testing.T) {
 
 // TestGetFilesFromDirectoriesSingleDir tests with a single directory.
 func TestGetFilesFromDirectoriesSingleDir(t *testing.T) {
-	testDataPath := filepath.Join("test_data")
+	testDataPath := filepath.Join("../../test_data")
 	output, err := GetFilesFromDirectories(context.Background(), []string{filepath.Join(testDataPath, "test1", "foo")})
 
 	if err != nil {
@@ -106,7 +116,7 @@ func TestGetFilesFromDirectoriesEmptyDir(t *testing.T) {
 // TestGetFilesFromDirectoriesNonExistent tests with non-existent directory.
 // Should log error but not fail - returns empty list.
 func TestGetFilesFromDirectoriesNonExistent(t *testing.T) {
-	testDataPath := filepath.Join("test_data")
+	testDataPath := filepath.Join("../../test_data")
 	nonExistentPath := filepath.Join(testDataPath, "nonexistent", "path", "that", "does", "not", "exist")
 	output, err := GetFilesFromDirectories(context.Background(), []string{nonExistentPath})
 
@@ -123,7 +133,7 @@ func TestGetFilesFromDirectoriesNonExistent(t *testing.T) {
 
 // TestGetFilesFromDirectoriesPartialFailure tests with mix of valid and invalid directories.
 func TestGetFilesFromDirectoriesPartialFailure(t *testing.T) {
-	testDataPath := filepath.Join("test_data")
+	testDataPath := filepath.Join("../../test_data")
 	// Create a temp directory that exists
 	tmpDir := t.TempDir()
 
@@ -142,7 +152,7 @@ func TestGetFilesFromDirectoriesPartialFailure(t *testing.T) {
 
 // TestGetFilesFromDirectoriesMultipleDirs tests multiple directories.
 func TestGetFilesFromDirectoriesMultipleDirs(t *testing.T) {
-	testDataPath := filepath.Join("test_data")
+	testDataPath := filepath.Join("../../test_data")
 	output, err := GetFilesFromDirectories(context.Background(), []string{
 		filepath.Join(testDataPath, "test1", "foo"),
 		filepath.Join(testDataPath, "test1", "bar"),

internal/backer/loadConfig_test.go

@@ -13,8 +13,12 @@ func TestLoadConfig(t *testing.T) {
 	C = Config{}
 
 	testDataPath := filepath.Join(t.TempDir(), "test_data")
-	os.MkdirAll(filepath.Join(testDataPath, "test1", "foo"), 0755)
-	os.MkdirAll(filepath.Join(testDataPath, "test1", "bar"), 0755)
+	if err := os.MkdirAll(filepath.Join(testDataPath, "test1", "foo"), 0755); err != nil {
+		t.Fatalf("Failed to create test directory foo: %v", err)
+	}
+	if err := os.MkdirAll(filepath.Join(testDataPath, "test1", "bar"), 0755); err != nil {
+		t.Fatalf("Failed to create test directory bar: %v", err)
+	}
 
 	// Create test config file
 	testConfigPath := filepath.Join(testDataPath, "test_config.json")

internal/backer/main_test.go

@@ -0,0 +1,103 @@
+package backer
+
+import (
+	"context"
+	"fmt"
+	"os"
+	"path/filepath"
+	"testing"
+)
+
+const testDataPath = "../../test_data"
+
+func TestMain(m *testing.M) {
+	if err := setupTestData(); err != nil {
+		fmt.Fprintf(os.Stderr, "FATAL: Failed to setup test data: %v\n", err)
+		os.Exit(1)
+	}
+
+	code := m.Run()
+
+	if err := cleanupTestData(); err != nil {
+		fmt.Fprintf(os.Stderr, "WARNING: Failed to cleanup test data: %v\n", err)
+	}
+
+	os.Exit(code)
+}
+
+func setupTestData() error {
+	testDir := filepath.Join(testDataPath, "test1")
+
+	dirs := []string{
+		filepath.Join(testDir, "foo", "mydir"),
+		filepath.Join(testDir, "bar", "goodbye.txt"),
+		filepath.Join(testDir, "hardlinks"),
+		filepath.Join(testDir, "symlinks"),
+		filepath.Join(testDir, "foo", "empty_dir"),
+		filepath.Join(testDataPath, "tmp"),
+	}
+
+	for _, dir := range dirs {
+		if err := os.MkdirAll(dir, 0755); err != nil {
+			return fmt.Errorf("Failed to create directory %s: %w", dir, err)
+		}
+	}
+
+	files := map[string]string{
+		filepath.Join(testDir, "foo", "hello_breakfast.txt"):   "hello breakfast content",
+		filepath.Join(testDir, "foo", "some_text.txt"):         "some text content",
+		filepath.Join(testDir, "foo", "mydir", "myfile.txt"):   "my file content",
+		filepath.Join(testDir, "bar", "hello.txt"):             "hello content",
+		filepath.Join(testDir, "bar", "goodbye.txt", "text.txt"): "text content",
+		filepath.Join(testDir, "hardlinks", "original.txt"):    "original content",
+		filepath.Join(testDir, "symlinks", "target.txt"):       "target content",
+	}
+
+	for path, content := range files {
+		if err := os.WriteFile(path, []byte(content), 0644); err != nil {
+			return fmt.Errorf("Failed to write file %s: %w", path, err)
+		}
+	}
+
+	orig := filepath.Join(testDir, "hardlinks", "original.txt")
+	link1 := filepath.Join(testDir, "hardlinks", "hardlink1.txt")
+	if err := os.Link(orig, link1); err != nil {
+		return fmt.Errorf("Failed to create hardlink %s -> %s: %w", link1, orig, err)
+	}
+
+	link2 := filepath.Join(testDir, "hardlinks", "hardlink2.txt")
+	if err := os.Link(orig, link2); err != nil {
+		return fmt.Errorf("Failed to create hardlink %s -> %s: %w", link2, orig, err)
+	}
+
+	target := filepath.Join(testDir, "symlinks", "target.txt")
+	link := filepath.Join(testDir, "symlinks", "link.txt")
+	if err := os.Symlink(target, link); err != nil {
+		return fmt.Errorf("Failed to create symlink %s -> %s: %w", link, target, err)
+	}
+
+	return nil
+}
+
+func cleanupTestData() error {
+	testDir := filepath.Join(testDataPath, "test1")
+
+	if err := os.RemoveAll(testDir); err != nil {
+		return fmt.Errorf("Failed to remove test directory %s: %w", testDir, err)
+	}
+
+	tmpDir := filepath.Join(testDataPath, "tmp")
+	if err := os.RemoveAll(tmpDir); err != nil {
+		return fmt.Errorf("Failed to remove tmp directory %s: %w", tmpDir, err)
+	}
+
+	return nil
+}
+
+func GetTestDataPath() string {
+	return testDataPath
+}
+
+func GetTestFilesFromDirectories(ctx context.Context, directories []string) ([]string, error) {
+	return GetFilesFromDirectories(ctx, directories)
+}

internal/backer/newServer.go

@@ -48,6 +48,23 @@ func isTLSError(err error) bool {
 	return strings.HasPrefix(err.Error(), "tls:")
 }
 
+// withStatusLogging wraps a handler to log 404 and 405 responses.
+func withStatusLogging(next http.Handler) http.Handler {
+	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		rw := &responseWriterWrapper{ResponseWriter: w, statusCode: http.StatusOK}
+		next.ServeHTTP(rw, r)
+
+		clientIP := getClientIP(r)
+
+		switch rw.statusCode {
+		case http.StatusNotFound:
+			log.Infof("Not Found from %s to %s", clientIP, r.RequestURI)
+		case http.StatusMethodNotAllowed:
+			log.Infof("Method Not Allowed from %s for %s %s", clientIP, r.Method, r.RequestURI)
+		}
+	})
+}
+
 // withServerHeader wraps a handler to add Server header to all responses.
 func withServerHeader(next http.Handler) http.Handler {
 	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -74,6 +91,13 @@ func NewServer(configPath string) (*ServerWrapper, error) {
 	mux := http.NewServeMux()
 
 	backupHandler := func(w http.ResponseWriter, r *http.Request) {
+		// Only GET method is allowed.
+		if r.Method != http.MethodGet {
+			http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
+
+			return
+		}
+
 		username, password, ok := r.BasicAuth()
 
 		// No auth? Good bye.
@@ -219,7 +243,7 @@ func NewServer(configPath string) (*ServerWrapper, error) {
 		srv := &ServerWrapper{
 			Server: &http.Server{
 				Addr:              fmt.Sprintf("%s:%d", C.Address, C.Port),
-				Handler:           withServerHeader(mux),
+				Handler:           withServerHeader(withStatusLogging(mux)),
 				ReadHeaderTimeout: readHeaderTimeout,
 				WriteTimeout:      time.Duration(C.BackupTimeout) * time.Minute,
 				IdleTimeout:       idleTimeout,
@@ -235,7 +259,7 @@ func NewServer(configPath string) (*ServerWrapper, error) {
 	srv := &ServerWrapper{
 		Server: &http.Server{
 			Addr:              fmt.Sprintf("%s:%d", C.Address, C.Port),
-			Handler:           withServerHeader(mux),
+			Handler:           withServerHeader(withStatusLogging(mux)),
 			ReadHeaderTimeout: readHeaderTimeout,
 			WriteTimeout:      time.Duration(C.BackupTimeout) * time.Minute,
 			IdleTimeout:       idleTimeout,
@@ -374,8 +398,20 @@ type (
 	ServerWrapper struct {
 		*http.Server
 	}
+
+	// responseWriterWrapper wraps [http.ResponseWriter] to capture status code.
+	responseWriterWrapper struct {
+		http.ResponseWriter
+		statusCode int
+	}
 )
 
+// WriteHeader captures the status code for logging.
+func (rw *responseWriterWrapper) WriteHeader(code int) {
+	rw.statusCode = code
+	rw.ResponseWriter.WriteHeader(code)
+}
+
 // Serve starts the server and classifies any errors returned.
 // TLS-related errors are logged at debug level, other errors at warn level.
 func (sw *ServerWrapper) Serve() error {

internal/backer/newServer_test.go

@@ -125,46 +125,43 @@ func TestGetClientIP(t *testing.T) {
 
 // TestGetCompressionAlgorithm tests the getCompressionAlgorithm function.
 func TestGetCompressionAlgorithm(t *testing.T) {
+	// Save original config
+	original := C
+	defer func() { C = original }()
+
 	tests := []struct {
-		name     string
-		ext      string
-		expected string
+		name              string
+		ext               string
+		expected          string
+		defaultCompression string
 	}{
-		{
-			name:     "tar.bz2",
-			ext:      "tar.bz2",
-			expected: "bzip2",
-		},
-		{
-			name:     "tar.zst",
-			ext:      "tar.zst",
-			expected: "zstd",
-		},
-		{
-			name:     "tar.lz4",
-			ext:      "tar.lz4",
-			expected: "lz4",
-		},
-		{
-			name:     "tar.xz",
-			ext:      "tar.xz",
-			expected: "xz",
-		},
-		{
-			name:     "tar.gz",
-			ext:      "tar.gz",
-			expected: "pgzip",
-		},
-		{
-			name:     "default with gzip config",
-			ext:      "",
-			expected: "gzip",
-		},
+		{name: "tar.bz2", ext: "tar.bz2", expected: "bzip2"},
+		{name: "tar.zstd", ext: "tar.zstd", expected: "zstd"},
+		{name: "tar.zst", ext: "tar.zst", expected: "zstd"},
+		{name: "tar.lz4", ext: "tar.lz4", expected: "lz4"},
+		{name: "tar.xz", ext: "tar.xz", expected: "xz"},
+		{name: "tar.gz", ext: "tar.gz", expected: "pgzip"},
+		{name: "bz2", ext: "bz2", expected: "bzip2"},
+		{name: "zst", ext: "zst", expected: "zstd"},
+		{name: "lz4", ext: "lz4", expected: "lz4"},
+		{name: "xz", ext: "xz", expected: "xz"},
+		{name: "gz", ext: "gz", expected: "pgzip"},
+		{name: "zstd", ext: "zstd", expected: "zstd"},
+		{name: "full path /archive.tar.gz", ext: "/archive.tar.gz", expected: "pgzip"},
+		{name: "full path /archive.tar.xz", ext: "/archive.tar.xz", expected: "xz"},
+		{name: "full path /archive", ext: "/archive", expected: "gzip"},
+		{name: "empty uses default", ext: "", expected: "gzip"},
+		{name: "unknown uses default", ext: "unknown", expected: "gzip"},
+		{name: "unknown with default zstd", ext: "unknown", expected: "zstd", defaultCompression: "zstd"},
 	}
 
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
-			C = Config{DefaultCompression: "gzip"}
+			if tt.defaultCompression != "" {
+				C = Config{DefaultCompression: tt.defaultCompression}
+			} else {
+				C = Config{DefaultCompression: "gzip"}
+			}
 			result := getCompressionAlgorithm(tt.ext)
 			if result != tt.expected {
 				t.Errorf("getCompressionAlgorithm(%q) = %q, expected %q", tt.ext, result, tt.expected)
@@ -191,6 +188,46 @@ func TestNewServerSuccess(t *testing.T) {
 	}
 }
 
+// TestWithServerHeader tests that the Server header is added to responses.
+func TestWithServerHeader(t *testing.T) {
+	handler := withServerHeader(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusOK)
+	}))
+
+	rec := &responseWriterWrapper{
+		ResponseWriter: &testResponseWriter{},
+		statusCode:     0,
+	}
+
+	req, _ := http.NewRequest("GET", "/test", nil)
+	handler.ServeHTTP(rec, req)
+
+	if rec.Header().Get("Server") != "backer" {
+		t.Error("Expected Server header to be set to 'backer'")
+	}
+}
+
+// testResponseWriter implements http.ResponseWriter for testing.
+type testResponseWriter struct {
+	header http.Header
+	status int
+}
+
+func (tw *testResponseWriter) Header() http.Header {
+	if tw.header == nil {
+		tw.header = make(http.Header)
+	}
+	return tw.header
+}
+
+func (tw *testResponseWriter) Write(data []byte) (int, error) {
+	return len(data), nil
+}
+
+func (tw *testResponseWriter) WriteHeader(status int) {
+	tw.status = status
+}
+
 // TestNewServerInvalidConfig tests server creation with invalid config.
 func TestNewServerInvalidConfig(t *testing.T) {
 	C = Config{} // Reset global config.
@@ -523,4 +560,125 @@ func TestCopyWithContextCancellation(t *testing.T) {
 	}
 }
 
+// TestStatusLogging404 tests that 404 responses are logged.
+func TestStatusLogging404(t *testing.T) {
+	C = Config{} // Reset global config.
+	excludePatterns = nil
+
+	// Create a minimal mux that returns 404 for all paths
+	testMux := http.NewServeMux()
+	testMux.HandleFunc("/archive", func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusOK)
+	})
+
+	wrappedMux := withStatusLogging(testMux)
+
+	// Use a ResponseRecorder to capture the response
+	rr := httptest.NewRecorder()
+	req, _ := http.NewRequest("GET", "/nonexistent", nil)
+	req.RemoteAddr = "127.0.0.1:12345"
+
+	wrappedMux.ServeHTTP(rr, req)
+
+	if rr.Code != http.StatusNotFound {
+		t.Errorf("Expected status 404, got %d", rr.Code)
+	}
+}
+
+// TestStatusLogging405 tests that 405 responses are logged.
+func TestStatusLogging405(t *testing.T) {
+	C = Config{} // Reset global config.
+	excludePatterns = nil
+
+	// Create a minimal mux that returns 405 for wrong method
+	testMux := http.NewServeMux()
+	testMux.HandleFunc("/archive", func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusMethodNotAllowed)
+	})
+
+	wrappedMux := withStatusLogging(testMux)
+
+	rr := httptest.NewRecorder()
+	req, _ := http.NewRequest("GET", "/archive", nil)
+	req.RemoteAddr = "127.0.0.1:12345"
+
+	wrappedMux.ServeHTTP(rr, req)
+
+	if rr.Code != http.StatusMethodNotAllowed {
+		t.Errorf("Expected status 405, got %d", rr.Code)
+	}
+}
+
+// TestMethodNotAllowedHandler tests that non-GET methods return 405.
+func TestMethodNotAllowedHandler(t *testing.T) {
+	C = Config{
+		Address:     "0.0.0.0",
+		Port:        8086,
+		Location:    "/archive",
+		User:        "testuser",
+		Password:    "testpass",
+		Directories: []string{"../../test_data/test1/foo"},
+	}
+	excludePatterns = nil
+
+	// Create test server
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		username, password, ok := r.BasicAuth()
+		if !ok || username != C.User || password != C.Password {
+			http.Error(w, "Unauthorized", http.StatusUnauthorized)
+
+			return
+		}
+
+		if r.Method != http.MethodGet {
+			w.Header().Set("Server", "backer")
+			http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
+
+			return
+		}
+
+		w.WriteHeader(http.StatusOK)
+	}))
+	defer server.Close()
+
+	// Test POST method
+	req, _ := http.NewRequest("POST", server.URL+"/archive", nil)
+	req.SetBasicAuth("testuser", "testpass")
+	resp, err := http.DefaultClient.Do(req)
+	if err != nil {
+		t.Fatalf("Request failed: %v", err)
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != http.StatusMethodNotAllowed {
+		t.Errorf("Expected status 405, got %d", resp.StatusCode)
+	}
+}
+
+// TestTimingSafeMatch tests the timingSafeMatch function for constant-time comparison.
+func TestTimingSafeMatch(t *testing.T) {
+	tests := []struct {
+		name     string
+		a        string
+		b        string
+		expected bool
+	}{
+		{"empty strings", "", "", true},
+		{"identical strings", "test", "test", true},
+		{"different strings", "test", "Test", false},
+		{"different length", "test", "testing", false},
+		{"first empty", "", "test", false},
+		{"second empty", "test", "", false},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			result := timingSafeMatch(tt.a, tt.b)
+			if result != tt.expected {
+				t.Errorf("timingSafeMatch(%q, %q) = %v, expected %v", tt.a, tt.b, result, tt.expected)
+			}
+		})
+	}
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/types_test.go

@@ -0,0 +1,17 @@
+package backer
+
+import (
+	"testing"
+)
+
+func TestErrDirectoryScanTimeout(t *testing.T) {
+	err := ErrDirectoryScanTimeout{ScannedFiles: 42}
+
+	if err.Error() != "Directory scan timed out" {
+		t.Errorf("Expected error message 'Directory scan timed out', got %q", err.Error())
+	}
+
+	if err.ScannedFiles != 42 {
+		t.Errorf("Expected ScannedFiles=42, got %d", err.ScannedFiles)
+	}
+}

internal/backer/utils_test.go

@@ -9,6 +9,8 @@ import (
 	"path/filepath"
 	"runtime"
 	"testing"
+
+	"github.com/klauspost/compress/zstd"
 )
 
 // TestCreateTarGzStreamBasic tests basic tar.gz creation with a single file.
@@ -247,6 +249,59 @@ func TestCreateTarGzStreamHardLink(t *testing.T) {
 	}
 }
 
+// TestCreateTarGzStreamHardLinkFromTestData tests hard link handling with test data.
+func TestCreateTarGzStreamHardLinkFromTestData(t *testing.T) {
+	if runtime.GOOS == "windows" {
+		t.Skip("Hard link detection not supported on Windows (inodes unavailable)")
+	}
+
+	testDataPath := filepath.Join("../../test_data", "test1", "hardlinks")
+	filepaths, err := GetFilesFromDirectories(context.Background(), []string{testDataPath})
+	if err != nil {
+		t.Fatalf("Failed to get files from directory: %v", err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarGzStream(ctx, filepaths)
+	defer reader.Close()
+
+	gzReader, err := gzip.NewReader(reader)
+	if err != nil {
+		t.Fatalf("Failed to create gzip reader: %v", err)
+	}
+	defer gzReader.Close()
+
+	tarReader := tar.NewReader(gzReader)
+
+	entries := 0
+	linkCount := 0
+
+	for {
+		header, err := tarReader.Next()
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			t.Fatalf("Failed to read tar entry: %v", err)
+		}
+
+		entries++
+		t.Logf("Entry %d: name=%s typeflag=%c linkname=%s", entries, header.Name, header.Typeflag, header.Linkname)
+
+		if header.Typeflag == tar.TypeLink {
+			linkCount++
+		}
+	}
+
+	if entries != 4 {
+		t.Errorf("Expected 4 entries (dir + 1 original + 2 hardlinks), got %d", entries)
+	}
+
+	if linkCount != 2 {
+		t.Errorf("Expected 2 hard link entries, got %d", linkCount)
+	}
+}
+
 // TestCreateTarGzStreamCancellation tests context cancellation.
 func TestCreateTarGzStreamCancellation(t *testing.T) {
 	tmpDir := t.TempDir()
@@ -564,4 +619,115 @@ func TestCreateTarXzStream(t *testing.T) {
 	}
 }
 
+// TestCreateTarPgzipStream tests pgzip (parallel gzip) compression.
+func TestCreateTarPgzipStream(t *testing.T) {
+	C = Config{CompressionLevel: 6}
+
+	tmpDir := t.TempDir()
+	testFile := filepath.Join(tmpDir, "test.txt")
+	content := []byte("test content for pgzip")
+	if err := os.WriteFile(testFile, content, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarPgzipStream(ctx, []string{testFile})
+	defer reader.Close()
+
+	gzReader, err := gzip.NewReader(reader)
+	if err != nil {
+		t.Fatalf("Failed to create gzip reader: %v", err)
+	}
+	defer gzReader.Close()
+
+	tarReader := tar.NewReader(gzReader)
+
+	header, err := tarReader.Next()
+	if err != nil {
+		t.Fatalf("Failed to read tar header: %v", err)
+	}
+
+	if header.Name != testFile && filepath.Base(header.Name) != "test.txt" {
+		t.Errorf("Expected file name 'test.txt', got '%s'", header.Name)
+	}
+
+	data, err := io.ReadAll(tarReader)
+	if err != nil {
+		t.Fatalf("Failed to read file content: %v", err)
+	}
+
+	if string(data) != string(content) {
+		t.Errorf("Expected content '%s', got '%s'", content, data)
+	}
+}
+
+// TestCreateTarZstdStream tests zstd compression.
+func TestCreateTarZstdStream(t *testing.T) {
+	C = Config{CompressionLevel: 6}
+
+	tmpDir := t.TempDir()
+	testFile := filepath.Join(tmpDir, "test.txt")
+	content := []byte("test content for zstd")
+	if err := os.WriteFile(testFile, content, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarZstdStream(ctx, []string{testFile})
+	defer reader.Close()
+
+	// zstd.Reader is in github.com/klauspost/compress/zstd
+	zstdReader, err := zstd.NewReader(reader)
+	if err != nil {
+		t.Fatalf("Failed to create zstd reader: %v", err)
+	}
+	defer zstdReader.Close()
+
+	tarReader := tar.NewReader(zstdReader)
+
+	header, err := tarReader.Next()
+	if err != nil {
+		t.Fatalf("Failed to read tar header: %v", err)
+	}
+
+	if header.Name != testFile && filepath.Base(header.Name) != "test.txt" {
+		t.Errorf("Expected file name 'test.txt', got '%s'", header.Name)
+	}
+
+	data, err := io.ReadAll(tarReader)
+	if err != nil {
+		t.Fatalf("Failed to read file content: %v", err)
+	}
+
+	if string(data) != string(content) {
+		t.Errorf("Expected content '%s', got '%s'", content, data)
+	}
+}
+
+// TestZstdLevel tests zstd level mapping.
+func TestZstdLevel(t *testing.T) {
+	tests := []struct {
+		input    int
+		expected zstd.EncoderLevel
+	}{
+		{1, zstd.SpeedFastest},
+		{2, zstd.SpeedFastest},
+		{3, zstd.SpeedFastest},
+		{4, zstd.SpeedDefault},
+		{5, zstd.SpeedDefault},
+		{6, zstd.SpeedDefault},
+		{7, zstd.SpeedBetterCompression},
+		{8, zstd.SpeedBetterCompression},
+		{9, zstd.SpeedBetterCompression},
+	}
+
+	for _, tt := range tests {
+		C = Config{CompressionLevel: tt.input}
+		result := zstdLevel(tt.input)
+		if result != tt.expected {
+			t.Errorf("zstdLevel(%d) = %v, expected %v", tt.input, result, tt.expected)
+		}
+	}
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/log/main.go

@@ -2,7 +2,6 @@
 package log
 
 import (
-	"context"
 	"fmt"
 	"log"
 	"log/slog"
@@ -27,11 +26,9 @@ var (
 	logLevel slog.Level
 )
 
-// slogWriter implements [io.Writer] to redirect standard log to slog.
 type (
-	slogWriter struct {
-		handler slog.Handler
-	}
+	// slogWriter implements [io.Writer] to redirect standard log to slog.
+	slogWriter struct{}
 )
 
 // Init sets up logger stuff.
@@ -154,9 +151,9 @@ func Debugf(format string, a ...any) {
 	slog.Debug(fmt.Sprintf(format, a...))
 }
 
-// DebugLogger returns a standard library logger that writes at the configured log level.
-// This is useful for assigning to http.Server.ErrorLog to route net/http errors
-// through our logger while respecting the configured log level.
+// DebugLogger returns a standard library logger that writes at debug/warn level.
+// TLS-related errors are logged at debug level, others at warn level.
+// Both are filtered by the configured log level: set loglevel to debug to see all messages.
 func DebugLogger() *log.Logger {
 	mu.RLock()
 	defer mu.RUnlock()
@@ -167,7 +164,9 @@ func DebugLogger() *log.Logger {
 
 	handler := slog.NewTextHandler(Log, opts)
 
-	return log.New(&slogWriter{handler: handler}, "", 0)
+	_ = handler // satisfy linter (handler configures level for potential future use).
+
+	return log.New(&slogWriter{}, "", 0)
 }
 
 // Write implements [io.Writer]. It differentiates TLS/SSL errors from other errors.
@@ -183,15 +182,12 @@ func (w *slogWriter) Write(p []byte) (int, error) {
 		strings.Contains(msg, "handshake") ||
 		strings.Contains(msg, "certificate")
 
-	level := slog.LevelWarn
 	if isTLSError {
-		level = slog.LevelDebug
+		Debug(msg)
+	} else {
+		Warn(msg)
 	}
 
-	record := slog.NewRecord(time.Now(), level, msg, 0)
-
-	_ = w.handler.Handle(context.TODO(), record)
-
 	return len(p), nil
 }
 

internal/log/main_test.go

@@ -363,7 +363,7 @@ func TestDebugLogger(t *testing.T) {
 	logFilePath := tmpFile.Name()
 	tmpFile.Close()
 
-	if err := Init("info", logFilePath); err != nil {
+	if err := Init("debug", logFilePath); err != nil {
 		t.Fatal(err)
 	}
 	defer Close()
@@ -385,4 +385,85 @@ func TestDebugLogger(t *testing.T) {
 	}
 }
 
+// TestDebugLoggerTLS tests that TLS errors are logged at debug level.
+func TestDebugLoggerTLS(t *testing.T) {
+	tmpFile, err := os.CreateTemp(t.TempDir(), "test_log_*.log")
+	if err != nil {
+		t.Fatal(err)
+	}
+	logFilePath := tmpFile.Name()
+	tmpFile.Close()
+
+	if err := Init("debug", logFilePath); err != nil {
+		t.Fatal(err)
+	}
+	defer Close()
+
+	logger := DebugLogger()
+	logger.Print("tls handshake timeout")
+
+	content, err := os.ReadFile(logFilePath)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if !strings.Contains(string(content), "tls handshake timeout") {
+		t.Error("Expected TLS message in log")
+	}
+}
+
+// TestDebugLoggerNonTLS tests that non-TLS errors are logged at warn level.
+func TestDebugLoggerNonTLS(t *testing.T) {
+	tmpFile, err := os.CreateTemp(t.TempDir(), "test_log_*.log")
+	if err != nil {
+		t.Fatal(err)
+	}
+	logFilePath := tmpFile.Name()
+	tmpFile.Close()
+
+	if err := Init("warn", logFilePath); err != nil {
+		t.Fatal(err)
+	}
+	defer Close()
+
+	logger := DebugLogger()
+	logger.Print("http: connection reset by peer")
+
+	content, err := os.ReadFile(logFilePath)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if !strings.Contains(string(content), "http: connection reset by peer") {
+		t.Error("Expected non-TLS message in log")
+	}
+}
+
+// TestDebugLoggerLevelFiltering tests that log level filtering works.
+func TestDebugLoggerLevelFiltering(t *testing.T) {
+	tmpFile, err := os.CreateTemp(t.TempDir(), "test_log_*.log")
+	if err != nil {
+		t.Fatal(err)
+	}
+	logFilePath := tmpFile.Name()
+	tmpFile.Close()
+
+	if err := Init("info", logFilePath); err != nil {
+		t.Fatal(err)
+	}
+	defer Close()
+
+	logger := DebugLogger()
+	logger.Print("tls handshake timeout") // TLS error at debug level - should be filtered
+
+	content, err := os.ReadFile(logFilePath)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if strings.Contains(string(content), "tls handshake timeout") {
+		t.Error("Debug-level TLS message should be filtered at info log level")
+	}
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

llms.txt

@@ -30,13 +30,13 @@ curl -u user:pass https://server:8086/archive -o backup.tar.gz
 │   │   ├── globals.go       # Global C Config variable
 │   │   ├── archive.go       # Common archive logic (createArchiveStream, writeFilesToTar)
 │   │   ├── gzip.go          # CreateTarGzStream, CreateTarPgzipStream
-│   │   ├── bzip2.go         # CreateTarBzip2Stream
-│   │   ├── zstd.go          # CreateTarZstdStream
-│   │   ├── lz4.go           # CreateTarLz4Stream
+│   │   ├── bzip2.go         # CreateTarBzip2Stream, mapCompressionLevelToBzip2
+│   │   ├── zstd.go          # CreateTarZstdStream, zstdLevel
+│   │   ├── lz4.go           # CreateTarLz4Stream, mapCompressionLevelToLz4
 │   │   ├── xz.go            # CreateTarXzStream
 │   │   ├── loadConfig.go    # HJSON/JSON config parsing with validation
 │   │   ├── newServer.go     # HTTP server, auth handler, context helpers
-│   │   ├── getFilesFromDirectories.go  # Directory walking, exclusion filtering
+│   │   ├── getFilesFromDirectories.go  # Directory walking, exclusion filtering, isExcluded
 │   │   └── *_test.go        # Test files
 │   └── log/main.go          # Logging wrapper around stdlib slog
 ├── contrib/                 # Init scripts
@@ -55,7 +55,7 @@ curl -u user:pass https://server:8086/archive -o backup.tar.gz
 |--------|---------|-------------|
 | address | 0.0.0.0 | Listen address |
 | port | 8086 | Listen port |
-| cert/key | - | TLS cert/key paths |
+| cert/key | - | TLS cert/key paths (required when nohttps=false) |
 | nohttps | false | Disable HTTPS |
 | location | /archive | API endpoint path |
 | user | - | Basic auth username (required) |
@@ -64,10 +64,23 @@ curl -u user:pass https://server:8086/archive -o backup.tar.gz
 | loglevel | info | Log verbosity (debug/info/warn/error) |
 | directories | - | Directories to backup (required) |
 | backup_timeout | 60 | Stream timeout in minutes (1-1440) |
-| compression_level | 9 | Gzip level (1-9) |
+| dir_scan_timeout | 60 | Directory scan timeout in minutes (1-1440) |
+| compression_level | 9 | Compression level (1-9) |
 | exclude_patterns | [] | Regex patterns to exclude from backup |
 | filename_prefix | backup | Prefix for backup filename in Content-Disposition header |
-| compression_algorithm | gzip | Compression algorithm (gzip, pgzip, bzip2, zstd, lz4 or xz) |
+| default_compression | gzip | Default compression algorithm (gzip, pgzip, bzip2, zstd, lz4, xz) |
+| compression_algorithm | gzip | Alias for default_compression (deprecated) |
+
+## Compression Level Mapping
+
+The `compression_level` config option (1-9) is mapped differently for each algorithm:
+
+| Algorithm | Level Mapping |
+|-----------|---------------|
+| gzip/pgzip/bzip2 | 1-9 directly maps to compression levels |
+| lz4 | 1-3 → Fastest (0), 4-6 → Default (1), 7-9 → Best (2) |
+| zstd | 1-3 → Fastest, 4-6 → Default, 7-9 → BetterCompression |
+| xz | Uses default compression (no configurable level) |
 
 ## Key Patterns
 
@@ -85,16 +98,36 @@ json.Marshal(tmp)
 json.Unmarshal(buf, &C)
 ```
 
-### Stream Cancellation
-Context-aware write and copy functions:
+### Context-Aware I/O
+Two helper functions in newServer.go handle context cancellation during streaming:
+- `writeWithContext(ctx, fn)` — checks context before calling fn
+- `copyWithContext(ctx, dst, src)` — buffered copy (32KB) with cancellation checks, computes MD5 hash
+
 ```go
 func writeWithContext(ctx context.Context, fn func() error) error
-func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) error
+func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) (int64, string, error)
 ```
 
+### HTTP Server Middleware
+Two middleware functions wrap the handler:
+- `withServerHeader` — adds "Server: backer" header to all responses
+- `withStatusLogging` — logs 404 and 405 responses
+
+```go
+Handler: withServerHeader(withStatusLogging(mux))
+```
+
+### Unmatched Route Logging
+The server wraps http.ResponseWriter to capture status codes:
+- **404 Not Found**: `"Not Found from {clientIP} to {path}"`
+- **405 Method Not Allowed**: `"Method Not Allowed from {clientIP} for {method} {path}"`
+
+Uses `responseWriterWrapper` that tracks status code via WriteHeader() method.
+
 ### Pipe Error Handling
 Uses `pipeErr` tracking to avoid noisy errors when reader closes early.
 Skips files that can't be read (logs warning, continues backup).
+- `isPipeClosedError()` checks for "io: read/write on closed pipe" errors
 
 ### Content-Disposition Filename
 Backup filenames include timestamp and configured prefix: `backup-20060102-150405.tar.gz` (extension depends on compression_algorithm).
@@ -102,18 +135,43 @@ Backup filenames include timestamp and configured prefix: `backup-20060102-15040
 ### Special File Types
 - **Device files** (character/block) are included with correct major/minor numbers.
 - **Sockets** are skipped (no entry created).
-- **Symlinks** are stored as symlinks (target path stored).
+- **Symlinks** are stored as symlinks (target path stored) - uses os.Lstat, no cycles possible.
 - **Named pipes** and other special files are included (tar header created).
+- **Hard links** are deduplicated - first occurrence stored with content, subsequent as link entries.
 - **Regular files** are compressed; other file types have no data section.
 
 ### Error Recovery
 - `writeWithContext` checks context before calling fn.
 - `copyWithContext` does buffered copy with cancellation checks.
 
+### Client IP Detection
+Checks `X-Forwarded-For` header first (for proxied requests), then falls back to `RemoteAddr`.
+
+### TLS Error Handling
+Uses `ServerWrapper` to intercept and classify errors from Serve() and ServeTLS():
+- TLS-specific errors logged at debug level to reduce log clutter
+- Uses errors.As to detect: tls.AlertError, tls.CertificateVerificationError, tls.ECHRejectionError, tls.RecordHeaderError
+
+### HTTP Routing & Compression Selection
+The server uses http.NewServeMux() for routing. Multiple routes support different compression formats:
+```go
+mux.Handle(C.Location, http.HandlerFunc(backupHandler))           // /archive
+mux.Handle(C.Location+".tar.gz", http.HandlerFunc(backupHandler))  // archive.tar.gz
+mux.Handle(C.Location+".tar.xz", http.HandlerFunc(backupHandler))   // archive.tar.xz
+mux.Handle(C.Location+".tar.bz2", http.HandlerFunc(backupHandler)) // archive.tar.bz2
+// ... etc (.tar.lz4, .tar.zst, .tar.zstd)
+```
+
+Compression algorithm determined by path extension using `path.Ext()`:
+- Full path: `/archive.tar.xz` → `.xz` → xz
+- Bare extension: `tar.xz` → `.xz` → xz
+- Default when no extension matches
+
 ### Performance Notes
 - Exclude patterns are precompiled after config load for faster matching.
 - Compression level is configurable (1-9); default 9 uses more CPU.
 - Large directory walks use `filepath.WalkDir` (efficient).
+- Context cancellation checked on each file during directory scan.
 
 ## Coding Style & Conventions
 
@@ -128,7 +186,8 @@ GetFilesFromDirectories()
 // Unexported (private): verbNoun or isNoun pattern
 writeWithContext()
 copyWithContext()
-isPipeClosedError()  // predicate helper
+isPipeClosedError()
+isExcluded()
 defaultConfigPath()
 ```
 
@@ -137,6 +196,7 @@ Struct fields use PascalCase, json tags use snake_case:
 ```go
 BackupTimeout     int `json:"backup_timeout"`
 CompressionLevel  int `json:"compression_level"`
+DirScanTimeout    int `json:"dir_scan_timeout"`
 ```
 
 ### Error Messages
@@ -149,6 +209,7 @@ CompressionLevel  int `json:"compression_level"`
 - Past tense: `"Failed to close gzip writer: %v", err`
 - Context included: `"Skipping %s: %v", filename, err`
 - Appropriate level: Warn for recoverable, Error for issues, Fatal for startup failures
+- Log levels: error, warn, info, debug
 
 ### Config Validation Pattern
 For each config option: set default first, then immediately validate:
@@ -166,7 +227,7 @@ if C.Port < 1 || C.Port > 65535 {
 
 ### Go Formatting
 - Tabs for indentation
-- Vim modeline at EOF: `/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */`
+- Vim modeline at EOF: `/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */
 
 ### Linter Suppressions (intentional)
 | Rule | Reason |
@@ -181,24 +242,38 @@ if C.Port < 1 || C.Port > 65535 {
 
 ## Testing
 ```bash
-go test ./...                    # Run tests
+go test ./...                    # Run tests (runs make clean first)
 go test -v -run TestName ./...   # Run specific test
 golangci-lint run ./...          # Lint
+make test                        # Clean + test (deterministic)
 ```
 
-Coverage: ~69% on internal/backer (main and log packages not covered by design)
+Coverage: ~64% on internal/backer (~100% on internal/log)
+
+### Test Data
+All test data is created by `TestMain` in `internal/backer/main_test.go` before tests run and cleaned up afterward. This ensures tests are independent of pre-existing files.
+
+- `test_data/test1/foo/` — test directory with files, subdirs, empty_dir
+- `test_data/test1/bar/` — test directory with files and a directory named `goodbye.txt/` (edge case: directory with file extension)
+- `test_data/test1/hardlinks/` — test directory with hard-linked files (original.txt, hardlink1.txt, hardlink2.txt)
+- `test_data/test1/symlinks/` — test directory with symbolic links
+- `test_data/test1/empty_dir/` — empty directory for testing
+- `test_data/tmp/` — reserved for test temporary files (use instead of system `/tmp`)
 
 ### Test Conventions
 - Reset global config: `C = Config{}`
+- Reset exclude patterns: `excludePatterns = nil`
 - Use `t.TempDir()` for temporary files
 - Helper function `writeTempConfig()` for config tests
+- Test data created by TestMain setup, cleaned after tests
+- Always check errors with meaningful diagnostics (use `t.Fatalf` for setup errors)
 
 ## Build
 ```bash
-make           # Build binary (version from git or "dev")
+make           # Clean + build binary (version from git or "dev")
 make build VERSION=0.1.0  # Build with specific version
-make clean     # Remove binary
-make test      # Run tests
+make clean     # Remove binary, test_data/tmp, test_data/test1
+make test      # Clean + test (deterministic)
 make upgrade   # Update dependencies
 make help      # Show targets
 ```
@@ -208,3 +283,20 @@ Version is injected at build time via ldflags:
 ```go
 var version = "dev"  // overridden by: -X main.version=$(VERSION)
 ```
+
+## Request Flow (Good Path)
+```
+Request → Method Check (GET) → Auth Check → Get Files (with timeout) → Create Archive → Stream → Complete
+```
+
+## Request Flow (Error Paths)
+| Request | Response | Logged |
+|---------|----------|--------|
+| Non-GET method | 405 Method Not Allowed | Yes (info) |
+| No auth | 401 Unauthorized | Yes (info) |
+| Wrong credentials | 401 Unauthorized | Yes (info) |
+| Unknown path | 404 Not Found | Yes (info) |
+| Dir scan timeout | 504 Request Timeout | Yes (error) |
+| Failed to get files | 500 Internal Server Error | Yes (error) |
+| Client disconnect | - | Yes (info) |
+| Backup completed | 200 OK | Yes (info) |

test_data/test1/bar/goodbye.txt/text.txt

@@ -1 +0,0 @@
-Fucked up beyond recognotion.

test_data/test1/foo/hello_breakfast.txt

@@ -1,2 +0,0 @@
-My dear Tweety,
-I'd like to greet you as my breakfast.

test_data/test1/foo/mydir/myfile.txt

@@ -1 +0,0 @@
-This is MY file!

test_data/test_config.json

@@ -5,10 +5,10 @@
     /* By default listen on port 8086, like first intel commercially successful cpu model.                            */
     "port": 8086,
 
-    /* Path location of certificate file                                                                              */
+    /* Path location of test certificate file                                                                       */
     "cert": "../../test_data/example.crt",
 
-    /* Path location of certificate key                                                                               */
+    /* Path location of test certificate key                                                                          */
     "key": "../../test_data/example.key",
 
     /* We really want to use https for backup archive tranferring.                                                    */