Add explicit statement that our tar is gnu-compatible to README.md

.gitignore

@@ -30,4 +30,6 @@ Thumbs.db
 *.test
 *.out
 
-/test_data/tmp
+# Test data (generated by tests)
+test_data/tmp
+test_data/test1

AGENTS.md

@@ -27,6 +27,36 @@ Backer is a Go-based HTTP/HTTPS backup server that creates compressed archives o
 - **Don't use `context` package unless absolutely necessary.** Try to find another solution first.
 - **Create functional and unit tests if possible/necessary** to verify changes that software code works at is should.
 
+### Test Rules
+
+- **Test data must be self-contained and created at test setup.** All test data should be created in `TestMain` in `internal/backer/main_test.go` before any tests run, not rely on pre-existing files in the repository. Use `test_data/test1/` as the test data directory.
+
+- **Use `test_data/tmp/` for temporary files.** When tests need temporary storage, use this directory (not system `/tmp`) to keep test data isolated and predictable.
+
+- **Always check errors with meaningful diagnostics.** Never discard errors with `_ =`. Use patterns like:
+  ```go
+  if err := os.MkdirAll(dir, 0755); err != nil {
+      t.Fatalf("Failed to create directory %s: %v", dir, err)
+  }
+  ```
+
+- **Tests must not depend on external state.** Test data should be created fresh for each test run and cleaned up afterward. Use `TestMain` with setup/cleanup functions.
+
+- **Use `t.Fatal` or `t.Fatalf` for setup errors.** When test setup fails, fail fast with clear diagnostic messages showing what failed and why.
+
+- **Reset global state between tests.** For tests that modify global variables (like `C` config), save the original value and restore it in a `defer`:
+  ```go
+  original := C
+  defer func() { C = original }()
+  ```
+
+- **Skip tests gracefully when features are unavailable.** For platform-specific features (hardlinks, symlinks), use `t.Skip()` when the feature is not supported:
+  ```go
+  if err := os.Link(orig, link); err != nil {
+      t.Skip("Hard links not supported, skipping test")
+  }
+  ```
+
 ## Build & Test Commands
 
 ```bash
@@ -207,6 +237,7 @@ Config is JSON (HJSON with comments supported). Default path depends on OS:
 | loglevel | `info` | Log verbosity: error/warn/info/debug |
 | directories | — | Directories to backup (required, validated for existence) |
 | backup_timeout | `60` | Stream timeout in minutes (1-1440) |
+| dir_scan_timeout | `60` | Directory scan timeout in minutes (1-1440) |
 | compression_level | `9` | Gzip level (1-9) |
 | exclude_patterns | `[]` | Regex patterns to exclude from backup |
 | filename_prefix | `backup` | Prefix for backup filename in Content-Disposition header |
@@ -247,10 +278,30 @@ Two helper functions in `newServer.go` handle context cancellation during stream
 - **Regular files**: streamed into tar archive with content
 - **Directories**: included as tar directory entries
 - **Symlinks**: stored as symlink entries (target path preserved). Backer never resolves or follows symlinks during directory traversal — it uses `os.Lstat` and `filepath.WalkDir` which do not follow symlinks. This means symlink cycles are impossible to occur.
+- **Broken symlinks**: archived with their target path (even if target doesn't exist)
 - **Device files** (char/block): included with correct major/minor
-- **Named pipes**: included (header only, no data)
+- **Named pipes (FIFOs)**: included (header only, no data)
 - **Sockets**: skipped (not archivable)
 - **Hard links**: deduplicated — if multiple files share the same inode, only the first is stored with content, subsequent hard links are stored as link entries pointing to the original
+- **GNU tar format**: archives use GNU tar format (`header.Format = tar.FormatGNU`) for compatibility with GNU tar utility
+
+### Archive Test Coverage
+
+All entity types are tested in `internal/backer/utils_test.go`:
+
+| Entity Type | Test Function |
+|------------|---------------|
+| Regular file | `TestCreateTarGzStreamBasic` |
+| Empty file | `TestCreateTarGzStreamEmptyFile` |
+| Directory | `TestCreateTarGzStreamDirectory` |
+| Symlink | `TestCreateTarGzStreamSymlink` |
+| Broken symlink | `TestCreateTarGzStreamBrokenSymlink` |
+| Symlink to directory | `TestCreateTarGzStreamSymlinkToDirectory` |
+| Hard link | `TestCreateTarGzStreamHardLink` |
+| Character device | `TestCreateTarGzStreamDeviceFile` (falls back to char device) |
+| Named pipe (FIFO) | `TestCreateTarGzStreamNamedPipe` |
+| Special characters in names | `TestCreateTarGzStreamSpecialCharacters` |
+| Nested files | `TestCreateTarGzStreamNestedFiles` |
 
 ### Pipe Error Recovery
 
@@ -275,13 +326,16 @@ Checks `X-Forwarded-For` header first (for proxied requests), then falls back to
 
 ### TLS Error Handling
 
-The server uses a `ServerWrapper` type to intercept and classify errors from `Serve()` and `ServeTLS()`. TLS-specific errors are logged at debug level to reduce log clutter:
+The server uses a `ServerWrapper` type to intercept and classify errors from `Serve()` and `ServeTLS()`. Two mechanisms:
 
-- Uses `errors.As` to detect TLS error types: `tls.AlertError`, `tls.CertificateVerificationError`, `tls.ECHRejectionError`, `tls.RecordHeaderError`
+1. **Server errors** (`Serve()` / `ServeTLS()`): Uses `errors.As` to detect TLS error types (`tls.AlertError`, `tls.CertificateVerificationError`, `tls.ECHRejectionError`, `tls.RecordHeaderError`) and checks for "tls:" prefix. TLS-specific errors logged at debug level, others at warn level.
-- Also checks for "tls:" prefix in error messages
-- Additionally uses `http.Server.ErrorLog` with a custom logger that writes at debug level for net/http's internal TLS-related messages
 
-This ensures client-side TLS errors (handshake failures, certificate issues) don't clutter production logs.
+2. **net/http internal errors** (`http.Server.ErrorLog`): Uses `DebugLogger()` which routes messages through `slogWriter.Write()`:
+   - TLS-related errors (keywords: "tls", "ssl", "handshake", "certificate") → logged at debug level
+   - Other errors → logged at warn level
+   - Both are filtered by configured `loglevel`. Set `loglevel: debug` to see all messages.
+
+This ensures client-side TLS errors (handshake failures, certificate issues) don't clutter production logs by default.
 
 ### Compression Level Mapping
 
@@ -302,7 +356,6 @@ The server uses `http.NewServeMux()` for routing. Multiple routes are registered
 
 ```go
 mux.Handle(C.Location, http.HandlerFunc(backupHandler))           // /archive
-mux.Handle(C.Location+"/", http.HandlerFunc(backupHandler))       // /archive/
 mux.Handle(C.Location+".tar.gz", http.HandlerFunc(backupHandler))  // /archive.tar.gz
 mux.Handle(C.Location+".tar.xz", http.HandlerFunc(backupHandler))   // /archive.tar.xz
 mux.Handle(C.Location+".tar.bz2", http.HandlerFunc(backupHandler)) // /archive.tar.bz2
@@ -318,6 +371,14 @@ The `getCompressionAlgorithm()` function handles both full paths and bare extens
 - Bare extension: `tar.xz` → `.tar.xz` → xz
 - Single ext: `xz` → `.xz` → xz
 
+### Unmatched Route Logging
+
+The server wraps the `http.ResponseWriter` to capture status codes and log unmatched routes:
+- **404 Not Found**: Logged as `"Not Found from {clientIP} to {path}"`
+- **405 Method Not Allowed**: Logged as `"Method Not Allowed from {clientIP} for {method} {path}"`
+
+This uses a `responseWriterWrapper` that tracks the status code via the `WriteHeader()` method.
+
 ## Dependencies
 
 - `github.com/hjson/hjson-go` v3.3.0 — HJSON config parsing
@@ -348,10 +409,14 @@ Tests live alongside source files as `*_test.go` (package name, not `_test` suff
 
 ### Test Data
 
-- `test_data/test_config.json` — valid test config (user: JohnnyGoode, password: SharpShooter)
+All test data is created by `TestMain` in `internal/backer/main_test.go` before tests run and cleaned up afterward. This ensures tests are independent of pre-existing files.
-- `test_data/test1/foo/` — test directory with files, subdirs, empty dirs
+
+- `test_data/test1/foo/` — test directory with files, subdirs, empty_dir
 - `test_data/test1/bar/` — test directory with files and a directory named `goodbye.txt/` (edge case: directory with file extension)
-- `test_data/example.crt` / `example.key` — TLS test certificates
+- `test_data/test1/hardlinks/` — test directory with hard-linked files (original.txt, hardlink1.txt, hardlink2.txt)
+- `test_data/test1/symlinks/` — test directory with symbolic links
+- `test_data/test1/empty_dir/` — empty directory for testing
+- `test_data/tmp/` — reserved for test temporary files (use instead of system `/tmp`)
 
 ## Service Deployment
 

Makefile

@@ -23,13 +23,15 @@ build:
 
 clean:
 	rm -f $(BINARY)
+	rm -rf test_data/tmp
+	rm -rf test_data/test1
 
 upgrade:
 	go get -u ./...
 	go mod tidy
 	go mod vendor
 
-test:
+test: clean
 	go test ./...
 
 help:

README.md

@@ -65,6 +65,7 @@ All options described in example config.
 | loglevel | "info" | Verbosity of logs. | Options: error, warn, info, debug. |
 | directories | — | Directories to backup. | Required. Array of paths. |
 | backup_timeout | 60 | Timeout in minutes for backup streaming. | Range: 1-1440 minutes. |
+| dir_scan_timeout | 60 | Timeout in minutes for getting file list from all backupped directories. | Range: 1-1440 minutes. |
 | compression_level | 9 | Compression level. | 1 (fastest) to 9 (best compression). |
 | exclude_patterns | [] | Regex patterns to exclude from backup. | E.g., `[".*\\.tmp$", "/node_modules/"]`. |
 | filename_prefix | "backup" | Prefix for backup filename. | E.g., "mybackup" produces `mybackup-20260325-092341.tar.gz`. |
@@ -127,9 +128,12 @@ Backer handles various file types when creating archives:
 - **Regular files**: Included with full content
 - **Directories**: Included as directory entries
 - **Symlinks**: Stored as symlink entries (target path preserved)
+- **Broken symlinks**: Archived with their target path (even if target doesn't exist)
 - **Device files**: Character and block devices included with correct major/minor
-- **Named pipes**: Included (header only, no data)
+- **Named pipes (FIFOs)**: Included (header only, no data)
+- **Sockets**: Skipped (not archivable)
 - **Hard links**: Deduplicated — multiple hard links to the same file are stored as link entries to avoid duplication
+- **GNU tar format**: Compatible with GNU tar utility
 
 ## Logging
 

data/config_example.json

@@ -47,6 +47,9 @@
     /* Regex patterns to exclude from backup.                                                                         */
     "exclude_patterns": [],
 
+    /* Maximum allowed time, spent for getting file list from backupped directories.                                  */
+    "dir_scan_timeout": 60,
+
     /* Directories to backup.                                                                                         */
     "directories": [
         "/path/to/directory1",

internal/backer/archive.go

@@ -9,6 +9,7 @@ import (
 	"os"
 	"path/filepath"
 	"strings"
+	"syscall"
 
 	"backer/internal/log"
 )
@@ -85,6 +86,8 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 		var (
 			err        error
 			linkTarget string
+			isSymlink  bool
+			isHardLink bool
 		)
 
 		select {
@@ -106,6 +109,7 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 			continue
 		}
 
+		// Handle symlinks - read target path and mark as symbolic link.
 		if st.Mode()&os.ModeSymlink != 0 {
 			linkTarget, err = os.Readlink(fpath)
 
@@ -114,6 +118,8 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 
 				continue
 			}
+
+			isSymlink = true
 		}
 
 		// Skip sockets explicitly (they cannot be backed up).
@@ -124,12 +130,16 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 		}
 
 		// Check for hard link - if inode was already seen, store as link instead of content.
-		if st.Mode().IsRegular() {
+		// Only check for regular files (not symlinks).
+		if st.Mode().IsRegular() && !isSymlink {
 			inode := getInode(st)
 			if inode != 0 {
 				if existingPath, seen := seenInodes[inode]; seen {
 					// This is a hard link to already-archived file.
-					linkTarget = existingPath
+					// To avoid "tar: Removing leading `/' from hard link targets" message, remove leading slash.
+					linkTarget = strings.TrimPrefix(filepath.ToSlash(filepath.Clean(existingPath)), "/")
+					isHardLink = true
+
 					log.Debugf("Hard link detected: %s -> %s", fpath, existingPath)
 				} else {
 					// First time seeing this inode - track it.
@@ -146,9 +156,13 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 			continue
 		}
 
-		// If linkTarget is set, ensure the header is marked as a link.
+		// Set correct type flag for symbolic links or hard links.
-		if linkTarget != "" {
+		if isSymlink {
+			header.Typeflag = tar.TypeSymlink
+			header.Linkname = linkTarget
+		} else if isHardLink {
 			header.Typeflag = tar.TypeLink
+			header.Linkname = linkTarget
 		}
 
 		header.Format = tar.FormatGNU
@@ -164,8 +178,8 @@ func writeFilesToTar(ctx context.Context, filepaths []string, tw *tar.Writer, cl
 			return
 		}
 
-		// Only stream file contents for regular files.
+		// Only stream file contents for regular files that are not hard links.
-		if mode.IsRegular() {
+		if mode.IsRegular() && linkTarget == "" {
 			f, err := os.Open(fpath)
 			if err != nil {
 				log.Warnf("Skipping %s: open failed: %v", fpath, err)
@@ -195,15 +209,12 @@ func getInode(fi os.FileInfo) uint64 {
 		return 0
 	}
 
-	// Use type assertion to get inode - works on Unix, returns 0 on Windows.
+	// Use reflection to get Dev and Ino fields from the syscall.Stat_t struct.
-	type inodeGetter interface {
+	st := fi.Sys()
-		Dev() uint64
+	// Try type assertion for the struct with fields.
-		Ino() uint64
+	switch v := st.(type) {
-	}
+	case *syscall.Stat_t:
-
+		return (v.Dev << 32) | v.Ino
-	if ig, ok := fi.Sys().(inodeGetter); ok {
-		// Combine device and inode for uniqueness across file systems.
-		return (ig.Dev() << 32) | ig.Ino()
 	}
 
 	return 0

internal/backer/getFilesFromDirectories.go

@@ -6,7 +6,6 @@ import (
 	"io/fs"
 	"os"
 	"path/filepath"
-	"time"
 
 	backerlog "backer/internal/log"
 )
@@ -81,13 +80,4 @@ func GetFilesFromDirectories(ctx context.Context, directories []string) ([]strin
 	return files, nil
 }
 
-// GetFilesFromDirectoriesWithTimeout makes a file list of given directories with timeout.
-// Convenience wrapper that creates context from timeout in minutes.
-func GetFilesFromDirectoriesWithTimeout(timeoutMinutes int, directories []string) ([]string, error) {
-	ctx, cancel := context.WithTimeout(context.Background(), time.Duration(timeoutMinutes)*time.Minute)
-	defer cancel()
-
-	return GetFilesFromDirectories(ctx, directories)
-}
-
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/getFilesFromDirectories_test.go

@@ -15,9 +15,18 @@ func normalizePath(p string) string {
 	return strings.ReplaceAll(p, "\\", "/")
 }
 
+// normalizePathSlice normalizes all paths in a slice.
+func normalizePathSlice(paths []string) []string {
+	result := make([]string, len(paths))
+	for i, p := range paths {
+		result[i] = normalizePath(p)
+	}
+	return result
+}
+
 // TestGetFilesFromDirectories checks if GetFilesFromDirectories returns the proper amount of strings, correct strings and no error.
 func TestGetFilesFromDirectories(t *testing.T) {
-	testDataPath := filepath.Join("..", "test_data")
+	testDataPath := "../../test_data"
 
 	// Log the paths we're checking for debugging
 	t.Logf("Checking directories: %v", []string{
@@ -61,7 +70,8 @@ func TestGetFilesFromDirectories(t *testing.T) {
 
 	for _, filepath := range output {
 		normalized := normalizePath(filepath)
-		if !slices.Contains(expectedData, normalized) {
+		normalizedExpected := normalizePathSlice(expectedData)
+		if !slices.Contains(normalizedExpected, normalized) {
 			t.Errorf("expectedData data does not contain string from output: %s", filepath)
 		}
 	}
@@ -69,7 +79,7 @@ func TestGetFilesFromDirectories(t *testing.T) {
 
 // TestGetFilesFromDirectoriesSingleDir tests with a single directory.
 func TestGetFilesFromDirectoriesSingleDir(t *testing.T) {
-	testDataPath := filepath.Join("test_data")
+	testDataPath := filepath.Join("../../test_data")
 	output, err := GetFilesFromDirectories(context.Background(), []string{filepath.Join(testDataPath, "test1", "foo")})
 
 	if err != nil {
@@ -106,7 +116,7 @@ func TestGetFilesFromDirectoriesEmptyDir(t *testing.T) {
 // TestGetFilesFromDirectoriesNonExistent tests with non-existent directory.
 // Should log error but not fail - returns empty list.
 func TestGetFilesFromDirectoriesNonExistent(t *testing.T) {
-	testDataPath := filepath.Join("test_data")
+	testDataPath := filepath.Join("../../test_data")
 	nonExistentPath := filepath.Join(testDataPath, "nonexistent", "path", "that", "does", "not", "exist")
 	output, err := GetFilesFromDirectories(context.Background(), []string{nonExistentPath})
 
@@ -123,7 +133,7 @@ func TestGetFilesFromDirectoriesNonExistent(t *testing.T) {
 
 // TestGetFilesFromDirectoriesPartialFailure tests with mix of valid and invalid directories.
 func TestGetFilesFromDirectoriesPartialFailure(t *testing.T) {
-	testDataPath := filepath.Join("test_data")
+	testDataPath := filepath.Join("../../test_data")
 	// Create a temp directory that exists
 	tmpDir := t.TempDir()
 
@@ -142,7 +152,7 @@ func TestGetFilesFromDirectoriesPartialFailure(t *testing.T) {
 
 // TestGetFilesFromDirectoriesMultipleDirs tests multiple directories.
 func TestGetFilesFromDirectoriesMultipleDirs(t *testing.T) {
-	testDataPath := filepath.Join("test_data")
+	testDataPath := filepath.Join("../../test_data")
 	output, err := GetFilesFromDirectories(context.Background(), []string{
 		filepath.Join(testDataPath, "test1", "foo"),
 		filepath.Join(testDataPath, "test1", "bar"),

internal/backer/loadConfig_test.go

@@ -13,8 +13,12 @@ func TestLoadConfig(t *testing.T) {
 	C = Config{}
 
 	testDataPath := filepath.Join(t.TempDir(), "test_data")
-	os.MkdirAll(filepath.Join(testDataPath, "test1", "foo"), 0755)
+	if err := os.MkdirAll(filepath.Join(testDataPath, "test1", "foo"), 0755); err != nil {
-	os.MkdirAll(filepath.Join(testDataPath, "test1", "bar"), 0755)
+		t.Fatalf("Failed to create test directory foo: %v", err)
+	}
+	if err := os.MkdirAll(filepath.Join(testDataPath, "test1", "bar"), 0755); err != nil {
+		t.Fatalf("Failed to create test directory bar: %v", err)
+	}
 
 	// Create test config file
 	testConfigPath := filepath.Join(testDataPath, "test_config.json")

internal/backer/main_test.go

@@ -0,0 +1,103 @@
+package backer
+
+import (
+	"context"
+	"fmt"
+	"os"
+	"path/filepath"
+	"testing"
+)
+
+const testDataPath = "../../test_data"
+
+func TestMain(m *testing.M) {
+	if err := setupTestData(); err != nil {
+		fmt.Fprintf(os.Stderr, "FATAL: Failed to setup test data: %v\n", err)
+		os.Exit(1)
+	}
+
+	code := m.Run()
+
+	if err := cleanupTestData(); err != nil {
+		fmt.Fprintf(os.Stderr, "WARNING: Failed to cleanup test data: %v\n", err)
+	}
+
+	os.Exit(code)
+}
+
+func setupTestData() error {
+	testDir := filepath.Join(testDataPath, "test1")
+
+	dirs := []string{
+		filepath.Join(testDir, "foo", "mydir"),
+		filepath.Join(testDir, "bar", "goodbye.txt"),
+		filepath.Join(testDir, "hardlinks"),
+		filepath.Join(testDir, "symlinks"),
+		filepath.Join(testDir, "foo", "empty_dir"),
+		filepath.Join(testDataPath, "tmp"),
+	}
+
+	for _, dir := range dirs {
+		if err := os.MkdirAll(dir, 0755); err != nil {
+			return fmt.Errorf("Failed to create directory %s: %w", dir, err)
+		}
+	}
+
+	files := map[string]string{
+		filepath.Join(testDir, "foo", "hello_breakfast.txt"):   "hello breakfast content",
+		filepath.Join(testDir, "foo", "some_text.txt"):         "some text content",
+		filepath.Join(testDir, "foo", "mydir", "myfile.txt"):   "my file content",
+		filepath.Join(testDir, "bar", "hello.txt"):             "hello content",
+		filepath.Join(testDir, "bar", "goodbye.txt", "text.txt"): "text content",
+		filepath.Join(testDir, "hardlinks", "original.txt"):    "original content",
+		filepath.Join(testDir, "symlinks", "target.txt"):       "target content",
+	}
+
+	for path, content := range files {
+		if err := os.WriteFile(path, []byte(content), 0644); err != nil {
+			return fmt.Errorf("Failed to write file %s: %w", path, err)
+		}
+	}
+
+	orig := filepath.Join(testDir, "hardlinks", "original.txt")
+	link1 := filepath.Join(testDir, "hardlinks", "hardlink1.txt")
+	if err := os.Link(orig, link1); err != nil {
+		return fmt.Errorf("Failed to create hardlink %s -> %s: %w", link1, orig, err)
+	}
+
+	link2 := filepath.Join(testDir, "hardlinks", "hardlink2.txt")
+	if err := os.Link(orig, link2); err != nil {
+		return fmt.Errorf("Failed to create hardlink %s -> %s: %w", link2, orig, err)
+	}
+
+	target := filepath.Join(testDir, "symlinks", "target.txt")
+	link := filepath.Join(testDir, "symlinks", "link.txt")
+	if err := os.Symlink(target, link); err != nil {
+		return fmt.Errorf("Failed to create symlink %s -> %s: %w", link, target, err)
+	}
+
+	return nil
+}
+
+func cleanupTestData() error {
+	testDir := filepath.Join(testDataPath, "test1")
+
+	if err := os.RemoveAll(testDir); err != nil {
+		return fmt.Errorf("Failed to remove test directory %s: %w", testDir, err)
+	}
+
+	tmpDir := filepath.Join(testDataPath, "tmp")
+	if err := os.RemoveAll(tmpDir); err != nil {
+		return fmt.Errorf("Failed to remove tmp directory %s: %w", tmpDir, err)
+	}
+
+	return nil
+}
+
+func GetTestDataPath() string {
+	return testDataPath
+}
+
+func GetTestFilesFromDirectories(ctx context.Context, directories []string) ([]string, error) {
+	return GetFilesFromDirectories(ctx, directories)
+}

internal/backer/newServer.go

@@ -48,6 +48,23 @@ func isTLSError(err error) bool {
 	return strings.HasPrefix(err.Error(), "tls:")
 }
 
+// withStatusLogging wraps a handler to log 404 and 405 responses.
+func withStatusLogging(next http.Handler) http.Handler {
+	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		rw := &responseWriterWrapper{ResponseWriter: w, statusCode: http.StatusOK}
+		next.ServeHTTP(rw, r)
+
+		clientIP := getClientIP(r)
+
+		switch rw.statusCode {
+		case http.StatusNotFound:
+			log.Infof("Not Found from %s to %s", clientIP, r.RequestURI)
+		case http.StatusMethodNotAllowed:
+			log.Infof("Method Not Allowed from %s for %s %s", clientIP, r.Method, r.RequestURI)
+		}
+	})
+}
+
 // withServerHeader wraps a handler to add Server header to all responses.
 func withServerHeader(next http.Handler) http.Handler {
 	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -74,6 +91,13 @@ func NewServer(configPath string) (*ServerWrapper, error) {
 	mux := http.NewServeMux()
 
 	backupHandler := func(w http.ResponseWriter, r *http.Request) {
+		// Only GET method is allowed.
+		if r.Method != http.MethodGet {
+			http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
+
+			return
+		}
+
 		username, password, ok := r.BasicAuth()
 
 		// No auth? Good bye.
@@ -219,7 +243,7 @@ func NewServer(configPath string) (*ServerWrapper, error) {
 		srv := &ServerWrapper{
 			Server: &http.Server{
 				Addr:              fmt.Sprintf("%s:%d", C.Address, C.Port),
-				Handler:           withServerHeader(mux),
+				Handler:           withServerHeader(withStatusLogging(mux)),
 				ReadHeaderTimeout: readHeaderTimeout,
 				WriteTimeout:      time.Duration(C.BackupTimeout) * time.Minute,
 				IdleTimeout:       idleTimeout,
@@ -235,7 +259,7 @@ func NewServer(configPath string) (*ServerWrapper, error) {
 	srv := &ServerWrapper{
 		Server: &http.Server{
 			Addr:              fmt.Sprintf("%s:%d", C.Address, C.Port),
-			Handler:           withServerHeader(mux),
+			Handler:           withServerHeader(withStatusLogging(mux)),
 			ReadHeaderTimeout: readHeaderTimeout,
 			WriteTimeout:      time.Duration(C.BackupTimeout) * time.Minute,
 			IdleTimeout:       idleTimeout,
@@ -374,8 +398,20 @@ type (
 	ServerWrapper struct {
 		*http.Server
 	}
+
+	// responseWriterWrapper wraps [http.ResponseWriter] to capture status code.
+	responseWriterWrapper struct {
+		http.ResponseWriter
+		statusCode int
+	}
 )
 
+// WriteHeader captures the status code for logging.
+func (rw *responseWriterWrapper) WriteHeader(code int) {
+	rw.statusCode = code
+	rw.ResponseWriter.WriteHeader(code)
+}
+
 // Serve starts the server and classifies any errors returned.
 // TLS-related errors are logged at debug level, other errors at warn level.
 func (sw *ServerWrapper) Serve() error {

internal/backer/newServer_test.go

@@ -125,46 +125,43 @@ func TestGetClientIP(t *testing.T) {
 
 // TestGetCompressionAlgorithm tests the getCompressionAlgorithm function.
 func TestGetCompressionAlgorithm(t *testing.T) {
+	// Save original config
+	original := C
+	defer func() { C = original }()
+
 	tests := []struct {
-		name     string
+		name              string
-		ext      string
+		ext               string
-		expected string
+		expected          string
+		defaultCompression string
 	}{
-		{
+		{name: "tar.bz2", ext: "tar.bz2", expected: "bzip2"},
-			name:     "tar.bz2",
+		{name: "tar.zstd", ext: "tar.zstd", expected: "zstd"},
-			ext:      "tar.bz2",
+		{name: "tar.zst", ext: "tar.zst", expected: "zstd"},
-			expected: "bzip2",
+		{name: "tar.lz4", ext: "tar.lz4", expected: "lz4"},
-		},
+		{name: "tar.xz", ext: "tar.xz", expected: "xz"},
-		{
+		{name: "tar.gz", ext: "tar.gz", expected: "pgzip"},
-			name:     "tar.zst",
+		{name: "bz2", ext: "bz2", expected: "bzip2"},
-			ext:      "tar.zst",
+		{name: "zst", ext: "zst", expected: "zstd"},
-			expected: "zstd",
+		{name: "lz4", ext: "lz4", expected: "lz4"},
-		},
+		{name: "xz", ext: "xz", expected: "xz"},
-		{
+		{name: "gz", ext: "gz", expected: "pgzip"},
-			name:     "tar.lz4",
+		{name: "zstd", ext: "zstd", expected: "zstd"},
-			ext:      "tar.lz4",
+		{name: "full path /archive.tar.gz", ext: "/archive.tar.gz", expected: "pgzip"},
-			expected: "lz4",
+		{name: "full path /archive.tar.xz", ext: "/archive.tar.xz", expected: "xz"},
-		},
+		{name: "full path /archive", ext: "/archive", expected: "gzip"},
-		{
+		{name: "empty uses default", ext: "", expected: "gzip"},
-			name:     "tar.xz",
+		{name: "unknown uses default", ext: "unknown", expected: "gzip"},
-			ext:      "tar.xz",
+		{name: "unknown with default zstd", ext: "unknown", expected: "zstd", defaultCompression: "zstd"},
-			expected: "xz",
-		},
-		{
-			name:     "tar.gz",
-			ext:      "tar.gz",
-			expected: "pgzip",
-		},
-		{
-			name:     "default with gzip config",
-			ext:      "",
-			expected: "gzip",
-		},
 	}
 
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
-			C = Config{DefaultCompression: "gzip"}
+			if tt.defaultCompression != "" {
+				C = Config{DefaultCompression: tt.defaultCompression}
+			} else {
+				C = Config{DefaultCompression: "gzip"}
+			}
 			result := getCompressionAlgorithm(tt.ext)
 			if result != tt.expected {
 				t.Errorf("getCompressionAlgorithm(%q) = %q, expected %q", tt.ext, result, tt.expected)
@@ -191,6 +188,46 @@ func TestNewServerSuccess(t *testing.T) {
 	}
 }
 
+// TestWithServerHeader tests that the Server header is added to responses.
+func TestWithServerHeader(t *testing.T) {
+	handler := withServerHeader(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusOK)
+	}))
+
+	rec := &responseWriterWrapper{
+		ResponseWriter: &testResponseWriter{},
+		statusCode:     0,
+	}
+
+	req, _ := http.NewRequest("GET", "/test", nil)
+	handler.ServeHTTP(rec, req)
+
+	if rec.Header().Get("Server") != "backer" {
+		t.Error("Expected Server header to be set to 'backer'")
+	}
+}
+
+// testResponseWriter implements http.ResponseWriter for testing.
+type testResponseWriter struct {
+	header http.Header
+	status int
+}
+
+func (tw *testResponseWriter) Header() http.Header {
+	if tw.header == nil {
+		tw.header = make(http.Header)
+	}
+	return tw.header
+}
+
+func (tw *testResponseWriter) Write(data []byte) (int, error) {
+	return len(data), nil
+}
+
+func (tw *testResponseWriter) WriteHeader(status int) {
+	tw.status = status
+}
+
 // TestNewServerInvalidConfig tests server creation with invalid config.
 func TestNewServerInvalidConfig(t *testing.T) {
 	C = Config{} // Reset global config.
@@ -523,4 +560,125 @@ func TestCopyWithContextCancellation(t *testing.T) {
 	}
 }
 
+// TestStatusLogging404 tests that 404 responses are logged.
+func TestStatusLogging404(t *testing.T) {
+	C = Config{} // Reset global config.
+	excludePatterns = nil
+
+	// Create a minimal mux that returns 404 for all paths
+	testMux := http.NewServeMux()
+	testMux.HandleFunc("/archive", func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusOK)
+	})
+
+	wrappedMux := withStatusLogging(testMux)
+
+	// Use a ResponseRecorder to capture the response
+	rr := httptest.NewRecorder()
+	req, _ := http.NewRequest("GET", "/nonexistent", nil)
+	req.RemoteAddr = "127.0.0.1:12345"
+
+	wrappedMux.ServeHTTP(rr, req)
+
+	if rr.Code != http.StatusNotFound {
+		t.Errorf("Expected status 404, got %d", rr.Code)
+	}
+}
+
+// TestStatusLogging405 tests that 405 responses are logged.
+func TestStatusLogging405(t *testing.T) {
+	C = Config{} // Reset global config.
+	excludePatterns = nil
+
+	// Create a minimal mux that returns 405 for wrong method
+	testMux := http.NewServeMux()
+	testMux.HandleFunc("/archive", func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusMethodNotAllowed)
+	})
+
+	wrappedMux := withStatusLogging(testMux)
+
+	rr := httptest.NewRecorder()
+	req, _ := http.NewRequest("GET", "/archive", nil)
+	req.RemoteAddr = "127.0.0.1:12345"
+
+	wrappedMux.ServeHTTP(rr, req)
+
+	if rr.Code != http.StatusMethodNotAllowed {
+		t.Errorf("Expected status 405, got %d", rr.Code)
+	}
+}
+
+// TestMethodNotAllowedHandler tests that non-GET methods return 405.
+func TestMethodNotAllowedHandler(t *testing.T) {
+	C = Config{
+		Address:     "0.0.0.0",
+		Port:        8086,
+		Location:    "/archive",
+		User:        "testuser",
+		Password:    "testpass",
+		Directories: []string{"../../test_data/test1/foo"},
+	}
+	excludePatterns = nil
+
+	// Create test server
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		username, password, ok := r.BasicAuth()
+		if !ok || username != C.User || password != C.Password {
+			http.Error(w, "Unauthorized", http.StatusUnauthorized)
+
+			return
+		}
+
+		if r.Method != http.MethodGet {
+			w.Header().Set("Server", "backer")
+			http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
+
+			return
+		}
+
+		w.WriteHeader(http.StatusOK)
+	}))
+	defer server.Close()
+
+	// Test POST method
+	req, _ := http.NewRequest("POST", server.URL+"/archive", nil)
+	req.SetBasicAuth("testuser", "testpass")
+	resp, err := http.DefaultClient.Do(req)
+	if err != nil {
+		t.Fatalf("Request failed: %v", err)
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != http.StatusMethodNotAllowed {
+		t.Errorf("Expected status 405, got %d", resp.StatusCode)
+	}
+}
+
+// TestTimingSafeMatch tests the timingSafeMatch function for constant-time comparison.
+func TestTimingSafeMatch(t *testing.T) {
+	tests := []struct {
+		name     string
+		a        string
+		b        string
+		expected bool
+	}{
+		{"empty strings", "", "", true},
+		{"identical strings", "test", "test", true},
+		{"different strings", "test", "Test", false},
+		{"different length", "test", "testing", false},
+		{"first empty", "", "test", false},
+		{"second empty", "test", "", false},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			result := timingSafeMatch(tt.a, tt.b)
+			if result != tt.expected {
+				t.Errorf("timingSafeMatch(%q, %q) = %v, expected %v", tt.a, tt.b, result, tt.expected)
+			}
+		})
+	}
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/backer/types_test.go

@@ -0,0 +1,17 @@
+package backer
+
+import (
+	"testing"
+)
+
+func TestErrDirectoryScanTimeout(t *testing.T) {
+	err := ErrDirectoryScanTimeout{ScannedFiles: 42}
+
+	if err.Error() != "Directory scan timed out" {
+		t.Errorf("Expected error message 'Directory scan timed out', got %q", err.Error())
+	}
+
+	if err.ScannedFiles != 42 {
+		t.Errorf("Expected ScannedFiles=42, got %d", err.ScannedFiles)
+	}
+}

internal/backer/utils_test.go

@@ -8,9 +8,19 @@ import (
 	"os"
 	"path/filepath"
 	"runtime"
+	"strings"
+	"syscall"
 	"testing"
+
+	"github.com/klauspost/compress/zstd"
 )
 
+// mkfifo creates a named pipe (FIFO).
+// Uses syscall.Mkfifo which is available on Unix.
+func mkfifo(path string, mode os.FileMode) error {
+	return syscall.Mkfifo(path, uint32(mode))
+}
+
 // TestCreateTarGzStreamBasic tests basic tar.gz creation with a single file.
 func TestCreateTarGzStreamBasic(t *testing.T) {
 	// Create temp file
@@ -64,6 +74,49 @@ func TestCreateTarGzStreamBasic(t *testing.T) {
 	}
 }
 
+// TestCreateTarGzStreamEmptyFile tests creation of archive with empty file.
+func TestCreateTarGzStreamEmptyFile(t *testing.T) {
+	tmpDir := t.TempDir()
+	emptyFile := filepath.Join(tmpDir, "empty.txt")
+	if err := os.WriteFile(emptyFile, []byte{}, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarGzStream(ctx, []string{emptyFile})
+	defer reader.Close()
+
+	gzReader, err := gzip.NewReader(reader)
+	if err != nil {
+		t.Fatalf("Failed to create gzip reader: %v", err)
+	}
+	defer gzReader.Close()
+
+	tarReader := tar.NewReader(gzReader)
+
+	header, err := tarReader.Next()
+	if err != nil {
+		t.Fatalf("Failed to read tar header: %v", err)
+	}
+
+	if header.Size != 0 {
+		t.Errorf("Expected empty file (size 0), got %d", header.Size)
+	}
+
+	if header.Typeflag != tar.TypeReg {
+		t.Errorf("Expected regular file type, got %c", header.Typeflag)
+	}
+
+	data, err := io.ReadAll(tarReader)
+	if err != nil {
+		t.Fatalf("Failed to read content: %v", err)
+	}
+
+	if len(data) != 0 {
+		t.Errorf("Expected empty content, got %d bytes", len(data))
+	}
+}
+
 // TestCreateTarGzStreamMultipleFiles tests with multiple files.
 func TestCreateTarGzStreamMultipleFiles(t *testing.T) {
 	tmpDir := t.TempDir()
@@ -180,6 +233,82 @@ func TestCreateTarGzStreamSymlink(t *testing.T) {
 	}
 }
 
+// TestCreateTarGzStreamBrokenSymlink tests handling of broken symlinks (target doesn't exist).
+func TestCreateTarGzStreamBrokenSymlink(t *testing.T) {
+	tmpDir := t.TempDir()
+	brokenLink := filepath.Join(tmpDir, "broken.txt")
+
+	if err := os.Symlink("/nonexistent/target", brokenLink); err != nil {
+		t.Skip("Symlinks not supported, skipping test")
+	}
+
+	ctx := context.Background()
+	reader := CreateTarGzStream(ctx, []string{brokenLink})
+	defer reader.Close()
+
+	gzReader, err := gzip.NewReader(reader)
+	if err != nil {
+		t.Fatalf("Failed to create gzip reader: %v", err)
+	}
+	defer gzReader.Close()
+
+	tarReader := tar.NewReader(gzReader)
+
+	header, err := tarReader.Next()
+	if err != nil {
+		t.Fatalf("Failed to read tar header: %v", err)
+	}
+
+	if header.Typeflag != tar.TypeSymlink {
+		t.Errorf("Expected symlink type, got %c", header.Typeflag)
+	}
+
+	// Broken symlinks are still stored - just the link target path
+	if header.Linkname != "/nonexistent/target" {
+		t.Errorf("Expected linkname '/nonexistent/target', got '%s'", header.Linkname)
+	}
+}
+
+// TestCreateTarGzStreamSymlinkToDirectory tests symlink pointing to a directory.
+func TestCreateTarGzStreamSymlinkToDirectory(t *testing.T) {
+	tmpDir := t.TempDir()
+	targetDir := filepath.Join(tmpDir, "targetdir")
+	linkToDir := filepath.Join(tmpDir, "linkdir")
+
+	if err := os.Mkdir(targetDir, 0755); err != nil {
+		t.Fatal(err)
+	}
+
+	if err := os.Symlink(targetDir, linkToDir); err != nil {
+		t.Skip("Symlinks not supported, skipping test")
+	}
+
+	ctx := context.Background()
+	reader := CreateTarGzStream(ctx, []string{linkToDir})
+	defer reader.Close()
+
+	gzReader, err := gzip.NewReader(reader)
+	if err != nil {
+		t.Fatalf("Failed to create gzip reader: %v", err)
+	}
+	defer gzReader.Close()
+
+	tarReader := tar.NewReader(gzReader)
+
+	header, err := tarReader.Next()
+	if err != nil {
+		t.Fatalf("Failed to read tar header: %v", err)
+	}
+
+	if header.Typeflag != tar.TypeSymlink {
+		t.Errorf("Expected symlink type, got %c", header.Typeflag)
+	}
+
+	if header.Linkname != targetDir {
+		t.Errorf("Expected linkname '%s', got '%s'", targetDir, header.Linkname)
+	}
+}
+
 // TestCreateTarGzStreamHardLink tests hard link handling - should deduplicate content.
 func TestCreateTarGzStreamHardLink(t *testing.T) {
 	if runtime.GOOS == "windows" {
@@ -229,9 +358,10 @@ func TestCreateTarGzStreamHardLink(t *testing.T) {
 
 		if header.Typeflag == tar.TypeLink {
 			linkCount++
-			// Verify it's a link to the original
+			// Verify it's a link to the original - code strips leading '/' to avoid tar warnings
-			if header.Linkname != originalFile && header.Linkname != filepath.ToSlash(originalFile) {
+			expectedLinkname := strings.TrimPrefix(filepath.ToSlash(filepath.Clean(originalFile)), "/")
-				t.Errorf("Expected linkname '%s', got '%s'", originalFile, header.Linkname)
+			if header.Linkname != expectedLinkname {
+				t.Errorf("Expected linkname '%s', got '%s'", expectedLinkname, header.Linkname)
 			}
 		}
 	}
@@ -247,6 +377,82 @@ func TestCreateTarGzStreamHardLink(t *testing.T) {
 	}
 }
 
+// TestCreateTarGzStreamHardLinkFromTestData tests hard link handling with test data.
+func TestCreateTarGzStreamHardLinkFromTestData(t *testing.T) {
+	if runtime.GOOS == "windows" {
+		t.Skip("Hard link detection not supported on Windows (inodes unavailable)")
+	}
+
+	testDataPath := filepath.Join("../../test_data", "test1", "hardlinks")
+	filepaths, err := GetFilesFromDirectories(context.Background(), []string{testDataPath})
+	if err != nil {
+		t.Fatalf("Failed to get files from directory: %v", err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarGzStream(ctx, filepaths)
+	defer reader.Close()
+
+	gzReader, err := gzip.NewReader(reader)
+	if err != nil {
+		t.Fatalf("Failed to create gzip reader: %v", err)
+	}
+	defer gzReader.Close()
+
+	tarReader := tar.NewReader(gzReader)
+
+	type entryInfo struct {
+		name      string
+		typeflag byte
+		linkname string
+	}
+	var entries []entryInfo
+
+	for {
+		header, err := tarReader.Next()
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			t.Fatalf("Failed to read tar entry: %v", err)
+		}
+
+		entries = append(entries, entryInfo{
+			name:      header.Name,
+			typeflag:  header.Typeflag,
+			linkname:  header.Linkname,
+		})
+		t.Logf("Entry: name=%s typeflag=%c linkname=%s", header.Name, header.Typeflag, header.Linkname)
+	}
+
+	// Check we have the expected number of entries
+	if len(entries) != 4 {
+		t.Errorf("Expected 4 entries (dir + 1 original + 2 hardlinks), got %d", len(entries))
+	}
+
+	// Find the original file (typeflag = TypeReg = '0') and verify hardlinks point to it
+	var originalName string
+	var hardlinkCount int
+
+	for _, e := range entries {
+		if e.typeflag == tar.TypeReg && e.linkname == "" {
+			originalName = filepath.Base(e.name)
+		}
+		if e.typeflag == tar.TypeLink {
+			hardlinkCount++
+			// Verify link points to the original file's base name
+			linkBase := filepath.Base(e.linkname)
+			if linkBase != originalName && linkBase != "original.txt" {
+				t.Errorf("Hard link %s points to %s, expected %s", filepath.Base(e.name), linkBase, originalName)
+			}
+		}
+	}
+
+	if hardlinkCount != 2 {
+		t.Errorf("Expected 2 hard link entries, got %d", hardlinkCount)
+	}
+}
+
 // TestCreateTarGzStreamCancellation tests context cancellation.
 func TestCreateTarGzStreamCancellation(t *testing.T) {
 	tmpDir := t.TempDir()
@@ -318,6 +524,64 @@ func TestCreateTarGzStreamEmptyList(t *testing.T) {
 	}
 }
 
+// TestCreateTarGzStreamSpecialCharacters tests files with special characters in names.
+func TestCreateTarGzStreamSpecialCharacters(t *testing.T) {
+	tmpDir := t.TempDir()
+
+	testCases := []struct {
+		name    string
+		expect string
+	}{
+		{"file with spaces.txt", "file with spaces.txt"},
+		{"file-with-dashes.txt", "file-with-dashes.txt"},
+		{"file_with_underscores.txt", "file_with_underscores.txt"},
+		{"file.multiple.dots.txt", "file.multiple.dots.txt"},
+	}
+
+	for _, tc := range testCases {
+		path := filepath.Join(tmpDir, tc.name)
+		if err := os.WriteFile(path, []byte(tc.name), 0644); err != nil {
+			t.Fatal(err)
+		}
+	}
+
+	ctx := context.Background()
+	filepaths := make([]string, len(testCases))
+	for i, tc := range testCases {
+		filepaths[i] = filepath.Join(tmpDir, tc.name)
+	}
+
+	reader := CreateTarGzStream(ctx, filepaths)
+	defer reader.Close()
+
+	gzReader, err := gzip.NewReader(reader)
+	if err != nil {
+		t.Fatalf("Failed to create gzip reader: %v", err)
+	}
+	defer gzReader.Close()
+
+	tarReader := tar.NewReader(gzReader)
+
+	found := make(map[string]bool)
+	for {
+		header, err := tarReader.Next()
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			t.Fatalf("Failed to read tar entry: %v", err)
+		}
+
+		found[filepath.Base(header.Name)] = true
+	}
+
+	for _, tc := range testCases {
+		if !found[tc.expect] {
+			t.Errorf("Expected to find '%s' in archive", tc.expect)
+		}
+	}
+}
+
 // TestCreateTarGzStreamNestedFiles tests with nested directory structure.
 func TestCreateTarGzStreamNestedFiles(t *testing.T) {
 	tmpDir := t.TempDir()
@@ -375,7 +639,43 @@ func TestCreateTarGzStreamNestedFiles(t *testing.T) {
 	}
 }
 
-// TestCreateTarGzStreamDeviceFile tests that device files are included in the archive.
+// TestCreateTarGzStreamNamedPipe tests that named pipes (FIFOs) are included in the archive.
+func TestCreateTarGzStreamNamedPipe(t *testing.T) {
+	tmpDir := t.TempDir()
+	namedPipe := filepath.Join(tmpDir, "testpipe")
+
+	if err := mkfifo(namedPipe, 0644); err != nil {
+		t.Skip("Named pipes not supported, skipping test")
+	}
+
+	ctx := context.Background()
+	reader := CreateTarGzStream(ctx, []string{namedPipe})
+	defer reader.Close()
+
+	gzReader, err := gzip.NewReader(reader)
+	if err != nil {
+		t.Fatalf("Failed to create gzip reader: %v", err)
+	}
+	defer gzReader.Close()
+
+	tarReader := tar.NewReader(gzReader)
+
+	header, err := tarReader.Next()
+	if err != nil {
+		t.Fatalf("Failed to read tar header: %v", err)
+	}
+
+	if header.Typeflag != tar.TypeFifo {
+		t.Errorf("Expected FIFO type, got %c", header.Typeflag)
+	}
+
+	// Ensure there are no more entries
+	if _, err := tarReader.Next(); err != io.EOF {
+		t.Error("Expected EOF after FIFO entry")
+	}
+}
+
+// TestCreateTarGzStreamDeviceFile tests that device files (char and block) are included in the archive.
 func TestCreateTarGzStreamDeviceFile(t *testing.T) {
 	// Use /dev/null as a test device file (character device).
 	devNull := "/dev/null"
@@ -383,8 +683,15 @@ func TestCreateTarGzStreamDeviceFile(t *testing.T) {
 	if err != nil {
 		t.Skipf("Skipping device test: %v", err)
 	}
-	if st.Mode()&os.ModeCharDevice == 0 {
+
-		t.Skipf("Skipping: %s is not a character device", devNull)
+	var expectedTypeflag byte
+	switch {
+	case st.Mode()&os.ModeCharDevice != 0:
+		expectedTypeflag = tar.TypeChar
+	case st.Mode()&os.ModeDevice != 0:
+		expectedTypeflag = tar.TypeBlock
+	default:
+		t.Skipf("Skipping: %s is not a device file", devNull)
 	}
 
 	ctx := context.Background()
@@ -399,23 +706,13 @@ func TestCreateTarGzStreamDeviceFile(t *testing.T) {
 
 	tarReader := tar.NewReader(gzReader)
 
-	// Read the device entry
 	header, err := tarReader.Next()
 	if err != nil {
 		t.Fatalf("Failed to read tar header: %v", err)
 	}
 
-	if header.Typeflag != tar.TypeChar {
+	if header.Typeflag != expectedTypeflag {
-		t.Errorf("Expected character device type, got %c", header.Typeflag)
+		t.Errorf("Expected device type %c, got %c", expectedTypeflag, header.Typeflag)
-	}
-
-	// Check that the name matches the file path (without leading slash)
-	expectedName := devNull
-	if expectedName[0] == '/' {
-		expectedName = expectedName[1:]
-	}
-	if header.Name != expectedName {
-		t.Errorf("Expected header name %q, got %q", expectedName, header.Name)
 	}
 
 	// Ensure there are no more entries
@@ -564,4 +861,115 @@ func TestCreateTarXzStream(t *testing.T) {
 	}
 }
 
+// TestCreateTarPgzipStream tests pgzip (parallel gzip) compression.
+func TestCreateTarPgzipStream(t *testing.T) {
+	C = Config{CompressionLevel: 6}
+
+	tmpDir := t.TempDir()
+	testFile := filepath.Join(tmpDir, "test.txt")
+	content := []byte("test content for pgzip")
+	if err := os.WriteFile(testFile, content, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarPgzipStream(ctx, []string{testFile})
+	defer reader.Close()
+
+	gzReader, err := gzip.NewReader(reader)
+	if err != nil {
+		t.Fatalf("Failed to create gzip reader: %v", err)
+	}
+	defer gzReader.Close()
+
+	tarReader := tar.NewReader(gzReader)
+
+	header, err := tarReader.Next()
+	if err != nil {
+		t.Fatalf("Failed to read tar header: %v", err)
+	}
+
+	if header.Name != testFile && filepath.Base(header.Name) != "test.txt" {
+		t.Errorf("Expected file name 'test.txt', got '%s'", header.Name)
+	}
+
+	data, err := io.ReadAll(tarReader)
+	if err != nil {
+		t.Fatalf("Failed to read file content: %v", err)
+	}
+
+	if string(data) != string(content) {
+		t.Errorf("Expected content '%s', got '%s'", content, data)
+	}
+}
+
+// TestCreateTarZstdStream tests zstd compression.
+func TestCreateTarZstdStream(t *testing.T) {
+	C = Config{CompressionLevel: 6}
+
+	tmpDir := t.TempDir()
+	testFile := filepath.Join(tmpDir, "test.txt")
+	content := []byte("test content for zstd")
+	if err := os.WriteFile(testFile, content, 0644); err != nil {
+		t.Fatal(err)
+	}
+
+	ctx := context.Background()
+	reader := CreateTarZstdStream(ctx, []string{testFile})
+	defer reader.Close()
+
+	// zstd.Reader is in github.com/klauspost/compress/zstd
+	zstdReader, err := zstd.NewReader(reader)
+	if err != nil {
+		t.Fatalf("Failed to create zstd reader: %v", err)
+	}
+	defer zstdReader.Close()
+
+	tarReader := tar.NewReader(zstdReader)
+
+	header, err := tarReader.Next()
+	if err != nil {
+		t.Fatalf("Failed to read tar header: %v", err)
+	}
+
+	if header.Name != testFile && filepath.Base(header.Name) != "test.txt" {
+		t.Errorf("Expected file name 'test.txt', got '%s'", header.Name)
+	}
+
+	data, err := io.ReadAll(tarReader)
+	if err != nil {
+		t.Fatalf("Failed to read file content: %v", err)
+	}
+
+	if string(data) != string(content) {
+		t.Errorf("Expected content '%s', got '%s'", content, data)
+	}
+}
+
+// TestZstdLevel tests zstd level mapping.
+func TestZstdLevel(t *testing.T) {
+	tests := []struct {
+		input    int
+		expected zstd.EncoderLevel
+	}{
+		{1, zstd.SpeedFastest},
+		{2, zstd.SpeedFastest},
+		{3, zstd.SpeedFastest},
+		{4, zstd.SpeedDefault},
+		{5, zstd.SpeedDefault},
+		{6, zstd.SpeedDefault},
+		{7, zstd.SpeedBetterCompression},
+		{8, zstd.SpeedBetterCompression},
+		{9, zstd.SpeedBetterCompression},
+	}
+
+	for _, tt := range tests {
+		C = Config{CompressionLevel: tt.input}
+		result := zstdLevel(tt.input)
+		if result != tt.expected {
+			t.Errorf("zstdLevel(%d) = %v, expected %v", tt.input, result, tt.expected)
+		}
+	}
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

internal/log/main.go

@@ -2,7 +2,6 @@
 package log
 
 import (
-	"context"
 	"fmt"
 	"log"
 	"log/slog"
@@ -27,11 +26,9 @@ var (
 	logLevel slog.Level
 )
 
-// slogWriter implements [io.Writer] to redirect standard log to slog.
 type (
-	slogWriter struct {
+	// slogWriter implements [io.Writer] to redirect standard log to slog.
-		handler slog.Handler
+	slogWriter struct{}
-	}
 )
 
 // Init sets up logger stuff.
@@ -154,9 +151,9 @@ func Debugf(format string, a ...any) {
 	slog.Debug(fmt.Sprintf(format, a...))
 }
 
-// DebugLogger returns a standard library logger that writes at the configured log level.
+// DebugLogger returns a standard library logger that writes at debug/warn level.
-// This is useful for assigning to http.Server.ErrorLog to route net/http errors
+// TLS-related errors are logged at debug level, others at warn level.
-// through our logger while respecting the configured log level.
+// Both are filtered by the configured log level: set loglevel to debug to see all messages.
 func DebugLogger() *log.Logger {
 	mu.RLock()
 	defer mu.RUnlock()
@@ -167,7 +164,9 @@ func DebugLogger() *log.Logger {
 
 	handler := slog.NewTextHandler(Log, opts)
 
-	return log.New(&slogWriter{handler: handler}, "", 0)
+	_ = handler // satisfy linter (handler configures level for potential future use).
+
+	return log.New(&slogWriter{}, "", 0)
 }
 
 // Write implements [io.Writer]. It differentiates TLS/SSL errors from other errors.
@@ -183,15 +182,12 @@ func (w *slogWriter) Write(p []byte) (int, error) {
 		strings.Contains(msg, "handshake") ||
 		strings.Contains(msg, "certificate")
 
-	level := slog.LevelWarn
 	if isTLSError {
-		level = slog.LevelDebug
+		Debug(msg)
+	} else {
+		Warn(msg)
 	}
 
-	record := slog.NewRecord(time.Now(), level, msg, 0)
-
-	_ = w.handler.Handle(context.TODO(), record)
-
 	return len(p), nil
 }
 

internal/log/main_test.go

@@ -363,7 +363,7 @@ func TestDebugLogger(t *testing.T) {
 	logFilePath := tmpFile.Name()
 	tmpFile.Close()
 
-	if err := Init("info", logFilePath); err != nil {
+	if err := Init("debug", logFilePath); err != nil {
 		t.Fatal(err)
 	}
 	defer Close()
@@ -385,4 +385,85 @@ func TestDebugLogger(t *testing.T) {
 	}
 }
 
+// TestDebugLoggerTLS tests that TLS errors are logged at debug level.
+func TestDebugLoggerTLS(t *testing.T) {
+	tmpFile, err := os.CreateTemp(t.TempDir(), "test_log_*.log")
+	if err != nil {
+		t.Fatal(err)
+	}
+	logFilePath := tmpFile.Name()
+	tmpFile.Close()
+
+	if err := Init("debug", logFilePath); err != nil {
+		t.Fatal(err)
+	}
+	defer Close()
+
+	logger := DebugLogger()
+	logger.Print("tls handshake timeout")
+
+	content, err := os.ReadFile(logFilePath)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if !strings.Contains(string(content), "tls handshake timeout") {
+		t.Error("Expected TLS message in log")
+	}
+}
+
+// TestDebugLoggerNonTLS tests that non-TLS errors are logged at warn level.
+func TestDebugLoggerNonTLS(t *testing.T) {
+	tmpFile, err := os.CreateTemp(t.TempDir(), "test_log_*.log")
+	if err != nil {
+		t.Fatal(err)
+	}
+	logFilePath := tmpFile.Name()
+	tmpFile.Close()
+
+	if err := Init("warn", logFilePath); err != nil {
+		t.Fatal(err)
+	}
+	defer Close()
+
+	logger := DebugLogger()
+	logger.Print("http: connection reset by peer")
+
+	content, err := os.ReadFile(logFilePath)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if !strings.Contains(string(content), "http: connection reset by peer") {
+		t.Error("Expected non-TLS message in log")
+	}
+}
+
+// TestDebugLoggerLevelFiltering tests that log level filtering works.
+func TestDebugLoggerLevelFiltering(t *testing.T) {
+	tmpFile, err := os.CreateTemp(t.TempDir(), "test_log_*.log")
+	if err != nil {
+		t.Fatal(err)
+	}
+	logFilePath := tmpFile.Name()
+	tmpFile.Close()
+
+	if err := Init("info", logFilePath); err != nil {
+		t.Fatal(err)
+	}
+	defer Close()
+
+	logger := DebugLogger()
+	logger.Print("tls handshake timeout") // TLS error at debug level - should be filtered
+
+	content, err := os.ReadFile(logFilePath)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if strings.Contains(string(content), "tls handshake timeout") {
+		t.Error("Debug-level TLS message should be filtered at info log level")
+	}
+}
+
 /* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */

llms.txt

@@ -30,13 +30,13 @@ curl -u user:pass https://server:8086/archive -o backup.tar.gz
 │   │   ├── globals.go       # Global C Config variable
 │   │   ├── archive.go       # Common archive logic (createArchiveStream, writeFilesToTar)
 │   │   ├── gzip.go          # CreateTarGzStream, CreateTarPgzipStream
-│   │   ├── bzip2.go         # CreateTarBzip2Stream
+│   │   ├── bzip2.go         # CreateTarBzip2Stream, mapCompressionLevelToBzip2
-│   │   ├── zstd.go          # CreateTarZstdStream
+│   │   ├── zstd.go          # CreateTarZstdStream, zstdLevel
-│   │   ├── lz4.go           # CreateTarLz4Stream
+│   │   ├── lz4.go           # CreateTarLz4Stream, mapCompressionLevelToLz4
 │   │   ├── xz.go            # CreateTarXzStream
 │   │   ├── loadConfig.go    # HJSON/JSON config parsing with validation
 │   │   ├── newServer.go     # HTTP server, auth handler, context helpers
-│   │   ├── getFilesFromDirectories.go  # Directory walking, exclusion filtering
+│   │   ├── getFilesFromDirectories.go  # Directory walking, exclusion filtering, isExcluded
 │   │   └── *_test.go        # Test files
 │   └── log/main.go          # Logging wrapper around stdlib slog
 ├── contrib/                 # Init scripts
@@ -55,7 +55,7 @@ curl -u user:pass https://server:8086/archive -o backup.tar.gz
 |--------|---------|-------------|
 | address | 0.0.0.0 | Listen address |
 | port | 8086 | Listen port |
-| cert/key | - | TLS cert/key paths |
+| cert/key | - | TLS cert/key paths (required when nohttps=false) |
 | nohttps | false | Disable HTTPS |
 | location | /archive | API endpoint path |
 | user | - | Basic auth username (required) |
@@ -64,10 +64,23 @@ curl -u user:pass https://server:8086/archive -o backup.tar.gz
 | loglevel | info | Log verbosity (debug/info/warn/error) |
 | directories | - | Directories to backup (required) |
 | backup_timeout | 60 | Stream timeout in minutes (1-1440) |
-| compression_level | 9 | Gzip level (1-9) |
+| dir_scan_timeout | 60 | Directory scan timeout in minutes (1-1440) |
+| compression_level | 9 | Compression level (1-9) |
 | exclude_patterns | [] | Regex patterns to exclude from backup |
 | filename_prefix | backup | Prefix for backup filename in Content-Disposition header |
-| compression_algorithm | gzip | Compression algorithm (gzip, pgzip, bzip2, zstd, lz4 or xz) |
+| default_compression | gzip | Default compression algorithm (gzip, pgzip, bzip2, zstd, lz4, xz) |
+| compression_algorithm | gzip | Alias for default_compression (deprecated) |
+
+## Compression Level Mapping
+
+The `compression_level` config option (1-9) is mapped differently for each algorithm:
+
+| Algorithm | Level Mapping |
+|-----------|---------------|
+| gzip/pgzip/bzip2 | 1-9 directly maps to compression levels |
+| lz4 | 1-3 → Fastest (0), 4-6 → Default (1), 7-9 → Best (2) |
+| zstd | 1-3 → Fastest, 4-6 → Default, 7-9 → BetterCompression |
+| xz | Uses default compression (no configurable level) |
 
 ## Key Patterns
 
@@ -85,35 +98,86 @@ json.Marshal(tmp)
 json.Unmarshal(buf, &C)
 ```
 
-### Stream Cancellation
+### Context-Aware I/O
-Context-aware write and copy functions:
+Two helper functions in newServer.go handle context cancellation during streaming:
+- `writeWithContext(ctx, fn)` — checks context before calling fn
+- `copyWithContext(ctx, dst, src)` — buffered copy (32KB) with cancellation checks, computes MD5 hash
+
 ```go
 func writeWithContext(ctx context.Context, fn func() error) error
-func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) error
+func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) (int64, string, error)
 ```
 
+### HTTP Server Middleware
+Two middleware functions wrap the handler:
+- `withServerHeader` — adds "Server: backer" header to all responses
+- `withStatusLogging` — logs 404 and 405 responses
+
+```go
+Handler: withServerHeader(withStatusLogging(mux))
+```
+
+### Unmatched Route Logging
+The server wraps http.ResponseWriter to capture status codes:
+- **404 Not Found**: `"Not Found from {clientIP} to {path}"`
+- **405 Method Not Allowed**: `"Method Not Allowed from {clientIP} for {method} {path}"`
+
+Uses `responseWriterWrapper` that tracks status code via WriteHeader() method.
+
 ### Pipe Error Handling
 Uses `pipeErr` tracking to avoid noisy errors when reader closes early.
 Skips files that can't be read (logs warning, continues backup).
+- `isPipeClosedError()` checks for "io: read/write on closed pipe" errors
 
 ### Content-Disposition Filename
 Backup filenames include timestamp and configured prefix: `backup-20060102-150405.tar.gz` (extension depends on compression_algorithm).
 
 ### Special File Types
-- **Device files** (character/block) are included with correct major/minor numbers.
+- **Regular files**: streamed into tar archive with content
-- **Sockets** are skipped (no entry created).
+- **Directories**: included as tar directory entries
-- **Symlinks** are stored as symlinks (target path stored).
+- **Symlinks**: stored as symlink entries (target path preserved) - uses os.Lstat, no cycles possible
-- **Named pipes** and other special files are included (tar header created).
+- **Broken symlinks**: archived with their target path (even if target doesn't exist)
-- **Regular files** are compressed; other file types have no data section.
+- **Device files** (char/block): included with correct major/minor numbers
+- **Named pipes (FIFOs)**: included (header only, no data)
+- **Sockets**: skipped (not archivable)
+- **Hard links**: deduplicated - first occurrence stored with content, subsequent as link entries
+- **GNU tar format**: uses `header.Format = tar.FormatGNU` for GNU tar compatibility
+
+### Test Coverage
+All entity types are tested: regular files, empty files, directories, symlinks, broken symlinks, symlinks to directories, hard links, device files, named pipes (FIFOs), special characters in filenames.
 
 ### Error Recovery
 - `writeWithContext` checks context before calling fn.
 - `copyWithContext` does buffered copy with cancellation checks.
 
+### Client IP Detection
+Checks `X-Forwarded-For` header first (for proxied requests), then falls back to `RemoteAddr`.
+
+### TLS Error Handling
+Uses `ServerWrapper` to intercept and classify errors from Serve() and ServeTLS():
+- TLS-specific errors logged at debug level to reduce log clutter
+- Uses errors.As to detect: tls.AlertError, tls.CertificateVerificationError, tls.ECHRejectionError, tls.RecordHeaderError
+
+### HTTP Routing & Compression Selection
+The server uses http.NewServeMux() for routing. Multiple routes support different compression formats:
+```go
+mux.Handle(C.Location, http.HandlerFunc(backupHandler))           // /archive
+mux.Handle(C.Location+".tar.gz", http.HandlerFunc(backupHandler))  // archive.tar.gz
+mux.Handle(C.Location+".tar.xz", http.HandlerFunc(backupHandler))   // archive.tar.xz
+mux.Handle(C.Location+".tar.bz2", http.HandlerFunc(backupHandler)) // archive.tar.bz2
+// ... etc (.tar.lz4, .tar.zst, .tar.zstd)
+```
+
+Compression algorithm determined by path extension using `path.Ext()`:
+- Full path: `/archive.tar.xz` → `.xz` → xz
+- Bare extension: `tar.xz` → `.xz` → xz
+- Default when no extension matches
+
 ### Performance Notes
 - Exclude patterns are precompiled after config load for faster matching.
 - Compression level is configurable (1-9); default 9 uses more CPU.
 - Large directory walks use `filepath.WalkDir` (efficient).
+- Context cancellation checked on each file during directory scan.
 
 ## Coding Style & Conventions
 
@@ -128,7 +192,8 @@ GetFilesFromDirectories()
 // Unexported (private): verbNoun or isNoun pattern
 writeWithContext()
 copyWithContext()
-isPipeClosedError()  // predicate helper
+isPipeClosedError()
+isExcluded()
 defaultConfigPath()
 ```
 
@@ -137,6 +202,7 @@ Struct fields use PascalCase, json tags use snake_case:
 ```go
 BackupTimeout     int `json:"backup_timeout"`
 CompressionLevel  int `json:"compression_level"`
+DirScanTimeout    int `json:"dir_scan_timeout"`
 ```
 
 ### Error Messages
@@ -149,6 +215,7 @@ CompressionLevel  int `json:"compression_level"`
 - Past tense: `"Failed to close gzip writer: %v", err`
 - Context included: `"Skipping %s: %v", filename, err`
 - Appropriate level: Warn for recoverable, Error for issues, Fatal for startup failures
+- Log levels: error, warn, info, debug
 
 ### Config Validation Pattern
 For each config option: set default first, then immediately validate:
@@ -166,7 +233,7 @@ if C.Port < 1 || C.Port > 65535 {
 
 ### Go Formatting
 - Tabs for indentation
-- Vim modeline at EOF: `/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */`
+- Vim modeline at EOF: `/* vim: setlocal ft=go noet ai ts=4 sw=4 sts=4: */
 
 ### Linter Suppressions (intentional)
 | Rule | Reason |
@@ -181,24 +248,38 @@ if C.Port < 1 || C.Port > 65535 {
 
 ## Testing
 ```bash
-go test ./...                    # Run tests
+go test ./...                    # Run tests (runs make clean first)
 go test -v -run TestName ./...   # Run specific test
 golangci-lint run ./...          # Lint
+make test                        # Clean + test (deterministic)
 ```
 
-Coverage: ~69% on internal/backer (main and log packages not covered by design)
+Coverage: ~64% on internal/backer (~100% on internal/log)
+
+### Test Data
+All test data is created by `TestMain` in `internal/backer/main_test.go` before tests run and cleaned up afterward. This ensures tests are independent of pre-existing files.
+
+- `test_data/test1/foo/` — test directory with files, subdirs, empty_dir
+- `test_data/test1/bar/` — test directory with files and a directory named `goodbye.txt/` (edge case: directory with file extension)
+- `test_data/test1/hardlinks/` — test directory with hard-linked files (original.txt, hardlink1.txt, hardlink2.txt)
+- `test_data/test1/symlinks/` — test directory with symbolic links
+- `test_data/test1/empty_dir/` — empty directory for testing
+- `test_data/tmp/` — reserved for test temporary files (use instead of system `/tmp`)
 
 ### Test Conventions
 - Reset global config: `C = Config{}`
+- Reset exclude patterns: `excludePatterns = nil`
 - Use `t.TempDir()` for temporary files
 - Helper function `writeTempConfig()` for config tests
+- Test data created by TestMain setup, cleaned after tests
+- Always check errors with meaningful diagnostics (use `t.Fatalf` for setup errors)
 
 ## Build
 ```bash
-make           # Build binary (version from git or "dev")
+make           # Clean + build binary (version from git or "dev")
 make build VERSION=0.1.0  # Build with specific version
-make clean     # Remove binary
+make clean     # Remove binary, test_data/tmp, test_data/test1
-make test      # Run tests
+make test      # Clean + test (deterministic)
 make upgrade   # Update dependencies
 make help      # Show targets
 ```
@@ -208,3 +289,20 @@ Version is injected at build time via ldflags:
 ```go
 var version = "dev"  // overridden by: -X main.version=$(VERSION)
 ```
+
+## Request Flow (Good Path)
+```
+Request → Method Check (GET) → Auth Check → Get Files (with timeout) → Create Archive → Stream → Complete
+```
+
+## Request Flow (Error Paths)
+| Request | Response | Logged |
+|---------|----------|--------|
+| Non-GET method | 405 Method Not Allowed | Yes (info) |
+| No auth | 401 Unauthorized | Yes (info) |
+| Wrong credentials | 401 Unauthorized | Yes (info) |
+| Unknown path | 404 Not Found | Yes (info) |
+| Dir scan timeout | 504 Request Timeout | Yes (error) |
+| Failed to get files | 500 Internal Server Error | Yes (error) |
+| Client disconnect | - | Yes (info) |
+| Backup completed | 200 OK | Yes (info) |

test_data/test1/bar/goodbye.txt/text.txt

@@ -1 +0,0 @@
-Fucked up beyond recognotion.

test_data/test1/foo/hello_breakfast.txt

@@ -1,2 +0,0 @@
-My dear Tweety,
-I'd like to greet you as my breakfast.

test_data/test1/foo/mydir/myfile.txt

@@ -1 +0,0 @@
-This is MY file!

test_data/test_config.json

@@ -5,10 +5,10 @@
     /* By default listen on port 8086, like first intel commercially successful cpu model.                            */
     "port": 8086,
 
-    /* Path location of certificate file                                                                              */
+    /* Path location of test certificate file                                                                       */
     "cert": "../../test_data/example.crt",
 
-    /* Path location of certificate key                                                                               */
+    /* Path location of test certificate key                                                                          */
     "key": "../../test_data/example.key",
 
     /* We really want to use https for backup archive tranferring.                                                    */