goc_marek/zfs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
fb9bb2fc82eed7c12e1daa51b1de686f0a9ebd14
zfs/readme.md
Marek Goc fb9bb2fc82 fix
2026-02-13 18:38:55 +01:00

12 KiB
Raw Blame History

ZFS Snapshot Manager

A distributed ZFS snapshot management system with S3-compatible storage support. This project provides client, server, and restore tools for managing ZFS snapshots across multiple machines.

Features

  • S3 Storage Support: Store snapshots in any S3-compatible storage (AWS S3, MinIO, Backblaze B2, Wasabi, DigitalOcean Spaces)
  • Local ZFS Storage: Option to use local ZFS datasets for maximum performance
  • Multi-client Architecture: Support for multiple clients with isolated storage and per-client quotas
  • Automatic Compression: Gzip compression for reduced storage costs
  • Snapshot Rotation: Automatic cleanup of old snapshots based on quota
  • Server-Managed Rotation Policies: Centralized control of client rotation policies - clients must use server-configured retention settings
  • API Key Authentication: Secure client-server communication

Project Structure

zfs/
├── cmd/
│   ├── zfs-server/     # Server executable
│   ├── zfs-client/     # Client executable
│   └── zfs-restore/    # Restore tool executable
├── internal/
│   ├── server/         # Server package (config, storage, HTTP handlers)
│   ├── client/         # Client package (snapshot creation, upload)
│   └── restore/        # Restore package (download, restore operations)
├── go.mod
├── go.sum
├── .env                # Configuration file
└── readme.md

Installation

Using Go Install

# Install server
go install git.ma-al.com/goc_marek/zfs/cmd/zfs-server@latest

# Install client
go install git.ma-al.com/goc_marek/zfs/cmd/zfs-client@latest

# Install restore tool
go install git.ma-al.com/goc_marek/zfs/cmd/zfs-restore@latest

Build from Source

# Clone the repository
git clone https://git.ma-al.com/goc_marek/zfs.git
cd zfs

# Build all binaries
go build -o bin/zfs-server ./cmd/zfs-server
go build -o bin/zfs-client ./cmd/zfs-client
go build -o bin/zfs-restore ./cmd/zfs-restore

Configuration

Server Configuration

Create a .env file in the working directory:

# S3 Configuration
S3_ENABLED=true
S3_ENDPOINT=s3.amazonaws.com
S3_ACCESS_KEY=YOUR_ACCESS_KEY
S3_SECRET_KEY=YOUR_SECRET_KEY
S3_BUCKET=zfs-snapshots
S3_USE_SSL=true

# Local ZFS fallback
ZFS_BASE_DATASET=backup

# Server settings
CONFIG_FILE=clients.json
METADATA_FILE=metadata.json
PORT=8080

Client Configuration

CLIENT_ID=client1
API_KEY=secret123
SERVER_URL=http://backup-server:8080
LOCAL_DATASET=tank/data
COMPRESS=true
STORAGE_TYPE=s3

Important

: The API_KEY in the client .env file must be the raw (unhashed) key. The server stores the SHA-256 hash in clients.json, and the client sends the raw key which the server then hashes for comparison. For example, if clients.json has api_key: "fcf730b6d95236ecd3c9fc2d92d7b6b2bb061514961aec041d6c7a7192f592e4" (hash of "secret123"), the client .env should have API_KEY=secret123.

Restore Tool Configuration

CLIENT_ID=client1
API_KEY=secret123
SERVER_URL=http://backup-server:8080

Usage

Server

# Start the backup server
zfs-server

# The server listens on port 8080 by default
# Endpoints:
#   POST /upload           - Request upload authorization
#   POST /upload-stream/   - Stream snapshot data
#   GET  /status           - Check client status
#   POST /rotate           - Rotate old snapshots
#   GET  /download         - Download a snapshot
#   GET  /rotation-policy  - Get client rotation policy
#   GET  /health           - Health check

Client Commands

The zfs-client tool provides the following commands for managing ZFS snapshots:

backup

Creates a snapshot and sends it to the server. Automatically uses incremental backup if a bookmark exists.

zfs-client backup

backup-full

Forces a full backup (no incremental). Use for the initial backup or when you want to resend the complete dataset.

zfs-client backup-full

backup-incremental

Creates an incremental backup from the last bookmark. Requires an existing bookmark from a previous full backup.

zfs-client backup-incremental

snapshot <type>

Creates a typed snapshot (hourly, daily, weekly, monthly) with automatic rotation. The rotation policy is fetched from the server if configured.

zfs-client snapshot hourly
zfs-client snapshot daily
zfs-client snapshot weekly
zfs-client snapshot monthly

rotate

Rotates local snapshots based on the retention policy. If the server has a rotation policy configured, it will be used; otherwise, default values apply.

zfs-client rotate

rotate-remote

Requests the server to rotate (delete old) remote snapshots to free up storage quota.

zfs-client rotate-remote

status

Displays the current backup status including storage usage, quota, and snapshot count from the server.

zfs-client status

bookmarks

Lists ZFS bookmarks on the local system. Bookmarks are used as reference points for incremental backups.

zfs-client bookmarks

help

Shows the help message with all available commands and options.

zfs-client help

Restore Tool Commands

The zfs-restore tool provides commands for listing and restoring snapshots from the backup server:

list

Lists all available snapshots for the configured client from the server.

zfs-restore list

Output example:

#  Snapshot ID                    Timestamp           Size
1  client1/tank_data_2024-02-13   2024-02-13 14:30    1.2 GB
2  client1/tank_data_2024-02-12   2024-02-12 14:30    1.1 GB

restore <number> <dataset>

Restores a snapshot by its list number to a specified ZFS dataset.

zfs-restore restore 1 tank/restored

Options:

  • --force or -f - Overwrite existing dataset if it exists
zfs-restore restore 1 tank/restored --force

latest <dataset>

Restores the most recent snapshot to a specified dataset.

zfs-restore latest tank/restored

save <number> <filename>

Downloads a snapshot and saves it to a local file without restoring.

zfs-restore save 1 backup.zfs.gz

mount <dataset> <mountpoint>

Mounts a restored ZFS dataset to a specified directory for file access.

zfs-restore mount tank/restored /mnt/restore

S3 Provider Configuration

AWS S3

S3_ENDPOINT=s3.amazonaws.com
S3_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE
S3_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
S3_BUCKET=my-zfs-backups
S3_USE_SSL=true

MinIO (Self-Hosted)

S3_ENDPOINT=minio.example.com:9000
S3_ACCESS_KEY=minioadmin
S3_SECRET_KEY=minioadmin
S3_BUCKET=zfs-snapshots
S3_USE_SSL=false

Setting Up MinIO Locally

Option A: Using Docker (Recommended)

# Create a directory for MinIO data
mkdir -p ~/minio-data

# Run MinIO container
docker run -d \
  --name minio \
  -p 9000:9000 \
  -p 9001:9001 \
  -v ~/minio-data:/data \
  -e MINIO_ROOT_USER=minioadmin \
  -e MINIO_ROOT_PASSWORD=minioadmin \
  minio/minio server /data --console-address ":9001"

Option B: Using Binary

# Download MinIO
wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
sudo mv minio /usr/local/bin/

# Create data directory
mkdir -p ~/minio-data

# Start MinIO
MINIO_ROOT_USER=minioadmin MINIO_ROOT_PASSWORD=minioadmin \
  minio server ~/minio-data --console-address ":9001"

Create the Bucket

After starting MinIO, create the bucket using the MinIO Client (mc) or web console:

# Install MinIO Client (mc)
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
sudo mv mc /usr/local/bin/

# Configure alias to local MinIO
mc alias set local http://localhost:9000 minioadmin minioadmin

# Create bucket
mc mb local/zfs

# Verify bucket was created
mc ls local

Alternatively, access the MinIO Web Console at http://localhost:9001 and create the bucket through the UI (login: minioadmin / minioadmin).

Backblaze B2

S3_ENDPOINT=s3.us-west-000.backblazeb2.com
S3_ACCESS_KEY=your_key_id
S3_SECRET_KEY=your_application_key
S3_BUCKET=zfs-backups
S3_USE_SSL=true

Wasabi

S3_ENDPOINT=s3.wasabisys.com
S3_ACCESS_KEY=your_access_key
S3_SECRET_KEY=your_secret_key
S3_BUCKET=zfs-backups
S3_USE_SSL=true

Client Configuration File

The server maintains a clients.json file with client configurations:

[
  {
    "client_id": "client1",
    "api_key": "hashed_key",
    "max_size_bytes": 107374182400,
    "dataset": "backup/client1",
    "enabled": true,
    "storage_type": "s3",
    "rotation_policy": {
      "keep_hourly": 24,
      "keep_daily": 7,
      "keep_weekly": 4,
      "keep_monthly": 12
    }
  }
]

Server-Managed Rotation Policy

When rotation_policy is configured for a client in clients.json, the client must use this policy and cannot override it. This enables centralized control of snapshot retention policies:

  • Server-Managed: If rotation_policy is set, the client fetches the policy from the server and applies it
  • Client-Autonomous: If no rotation_policy is set, the client uses its default policy

The rotation policy fields are:

  • keep_hourly: Number of hourly snapshots to keep (default: 24)
  • keep_daily: Number of daily snapshots to keep (default: 7)
  • keep_weekly: Number of weekly snapshots to keep (default: 4)
  • keep_monthly: Number of monthly snapshots to keep (default: 12)

API Endpoint

The server exposes a /rotation-policy endpoint for clients to fetch their configured policy:

GET /rotation-policy?client_id=client1&api_key=secret123

Response:

{
  "success": true,
  "message": "Server-managed rotation policy",
  "rotation_policy": {
    "keep_hourly": 24,
    "keep_daily": 7,
    "keep_weekly": 4,
    "keep_monthly": 12
  },
  "server_managed": true
}

Architecture

┌─────────────┐    ZFS send      ┌──────────────────┐
│  Client 1   │───────┬─────────▶│  Backup Server   │
│ (S3 mode)   │       │          │                  │
└─────────────┘       │          │  ┌────────────┐  │
                      │          │  │ S3 Backend │  │
┌─────────────┐       │  HTTP    │  └─────┬──────┘  │
│  Client 2   │───────┤  Stream  │        │         │
│ (S3 mode)   │       │          │        ▼         │
└─────────────┘       │          │  ┌────────────┐  │
                      │          │  │   MinIO    │  │
┌─────────────┐       │          │  │     or     │  │
│  Client 3   │───────┘          │  │   AWS S3   │  │
│ (Local ZFS) │─────────────────▶│  └────────────┘  │
└─────────────┘   ZFS recv       │                  │
                                 │  ┌────────────┐  │
                                 │  │  Local ZFS │  │
                                 │  │   Backend  │  │
                                 │  └────────────┘  │
                                 └──────────────────┘

Storage Format

Snapshots are stored in S3 with the following naming convention:

s3://bucket/client1/tank_data_2024-02-13_14:30:00.zfs.gz
                ^         ^           ^
              client   dataset    timestamp

Security

  • API keys are hashed using SHA-256
  • S3 bucket policies can restrict access to backup server only
  • Server-side encryption available in S3
  • Client-side encryption possible via custom compression pipeline

Monitoring

Health Check

curl http://localhost:8080/health

Server Logs

# SystemD
journalctl -u zfs-server -f

# Docker
docker logs -f zfs-server

Development

Project Layout

  • cmd/ - Main applications (entry points)
  • internal/ - Private application code
    • server/ - Server logic, HTTP handlers, storage backends
    • client/ - Client logic for creating and uploading snapshots
    • restore/ - Restore logic for downloading and restoring snapshots

Building

# Build all
go build ./...

# Run tests
go test ./...

# Lint
go vet ./...

License

MIT License

Reference in New Issue View Git Blame Copy Permalink