simplyfy
This commit is contained in:
83
readme.md
83
readme.md
@@ -4,13 +4,14 @@ A distributed ZFS snapshot management system with S3-compatible storage support.
|
||||
|
||||
## Features
|
||||
|
||||
- **S3 Storage Support**: Store snapshots in any S3-compatible storage (AWS S3, MinIO, Backblaze B2, Wasabi, DigitalOcean Spaces)
|
||||
- **S3 Storage Support**: Store snapshots in any S3-compatible storage using AWS SDK v2 (AWS S3, MinIO, Backblaze B2, Wasabi, DigitalOcean Spaces)
|
||||
- **Local ZFS Storage**: Option to use local ZFS datasets for maximum performance
|
||||
- **Multi-client Architecture**: Support for multiple clients with isolated storage and per-client quotas
|
||||
- **Automatic Compression**: Gzip compression for reduced storage costs
|
||||
- **Automatic Compression**: LZ4 compression for reduced storage costs and faster transfers
|
||||
- **Snapshot Rotation**: Automatic cleanup of old snapshots based on quota
|
||||
- **Server-Managed Rotation Policies**: Centralized control of client rotation policies - clients must use server-configured retention settings
|
||||
- **API Key Authentication**: Secure client-server communication
|
||||
- **Simple CLI**: Just use `zfs-client snap` to backup - automatically handles full/incremental
|
||||
|
||||
## Project Structure
|
||||
|
||||
@@ -93,11 +94,19 @@ API_KEY=secret123
|
||||
SERVER_URL=http://backup-server:8080
|
||||
LOCAL_DATASET=tank/data
|
||||
COMPRESS=true
|
||||
|
||||
# Optional: Direct S3 upload (bypasses server storage)
|
||||
S3_ENDPOINT=https://s3.amazonaws.com
|
||||
S3_REGION=us-east-1
|
||||
S3_BUCKET=zfs-backups
|
||||
S3_ACCESS_KEY=your_access_key
|
||||
S3_SECRET_KEY=your_secret_key
|
||||
```
|
||||
|
||||
> **Important**:
|
||||
> - The `API_KEY` in the client `.env` file must be the **raw (unhashed)** key. The server stores the SHA-256 hash in the database.
|
||||
> - **Storage type is determined by the server**, not the client. The server decides whether to use S3 or local ZFS storage based on its configuration.
|
||||
> - The client automatically handles full vs incremental backups based on whether a bookmark exists.
|
||||
|
||||
### Restore Tool Configuration
|
||||
|
||||
@@ -128,52 +137,18 @@ zfs-server
|
||||
|
||||
### Client Commands
|
||||
|
||||
The `zfs-client` tool provides the following commands for managing ZFS snapshots:
|
||||
The `zfs-client` tool provides simple commands for creating and sending ZFS snapshots:
|
||||
|
||||
#### `backup`
|
||||
Creates a snapshot and sends it to the server. Automatically uses incremental backup if a bookmark exists.
|
||||
#### `snap`
|
||||
Creates a snapshot and sends it to the server. Automatically detects if this is the first backup (full) or subsequent backup (incremental).
|
||||
|
||||
```bash
|
||||
zfs-client backup
|
||||
zfs-client snap
|
||||
```
|
||||
|
||||
#### `backup-full`
|
||||
Forces a full backup (no incremental). Use for the initial backup or when you want to resend the complete dataset.
|
||||
On first run, it will print: `→ No previous backup found, doing FULL backup...`
|
||||
|
||||
```bash
|
||||
zfs-client backup-full
|
||||
```
|
||||
|
||||
#### `backup-incremental`
|
||||
Creates an incremental backup from the last bookmark. Requires an existing bookmark from a previous full backup.
|
||||
|
||||
```bash
|
||||
zfs-client backup-incremental
|
||||
```
|
||||
|
||||
#### `snapshot <type>`
|
||||
Creates a typed snapshot (hourly, daily, weekly, monthly) with automatic rotation. The rotation policy is fetched from the server if configured.
|
||||
|
||||
```bash
|
||||
zfs-client snapshot hourly
|
||||
zfs-client snapshot daily
|
||||
zfs-client snapshot weekly
|
||||
zfs-client snapshot monthly
|
||||
```
|
||||
|
||||
#### `rotate`
|
||||
Rotates local snapshots based on the retention policy. If the server has a rotation policy configured, it will be used; otherwise, default values apply.
|
||||
|
||||
```bash
|
||||
zfs-client rotate
|
||||
```
|
||||
|
||||
#### `rotate-remote`
|
||||
Requests the server to rotate (delete old) remote snapshots to free up storage quota.
|
||||
|
||||
```bash
|
||||
zfs-client rotate-remote
|
||||
```
|
||||
On subsequent runs, it automatically does incremental backups from the last bookmark.
|
||||
|
||||
#### `status`
|
||||
Displays the current backup status including storage usage, quota, and snapshot count from the server.
|
||||
@@ -182,13 +157,6 @@ Displays the current backup status including storage usage, quota, and snapshot
|
||||
zfs-client status
|
||||
```
|
||||
|
||||
#### `bookmarks`
|
||||
Lists ZFS bookmarks on the local system. Bookmarks are used as reference points for incremental backups.
|
||||
|
||||
```bash
|
||||
zfs-client bookmarks
|
||||
```
|
||||
|
||||
#### `help`
|
||||
Shows the help message with all available commands and options.
|
||||
|
||||
@@ -196,6 +164,23 @@ Shows the help message with all available commands and options.
|
||||
zfs-client help
|
||||
```
|
||||
|
||||
### Client Configuration
|
||||
|
||||
```env
|
||||
CLIENT_ID=client1
|
||||
API_KEY=secret123
|
||||
SERVER_URL=http://backup-server:8080
|
||||
LOCAL_DATASET=tank/data
|
||||
COMPRESS=true
|
||||
|
||||
# Optional: S3 direct upload (bypasses server)
|
||||
S3_ENDPOINT=https://s3.amazonaws.com
|
||||
S3_REGION=us-east-1
|
||||
S3_BUCKET=zfs-backups
|
||||
S3_ACCESS_KEY=your_access_key
|
||||
S3_SECRET_KEY=your_secret_key
|
||||
```
|
||||
|
||||
### Restore Tool Commands
|
||||
|
||||
The `zfs-restore` tool provides commands for listing and restoring snapshots from the backup server:
|
||||
|
||||
Reference in New Issue
Block a user