Writing a Bash Script to Backup to Backblaze B2

In this tutorial, we’ll walk through the creation of a bash script that utilizes Restic to perform backups to Backblaze B2 cloud storage and sends email notifications in case of failure. Restic is a modern backup program that offers efficient deduplication and encryption features.

Prerequisites

Before getting started, make sure you have the following:

A Backblaze B2 account and access keys
Restic installed on your system
Access to an SMTP server for sending emails

Script Overview

The bash script should perform the following tasks:

Sets up the necessary variables, such as AWS access keys, repository name, and email settings.
Checks for the existence of required tools and install any missing dependencies.
Executes Restic commands for backup, retention policies, and cleanup.
Sends email notifications in case of backup failure.

Step-by-Step Guide

Step 1: Setting up Variables

Before running the script, ensure you fill in the necessary variables in the script, such as AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, REPO_NAME, REPO_PWFILE, REPO_URL, BACKUP_FILE_LIST, EXCLUDE_FILE_LIST, RECIPIENT_EMAIL, SMTP_SERVER, SMTP_USERNAME, and SMTP_PASSWORD.

#!/bin/bash

AWS_ACCESS_KEY_ID=""
AWS_SECRET_ACCESS_KEY=""
REPO_NAME="bck-example"
REPO_PWFILE="/root/.restic"
REPO_URL="s3:s3.eu-central-003.backblazeb2.com/$REPO_NAME"
BACKUP_FILE_LIST="./backups.txt"
EXCLUDE_FILE_LIST="./exclude.txt"
RECIPIENT_EMAIL="[email protected]"
SMTP_SERVER="mail.jnns.de:465"
SMTP_USERNAME="[email protected]"
SMTP_PASSWORD=""

Step 2: Defining Helper Functions

The script defines several helper functions:

check_file_exists: Checks if a file exists. check_command_installed: Checks if a command is installed and installs it if not. run_restic: Executes Restic commands and handles errors. urlencode: Encodes special characters in a string for use in URLs. send_email: Sends email notifications in case of failure.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
check_file_exists() {
  [[ -f "$1" ]] || { echo "Error: File $1 not found. Exiting."; exit 1; }
}

check_command_installed() {
  local command_name="$1"
  if ! command -v "$command_name" &> /dev/null; then
    echo "$command_name is not installed. Installing..."
    apt install -y $command_name
  fi
}

run_restic() {
  local output
  output=$(restic -p "$REPO_PWFILE" -r "$REPO_URL" "$@" 2>&1) || { echo "Error: $output"; return 1; }
  echo "$output"
}

urlencode() {
  local string="$1"
  local encoded_string=""
  local char

  for ((i = 0; i < ${#string}; i++)); do
    char="${string:i:1}"
    case "$char" in
      [a-zA-Z0-9.~_-])
        encoded_string+="$char"
        ;;
      *)
        printf -v encoded_char '%%%02x' "'$char"
        encoded_string+="$encoded_char"
        ;;
    esac
  done

  printf "%s" "$encoded_string"
}

send_email() {
  local hostname=$(hostname)
  local subject="Backup Failure on \"$hostname\""
  local body="Backup process failed on $hostname:\n\n$output"

  echo -e "$body" | s-nail -s "$subject" \
    -S v15-compat \
    -S mta=smtps://"$(urlencode $SMTP_USERNAME)":"$SMTP_PASSWORD"@"$SMTP_SERVER" \
    -S from="$SMTP_USERNAME" \
    "$RECIPIENT_EMAIL"
}

Step 3: Installing Dependencies

The script checks if required commands (s-nail and restic) are installed and installs them if necessary.

1
2
check_command_installed "s-nail"
check_command_installed "restic"

Step 4: Backup Process

The script initiates the backup process using Restic. It unlocks the repository if necessary, initializes it if it’s not yet initialized, performs the backup, and applies retention policies.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
check_file_exists "$REPO_PWFILE"
check_file_exists "$BACKUP_FILE_LIST"

if [ -f "$EXCLUDE_FILE_LIST" ]; then
  echo "Using exclude file: $EXCLUDE_FILE_LIST"
  EXCLUDE_OPTION="--exclude-file $EXCLUDE_FILE_LIST"
else
  echo "No exclude file found. Proceeding without exclusions."
  EXCLUDE_OPTION=""
fi

export AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY

restic cache --quiet --cleanup
restic self-update --quiet

echo "Attempting to unlock or initialize repository..."
output=$(run_restic unlock) || { 
  echo "Unlock failed. Running init..."
  output=$(run_restic init) || { 
    echo "Error: Unable to initialize repository."
    send_email
    exit 1
  }
}

echo "Starting backup process..."
output=$(run_restic backup --files-from "$BACKUP_FILE_LIST" $EXCLUDE_OPTION) || { 
  echo "Backup process failed."
  send_email
  exit 1
}

echo "Performing cleanup and retention policies..."
output=$(run_restic forget --keep-daily 7 --keep-weekly 1 --keep-monthly 1 --prune) || { 
  echo "Retention policy execution failed."
  send_email
  exit 1
}

unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY

echo "Backup process completed successfully."

Conclusion

With this bash script, you can automate your backups using Restic and ensure that you’re notified promptly in case of any failures. Feel free to customize the script further to suit your specific backup needs and preferences.

Full source code can be found on my Github: https://github.com/Jnnshschl/SimpleResticB2Backup

Prerequisites#

Script Overview#

Step-by-Step Guide#

Step 1: Setting up Variables#

Step 2: Defining Helper Functions#

Step 3: Installing Dependencies#

Step 4: Backup Process#

Conclusion#