11 Commits 915059011c ... 694f2eabb0

Author SHA1 Message Date
  Emmanuel Bouthenot 694f2eabb0 Update Changelog 1 year ago
  Emmanuel Bouthenot df31000fa3 Minor changes in Readme.md 1 year ago
  Emmanuel Bouthenot 5b1f6b9e15 Switch encryption from OpenSSL to GnuPG 1 year ago
  Emmanuel Bouthenot 3688f4ad2f Fix mail subject 1 year ago
  Emmanuel Bouthenot a602345e90 Fix double dot in database dumps filenames 1 year ago
  Emmanuel Bouthenot fa64f27fbb Add home page 1 year ago
  Emmanuel Bouthenot d310a31183 Minor changes in comments 1 year ago
  Emmanuel Bouthenot cf88d39d1a Update copyright year 1 year ago
  Emmanuel Bouthenot f918ff1b62 Change default BACKUPDIR 1 year ago
  Emmanuel B 6babbafc1f Merge pull request #11 from DanScharon/DanScharon-patch-1 1 year ago
  Daniel Scharon a84ec011bd add pigz in comment among possible values for compression type 1 year ago
3 changed files with 86 additions and 35 deletions
  1. 8 7
      Changelog.md
  2. 26 5
      Readme.md
  3. 52 23
      autopostgresqlbackup

+ 8 - 7
Changelog.md

@@ -2,19 +2,20 @@
 
 ## Version 2.0
 
-* Huge code cleanup and refactoring
+* Huge code cleanup and refactoring (Closes: [#2](https://github.com/k0lter/autopostgresqlbackup/issues/2))
 
 ### Added features
 
-* Compressing and/or encrypting dumps on the fly (Closes: #1)
-* The day of the week for the weekly backups is now configurable (1 for Monday, 0 disable weekly backups) (Closes: #1, #9)
-* The day of the month for the monthly backups is now configurable (instead of 1 by default, 0 disable the monthly backups) (Closes: #1, #8)
-* Support for any compression tool that supports to read data to be compressed from stdin and outputs it to stdout (Closes: #3, #6)
+* Compressing and/or encrypting dumps on the fly (Closes: [#1](https://github.com/k0lter/autopostgresqlbackup/issues/1))
+* The day of the week for the weekly backups is now configurable (1 for Monday, 0 disable weekly backups) (Closes: [#1](https://github.com/k0lter/autopostgresqlbackup/issues/1), [#9](https://github.com/k0lter/autopostgresqlbackup/issues/9))
+* The day of the month for the monthly backups is now configurable (instead of 1 by default, 0 disable the monthly backups) (Closes: [#1](https://github.com/k0lter/autopostgresqlbackup/issues/1), [#8](https://github.com/k0lter/autopostgresqlbackup/issues/8))
+* Support for any compression tool that supports to read data to be compressed from stdin and outputs it to stdout (Closes: [#3](https://github.com/k0lter/autopostgresqlbackup/issues/3), [#6](https://github.com/k0lter/autopostgresqlbackup/issues/6))
 * Daily, weekly and monthly dumps keeped are now configurable (by default, 14 daily, 5 weekly and 6 monthly backups)
+* Switch encryption from OpenSSL to GnuPG, see [details](https://github.com/k0lter/autopostgresqlbackup#openssl-encryption) (Closes: [#10](https://github.com/k0lter/autopostgresqlbackup/issues/10))
 
 ### Removed features
 
 * It's no longer possible to dump all databases in a single file
-* Copying the last dump in the latest/ directory is no longer supported
+* Copying the lastest dump in the latest/ directory is no longer supported
 * Specifying the databases names to dump during the montly backup is no longer supported
-* It is no longer supported to send backup files by email (MAILCONTENT=files). It is probably no a good idea to send backup files by email (for various reasons: file size, privacy, data leaks, etc.) but I guess that it could easily be implemented in a POSTBACKUP script.
+* It is no longer supported to send backup files by email (MAILCONTENT=files). It was anyway probably not a good idea to send backup files by email (for various reasons: file size, privacy, data leaks, etc.) but I guess that it could easily be implemented in a POSTBACKUP script if needed.

+ 26 - 5
Readme.md

@@ -4,6 +4,8 @@ AutoPostgreSQLBackup is a shell script (usually executed from a cron job) design
 
 AutoPostgreSQLBackup extract databases into flat files in a daily, weekly or monthly basis.
 
+Version 2.0 is a full rewrite.
+
 It supports:
  * Email notification
  * Compression on the fly
@@ -15,23 +17,42 @@ It supports:
 
 ## Usage
 
-On Debian derived operating systems:
+On Debian (or derived):
 
-Install it : `apt install autopostgresqlbackup`
+Install: `apt install autopostgresqlbackup`
 
-If the default options are not suitable for you, change them: `$EDITOR /etc/default/autopostgresqlbackup`
+If the default options are not suitable for you, change them: `${EDITOR} /etc/default/autopostgresqlbackup`
 
 That's it!
 
 ## History
 
- * 2022: Almost full rewrite with better error handling and new features (see Changelog.md for details)
+ * 2023: Almost full rewrite with better error handling and new features (see Changelog.md for details)
  * 2019: Creation of a fork/standelone project on Github (https://github.com/k0lter/autopostgresqlbackup)
  * Since 2011: Various patches (fixes and new features) were added in the Debian package
  * 2011: AutoPostgreSQLBackup was included in Debian
- * 2005: AutoPostgreSQLBackup was written by AutoPostgreSQLBackup (with some contributions of Friedrich Lobenstock)
+ * 2005: AutoPostgreSQLBackup was written by Aaron Axelsen (with some contributions of Friedrich Lobenstock)
    * Project webpage was http://autopgsqlbackup.frozenpc.net/ (offline)
 
+## Encryption
+
+Encryption (asymmetric) is now done with GnuPG, you just need to add the
+public key (armored or not) you want to encrypt the data to in the file pointed by the `${ENCRYPTION_PUBLIC_KEY}` configuration setting.
+
+Export your public key:
+
+`gpg --export 0xY0URK3Y1D --output mypubkey.gpg`
+
+or
+
+`gpg --export --armor 0xY0URK3Y1D --output mypubkey.asc`
+
+then copy `mypubkey.asc` or `mypubkey.gpg` to the path pointed by the `${ENCRYPTION_PUBLIC_KEY}` configuration setting and set the `${ENCRYPTION}` setting to `yes`.
+
+## OpenSSL Encryption
+
+Starting from version 2.0 encryption with OpenSSL is no longer supported as [it was discovered](https://github.com/k0lter/autopostgresqlbackup/issues/10) (but also [known for quite some time](https://github.com/cytopia/mysqldump-secure/issues/21)) that encrypting large files with [OpenSSL silently fail](https://github.com/openssl/openssl/issues/2515) and that decrypting these files is [close to be impossible](https://github.com/imreFitos/large_smime_decrypt).
+
 ## Authors
 
  * Emmanuel Bouthenot (Current maintainer)

+ 52 - 23
autopostgresqlbackup

@@ -5,7 +5,7 @@
 # https://github.com/k0lter/autopostgresqlbackup
 # Copyright (c) 2005 Aaron Axelsen <axelseaa@amadmax.com>
 #               2005 Friedrich Lobenstock <fl@fl.priv.at>
-#               2013-2022 Emmanuel Bouthenot <kolter@openics.org>
+#               2013-2023 Emmanuel Bouthenot <kolter@openics.org>
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -45,7 +45,7 @@ DBNAMES="all"
 GLOBALS_OBJECTS="postgres_globals"
 
 # Backup directory location e.g /backups
-BACKUPDIR="/backups"
+BACKUPDIR="/var/backups"
 
 # Email Address to send mail to? (user@domain.com)
 MAILADDR="user@domain.com"
@@ -80,7 +80,7 @@ BRWEEKLY=5
 # Default is 12 months
 BRMONTHLY=12
 
-# Choose Compression type. (gzip, bzip2, xz or zstd)
+# Choose Compression type. (gzip, pigz, bzip2, xz or zstd)
 COMP=gzip
 
 # Compression options
@@ -96,26 +96,21 @@ EXT="sql"
 PERM=600
 
 # Encryption settings
-# (inspired by http://blog.altudov.com/2010/09/27/using-openssl-for-asymmetric-encryption-of-backups/)
 #
 # It is recommended to backup into a staging directory, and then use the
 # POSTBACKUP script to sync the encrypted files to the desired location.
 #
-# Encryption uses private/public keys. You can generate the key pairs like the following:
-# openssl req -x509 -nodes -days 100000 -newkey rsa:2048 -keyout backup.key -out backup.crt -subj '/'
+# For now the only encryption method supported is using GnuPG
 #
 # Decryption:
-# openssl smime -decrypt -in backup.sql.gz.enc -binary -inform DEM -inkey backup.key -out backup.sql.gz
-
+# gpg --decrypt --output backup.sql.gz backup.sql.gz.enc
+#
 # Enable encryption
 ENCRYPTION=no
 
 # Encryption public key (path to the key)
 ENCRYPTION_PUBLIC_KEY=""
 
-# Encryption Cipher (see enc manpage)
-ENCRYPTION_CIPHER="aes256"
-
 # Suffix for encyrpted files
 ENCRYPTION_SUFFIX=".enc"
 
@@ -127,10 +122,6 @@ ENCRYPTION_SUFFIX=".enc"
 # }}}
 
 # {{{ OS Specific
-#=====================================================================
-# Debian specific options ===
-#=====================================================================
-
 if [ -f /etc/default/autopostgresqlbackup ]; then
     # shellcheck source=/dev/null
     . /etc/default/autopostgresqlbackup
@@ -187,6 +178,7 @@ fi
 
 # {{{ Defaults
 PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/postgres/bin:/usr/local/pgsql/bin
+HOMEPAGE="https://github.com/k0lter/autopostgresqlbackup"
 NAME="AutoPostgreSQLBackup"         # Script name
 VERSION="2.0"                       # Version Number
 DATE="$(date '+%Y-%m-%d_%Hh%Mm')"   # Datestamp e.g 2002-09-21
@@ -207,6 +199,9 @@ LOG_FILE="${LOG_DIR}/${NAME}_${DBHOST//\//_}-$(date '+%Y-%m-%d_%Hh%Mm').log"
 # Debug mode
 DEBUG="no"
 
+# Encryption prerequisites
+GPG_HOMEDIR=
+
 # pg_dump options
 if [ -n "${OPT}" ]; then
     IFS=" " read -r -a PG_OPTIONS <<< "${OPT}"
@@ -305,6 +300,15 @@ log_warn() {
 }
 # }}}
 
+# {{{ gpg_setup()
+gpg_setup() {
+    GPG_HOMEDIR="$(mktemp --quiet --directory -t "${NAME}.XXXXXX")"
+    chmod 700 "${GPG_HOMEDIR}"
+    log_debug "With encryption enabled creating a temporary GnuPG home in ${GPG_HOMEDIR}"
+    gpg --quiet --homedir "${GPG_HOMEDIR}" --quick-gen-key --batch --passphrase-file /dev/null "root@$(hostname --fqdn)"
+}
+# }}}
+
 # {{{ dblist()
 dblist () {
     local cmd_prog cmd_args raw_dblist dblist dbexcl databases
@@ -388,8 +392,8 @@ dbdump () {
 
 # {{{ encryption()
 encryption() {
-    log_debug "Encrypting using cypher ${ENCRYPTION_CIPHER} and public key ${ENCRYPTION_PUBLIC_KEY}"
-    openssl smime -encrypt -${ENCRYPTION_CIPHER} -binary -outform DEM "${ENCRYPTION_PUBLIC_KEY}" 2>&7
+    log_debug "Encrypting using public key ${ENCRYPTION_PUBLIC_KEY}"
+    gpg --homedir "${GPG_HOMEDIR}" --encrypt --passphrase-file /dev/null --recipient-file "${ENCRYPTION_PUBLIC_KEY}" 2>&7
 }
 # }}}
 
@@ -429,7 +433,7 @@ dump() {
                 comp_ext=".zstd"
                 ;;
         esac
-        dump_file="${dump_file}.${comp_ext}"
+        dump_file="${dump_file}${comp_ext}"
     fi
 
     if [ "${ENCRYPTION}" = "yes" ]; then
@@ -545,6 +549,7 @@ fi
 
 # {{{ main()
 log_info "${NAME} version ${VERSION}"
+log_info "Homepage: ${HOMEPAGE}"
 log_info "Backup of Database Server - ${HOST}"
 
 if [ -n "${COMP}" ]; then
@@ -554,9 +559,28 @@ if [ -n "${COMP}" ]; then
     fi
 fi
 
-if [ "${ENCRYPTION}" = "yes" ] && ! command -v "openssl" >/dev/null ; then
-    log_warn "Disabling encryption, 'openssl' command not found"
-    ENCRYPTION="no"
+if [ "${ENCRYPTION}" = "yes" ]; then
+    if [ ! -s "${ENCRYPTION_PUBLIC_KEY}" ]; then
+        log_warn "Disabling encryption, '${ENCRYPTION_PUBLIC_KEY}' is empty or does not exists"
+        ENCRYPTION="no"
+    elif ! command -v "gpg" >/dev/null ; then
+        log_warn "Disabling encryption, 'gpg' command not found"
+        ENCRYPTION="no"
+    else
+        gpg_setup
+        if ! keyinfo="$(gpg --quiet --homedir "${GPG_HOMEDIR}" "${ENCRYPTION_PUBLIC_KEY}" 2>/dev/null)"; then
+            log_warn "Disabling encryption, key in '${ENCRYPTION_PUBLIC_KEY}' does not seems to be a valid public key"
+            ENCRYPTION="no"
+            if command -v "openssl" >/dev/null && openssl x509 -noout -in "${ENCRYPTION_PUBLIC_KEY}" >/dev/null 2>&1; then
+                log_warn "public key in '${ENCRYPTION_PUBLIC_KEY}' seems to be in PEM format"
+                log_warn "Encryption using openssl is no longer supported: see ${HOMEPAGE}#openssl-encryption"
+            fi
+        else
+            keyfp="$(echo "${keyinfo}" | sed -r -n 's/^\s*([a-z0-9]+)\s*$/\1/pi')"
+            keyuid="$(echo "${keyinfo}" | sed -r -n 's/^\s*uid\s+(\S.*)$/\1/pi' | head -n1)"
+            log_info "Encryption public key is: 0x${keyfp} (${keyuid})"
+        fi
+    fi
 fi
 
 log_info "Backup Start: $(date)"
@@ -630,17 +654,22 @@ if [ "${DEBUG}" = "no" ] && grep -q '^err|' "${LOG_FILE}" ; then
             fi
         done
         printf "\nFor more information, try to run %s in debug mode, see \`%s -h\`\n" "${NAME}" "$(basename "$0")"
-    ) | mail -s "${NAME} - log" "${MAILADDR}"
+    ) | mail -s "${NAME} issues on $(hostname --fqdn)" "${MAILADDR}"
 fi
 # }}}
 
-# {{{ Cleanup logs and exit()
+# {{{ Cleanup and exit()
 if [ -s "${LOGERR}" ]; then
     rc=1
 else
     rc=0
 fi
 
+# Cleanup GnuPG home dir
+if [ -d "${GPG_HOMEDIR}" ]; then
+    rm -rf "${GPG_HOMEDIR}"
+fi
+
 # Clean up log files
 rm -f "${LOG_FILE}"