Linux: Script to backup your important file(s) and ftp them

For those who got the Open Source fever, sharing only makes you hot!

Moderator: neilg

Linux: Script to backup your important file(s) and ftp them

Postby neilg » Sun Jul 03, 2005 12:43 pm

This is not directly Compiere related however I post it because some of us have adopted Linux as our preferred workstation software and it becomes an issue that needs to be solved.

The problem is: I need to backup some important system files and critical files I use on a daily basis. I've got a ftp server where I can upload files to. How do I do this easily and with one command?

Here is a shell script written by someone else adapted to solve this problem. I've adopted using symbolic links to specify which files I wish to backup. You create a folder called '/mydir/backup' then you create a folder for each 'set' or 'package'. E.g. you could have one called 'daily'. Then you will have a folder '/mydir/backup/daily'. Then
Code: Select all
cd /mydir/backup/daily
ln -s /myfiles/importantDir1
ln -s /etc/hosts #backup this important file

Now when you wish to backup this set type in
Code: Select all
backupshellscriptname.sh daily
This will do all the work for you.

For the unitiated a symbolic link is (from dictionary.com):
<file system> (Or "symlink", "soft link" (by contrast with
"hard link"), "shortcut", "alias") A special type of
Unix file which refers to another file by its pathname. A
symbolic link is created with the "ln" (link) command:

ln -s OLDNAME NEWNAME

Where OLDNAME is the target of the link (usually a pathname)
and NEWNAME is the pathname of the link itself.


Here is the script, you will need to change certain settings for your needs (read the whole script I've patched it in a few places). Save it in /usr/bin as 'backupshellscriptname.sh' (change the name to suite your taste)
Code: Select all
#!/bin/bash
# Shell script (BASH) to backup the selected directory on server and upload to
# another ftp server securely. This script uses the gpg command to
# encrypt the .tar.gz file before upload take place.
#
# In order to run this script you must have following tools installed:
# - /usr/bin/ncftpput
# - /bin/tar
# - /usr/bin/mail
# - /usr/bin/gpg
#
# Script also mails back the ftp operation failed or not
#
# Installation:
# Customize the script according to your need. You need to setup ftp
# server, password etc. Next, you need to setup gpg user name and
# import public key so that you can encrypt the files. Usually following two
# commands needed for gpg:
# gpg --import userkey
# gpg --edit-key KEY_ID|USER_ID
# Command>trust
#
# --------------------------------------------------------------------
# This is a free shell script under GNU GPL version 2.0 or above
#
# Copyright (C) 2005 Vivek G Gite
#
# Feedback/comment/suggestions : http://cyberciti.biz/fb/
# ---------------------------------------------------------------------

# Package to backup which will be a folder listed in BASE below

PACKAGE=$1

# folder with a set of folders
# each folder contains one ore more symbolic links to the actual
# files which are to be backed up
#
BASE="/mydir/backup"

if [ "$#" != "1" ]; then
   echo "Usage: $0 packageName for $BASE"
   echo "Must be one of:"
   echo "==============="
   ls -d $BASE/*
   exit 1
fi

PACKAGE=$1
BACKUP=$BASE/$PACKAGE
echo "Backup for:"
echo "==============="
ls -d $BACKUP/*
echo "==============="

# Remote ftp server
FTPH="myFtpSite"

# Remote ftp user name
FTPU="myFtpUser"

# Remote ftp user password
FTPP="myFtpPassword"

# Local gpg user_id
#GPGU="me"

# Remote directory, blank for default remote dir
# If dir does not exist it will be created automatically by ncftpput :)
FTPD=""

# Temporary directory to store tar.gz file and process it
TMPD="/tmp1"

# Mail message
# Admin email me@mycorp.com or pager@yourmobile.com
MTO="root"
# Mail subject
MSUB="Backup $(hostname) report"
# Admin info, URL email id; change it according to your need :)
ADMIN_INFO="For support visit http://cyberciti.biz/fb/ or write an email to nobody@cyberciti.biz"

# Only change if your UNIX stores bin in diffrent location
NCFTP="/usr/bin/ncftpput"
TAR="/bin/tar"  # must be gnu tar
MAILC="/usr/bin/mail"
#GPG="/usr/bin/gpg"

#######################################################################

#######################################################################
#FILE="$(hostname).$(date +"%d-%m-%Y").tar.gz"
FILE="$PACKAGE.tar"
ZIPPEDOUT="$TMPD/$FILE.gz"
TAROUT="$TMPD/$FILE"
FOUT="$TMPD/$FILE.1"
MFILE="/tmp/ftpout.$$.txt"
MESS=""

echo "Backup file is $OUT"
echo "log file is $MFILE"

if [ ! -x $TAR ]; then
  echo "$TAR command not found, contact $ADMIN_INFO"
  exit 1
fi

if [ ! -x $NCFTP ]; then
  echo "$NCFTP command not found, contact $ADMIN_INFO"
  exit 1
fi

#if [ ! -x $GPG ] ; then
#  echo "$GPG command not found, contact $ADMIN_INFO"
#  exit 1
#fi

rm -f $TAROUT
echo "$TAROUT cfh $OUT $BACKUP"
$TAR cfh $TAROUT $BACKUP    #follow symlinks
#backup the backup symbolic links
$TAR uf $TAROUT $BASE   #backup the symlinks for the backup. dont follow sym links
#
gzip -f9 $TAROUT
if [ $? -ne 0 ];
then
   MESS="$TAR failed to create backup. Nothing uploaded to remote FTP $FTPH server"
else
   # Encrypt the .tar.gz file before upload
   #$GPG -e -r $GPGU -o $FOUT $OUT
   echo "File before encryption `ls -l $ZIPPEDOUT`"
   openssl enc -aes-256-cbc -in $ZIPPEDOUT -out $FOUT -pass stdin < /root/crypt/.secret
   echo "File after encryption `ls -l $FOUT`"
   $NCFTP -m -u "$FTPU" -p "$FTPP" "$FTPH" "$FTPD" "$FOUT"
   OSTAT="$?"
   case $OSTAT in
   0) MESS="Success.";;
   1) MESS="Could not connect to remote host $FTPH.";;
        2) MESS="Could not connect to remote host $FTPH - timed out.";;
        3) MESS="Transfer failed.";;
        4) MESS="Transfer failed - timed out.";;
        5) MESS="Directory change failed.";;
        6) MESS="Directory change failed - timed out.";;
        7) MESS="Malformed URL.";;
        8) MESS="Usage error. May be your version of ncftpput ($NCFTP) is old";;
        9) MESS="Error in login configuration file.";;
        10)MESS="Library initialization failed.";;
        11) MESS="Session initialization failed.";;
   *) MESS="Unknown error, contact admin $ADMIN_INFO";;
   esac
fi

>$MFILE
echo "Backup status for $(hostname) as on $(date):" >>$MFILE
echo "" >>$MFILE
echo "Backup File : $FOUT" >>$MFILE
echo "Backup ftp server : $FTPH" >>$MFILE
echo "Backup status message : $MESS" >>$MFILE
echo "" >>$MFILE
echo "-- Automatically generated by $(basename $0)" >>$MFILE

# send an email to admin
#$MAILC -s "$MSUB" $MTO <$MFILE
# remove the files
[ -f $MFILE ] && rm -f $MFILE || :
[ -f $FOUT ] && rm -f $FOUT || :
[ -f $OUT ] && rm -f $OUT || :
# EOF : script

This uses a different encryption command to the original GPG based encryption. The GPG one can be used but you'll need to figure it out on your own.

You should put your encryption password in /root/crypt/.secret which you shouldn't backup. Write it down and keep in a safe place. Also make sure no-one has access to it and type in
Code: Select all
chmod -R go-rwx /root/crypt
You should also write the following command down for when you need to restore:

Code: Select all
openssl enc -d -aes-256-cbc -in daily.tar.gz.1 -out daily.tar.gz


So to summarize you backup & ftp as follows:

Code: Select all
backupshellscriptname.sh daily


Comments and suggestions to neilg@knowledgesuccess.com. Tested on Compiere: version indepedant ! :)
neilg
Regular
 
Posts: 141
Joined: Thu Jun 16, 2005 1:58 pm

Testing the integrity of your file

Postby neilg » Sun Jul 17, 2005 7:35 pm

You should from time to time test the integrity of your backup. FTP the file back from where you save it then use this script:

Code: Select all
#
# unencrypt the file originally encrypted using bk.sh
# bkrest.sh (or bktest.sh would be more appropriate)
# by Neil Gordon 2005/07/17 13:40
#

TMPD="/tmp1"

if [ "$#" != "1" ]; then
   echo "Usage: $0 packageName for $TMPD"
   exit 1
fi

ENCFILE=$TMPD/$1.tar.1
FOUT=$TMPD/$1.tar.gz
echo "Unencrypting file: $ENCFILE"
echo "Output file: $FOUT"


openssl enc -d -aes-256-cbc -in $ENCFILE -out $FOUT -pass stdin < /root/crypt/.secret
echo "File after decryption `ls -l $FOUT`"
echo "CATALOG"
echo "================================================================"
tar -ztvf $FOUT
neilg
Regular
 
Posts: 141
Joined: Thu Jun 16, 2005 1:58 pm


Return to Contributor's Corner

Who is online

Users browsing this forum: No registered users and 1 guest