Below is the method described to
take huge backup over network using “RSYNC”. Following
are the pre-requests which requires to perform below operation.

(1)SSH-Keygen configuration needs to
be setup between two servers which offers password less authentication
mechanism.

Note: Rsync Command is used here and if
run first time will synchronize full data over the network to remote server's
Rsync directory & then after it will compare both server's data à delete from backup if it is been
deleted from original data server & copy (synchronize) only newer files
which have been modified / created on original data server to the remote RSYNC
Backup Server's Rsync Directory.

(2) Data Partition should be on LVM
& Volume Group Manager (VG) contains free space same as Data Logical
Volume(LV) to take advantage of volume snapshot backup which is offered by LVM
File System.   èNot
mandatory

Scenario:

Server1=192.168.181.40 (contains Huge Data of mail server
/ nfs / samba / ftp server's data which is public
share & gets deleted / modified / new files created by users
everyday)

Server2=192.168.181.121(Backup Server with Huge Blank Space
as data can not be stored in compress mode here to synchronies with remote
server's data directory)

(1) First step is to establish
keyless authentication mechanism between original data server to backup server.
So go to server2 & perform below steps -

Server2:

[root@back_server /]# mkdir -p /backup/aibackups/LONAISWMFT001/wmfdata/cyrus
ècreate
backup RSYNC directory to be synchronize with original data using
Rsync

[root@back_server /]# useradd -d /backup/aibackups/ backup                               ècreate
user backup on that directory

[root@back_server /]# su - backup

[backup@back_server ~]$ pwd

/backup/aibackups/

[backup@back_server ~]$ exit

[root@back_server /]# chown -R backup:backup
backup/                          èchange
ownership of Rsync directory

[root@back_server /]# su - backup

[backup@back_server ~]$ ssh-keygen -t rsa                                            ècreate
“rsa” key for user “backup” on backup
server

Generating public/private rsa key pair.     
                                   
èJust
press ENTER in all the questions it asks

Enter file in which to save the key
(/backup/aibackups//.ssh/id_rsa):

Created directory
'/backup/aibackups//.ssh'.

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved
in /backup/aibackups//.ssh/id_rsa.

Your public key has been saved in
/backup/aibackups//.ssh/id_rsa.pub.

The key fingerprint
is:

d9:09:07:c3:9b:e7:f3:bc:5f:f9:74:94:4e:5b:53:96 backup@back_server.oracle

[backup@back_server ~]$ cd .ssh/         
                                               
èAbove
command will create a hidden directory “.ssh”

[backup@back_server .ssh]$ ls -l

total 16

-rw-------  1
backup backup 883 May 
6 16:11 id_rsa

-rw-r--r--  1 backup backup 230
May  6 16:11 id_rsa.pub

(2) Same steps needs to be perform
on Data Server.

Server1:

[root@lonaiswmft001 ~]# ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key
(/root/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved
in /root/.ssh/id_rsa.

Your public key has been saved in
/root/.ssh/id_rsa.pub.

The key fingerprint
is:

cf:23:a4:80:ef:7d:45:aa:ac:98:01:8a:18:e9:83:7d
root@lonaiswmft001

[root@lonaiswmft001 ~]# cd .ssh/

[root@lonaiswmft001 .ssh]# ll

total 8

-rw-------  1
root root 887 May 
6 16:43 id_rsa

-rw-r--r--  1 root root 228
May  6 16:43 id_rsa.pub

 [root@lonaiswmft001 .ssh]# scp -r -p id_rsa.pub root@192.168.181.121:/               ècopy
file “id_rsa.pub” from Original Data Server to remote
backup server.

The authenticity of host
'192.168.181.121 (192.168.181.121)' can't be established.

RSA key fingerprint is 8e:79:cd:84:c4:6e:a4:7d:76:0f:cf:11:e5:ef:8a:00.

Are you sure you want to continue
connecting (yes/no)? yes

Warning: Permanently added
'192.168.181.121' (RSA) to the list of known hosts.

root@192.168.181.121's
password:

id_rsa.pub                                                                                 
100%  228     0.2KB/s  
00:00

 (3) Now create authentication file
using key file from server1 to server2 using user backup.

 Server2:

[root@back_server /]# mv /id_rsa.pub
/backup/aibackups/.ssh/authorized_keys                  èmove
file “id_rsa.pub” to rsync
directory & rename it as “authorized_keys”

 [root@back_server /]# su - backup

 [backup@back_server ~]$ cd .ssh/

 [backup@back_server .ssh]$ ls -l

total 24

-rw-r--r--  1 root   root   228 May 
6  2008
authorized_keys

-rw-------  1
backup backup 883 May 
6 16:11 id_rsa

-rw-r--r--  1 backup backup 230
May  6 16:11 id_rsa.pub

 (4) Now, Check keyless
authentication between two servers using user “backup”

 Server1:

 [root@lonaiswmft001 .ssh]# ssh -v backup@192.168.181.121

OpenSSH_3.9p1, OpenSSL 0.9.7a Feb 19 2003

debug1: Reading configuration data
/etc/ssh/ssh_config

debug1: Applying options for
*

debug1: Connecting to
192.168.181.121 [192.168.181.121] port 22.

debug1: Connection
established.

debug1: permanently_set_uid: 0/0

debug1: identity
file /root/.ssh/identity type -1

debug1: identity
file /root/.ssh/id_rsa type
1

debug1: identity
file /root/.ssh/id_dsa type
-1

debug1: Remote protocol version
1.99, remote software version OpenSSH_3.9p1

debug1: match: OpenSSH_3.9p1 pat
OpenSSH*

debug1: Enabling compatibility mode
for protocol 2.0

debug1: Local version string
SSH-2.0-OpenSSH_3.9p1

debug1: SSH2_MSG_KEXINIT
sent

debug1: SSH2_MSG_KEXINIT
received

debug1: kex: server->client aes128-cbc hmac-md5
none

debug1: kex: client->server aes128-cbc hmac-md5
none

debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent

debug1: expecting
SSH2_MSG_KEX_DH_GEX_GROUP

debug1: SSH2_MSG_KEX_DH_GEX_INIT
sent

debug1: expecting
SSH2_MSG_KEX_DH_GEX_REPLY

debug1: Host '192.168.181.121' is
known and matches the RSA host key.

debug1: Found key in
/root/.ssh/known_hosts:1

debug1: ssh_rsa_verify: signature correct

debug1: SSH2_MSG_NEWKEYS
sent

debug1: expecting
SSH2_MSG_NEWKEYS

debug1: SSH2_MSG_NEWKEYS
received

debug1: SSH2_MSG_SERVICE_REQUEST
sent

debug1: SSH2_MSG_SERVICE_ACCEPT
received

debug1: Authentications that can
continue: publickey,gssapi-with-mic,password

debug1: Next authentication method:
gssapi-with-mic

debug1: Authentications that can
continue: publickey,gssapi-with-mic,password

debug1: Authentications that can
continue: publickey,gssapi-with-mic,password

debug1: Next authentication method:
publickey

debug1: Trying private key:
/root/.ssh/identity

debug1: Offering public key:
/root/.ssh/id_rsa

debug1:
Server accepts key: pkalg ssh-rsa blen 149

debug1:
read PEM private key done: type RSA

debug1:
Authentication succeeded (publickey).

debug1: channel 0: new
[client-session]

debug1: Entering interactive
session.

[backup@back_server ~]$          èyou
will reach to remote server’s rsync directory using
login as user “backup” without supplying password for user “backup”

Note: When we have create user “backup”
we have yet not set password for user “backup” and it is not needed as
well.

[backup@back_server ~]$exit

[root@lonaiswmft001
.ssh]#

 (5) After successful authentication
without password from data server (server1) to RSYNC Backup Server (server2)
using user “backup”, we can now perform Rsync operation to backup
data.

 Server1:

 [root@lonaiswmft001 ~]# fdisk -l

 Disk /dev/sda: 80.0 GB, 80032038912 bytes

255 heads, 63 sectors/track, 9730
cylinders

Units = cylinders of 16065 * 512 =
8225280 bytes

 Device Boot      Start         End      Blocks  
Id 
System

/dev/sda1   *          
1          13      104391  
83 
Linux

/dev/sda2              14         395     3068415  
82  Linux
swap

/dev/sda3             396        9730   
74983387+ 
8e  Linux LVM

 [root@lonaiswmft001 ~]# df -h

Filesystem                                                       
Size 
Used Avail Use% Mounted on

/dev/mapper/VolGroup00-LogVol00
                      2.0G  282M  1.6G 
15% /

/dev/sda1                                                         
  99M   13M  
82M  13%
/boot

none                                                                
  755M     0 
755M   0% /dev/shm

/dev/mapper/VolGroup00-LogVol01
                      2.0G  303M  1.6G 
17% /home

/dev/mapper/VolGroup00-LogVol02
                      7.7G  3.8G  3.6G 
51% /usr

/dev/mapper/VolGroup00-LogVol03
                      9.9G  1.9G  7.5G 
20% /var

/dev/mapper/VolGroup00-LogVol04
                      5.0G  1.3G  3.4G 
28% /config

/dev/mapper/VolGroup00-LogVol05
                      9.9G  4.0G 
5.5G  42% /wmfdata            èAssume
that logical volume(LV) “/dev/VolGroup00/LogVol05” which is mounted on /wmfdata
contains data which needs to by backuped using RSYNC
to remote backup server.

 [root@lonaiswmft001 ~]# lvscan                                                                                     
èto
scan all available LV

 
ACTIVE           
'/dev/VolGroup00/LogVol00' [1.97 GB] inherit

 
ACTIVE           
'/dev/VolGroup00/LogVol01' [1.97 GB] inherit

 
ACTIVE           
'/dev/VolGroup00/LogVol02' [7.81 GB] inherit

 
ACTIVE           
'/dev/VolGroup00/LogVol03' [9.97 GB] inherit

 
ACTIVE           
'/dev/VolGroup00/LogVol04' [5.00 GB] inherit

 
ACTIVE           
'/dev/VolGroup00/LogVol05' [10.00 GB] inherit

 Note: LVM is a unix file system which offers many advantages over the
common file systems like –

• Online extend of partition size
  without losing / backup your data.
• Creates snapshot of the partition
  without unmounting it for taking FULL Backup of that
  partition.
• combines multiple hard disks to create a
  large partition.

So, here, we are taking snapshot
benefit of LVM.

 [root@lonaiswmft001 ~]# lvcreate --size 10G --snapshot
--name data_snapshot
/dev/VolGroup00/LogVol05

 
Logical volume "data_snapshot"
created

 [root@lonaiswmft001 ~]# lvscan

 
ACTIVE           
'/dev/VolGroup00/LogVol00' [1.97 GB] inherit

 
ACTIVE           
'/dev/VolGroup00/LogVol01' [1.97 GB] inherit

 
ACTIVE           
'/dev/VolGroup00/LogVol02' [7.81 GB] inherit

 
ACTIVE           
'/dev/VolGroup00/LogVol03' [9.97 GB] inherit

 
ACTIVE           
'/dev/VolGroup00/LogVol04' [5.00 GB] inherit

 
ACTIVE   Original '/dev/VolGroup00/LogVol05' [10.00
GB] inherit

  ACTIVE  
Snapshot '/dev/VolGroup00/data_snapshot' [10.00 GB] inherit

 [root@lonaiswmft001 ~]# mkdir /data_snapshot                                                   
ècreate
a blank directory

 [root@lonaiswmft001 ~]# mount /dev/VolGroup00/data_snapshot /data_snapshot

 [root@lonaiswmft001 ~]# df -h

Filesystem                                                       
Size 
Used Avail Use% Mounted on

/dev/mapper/VolGroup00-LogVol00
                    2.0G  282M  1.6G 
15% /

/dev/sda1                                                         
99M   13M   82M  13% /boot

none                                                                
755M     0  755M  
0% /dev/shm

/dev/mapper/VolGroup00-LogVol01                      2.0G  303M  1.6G 
17% /home

/dev/mapper/VolGroup00-LogVol02                      7.7G  3.8G  3.6G 
51% /usr

/dev/mapper/VolGroup00-LogVol03                      9.9G  1.9G  7.5G 
20% /var

/dev/mapper/VolGroup00-LogVol04                      5.0G  1.3G  3.4G 
28% /config

/dev/mapper/VolGroup00-LogVol05                      9.9G  4.0G 
5.5G  42%
/wmfdata

/dev/mapper/VolGroup00-data_snapshot              9.9G  4.0G 
5.5G  42% /data_snapshot

 [root@lonaiswmft001 ~]# rsync -avze ssh --delete /data_snapshot/data/cyrus/
backup@192.168.181.121:/backup/aibackups/LONAISWMFT001/wmfdata/cyrus                èperform
remote backup using rsync to remote rsync directory & synchronize all the data from original
data server to remote Backup RSYNC Server.

building file list ...
done

./

cyrus_lib/

cyrus_lib/.bash_history

cyrus_lib/annotations.db

cyrus_lib/backup/

cyrus_lib/backup/mboxlist.1.gz

cyrus_lib/backup/mboxlist.gz

cyrus_lib/db/

cyrus_lib/db.backup1/

cyrus_lib/db.backup1/annotations.db

cyrus_lib/db.backup1/log.0000000001

cyrus_spool/mail/v/user/vishal^kesariya/Sent/9.

cyrus_spool/mail/v/user/vishal^kesariya/Sent/cyrus.cache

cyrus_spool/mail/v/user/vishal^kesariya/Sent/cyrus.header

cyrus_spool/mail/v/user/vishal^kesariya/Sent/cyrus.index

cyrus_spool/mail/v/user/vishal^kesariya/Sent/cyrus.squat

cyrus_spool/mail/v/user/vishal^kesariya/Trash/…………………………………………………………………so
on…………………..

cyrus_spool/mail/v/user/vishal^kesariya/Trash/190.

cyrus_spool/mail/v/user/vishal^kesariya/Trash/cyrus.cache

cyrus_spool/mail/v/user/vishal^kesariya/Trash/cyrus.header

cyrus_spool/mail/v/user/vishal^kesariya/Trash/cyrus.index

cyrus_spool/mail/v/user/vishal^kesariya/Trash/cyrus.squat

cyrus_spool/mail/v/user/vishal^kesariya/cyrus.cache

cyrus_spool/mail/v/user/vishal^kesariya/cyrus.header

cyrus_spool/mail/v/user/vishal^kesariya/cyrus.index

cyrus_spool/mail/v/user/vishal^kesariya/cyrus.squat

 sent 13143983 bytes  received 53620 bytes  2030400.46 bytes/sec

total size is 93210136  speedup is 7.06

 (6) After completing successful
backup you must unmount your snapshot partition &
also remove it.

 [root@lonaiswmft001 ~]# umount /data_snapshot/

 [root@lonaiswmft001 ~]# lvremove -f
/dev/VolGroup00/data_snapshot

 
Logical volume "data_snapshot" successfully
removed

 (7) Below is the SHELL SCRIPT which
you can run it daily by scheduling it in crontab
created from above commands & generate a log file so that you can verify
daily backups whether it was successful / any error
reported.

 [root@lonaiswmft001 ~]# cat /etc/scripts/mail-snapshot.sh

 #!/bin/bash

 echo "`date` **** CYRUS BACKUP START
****" >> /var/log/cyrus-backup.log

#Uncommnet
the below line to automate the backup

echo "`date` Stopping cyrus." >> /var/log/cyrus-backup.log

/etc/init.d/cyrus-imapd stop >> /var/log/cyrus-backup.log  
                                   
èTo
stop all the transactions on data

 echo "`date` Creating snapshot volume."
>> /var/log/cyrus-backup.log

lvcreate --size 10G --snapshot --name data_snapshot /dev/VolGroup00/LogVol05   

sleep 20

 echo "`date` Starting cyrus." >> /var/log/cyrus-backup.log

/etc/init.d/cyrus-imapd start >> /var/log/cyrus-backup.log 
                                   
èStart
transactions again after creating snapshot of data volume

 echo "`date` Mounting snapshot volume."
>> /var/log/cyrus-backup.log

mount /dev/VolGroup00/data_snapshot
/data_snapshot/ >> /var/log/cyrus-backup.log

 echo "`date` Syncing to backup server."
>> /var/log/cyrus-backup.log

rsync -avze ssh
--delete /data_snapshot/data/cyrus/
backup@192.168.181.121:/backup/aibackups/LONAISWMFT001/wmfdata/cyrus

 echo "`date` Unmounting snapshot volume." >> /var/log/cyrus-backup.log

umount /data_snapshot/ >> /var/log/cyrus-backup.log

 echo "`date` Removing snapshot volume."
>> /var/log/cyrus-backup.log

 lvremove -f /dev/VolGroup00/data_snapshot >>
/var/log/cyrus-backup.log

 echo "`date` **** CYRUS BACKUP STOP
****" >> /var/log/cyrus-backup.log

Attachment	Size
RSYNC-Backup-method_script.doc	59 KB