［每周討論專題］－－第六期－－WEB伺服器的數據遠程同步
［每周討論專題］－－第六期－－WEB伺服器的數據遠程同步
每周討論專題【第六期】.......................................................點這裡查看其他討論專題
WEB伺服器的數據遠程同步
本期討論主旨，WEB伺服器的數據遠程同步！！！
感謝 wingger 點題
本期主持：wingger
《解決方案》
［每周討論專題］－－第六期－－WEB伺服器的數據遠程同步
兩個WEB遠 程同步可以嗎
或是如何負載勻橫
《解決方案》
［每周討論專題］－－第六期－－WEB伺服器的數據遠程同步
I think only web can't do so.maybe you can use scp or sftp/ftp do so.
《解決方案》
［每周討論專題］－－第六期－－WEB伺服器的數據遠程同步
如果是數據同步，而且使用了資料庫的話，可以考慮採用資料庫的同步複製功能；
當然也可以對web系統自己預留介面，定期自動查詢同步。最笨了，而且性能不好控制
《解決方案》
［每周討論專題］－－第六期－－WEB伺服器的數據遠程同步
rsync這個軟體可以實現mirror.
《解決方案》
［每周討論專題］－－第六期－－WEB伺服器的數據遠程同步
使用rsync
下面是rsync網站上的一些例子
每隔七天將數據往中心伺服器做增量備份
#!/bin/sh
# This script does personal backups to a rsync backup server. You will end up
# with a 7 day rotating incremental backup. The incrementals will go
# into subdirectories named after the day of the week, and the current
# full backup goes into a directory called "current"
# tridge@linuxcare.com
# directory to backup
BDIR=/home/$USER
# excludes file - this contains a wildcard pattern per line of files to exclude
EXCLUDES=$HOME/cron/excludes
# the name of the backup machine
BSERVER=owl
# your password on the backup server
export RSYNC_PASSWORD=XXXXXX
########################################################################
BACKUPDIR=`date +%A`
OPTS="--force --ignore-errors --delete-excluded --exclude-from=$EXCLUDES
--delete --backup --backup-dir=/$BACKUPDIR -a"
export PATH=$PATH:/bin:/usr/bin:/usr/local/bin
# the following line clears the last weeks incremental directory
[ -d $HOME/emptydir ] || mkdir $HOME/emptydir
rsync --delete -a $HOME/emptydir/ $BSERVER::$USER/$BACKUPDIR/
rmdir $HOME/emptydir
# now the actual transfer
rsync $OPTS $BDIR $BSERVER::$USER/current
備份至一個空閑的硬碟
I do local backups on several of my machines using rsync. I have an
extra disk installed that can hold all the contents of the main
disk. I then have a nightly cron job that backs up the main disk to
the backup. This is the script I use on one of those machines.
#!/bin/sh
export PATH=/usr/local/bin:/usr/bin:/bin
LIST="rootfs usr data data2"
for d in $LIST; do
mount /backup/$d
rsync -ax --exclude fstab --delete /$d/ /backup/$d/
umount /backup/$d
done
DAY=`date "+%A"`
rsync -a --delete /usr/local/apache /data2/backups/$DAY
rsync -a --delete /data/solid /data2/backups/$DAY
The first part does the backup on the spare disk. The second part
backs up the critical parts to daily directories. I also backup the
critical parts using a rsync over ssh to a remote machine.
對vger的cvs樹進行鏡像
The vger.rutgers.edu cvs tree is mirrored onto cvs.samba.org via
anonymous rsync using the following script.
#!/bin/bash
cd /var/www/cvs/vger/
PATH=/usr/local/bin:/usr/freeware/bin:/usr/bin:/bin
RUN=`lps x | grep rsync | grep -v grep | wc -l`
if [ "$RUN" -gt 0 ]; then
echo already running
exit 1
fi
rsync -az vger.rutgers.edu::cvs/CVSROOT/ChangeLog $HOME/ChangeLog
sum1=`sum $HOME/ChangeLog`
sum2=`sum /var/www/cvs/vger/CVSROOT/ChangeLog`
if [ "$sum1" = "$sum2" ]; then
echo nothing to do
exit 0
fi
rsync -az --delete --force vger.rutgers.edu::cvs/ /var/www/cvs/vger/
exit 0
Note in particular the initial rsync of the ChangeLog to determine if
anything has changed. This could be omitted but it would mean that the
rsyncd on vger would have to build a complete listing of the cvs area
at each run. As most of the time nothing will have changed I wanted to
save the time on vger by only doing a full rsync if the ChangeLog has
changed. This helped quite a lot because vger is low on memory and
generally quite heavily loaded, so doing a listing on such a large
tree every hour would have been excessive.
在家自動備份
我用rsync to backup my wifes home directory across a modem link each
night. The cron job looks like this
#!/bin/sh
cd ~susan
{
echo
date
dest=~/backup/`date +%A`
mkdir $dest.new
find . -xdev -type f \( -mtime 0 -or -mtime 1 \) -exec cp -aPv "{}"
$dest.new \;
cnt=`find $dest.new -type f | wc -l`
if [ $cnt -gt 0 ]; then
rm -rf $dest
mv $dest.new $dest
fi
rm -rf $dest.new
rsync -Cavze ssh . samba:backup
} >;>; ~/backup/backup.log 2>;&1
note that most of this script isn't anything to do with rsync, it just
creates a daily backup of Susans work in a ~susan/backup/ directory so
she can retrieve any version from the last week. The last line does
the rsync of her directory across the modem link to the host
samba. Note that I am using the -C option which allows me to add
entries to .cvsignore for stuff that doesn't need to be backed up.
Fancy footwork with remote file lists
One little known feature of rsync is the fact that when run over a
remote shell (such as rsh or ssh) you can give any shell command as
the remote file list. The shell command is expanded by your remote
shell before rsync is called. For example, see if you can work out
what this does:
rsync -avR remote:'`find /home -name "*."`' /tmp/
note that that is backquotes enclosed by quotes (some browsers don't
show that correctly).
《解決方案》
［每周討論專題］－－第六期－－WEB伺服器的數據遠程同步
Rsync for Windows（我沒測試成功，有興趣的可以試試，我有空會把這篇翻譯成中文）
導言
我寫這份文檔可以幫助那些想通過使用Rsync備份 windowsrs 到LINUX servers的人或 Windows 工作站/伺服器. 這是經過測試過的。
--------------------------------------------------------------------------------
1: Cygwin環境安裝
你需要下載和安裝Cygwin系統 .
安裝時記住要選擇 Rsync from the +Net package list, 和a suitable Editor (因為這個系統不會默認安裝). 要用到一些編輯器 Pico (在 +Mail之下的part of Pine living ) 和 Nano (living under +Editors).
下面這段都是講如何在win下設置環境變數的
在環境變數中增加C:\Cygwin\bin.
否則 會報錯：apps called from outside Cygwin will fail.
On Windows 2000/XP, open the Control Panel and double click on the System applet. Click on the Advanced tab, then click the Environment Variables button. Double click on the PATH statement in the 'System Variable' screen (lower of the two), add the path on the end, and click OK. Click OK to close the Environment Variables screen, then click OK to close the System Properties dialogue box. The path will be dynamically reloaded (no need to reboot).
注意：
If the end of the path looks something like this: C:\Somepath don't forget to add ; before you add the C:\Cygwin\bin; e.g. C:\Somepath;C:\Cygwin\bin;
--------------------------------------------------------------------------------
2: 服務端分LINUX和WIN服務端兩種
Linux Rsync 服務安裝
1. 確認 Rsync 已經安裝在你的LINUX上 .
2. Choose the path for your backup area: This can be on a per-user basis (backup a Users data to their /home/user area) or on a system level basis (a single machine backing up to one directory.)
Real world example:
All rsync data on the Gaztronics Server sits on /dev/hda2 as /data/rsync_dump
3. Create the /etc/rsyncd.conf and /etc/rsyncd.secrets files.
Here is an example of an rsyncd.conf file where the backup area drops into the user 'Fred's' home drive:
path = /home/fred/backup
comment = Fred's Offsite storage area (requires authentication)
uid = fred
gid = users
read only = false
auth users = fred
secrets file = /etc/rsyncd.secrets
The permissions for this file should be: -rw-r--r-- (644) and root root.
The corresponding rsyncd.secrets file contains the following entry:
fred:BackUpPassword
The permissions for this file should be: -rw------- (600) and root root.
4. Start Rsync in daemon mode.
Linux Tip: Linux Distros usually run rsync from xinetd. You might need to run /usr/sbin/setup (Red Hat) and select 'rsync' in the 'System Services'; or you can edit the 'rsync' file in /etc/xinetd.d and set disable = no (don't forget to restart xinetd!).
Windows Rsync 服務安裝
Setting up Rsync as a Server under Windows is a little more tricky, due to the differences in security and paths, and the inability to use the 'authenticate user' mode of Rsync. For this reason, I would not recommend setting up an Rsync Server on Windows for use over the Internet. Keep the installation within a secure Local Area Network.
Method update: Windows 2003 Server has thrown a spanner in the works. Microsoft have set the paranoid level to maximum which has resulted in Cygwin based services failing to start. This method has been updated to take this into account.
This method comes without warranty, but it should work for: Windows NT 4.0 Server; Windows NT 4.0 Workstation; Windows 2000 Server; Windows 2000 Workstation; Windows XP Workstation.
Method 1. Without installing Cygwin in the full: You might be interested in this Rsync Server project. The Windows package installs the bare essentials to create an Rsync Server.
This method has yet to be tested with 2003 Server!
Method 2. Install Cygwin in full and use its directory structure for storage. (This is recommended if you are new to Linux as the directory permissions are stored *nix style and you will need Cygwin to change them - you cannot change them from Windows!)
Both methods require a valid rsyncd.conf, such as the one below.
--------------------------------------------------------------------------------
Method 2. Step 1: Install Cygwin as in Section 1.
Method 2. Step 2: Choose an area to backup the files to. (In this example I have used /var/rsync_dump in the Cygwin tree.)
Method 2. Step 3: Create the /etc/rsyncd.conf file, as in the example below:
use chroot = false
strict modes = false
path = /cygdrive/c/cygwin/var/rsync_dump
comment = Rsync storage area
read only = false
Note: The path = /cygdrive/c/cygwin/var/rsync_dump looks a little odd. This is Cygwin convention for defining Windows paths from within a *nix emulator.
Method 2. Step 4: If you are setting up on Windows 2003 Server (otherwise skip to the next step):
(i) Open the Windows File Explorer and go to the C: drive.
(ii) Right click on the 'Cygwin' directory and select 'Properties'.
(iii) Click on the 'Security' tab. The user 'Administrator' should be the first in the list and it will not have any permissions set for this folder.
(If the user 'Administrator' is not listed, you will need to add it.)
(iv) Tick the 'Allow - Full Control' box in the "Permissions for Administrator" window.
(v) Click the Advanced button and tick the box for "Replace permission entries on all child objects with entries shown here that apply to child objects".
(vi) Click the Apply button to set the permissions.
(vii) Click the OK button to close the Advanced settings dialogue box.
(viii) Click the OK button to close the Cygwin properties dialogue box.
Method 2. Step 5 Install Rsync as a Service from a 'Command Prompt' window with the following command line:
cygrunsrv.exe -I "Rsync" -p /cygdrive/c/cygwin/bin/rsync.exe -a "--config=/cygdrive/c/cygwin/etc/rsyncd.conf --daemon --no-detach"
-f "Rsync" -u Administrator -w password
Note: This is all one line!
The section -u Administrator -w password installs the service to run as the user 'Administrator' (password is the Administrator's account password) and is required by Windows 2003 Server, else the service will fail to start correctly. The extra settings have not been needed for Windows NT 4.0 or Windows 2000, but may be a good idea.
Windows Tip: Once you have the service running as Administrator, you can, for good security practice, create an account with Admin priveleges which Rsync can use. Remember to change the permissions on the C:\Cygwin directory and for the Service.
Method 2. Final Step From the same command prompt, start the Service with net start rsync.
If all has gone well, you should be ready to accept incoming client connections.
--------------------------------------------------------------------------------
3: 客戶端
In order to backup your Windows machine effectively, I would recommend the use of a batch file, as in the following:
@ cls
@ echo off
rem Rsync job control file
C:\Cygwin\bin\rsync -vrtz --password-file=c:\cygwin\secret --delete /cygdrive/d/Data fred@company.com::computername
An explanation:
C:\Cygwin\bin\rsync - is the full path to 'rysync.exe'.
-vrtz - See the rsync documentation for details.
--password-file=c:\cygwin\secret - Path to 'secret' file. (Note: Remember this is for backup to a Linux based Rsync Server; a Windows based Rsync Server cannot authenticate!)
--delete - delete remote files that are deleted locally.
/cygdrive/d/Data - in this example means D:\Data.
fred@company.com::computername - is the user ID, hostname (can be IP address if over Local Network), and the module connection name (in this example 'computername').
Windows Tip: We assume you wish to run the backup automatically! In that case, you will have to create the 'secret' file that Rsync uses for its authenticating password. Here is how to do that:
1. Login to Cygwin.
2. Create a file called 'secret' in the root of the Cygwin application (i.e. cd /) with the Rsync server password (in this example: 'BackUpPassword') and give it 600 permissions (chmod 600 secret). This is the file '--password-file=c:\cygwin\secret' as referred to above.
3. Exit Cygwin.
Windows Tip: You may call this batch file from the Startup Group, the Scheduler (and AT scheduler), or from the Logon / Logoff features of the Group Policy (Win2k/XP) by running gpedit.msc.
--------------------------------------------------------------------------------
Further reading: check out the documentation on the Rsync website.
Happy backing up!!
《解決方案》
［每周討論專題］－－第六期－－WEB伺服器的數據遠程同步
版主去哪了，
《解決方案》
［每周討論專題］－－第六期－－WEB伺服器的數據遠程同步
其實遠程數據同步的方案是比較多的。
如果數據量不大用rsync就完全可以實現了，可以參看
http://www.fanqiang.com/a1/b1/20011019/0800011478.html
另外如果數據量大的話可以採用rsync＋sgi－fam
sgi－fam在redhat AS3 裡面就有，不用編譯內核
具體做法可以參看
http://oldsite.linuxaid.com.cn/training/showtri.jsp?i=221
如果是專業的網站就可以考慮用商業的數據同步軟體來做
比如snapmirror等等，甚至可以用netapp等專業的存儲設備加以實現。
《解決方案》
［每周討論專題］－－第六期－－WEB伺服器的數據遠程同步
如果用root根用戶，用rsync之前最好小心點，此命令像rm命令一樣「殺傷力」具大。
