Skip to content

Commit 100e804

Browse files
committed
sync cpdup with dragonfly current.
1 parent 2a1a289 commit 100e804

15 files changed

Lines changed: 2124 additions & 1572 deletions

File tree

bin/cpdup/BACKUPS

Lines changed: 90 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,3 @@
1-
$DragonFly: src/bin/cpdup/BACKUPS,v 1.4 2007/05/17 08:19:00 swildner Exp $
2-
31
INCREMENTAL BACKUP HOWTO
42

53
This document describes one of several ways to set up a LAN backup and
@@ -17,7 +15,8 @@ $DragonFly: src/bin/cpdup/BACKUPS,v 1.4 2007/05/17 08:19:00 swildner Exp $
1715
The easiest way to create a LAN backup box is to NFS mount all your
1816
backup clients onto the backup box. It is also possible to use cpdup's
1917
remote host feature to access your client boxes but that requires root
20-
access to the client boxes and is not described here.
18+
access to the client boxes and is not described here. (But see the
19+
sections "OFF-SITE BACKUPS" and "SSH SECURITY TIPS" below.)
2120

2221
Create a directory on the backup machine called /nfs, a subdirectory
2322
foreach remote client, and subdirectories for each partition on each
@@ -34,7 +33,7 @@ $DragonFly: src/bin/cpdup/BACKUPS,v 1.4 2007/05/17 08:19:00 swildner Exp $
3433
Before you actually do the NFS mount, create a dummy file for each
3534
mount point that can be used by scripts to detect when an NFS mount
3635
has not been done. Scripts can thus avoid a common failure scenario
37-
and not accidently cpdup an empty mount point to the backup partition
36+
and not accidentally cpdup an empty mount point to the backup partition
3837
(destroying that day's backup in the process).
3938

4039
touch /nfs/box1/home/NOT_MOUNTED
@@ -84,7 +83,7 @@ $DragonFly: src/bin/cpdup/BACKUPS,v 1.4 2007/05/17 08:19:00 swildner Exp $
8483
PART 3 - DO AN INCREMENTAL BACKUP
8584

8685
An incremental backup is exactly the same as a level 0 backup EXCEPT
87-
you use the -H option to specify the location of the most recent
86+
you use the -H option to specify the location of the most recent
8887
completed backup. We simply maintain the handy softlink pointing at
8988
the most recent completed backup and the cpdup required to do this
9089
becomes trivial.
@@ -115,7 +114,7 @@ $DragonFly: src/bin/cpdup/BACKUPS,v 1.4 2007/05/17 08:19:00 swildner Exp $
115114
and copy the file instead. Generally speaking only a few special cases
116115
will hit the hardlink limit for a filesystem. For example, the
117116
CVS/Root file in a checked out cvs repository is often hardlinked, and
118-
the sheer number of hardlinked 'Root' files multiplied by the number
117+
the sheer number of hardlinked 'Root' files multiplied by the number
119118
of backups can often hit the filesystem hardlink limit.
120119

121120
PART 4 - DO AN INCREMENTAL VERIFIED BACKUP
@@ -124,7 +123,7 @@ $DragonFly: src/bin/cpdup/BACKUPS,v 1.4 2007/05/17 08:19:00 swildner Exp $
124123
might exist on the physical /backup disk in only one place even though
125124
it may be present in dozens of daily mirrors. To ensure that the
126125
file being hardlinked does not get corrupted cpdup's -f option can be
127-
used in conjunction with -H to force cpdup to validate the contents
126+
used in conjunction with -H to force cpdup to validate the contents
128127
of the file, even if all the stat info looks identical.
129128

130129
cpdup -f -H /backup/mirrors/box1 ...
@@ -136,7 +135,7 @@ $DragonFly: src/bin/cpdup/BACKUPS,v 1.4 2007/05/17 08:19:00 swildner Exp $
136135
will have a distinct copy of all the file data and you will quickly
137136
run out of space in your /backup partition.
138137

139-
MAINTAINANCE OF THE "/backup" DIRECTORY
138+
MAINTENANCE OF THE "/backup" DIRECTORY
140139

141140
Now, clearly you are going to run out of space in /backup if you keep
142141
doing this, but you may be surprised at just how many daily incrementals
@@ -153,14 +152,14 @@ $DragonFly: src/bin/cpdup/BACKUPS,v 1.4 2007/05/17 08:19:00 swildner Exp $
153152
Making an off-site backup involves similar methodology, but you use
154153
cpdup's remote host capability to generate the backup. To avoid
155154
complications it is usually best to take a mirror already generated on
156-
your LAN backup box and copy that to the remote box.
155+
your LAN backup box and copy that to the remote box.
157156

158157
The remote backup box does not use NFS, so setup is trivial. Just
159158
create your super-large /backup partition and mkdir /backup/mirrors.
160159
Your LAN backup box will need root access via ssh to your remote backup
161-
box.
160+
box. See the section "SSH SECURITY TIPS" below.
162161

163-
You can use the handy softlink to get the latest 'box1.date' mirror
162+
You can use the handy softlink to get the latest 'box1.date' mirror
164163
directory and since the mirror is all in one partition you can just
165164
cpdup the entire machine in one command. Use the same dated directory
166165
name on the remote box, so:
@@ -231,7 +230,7 @@ $DragonFly: src/bin/cpdup/BACKUPS,v 1.4 2007/05/17 08:19:00 swildner Exp $
231230

232231
Once done you can do incremental backups using './do_mirror 1' to do a
233232
verified incremental, or './do_mirror 2' to do a stat-optimized
234-
incremental. You can enable the cron jobs that run do_mirror and
233+
incremental. You can enable the cron jobs that run do_mirror and
235234
do_cleanup now.
236235

237236
--
@@ -261,3 +260,82 @@ $DragonFly: src/bin/cpdup/BACKUPS,v 1.4 2007/05/17 08:19:00 swildner Exp $
261260
should do a tower-of-hanoi removal
262261

263262

263+
SSH SECURITY TIPS
264+
265+
To allow root access via ssh, add the following line to your sshd
266+
configuration on the client boxes (typically /etc/ssh/sshd_config):
267+
268+
PermitRootLogin forced-commands-only
269+
270+
If your OpenSSH version is too old to recognize that setting, you
271+
should update to a more recent version immediately.
272+
Restart sshd for the settings to take effect.
273+
274+
On the backup machine, create a special backup key for root:
275+
276+
mkdir /root/.ssh # if it doesn't already exist
277+
cd /root/.ssh
278+
ssh-keygen -t dsa -N "" -f backup-key
279+
280+
You now have a key pair, consisting of a secret key called "backup-key"
281+
and a public key called "backup-key.pub". The secret key must *NEVER*
282+
leave the backup machine nor be disclosed in any way! Note that we
283+
haven't procted the secret key with a passphrase (-N "") because it
284+
will be used by cron jobs where no passphrase can be entered.
285+
286+
On the client boxes, create a file /root/.ssh/authorized_keys.
287+
It should contain just this line:
288+
289+
command="/usr/local/bin/cpdup -S",from="<BAKHOST>",no-pty,
290+
no-port-forwarding,no-X11-forwarding,no-agent-forwarding <PUBKEY>
291+
292+
This must be on one long line; it has been broken up here for
293+
readability only. Note that the options must be separated by commas
294+
*ONLY* (no spaces). Replace <BAKHOST> with the IP address or DNS name
295+
of the backup machine. Replace <PUBKEY> with the contents of the
296+
file /root/.ssh/backup-key.pub from the backup machine (the public key,
297+
not the secret key!). It typically starts with "ssh-dss" followed by
298+
a long character sequence that looks like line noise, followed by a
299+
comment that typically indicates who created the key.
300+
301+
The format of the authorized_keys file is documented in the sshd(8)
302+
manual page. Please refer to it for more details.
303+
304+
If you have done all of the above correctly, then the root user on the
305+
backup machine will be able to log into the client boxes as root and
306+
execute "/usr/local/bin/cpdup -S", but nothing else.
307+
308+
To further improve security, you can place the slave cpdup on the client
309+
machine into read-only mode by adding the -R option. In this case, the
310+
line from the authorized_keys file should begin as follows:
311+
312+
command="/usr/local/bin/cpdup -RS",from="<BAKHOST>",etc...
313+
314+
If you do that, your backup server can only pull backups from the client
315+
machine, but it won't be able to change anything on it. That is, you
316+
cannot use the client machine as a remote target. So, if an attacker
317+
manages to be able to execute commands on your backup machine, he won't
318+
be able to do any harm to your clients. This also protects against
319+
human errors, e.g. accidentally swapping source and destination.
320+
321+
By the way, it doesn't really matter much whether you specify the -R
322+
option when running cpdup on the backup machine. If you do it, then
323+
the -R option will be passed to the slave, but the command="..." entry
324+
from the authorized_keys file overides it anyway, so the slave always
325+
runs with the -R option.
326+
327+
When using cpdup on the backup machine, make sure that the right key is
328+
used by passing the -i option to the ssh command:
329+
330+
cpdup -F -i/root/.ssh/backup-key ...
331+
332+
If one or both of the machines involved has a slow processor, it might
333+
be worthwhile to use a faster encryption algorithm, for example:
334+
335+
cpdup -F -cblowfish-cbc ...
336+
337+
If your OpenSSH version has been patched to support unencrypted transfers
338+
*AND* you trust the physical network between the machines involved, you
339+
might want to disable encryption altogether:
340+
341+
cpdup -F -cnone ...

bin/cpdup/Makefile

Lines changed: 9 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,17 @@
1-
# $DragonFly: src/bin/cpdup/Makefile,v 1.7 2008/04/10 22:09:08 dillon Exp $
2-
31
PROG= cpdup
4-
SRCS= cpdup.c hcproto.c hclink.c misc.c fsmid.c
5-
6-
.if !defined(NOPTHREADS) && !defined(BOOTSTRAPPING)
7-
CFLAGS += -DUSE_PTHREADS=1
8-
LIBADD+= pthread
9-
.endif
2+
SRCS= cpdup.c hcproto.c hclink.c misc.c
103

4+
.if defined(BOOTSTRAPPING) || defined(RESCUE)
5+
# For boostrapping buildworld the md5 functionality is not needed
6+
CFLAGS+= -DNOMD5
7+
.else
118
.if !defined(NOMD5)
12-
SRCS+= md5.c
9+
SRCS+= md5.c
10+
LIBADD+= crypto md
11+
.endif
1312
.endif
1413

15-
WARNS?= 1
16-
17-
LIBADD+= md
14+
WARNS?= 1
1815

1916
.include <bsd.prog.mk>
2017

bin/cpdup/PORTING

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,18 +15,16 @@ $DragonFly: src/bin/cpdup/PORTING,v 1.2 2008/04/11 17:33:11 dillon Exp $
1515
This script should suffice when compiling under linux. For some reason
1616
that I cannot fathom, linux defaults to using a 32 bit off_t. It makes
1717
no sense at all to me why they would want to do this, BSDs went to
18-
a 64 bit default off_t over 15 years ago. In anycase, fixing it
18+
a 64 bit default off_t over 15 years ago. In anycase, fixing it
1919
requires a few -D options.
2020

2121
#!/bin/csh
2222
#
2323
cd cpdup
2424
rm -f md5.c
2525
rm -f *.o
26-
cc -D__unused= -D_GNU_SOURCE -D__USE_FILE_OFFSET64 -DNOMD5 \
27-
-DUSE_PTHREADS -pthread *.c -c
28-
cc -D__unused= -D_GNU_SOURCE -D__USE_FILE_OFFSET64 -DNOMD5 \
29-
-DUSE_PTHREADS -pthread *.o -o ~/bin/cpdup
26+
cc -D__unused= -D_GNU_SOURCE -D__USE_FILE_OFFSET64 -DNOMD5 *.c -c
27+
cc -D__unused= -D_GNU_SOURCE -D__USE_FILE_OFFSET64 -DNOMD5 *.o -o ~/bin/cpdup
3028

3129
BACKUP SCRIPT MODIFICATIONS - you will almost certainly have to adjust
3230
the do_cleanup script to extract the proper field(s) from the df output.

0 commit comments

Comments
 (0)