[ale] has anyone successfully used RSYNC to....

Wolf Halton wolf.halton at gmail.com
Sun Mar 4 20:48:01 EST 2012 

On Sun, Mar 4, 2012 at 6:05 PM, Michael H. Warfield <mhw at wittsend.com> wrote:
> On Sun, 2012-03-04 at 16:39 -0500, Wolf Halton wrote:
>> Replacing the booted os with an archived copy may also replace /proc.
>> Though  there may be a way  to avoid that.
>
> No.
>
> /proc is the proc file system.  It's special.  Like /sys (sysfs) and
> pty, it's a pseudo filesystem.
>
> 1) It should NOT be backed up.  That's worthless.  My backups using
> rsync always include -x and then only specify the specific file systems
> I want to back up.
>
> 2) /proc and /sys will always be recreated by the OS.  They are not real
> file systems.
>
> Mike
>

Mike,

So, you _can_ restore a partition holding a running operating system with rsync?
I knew /sys and /proc are special pseudo file-systems.  I never tried
replacing a running system with an rsynced copy.  Though to be honest
I am not sure the OP is really about that, but more about rsyncing a
data-storage partition, which does seem likely to succeed, as long as
the files are not in use when the rsync is done.

Wolf



>> http://sourcefreedom.com
>> On Mar 4, 2012 1:48 PM, "Courtney Thomas" <courtneycthomas at bellsouth.net>
>> wrote:
>>
>> > On 03/03/12 15:03, Wolf Halton wrote:
>> > > On Fri, Mar 2, 2012 at 2:23 PM, Ed Cashin<ecashin at noserose.net>  wrote:
>> > >> I've done that, but more often, "NEWdrive" is bigger than "HD1" and
>> > >> "HD2".  I don't have the exact commands handy, but I can describe
>> > >> them, and whatever anybody tells you about this, make sure you
>> > >> actually do the procedure you choose *before* you're forced to.
>> > >> There's inevitably some learning and tweaking, even if you get recipes
>> > >> from other people.
>> > >>
>> > >> So in general, for rsync, I used "-a" plus the option that preserves
>> > >> sparse files ("-S") and the one ("-H") that preserves hard links, and
>> > >> often the one ("-c") that uses checksums to ensure data integrity.  Of
>> > >> course, because rsync works over ssh, it's easiest to keep HD2 in a
>> > >> separate machine, so that you can rsync over the network.  I use "-x"
>> > >> to stay on one filesystem instead of going into stuff like /proc.
>> > >>
>> > >> Some UNTESTED illustrative example commands from memory are below,
>> > >> assuming the only real filesystems on the localhost are / and /home.
>> > >>
>> > >>   rsync -avSHx / otherhost:/backup-of-hd1
>> > >>   rsync -avSHx /home/ otherhost:/backup-of-hd1/home
>> > >>
>> > >> The most tricky thing about rsync is probably that trailing slash in
>> > >> the second command.  It means "the contents of".  Without it, rsync
>> > >> would make a new directory called "home" inside /backup-of-hd1/home on
>> > >> otherhost.
>> > >>
>> > >> For the boot sector, I would copy it to a local file before the rsync
>> > >> with dd.  But I no longer think that's as good as getting very
>> > >> comfortable with the installation of a boot loader like grub or lilo.
>> > >> There's rarely information in the boot sector that cannot be replaced
>> > >> with something just as good when you're moving onto NEWdrive.  Most of
>> > >> the time you can live boot a CD and run grub or something to install
>> > >> the boot loader in the MBR of NEWdrive.
>> > >>
>> > >> For the partition table, I used to use sfdisk to dump the partition
>> > >> info to a format that sfdisk can read to recreate the same
>> > >> partitioning scheme.  Now I don't think that's necessary either,
>> > >> because if you're doing a high-level file-based backup like rsync
>> > >> does, you can just partition NEWdrive however you want, and it's
>> > >> probably going to be bigger anyway, so good.  And maybe you've learned
>> > >> something and NEWdrive will have a better partitioning scheme than HD1
>> > >> did.
>> > >>
>> > >> For formatting, I think you mean filesystem creation.  In that case,
>> > >> again, maybe there's a better choice for the filesystem on NEWdrive
>> > >> than the one you were using on HD1, so it depends.  Because the backup
>> > >> is at the file level, you can choose whichever filesystem is best for
>> > >> your needs.  For example, maybe you have 3 partitions on MYNEWDRIVE
>> > >> and want ext3, so you could do something like "mkfs -t ext3
>> > >> /dev/MYNEWDRIVE1" and "mkswap /dev/MYNEWDRIVE2" and "mkfs -t ext3
>> > >> /dev/MYNEWDRIVE3".
>> > >>
>> > >> After that, mount that new filesystem use the same rsync options to
>> > >> copy the files from the mounted filesystem of HD2 to the new
>> > >> filesystem on NEWdrive.  Something like this is what you'd do on
>> > >> "otherhost" after those filesystem creation commands:
>> > >>
>> > >>   mkdir /mnt/NEWdrive
>> > >>   mount /dev/MYNEWDRIVE1 /mnt/NEWdrive
>> > >>   mount /dev/MYNEWDRIVE3 /mnt/NEWdrive/home
>> > >>   rsync -avSHx --progress /backup-of-hd1/ /mnt/NEWdrive
>> > >>   vi /mnt/NEWdrive/etc/fstab  # if you've changed the partitioning
>> > >> scheme or filesystems
>> > >>
>> > >> These commands are pretty darn close, but they're untested and
>> > >> provided without warranty.  If anyone sees errors, please chime in.
>> > >>
>> > >> OP, please practice using them inside virtual machines or something
>> > >> until you're as comfortable as anyone else if you intend to do this
>> > >> when other people are counting on it working successfully.
>> > >>
>> > >> On Fri, Mar 2, 2012 at 1:38 PM, Courtney Thomas
>> > >> <courtneycthomas at bellsouth.net>  wrote:
>> > >>> ...(1) back up all their filesystems from HD1 to an archive hard drive
>> > HD2,
>> > >>>
>> > >>> (2) then replaced HD1 with a NEWdrive which was partitioned, labeled
>> > and
>> > >>> formatted exactly duplicating the replaced HD1, and then
>> > >>>
>> > >>> (3) RSYNCed the archived files on HD2 to the NEWdrive, with the result
>> > that:
>> > >>>
>> > >>> the NEWdrive was not only without fault, but that the previous
>> > >>> functionality
>> > >>> of HD1 was exactly duplicated ?
>> > >>>
>> > >>> If yes to all 3, would anyone care to share
>> > >>>
>> > >>> (1) the RSYNC backup command for HD1 ->  HD2
>> > >>> (2) their partition, label and format commands for NEWdrive
>> > >>> (3) the RSYNC restore command for HD2 ->  NEWdrive
>> > >>>
>> > >>> This is obviously of overwhelming importance to all and if anyone can
>> > >>> point to a
>> > >>> reliable source, or provide it outright themself, for the above sought
>> > >>> command
>> > >>> summary, my gratitude would be boundless  :-)
>> > >>>
>> > >>> I am aware that rsync, fdisk, e2label and mkfs are all documented
>> > commands,
>> > >>> but a successful usage, as herein outlined, is, at least in my case,
>> > >>> frustrating
>> > >>> days away of trial and error, having already wasting such time in
>> > >>> pursuit of such
>> > >>> an outcome using 'dump'.
>> > >>>
>> > >>> Sincerely,
>> > >>>
>> > >>> Courtney
>> > >>> _______________________________________________
>> > >>> Ale mailing list
>> > >>> Ale at ale.org
>> > >>> http://mail.ale.org/mailman/listinfo/ale
>> > >>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> > >>> http://mail.ale.org/mailman/listinfo
>> > >>
>> > >>
>> > >> --
>> > >>    Ed Cashin<ecashin at noserose.net>
>> > >>    http://noserose.net/e/
>> > >>    http://www.coraid.com/
>> > >>
>> > >> _______________________________________________
>> > >> Ale mailing list
>> > >> Ale at ale.org
>> > >> http://mail.ale.org/mailman/listinfo/ale
>> > >> See JOBS, ANNOUNCE and SCHOOLS lists at
>> > >> http://mail.ale.org/mailman/listinfo
>> > > I am using rsync to back up zipped copies of configs and whatever else
>> > > is needed.  This is not a 1:1 of the whole originating disc
>> > > partitions, as you say you want - but just the important pieces.
>> > > Presuming the original drive doesn't fail, but only some piece of the
>> > > puzzle failed, it is still a manual task to recover the piece you
>> > > need.  My thinking is that if the original drive fails, I have either
>> > > ZRAID or a system snapshot to return me to the original OS state, and
>> > > the only thing I am missing is the recent data.
>> > >
>> > > tar xzf thearchive.tar.gz
>> > > rsync -avz the-directory-you-need  root at yourserver.com:
>> > /path-towhere-it-goes/
>> > >
>> > > This is one of the most common errors with rsync as well, in that it
>> > > is a basic reversal of the original rsync command. B ->  A rather than
>> > > A ->  B
>> > > If you are going to be running your rsync from a 3rd machine, as you
>> > > would have to be doing if you are replacing whole disc-images, I
>> > > think, you would have full paths at both ends of your command.
>> > >
>> > > Ed ++
>> > > Always test major file-moving on VMs before you trust the process on
>> > > your production machines.
>> > >
>> > > Wolf
>> > >
>> > Wolf,
>> >
>> > Thank you for your response.
>> >
>> > Having not used rsync, why are 3 machines necessary to make whole disk
>> > copies ?
>> >
>> > Why can't an unused disk on the syncing machine be sync'd with the
>> > desired disk, wherever it's located, including on the syncing machine ?
>> >
>> > Cordially,
>> >
>> > Courtney
>> > _______________________________________________
>> > Ale mailing list
>> > Ale at ale.org
>> > http://mail.ale.org/mailman/listinfo/ale
>> > See JOBS, ANNOUNCE and SCHOOLS lists at
>> > http://mail.ale.org/mailman/listinfo
>> >
>>
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> http://mail.ale.org/mailman/listinfo/ale
>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> http://mail.ale.org/mailman/listinfo
>
> --
> Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
>   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
>   NIC whois: MHW9          | An optimist believes we live in the best of all
>  PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>



-- 
This Apt Has Super Cow Powers - http://sourcefreedom.com
Advancing Libraries Together - http://LYRASIS.org

More information about the Ale mailing list