[ale] rsync

Wed Aug 28 14:08:06 EDT 2013

Thanks all.

In this case, the remote mount to S3 is using FUSE, so it's pretty slow
going, but it seems to be working, touch wood. Long way to go. Wish there
was a better way to do this. AWS offers a local copy service, but not
between EBS and S3.

ed

On Wed, Aug 28, 2013 at 1:58 PM, Lightner, Jeff <JLightner at water.com> wrote:

>   On your point 2.  ****
>
> ** **
>
> An NFS mount is still transferring data over the network so bandwidth is
> still a concern.   True enough if you do rsync from local physical drive
> mount to NFS mount locally you’d get little benefit (and probably some
> performance loss) from doing compression in rsync.   However, it seems it
> would be more to the point to suggest doing the rsync to the host that is
> sharing out the NFS filesystem in such a case.    (Of course point 1
> obviates point 2 anyway but I thought it worth mentioning for complete
> detail.)****
>
> ** **
>
> *From:* ale-bounces at ale.org [mailto:ale-bounces at ale.org] *On Behalf Of *Brian
> Mathis
> *Sent:* Wednesday, August 28, 2013 12:00 PM
> *To:* Atlanta Linux Enthusiasts
> *Subject:* Re: [ale] rsync****
>
> ** **
>
> Compression won't help for 2 reasons:****
>
> 1) Image data is already compressed****
>
> 2) It sounds like you're syncing between local mounts, in which case the
> data would be uncompressed before writing to the destination, which is the
> local mount, which then sends it over the network.  I hope rsync is smart
> enough to ignore the compression option when syncing between local sources.
> ****
>
> Compression is only going to help if you're syncing between 2 separate
> machines and the data is being transferred over the network by rsync itself
> -- not when using a network file system.****
>
> ** **
>
>
> ****
>
> ❧ Brian Mathis****
>
> ** **
>
> On Wed, Aug 28, 2013 at 9:40 AM, Edward Holcroft <eholcroft at mkainc.com>
> wrote:****
>
> I am rsyncing a many files, mostly photos from an EBS volume to an S3
> bucket (locally mounted using FUSE) on AWS.****
>
> ** **
>
> Given that it's primarily photographic data (jpg), is it still a good idea
> to use compression (z- option), or will I not gain any real benefit by
> trying to compress jpegs?****
>
> ** **
>
> Alternately, if anyone has a better way of getting 600GB of data from EBS
> to S3, please let me know.****
>
> ** **
>
> ed****
>
> ** **
>
> -- ****
>
> Edward Holcroft | Madsen Kneppers & Associates Inc.
> 3020 Holcomb Bridge Rd. NW | Norcross, GA 30071
> O (770) 446-9606 | M (770) 630-0949****
>
>
> MADSEN, KNEPPERS & ASSOCIATES USA, MKA Canada Inc. WARNING/CONFIDENTIALITY
> NOTICE: This message may be confidential and/or privileged. If you are not
> the intended recipient, please notify the sender immediately then delete it
> - you should not copy or use it for any purpose or disclose its content to
> any other person. Internet communications are not secure. You should scan
> this message and any attachments for viruses. Any unauthorized use or
> interception of this e-mail is illegal.
>
>
>
>
>
>   _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo****
>
> ** **
>
>
>
>
>
> Athena®, Created for the Cause™
>
> Making a Difference in the Fight Against Breast Cancer
>
>
>
>
>
> *How and Why I Should Support Bottled Water!
> *Do not relinquish your right to choose bottled water as a healthy
> alternative to beverages that contain sugar, calories, etc. Your support of
> bottled water will make a difference! Your signatures count! Go to
> http://www.bottledwatermatters.org/luv-bottledwater-iframe/dswaters and
> sign a petition to support your right to always choose bottled water. Help
> fight federal and state issues, such as bottle deposits (or taxes) and
> organizations that want to ban the sale of bottled water. Support community
> curbside recycling programs. Support bottled water as a healthy way to
> maintain proper hydration. Our goal is 50,000 signatures. Share this
> petition with your friends and family today!
>
>
>
> ---------------------------------
> CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential
> information and is for the sole use of the intended recipient(s). If you
> are not the intended recipient, any disclosure, copying, distribution, or
> use of the contents of this information is prohibited and may be unlawful.
> If you have received this electronic transmission in error, please reply
> immediately to the sender that you have received the message in error, and
> delete it. Thank you.
> ----------------------------------****
>
>
>
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
>

-- 
Edward Holcroft | Madsen Kneppers & Associates Inc.
3020 Holcomb Bridge Rd. NW | Norcross, GA 30071
O (770) 446-9606 | M (770) 630-0949

-- 
MADSEN, KNEPPERS & ASSOCIATES USA, MKA Canada Inc. WARNING/CONFIDENTIALITY 
NOTICE: This message may be confidential and/or privileged. If you are not 
the intended recipient, please notify the sender immediately then delete it 
- you should not copy or use it for any purpose or disclose its content to 
any other person. Internet communications are not secure. You should scan 
this message and any attachments for viruses. Any unauthorized use or 
interception of this e-mail is illegal.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20130828/58a83734/attachment-0001.html>