[ale] OT converting old videos, cataloging dvds, storing forever
Jim Kinney
jim.kinney at gmail.com
Tue Oct 16 13:11:03 EDT 2012
Um. Magnetic tape is cheap and very reliable. Storage silos from ebay can
hold multiple terrabytes and are Linux happy. 20 year lifespan on tape in
decent environment. So a linux storage system with enough drive space for
staging restores from tape for play.
On Oct 16, 2012 12:33 PM, "Ron Frazier (ALE)" <
atllinuxenthinfo at techstarship.com> wrote:
> HI all,
>
> After a few hours of Googling and reading forums and such, I've got some
> new data that I wanted to share. My little video conversion project
> seems to be hitting a snag. The 800 hours of videos I have are in the
> category of important to me, but not critical. If I lose them, I won't
> die tomorrow. So, the snag I'm hitting is the cost in terms of storage
> cost and time to convert them. I don't have a huge budget, certainly
> not all at once. I wanted to share my results, and see if you guys
> think I'm missing something.
>
> In terms of time, I think it will take me 10 - 15 minutes of tinkering
> with equipment, software, discs, cases, labels, etc. per one hour show
> to convert. More if I do edits and cut commercials and sync the sound,
> etc. Perhaps less if I store only to HDD and don't edit. That adds up
> to 200 hours of my time, a non trivial amount. Obviously the video tape
> must play, but I don't necessarily have to watch it.
>
> Here is what I believe to be the state of the art in storage options.
>
> I'm not considering other types of tape at this point. I'm also not
> considering SSD, based on data retention issues I mentioned in another
> post.
> Option 1 is using HDD's to store the videos.
> Option 2 is some other type of disc, bluray, dvd, or mdisc.
> Option 3 is online storage.
> Based on my reading, I have concerns about the long term reliability of
> bluray, so I'm probably dismissing it.
>
> Let's discuss discs first. Assume a single layer single sided DVD can
> hold ~1 hour of good quality video at a data rate of about 8 Mbps. For
> a HDD, assume 4.7 GB / hr of video. Note, in some of the forums where
> people burn LOTS of DVD's and spend hours checking their error rates
> etc, it was mentioned that 16x write speed DVD's are not reliable. So,
> if available, I'm mentioning only 8x DVD's here. There are also many
> types of dyes, and very confusing data, which I'm not going to try to
> sort out here as I don't understand it all. There are may types of
> printable surfaces on the top of the disc, for automated printing of
> labels. Lightscribe and inkjet surfaces are examples. These usually
> increase the cost. I don't need them. For my purposes, writing on the
> disc with a DVD safe sharpie pen is enough. Based on my reading, double
> layer discs suffer more compatibility problems so I'm not considering
> them. Double sided discs are much more likely to get scratched and
> cannot easily be labeled so I'm not considering them. It is actually
> very hard to figure out all the DVD media options, but I've selected a
> few to look at. Prices are Amazon, if they sell the product in question.
>
> Gotta account for the case for the DVD, about $ 0.25 each.
>
> The DVD-R format seems to be the most universally compatible. I'm not
> trying to list all discs here, just get representative prices. Of
> conventionally constructed discs, Verbatim seems to have a good
> reputation, but there are varying reviews. Longevity estimates for
> conventional disc media range from 5 years anecdotal to 50 years from
> the factories. I will say that I have some 5 year old Maxell DVD-R's
> that are still readable, but I will probably copy them to archival discs.
>
> VERBATIM 94852 DVD-r media 8x 4.7gb 50-pk spindle (shiny silver) - $ 19
> / 50 = $ 0.38 / disc. Cost with case (cwc) = $ 0.63 / disc
> Verbatim Shiny Silver DataLife Plus 8x DVD-R Spindle, 50 Discs - also $
> 19 / 50
>
> It seems that JVC Taiyo Yuden is the top of the heap in terms of quality
> for conventional media. Taiyo Yuden bought part of JVC as I understand
> it and they're now shipping with JVC part numbers and packaging. Many
> people still know the Taiyo Yuden name and so that's usually listed in
> the product listings. Apparently, some TY discs themselves are
> unlabeled, so if you want TY, make sure you get TY.
>
> 100 Taiyo Yuden/JVC 8X DVD-R 4.7GB Silver Thermal Lacquer - $ 29 / 100 =
> $ 0.29 / disc. cwc = $ 0.54 / disc Better product, cheaper price. Cool.
>
> Still, I don't consider these discs long term storage. The next step up
> is archival grade discs, which are much more expensive. Archival grade
> discs usually have a solid gold layer to minimize corrosion and maximize
> reflectivity. Some have gold only. Some have gold and silver, which is
> cheaper - less gold, more silver. They also usually have an extra hard
> coating to prevent scratches. Conventional DVD's are insanely easy to
> scratch.
>
> Verbatim 95355 UltraLife 4.7 GB 8x Gold Archival Grade DVD-R, 50-Disc
> Spindle - $ 93 / 50 = $ 1.86 / disc. cwc = $ 2.11 / disc
>
> My research tells me that MAM-A / Mitsui is state of the art in archival
> grade discs. They have lots of data on their website. Their gold /
> silver DVD discs are projected to last 83 years. Their gold DVD discs
> are projected to last 100 years. And their gold CD discs are projected
> to last 300 years. Also, note the same risk as above in buying
> unlabeled discs.
>
> http://www.mam-a.com/
>
> MAM-A Mitsui Gold Archive 8x DVD-R with no logo in bulk - 50 Count = $
> 117 / 50 = $ 2.34 / disc. cwc = $ 2.59 / disc (100 year discs)
> MAM-A 8x SILVER PLUS GOLD DVD-R Branded Gold Surface 50pk Beehive with
> Hard Coat - $ 93 / 50 = 1.86 / disc. cwc = $ 2.11 / disc (83 year
> discs) (factory price)
> MAM-A 84019 Silver plus Gold DVD-R 16x 4.7GB MAM-A Logo 50pk Spindle -
> 84019 - $ 80 / 50 = $ 1.60 / disc. cwc = $ 1.85 / disc (83 year
> discs) (dealer price)
>
> This last one is probably the best compromise between price and longevity.
>
> Finally, if you want longevity for 1000 years, there's m-disc, as
> mentioned previously. This is overkill for what I'm doing, but here are
> the prices.
>
> M-DISC DVD+R 4X Cake Box 50 Pack - $ 136 / 50 = $ 2.72 / disc. cwc = $
> 2.97 / disc. That's only $ 0.38 / disc more than the 100 year discs.
> Not to bad if you need that kind of lifespan.
>
> So, the best I can do for an archival grade disc is $ 1.85 / disc.
>
> Now, what about the HDD option. Here's a 3 TB drive I found on Amazon.
>
> Western Digital Caviar Green 3 TB SATA III 64 MB Cache Bare/OEM Desktop
> Hard Drive - WD30EZRX - $ 140
>
> Let's say I want two for redundancy, since this is NOT archival media.
> I would definitely spinrite this thing a couple of times / year to
> refresh the magnetic fields. That process would probably take 5 days.
> 2 HDD - $ 280
>
> How about a nice dual drive case with fan.
>
> Vantec Dual 3.5-Inch SATA to USB 3.0 and eSATA with JBOD/RAID 0/1
> External Hard Drive Enclosure (NST-400MX-S3R) - $ 83
>
> So the total cost is - $ 363
>
> And we can store 3000 GB / 4.7 GB per hour = 638 hours of video
> approximately equivalent to 638 DVD's.
>
> So, cost per equivalent DVD works out to - $ 363 / 638 = $ 0.57 / "disc"
> or hour. This is slightly more than the cost of cheap DVD's when cost
> of cases is accounted for but substantially less than the cost of
> archival DVD's.
>
> So, here's how I see my choices.
>
> - 83 year archival DVD's
>
> * relatively low up front cost for 50 pack
> * substantially larger long term cost
> * some additional cost for storage rack, etc. (and where do I put it)
> * discs are easy to access and play in dvd player
>
> - two big hard drives
>
> * relatively high up front cost
> * substantially lower long term cost
> * minimal space required
> * movies are harder to access since I have to hook up computer to tv
> and start player, etc.
>
> There is the possibility of online storage. Storing 5 GB on Amazon S3
> will cost me $ 0.75 / month for 1 hour of video. It will cost me about
> $ 0.75 each time I retrieve it. Given the quantity of data, not very
> practical.
>
> None of these choices are really great for me. What I may do is to
> select the best 50 - 100 hours of source video and put them onto 83 year
> archival discs. Even that would cost me $ 185. I'm not sure it's worth
> it. If I'm comparing the HDD to the 83 year archival discs, the hard
> drive becomes more economical than the discs once I store 196 hours of
> video. At this point, it seems there are no good (within my budget)
> options to store 800 hours of video. One other option is to convert
> everything to cheap DVD's, then refresh it and recopy it every 5 - 10
> years. That doesn't sound too attractive either.
>
> PS, from a hardware perspective, the Hauppauge USB-Live2 analog video
> digitizer looks cool. Haven't tried it and don't know about it's
> drivers or software. I can get mdisc / bluray drives in the $ 50 - 100
> range.
>
> Let me know what you think, and whether I've left out anything major.
> This took two hours to type and I didn't proofread it, so sorry for any
> errors.
>
> Sincerely,
>
> Ron
>
>
> On 10/15/2012 4:45 PM, JD wrote:
> >> I also wanted to share a great way to track and catalog DVD's that
> >> you've purchased. It's an android app called MyMovies (free - limited
> >> to 50 titles) and MyMovies Pro ($6 - unlimited). It's ultra cool. Just
> >>
> > Hopefully, someone else has found a fantastic Linux video editor and
> will be
> > able to let us all know about it.
> >
> >
> >
> > Previously I used a F/LOSS web server + back-end DB to catalog DVDs and
> > recordings. Over time, that became too much effort and I didn't like
> that the
> > program forced me to a specific DB and the export was only to HTML
> files. Tried
> > a Java desktop client too, but it was toooo damn slow. It didn't play
> nice on
> > my network.
> >
> > Ended up using the time-proven 'ls -Rl' technique with numbered DVDs.
> With a
> > large number of data DVDs holding TV recordings in xvid/avi or
> h.264/mkv, a
> > quick 'egrep -i' searches by title for the specific numbered disk
> containing the
> > recorded show/movie. As a CLI person, the most-used desktops all have a
> script
> > that will ssh over to the main box with all the dvd-0xyz.txt files and
> perform
> > the query. For more information, XBMC has the catalog with actors,
> directors,
> > year, genre, title, etc .... grep is just so useful and text is so
> easy to
> > search.
> >
> > I love that the grep clearly returns the numbered data DVD I need to
> load.
> >
> > As to converting VHS to MPEG or XVID or h.264, almost any $20 USB TV
> adapter can
> > record SD content to MPEG2. Just pick from a MythTV compatible device
> to be
> > happiest long term. If you want to record OTA HD content, I'd suggest
> more
> > effort for the selected hardware. I love the HD-Homerun network tuners.
> Those
> > work well with OTA except 1 channel that doesn't come in at my house. It
> is
> > hardly a major issue, since 69 other channels do come in OTA from
> Marietta.
> >
> > For finding commercials and editing them out, I searched for Linux-based
> > solutions for years and finally gave up. Comskip does run nice under
> WINE, I've
> > never found a Linux-based editor that will accept the comskip created
> cut files
> > (there's like 6 different formats that comskip will create) as the
> starting
> > point for where to cut commercials. Comskip is about 90% accurate.
> Comskip can
> > output cut marker files that XBMC and mplayer will honor without
> altering the
> > source video file. That can be handy when you just want to skip
> commercials, but
> > not waste time actually modifying the file contents.
> >
> > Instead of fighting it, I bought - yes, purchased a Windows tool called
> > VideoRedo TV Suite. It appears to use comskip code internally to create
> cut
> > markers. There's a point where $50 is worth it to me. That $50 program
> has
> > been heavily automated here to generate the cut marker files overnight.
> It takes
> > about 45 seconds of human effort to confirm the cut locations and
> request the
> > modified file be saved. If you are willing to live with 90% accurate
> cuts, it
> > could all be automated. Networks have changed up the markers commonly
> searched
> > by VRDP and comskip to trick them into keeping commercials and removing
> some of
> > the desired content. Older stations with less money like RTV are almost
> 100%
> > accurate with cut locations.
> >
> > At this point, my systems are almost automatic with just a few manual
> > "confirmation" places. WTV --> MPEG2 --> h.264/mkv with CC1 and CC3
> closed
> > captions included. If there are multiple audio channels in the stream,
> that can
> > require special processing, but that usually happens with a few PBS
> programs
> > with audio tracks for the deaf or background-only audio.
> >
> > I've blogged about almost all of these solutions over the years. The
> most recent
> > articles are about building your own DB4 antenna for $20 or so that
> actually
> > beat some Antennas Direct models.
> >
> >
> > And just to be clear, I also wimped out and use Windows7 to record TV.
> It is
> > just so easy to setup with free schedule data. That OS runs inside a
> virtual
> > machine under KVM and has been doing it nicely for over a year. THAT
> was not an
> > easy thing to accomplish to get satisfactory results. Microsoft did
> (does?) some
> > things trying to protect Joe-end-user from doing foolish things, like
> blocking
> > recording directly to NAS storage.
> >
> > I suspect there are hundreds of years of knowledge lurking related to
> this problem.
> > _______________________________________________
> > Ale mailing list
> > Ale at ale.org
> > http://mail.ale.org/mailman/listinfo/ale
> > See JOBS, ANNOUNCE and SCHOOLS lists at
> > http://mail.ale.org/mailman/listinfo
> >
> >
>
> --
>
> (To whom it may concern. My email address has changed. Replying to former
> messages prior to 03/31/12 with my personal address will go to the wrong
> address. Please send all personal correspondence to the new address.)
>
> (PS - If you email me and don't get a quick response, you might want to
> call on the phone. I get about 300 emails per day from alternate energy
> mailing lists and such. I don't always see new email messages very
> quickly.)
>
> Ron Frazier
> 770-205-9422 (O) Leave a message.
> linuxdude AT techstarship.com
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.ale.org/pipermail/ale/attachments/20121016/b2d8caf1/attachment-0001.html
More information about the Ale
mailing list