[mythtv-users] Thoughts on a reliable (backed up) mythtv system

[Brad Templeton]

All this talk about RAID has made me think about what other
techniques might be interesting for making a backed up mythtv
system.   RAID is very good at dealing with one problem -- hard
disk failure, and can provide a read speedup, but it doesn't
solve any other problems, some of them very real.  It's a good
solution for high availability data centers that dare not go down
because of a disk failure, but I wonder at its value for a home
MythTV system, where the worst consequence is the missing of a TV
recording once every decade or so, and short inabilities to watch
shows.


I believe the best form of backup, while more expensive in disk
cost than RAID, is remote mirroring.   At least to another computer
on the same LAN, and even better to a disk in another building.
Mirroring with short-term keeping of deleted files, as well.

Mirroring, unlike RAID, deals with accidental file deletion.  It can
deal with malicious destruction of a machine or files by an intrusion.
It can deal with file corruption due to software bugs.  It can deal
with power spikes, power supply failures and controller failures that
wipe out an entire machine.  It can deal with overheating that
fries multiple drives.  It can deal with fire burning down a room,
or in the case of offsite backup, a building.   It can deal with
thieves or government goons coming in and stealing a computer.

While mirroring done every few hours will not recover very recently
generated files (though there are means for this) it also means
that mirroring can be done to drives that are normally spun down, and
only spun up when it's time to mirror.   As noted, with power costing
around $18/year/drive in California, that can make quite a difference
in cost over the 3-4 year life of a drive.

It also seems like it would be possible to do some very smart
mirroring with mythtv.  To wit:

    a) If a program is going to air again according to the schedules,
       with no conflict, you need not back it up.   You can just re-record
       it.    If you still haven't deleted it by the time there are no
       new planned airings, you could back it up then.

    b) You could consider transcoding before backup, to keep backups small.
       The theory being that if you lose a hard drive you can tolerate
       the slight loss in quality.     (Many transocde already of course.)

    c) With a bit of work, one could identify programs which, in a pinch,
       can be had other ways (rent the DVD, pay-to-download) and know you
       don't have to back those up, again feeling that if you lose a
       hard drive before you see them, you can get them with a bit more
       work and a spot of money.   Over time, almost all programs are
       going to end up this way, and the database listing what's available
       should be easy to build from online catalogs etc.

    d) Perhaps most interestingly, one could do a network backup system.
       In a network backup, large groups of users publish signatures to
       indicate files they have.   Typically MD5s but read on.  Files
       nobody else has are backed up to other user's systems in a space
       trading regime, encrypted for private files, open for public files.

       If one user loses a file, they can just get it automatically from
       one of the other users in the backup group.

       Now there are some wrinkles.  TV recordings will not be identical
       and have the same MD5s.   Even Digital TV recordings would have
       drop-outs if you could sync their starts exactly.   So signatures
       would be things like station, start, duration and quality for a
       TV recording.

       The other wrinkle is copyright lawsuits.  This is what mp3.com
       got creamed for, or similar to it.  There are backup network
       programs (like BackupPC) and services, but they use the MD5.
       However, a system like this should be legal, you would think,
       but that's not guarantee.    The system actually would be legal,
       it would be the users who were violating copyright if anybody
       was, making it a lot safer to build and provide such a P2P system.
       The main issue would be if people could get recordings who
       never actually recorded them in the first place.

    e) Of course, many people use the possibly infringing P2P websites
       as a "backup" for MythTV, figuring if they lose a show they
       recorded, there is no major ethical qualm, and an acceptable
       litigation risk, in going to one of the P2P sites to recover the
       show.   (Though this is usually not possible for HDTV.)
       
    f) As a minor plus, if you "back up" your backend's videos to spare
       drives that spin-up-on-demand in your frontends, your frontends
       can play those videos directly.