[mythtv-users] Mythtv .19 live watching

[f-myth-users at media.mit.edu]

    > Date: Wed, 19 Jul 2006 08:22:42 -0500
    > From: "David Schmidt" <david.schmidt.in.dallas at gmail.com>

    > On 7/17/06, Alex Brekken <brekkal at gmail.com> wrote:
    > > Hmmm, I've been running 18.1 at full disk for over a year now and never had
    > > a problem.  I wasn't aware others were having problems with this either.
    > > I'll keep it in mind though when I move to .19.
    > >
    > >
    > >  On 7/16/06, chris at cpr.homelinux.net <chris at cpr.homelinux.net> wrote:
    > > >
    > >
    > >
    > > Second, both 0.18 and 0.19 users have complained again and again
    > > about the auto-expire not working and how it causes their partition
    > > to hit 100% and kill the machine.  I therefore don't trust
    > > auto-expire and manually manage the recorded programs.
    > >

    > I can also testify that on 18.1 auto-expire "just works."  I've been
    > running it on my proof-of-concept system for 4-5 months now without
    > issue.  The /video drive on it is only 20G (SD), so you better believe
    > it expires *A LOT*!!!

My problem with the 0.18.1 autoexpirer was in a different direction,
and I bug-reported it a few months ago (and could find the thread if
someone cares & can't).

In my case, a transient filesystem issue (I'd say NFS, but the expirer
runs on the MBE and my MBE's video store is local, not NFS-mounted)
caused -all- values for the filesystem (amount used, amount total,
amount free) to return zero, and the autoexpirer then expired
-everything-, without noticing that (a) these values made no sense,
and (b) it wasn't making progress anyway.  [I say "all values" because
a "Recorded Programs" page in MythWeb just happened to have said
"using 0 B ... out of 0 B" at the time I noticed the issue; I didn't
actually catch df itself emitting these numbers, so I'm -actually-
guessing that it was something internal to Myth that failed, since
if jfs did this on a regular basis, users would scream bloody murder.]

The only thing that saved me was that (a) most programs weren't set to
allow autoexpirary and (b) those that were just happened to actually
be symlinks into another NFS-mounted filesystem, so the only things
that got expired were symlinks, not real data.  [AND, of course, all
their database information, but fortunately I had that backed up and
could (with a bunch of inconvenience) merge that back into the
database.]  If I hadn't been very lucky, the autoexpirer would have
simply wiped out all the data on the machine, instantly.

Since then, I have defaulted to NEVER allowing autoexpiration of
anything, since I apparently cannot do the easier move of turning
the autoexpirer off entirely.  (Short of editing & recompiling, that
is, and I'm trying to stay with the currently-installed package until
I migrate to 19 or 20.)

I also suggested, in that bug report, that the autoexpirer have some
basic sanity checks installed, such as NOT running if any filesystem
parameter other than freespace is ALSO zero or negative (e.g., "you're
not getting valid data from the filesystem, so don't go off and trash
the world in your confusion"---and yes, I've seen negative values with
certain incompatible versions of networked filesystems!), which might
take care of problems with people who have NFS-mounted video stores if
something momentarily goes wrong with the mount.  [For example, if the
mount goes offline, the expirer may read the underlying values of the
mountpoint and get very small values [perhaps zero, perhaps not?], and
even if it can't get at the video data itself, it will still happily
wipe out the database entries for them.]

I also suggested that the autoexpirer ensure that freespace actually
-increases- after -every- deletion and abort with a message to the log
if this isn't happening, so it might perhaps only expire -one-
recording and not -every- recording if something else is confused.
(Unless we're doing the very slow delete-via-truncation for ext3fs
[was this ever actually committed?], any deletion should raise
freespace if we check immediately thereafter, even if several
high-rate streams are being written simultaneously, unless we're
deleting something only a couple seconds long.)

My basic point was, "anything that can autonomously remove large
quantities of valuable data needs very paranoid error-checking, even
for cases that ``can't happen'', and the autoexpirer doesn't have it."

So, yes, even if the autoexpiry logic of "what to delete" is perfect,
I have personally seen transients that confused the autoexpirer
into deleting way too much if something else is momentarily peculiar.
Since I don't know for sure what caused the peculiar behavior (but I
doubt -very- much that jfs just randomly claimed to have 0 free and
0 total or we'd have heard other bug reports---therefore I'm blaming
Myth 100% for this regardless), and since peculiar behavior can cause
Myth to instantly delete all data, I've tried as hard as I can to turn
this mechanism off, since even a low probability can have catastrophic
consequences.