[mythtv] Scheduler needs table keys?

f-myth-users at media.mit.edu f-myth-users at media.mit.edu
Mon Jan 29 05:28:15 UTC 2007


    > Date: Sun, 28 Jan 2007 21:28:13 -0500
    > From: Chris Pinkham <cpinkham at bc2va.org>

    > * On Sun Jan 28, 2007 at 09:14:53PM -0500, f-myth-users at media.mit.edu wrote:
    > >> From: Chris Pinkham <cpinkham at bc2va.org>

    > > As a stopgap until the real fix goes in (or as something I might be
    > > able to do immediately to my .18), do you or anyone have any idea what
    > > -not- writing recorded.filesize will do?  If it's a complete nonstarter
    > > 'cause it'll break everything else, fine, but if it would be possible
    > > to just patch out that pair of lines you indicated and live with it
    > > until the table locking gets sorted out, well...

    > Shouldn't cause any big issues that I know about.

Wow, that's the best news about this I've heard in months!  Should I
make sure that commflagging waits until the recording is -complete-
before starting, or can it run concurrently?  (Granted, if making this
change means that rescheds don't corrupt recordings, I can probably go
back to many simultaneous commflaggers, so even if they have to run
only after a recording ends, they'll still keep up.  At the moment, I
have the commflagging throttled way back because until recently I
thought all my problems were simple disk contention, hence was doing
everything I could to fix such issues [and having no luck].)

I'm currently running a .deb-based 0.18.1, but I've got a spare
partition (courtesy of all my multispindling work to try to fix this)
available and will try compiling 0.18.1 from source in the spare [so I
can switch back & forth trivially] & running with that (to make sure
the current bad behavior persists).  Assuming I see identical results
with my compiled version & the packaged version, I'll then patch out
the recorded.filesize updates and see if that fixes the corrupted
recordings during rescheds.

						       The recorders should
    be doing one final curRecording->SetFilesize() when the recording is
    done anyway.  The filesize is updated during recording for a couple
    reasons including if the backend is stopped prematurely, that is part
    of the reason we write out the seektable while recording also.  The
    seektable is also written out so that a player can get the seektable
    from the DB without having to request it from the backend.

I pretty much never have a backend crash during recording, but if I
did, I take it that recorded.filesize might never get updated?  I
s'pose that could be bad, but I could manually do so if I absolutely
had to after a crash...  [Oh wait, the next paragraph below seems to
indicate that this isn't an issue as long as I poke the frontend after
a backend crash---which seems pretty likely to happen anyway as I look
around to see what happened...]

    There is code in mythbackend to fill in the filesize if it is missing
    whenever the frontend requests the list of recordings, so if you view
    the list of recordings while a recording is in progress, the filesize
    field will be populated with the current filesize which will then get
    updated with the final filesize by the recorder when the recording ends.

Good.

    Before the filesize was stored in the database, the master backend would
    have to check the filesizes of every recording whenever the recordings
    list was requested by the frontend.  So if anyone thinks it is slow to
    start up the Watch Recordings screen now, think about how it was back
    then when the backend had to stat every recording's file to get its
    filesize before returning the recordings list. :)

It's not clear to me that this stat must be slow.  (Was it profiled
before this change went in?)  I can see that it might be over a slow
NFS system, but presumably the MBE almost always has a local disk or a
fast NFS FS backing it or it'd have other problems.  [Hell, I just
tried doing a Dired of my MBE's recording directory, from the FE/SBE
-while- burning a DVD on the FE.  (Dired is an Emacs utility that does
the equivalent of "ls -al" and then parses the results into a buffer.)
That completed in under two seconds and maybe under one, for a listing
that included 352 entries (half of them .png's for the thumbnails),
even though it had to both ship the results over a 100baseT NFS
connection -and- process them inside Emacs.  It's hard to believe that
statting a few hundred files would significantly slow down Watch
Recordings by that much, but I guess for the wrong kind of filesystem
or thousands of recordings or something...]

[Actually, I guess it -might- be slow if the BE did some iteration
over a DB and then statted each file individually and there was
something inefficient there---but clearly ls can do it efficiently
and (worst case) one approach would be to use ls itself, or do
whatever it's doing, grab the entire directory (including lots of
files Myth doesn't care about, e.g., the thumbnails), and then pick
through the results.  I mean, really, how many times have you waited
more than a second for "ls -al" on any directory with fewer than a
thousand files in it?  There's clearly something I'm missing here.]


More information about the mythtv-dev mailing list