[mythtv] [PATCH] Add streaming output to transcoder

[Geoffrey Hausheer]

On Sat, 16 Aug 2003 21:31:57 -0700, "Chris Petersen
lists-at-forevermore.net |mythtv/1.0-Allow|"
<o8f8hesx3k0t at sneakemail.com> said:
> Is there any way to buffer the fifos themselves, rather than filling up
> RAM?  basically, spit audio and video at the exact same time, decoding
> only as much of the nuv as the slowest fifo.  After all, it's not like
> we NEED the audio or video to finish faster.
That's what I mean when I talked about adding a 'sync' option.  The audio
and video will be synchronized (so I will send one audio and one video
frame, and won't send another set until the first set has been consumed
(thus mythtranscode is enforcing sync).  However, the reading program
needs to be written with this in mind.  Fifos are a pain, because they
are always blocking.  Thus when reading/writing more than one, you need
to be careful not to deadlock.  In the case of mencdoer (and probably
other programs), they are written with the assumption that their input
are files, and thus they can read each stream at whatever rate they want.
 To make that work with fifos, the buffer (any difference in frame number
between audio and video) needs to be kept somewhere.  RAM is the best
place assuming the number isn't too big.  In the mencoder case, it is
necessary to buffer ~2 seconds of video (~50MB) to make it happy.  other
programs may be less.  I could build the buffers on disk, and have a
queued fifo, but it would really slow things down.  Unfortunately, the
design in mythtranscode today is a circular-queue, which means there is
no unused memory that can get swapped out, even if the queues are mostly
empty.  That is a minor optimization I may look at in the future, but to
me it is overengineering the problem (i.e it is only worth it if people
are actually running out of memory with the current implemetation)

> My guess is that most
> encoding programs will be smart enough to buffer their reads against a
> slow input, and just wait on the fifo to spit out more data.
Well, unfortunately, this is a bad guess.  Most encoders were designed to
use files as inputs.  In the case where streams were considered, it is
normally a single stream with video and audio muxed together.  Most
encoders are not designed to work with multiple fifos, and it is not
trivial to get working right, as evidenced by all the hoops I had to jump
through to ge mencoder working.  Implementing a reader which doen't
deadlock isn't much easier than impelementing a writer.

> 
> I'm just transcoding an hour-long show (before commercial cut) and
> seeing my RAM get eaten up pretty quickly (well, relatively) after the
> audio encode speeds up.  I have a gig of RAM, so I'm not too worried,
> but some people might have less (my myth box only has 512).
Umm, this doesn't sound right.  I expect mythtranscode to take up about
65MB of RAM (asusming a 640x480 nuv file).  I can possibly reduce that
number in the future for mencoder.  For the 'sync' option, there would be
no buffering, which should drop the usage down under 20MB (just myth
overhead).  If mythtranscode is taking up a lot of RAM, you should see
messages about creating new buffers, which implies something isn't
working correctly. 

> Also, it might be nice to have mythtranscode give some sort of
> progress/timestamp on the console, so we could tell how far it's
> gotten.  It's pretty easy to do this by just printing data, and then a
> \r to reset the cursor but not add a newline.
I could do this, but since mencoder and ffmpeg both give status, I
figured that would be sufficient.

.Geoff