[mythtv] Lossless MPEG2 editing!

[Cory Papenfuss]

>> - GOP vs. non-GOP accurate edits.  MPEG2 streams have I, P, and B frames. 
>> Only the I frames can "stand alone."  The other two rely on 
>> previous/future combinations of other frames to generate a frame.
>
> 	Yes, but...
>
> a) If I want to chop say 5 seconds off the end of a sequence,
> the edit needn't and on an I-frame. If playing forward, it will
> be frame accurate. If playing backwards from the end, there is
> garbage until the next I-frame, but from my point of view that
> is an imperfect scenario anyway (e.g. if the audio is even able
> to be played back, it sounds strange, so a video glitch is OK)
 	... but a B-frame (Bi-directional prediction IIRC) can still
require I or P frames after the cutpoint.  The normal pattern for the GOP 
is IBBPBBPBBP or something like that, so it's still not going to be 
frame-accurate.  In fact, the Conexant MPEG2 cards *do* have the option 
(from the API at least) of using open-GOPs.  Then each GOP isn't even 
independent of each other and B/P frames can reference frames outside 
their current GOP.

>
> b) If I want to take a few seconds out of the middle of a sequence,
> then there are 2 simple "hacky" scenarios:
>
> 1) Frame accurate start of cut, non-frame-accurate end of cut
> (join sequence at next I frame like the crude original utility)
 	I don't see how this would work.  It'll definately have issues 
with missing references.

>
> 2) Frame accurate start and end of cut, by generating a new
> I-frame at the end of the cut
 	Closer, but most likely still not enough.

> 	Now, these are far from perfect. They both change the bitrate
> of the original stream, and neither would seek backwards correctly,
> but I think they are a workable alternative to transcoding.
> Until someone codes up a self-contained correct/perfect solution!
>
 	I'm not saying I have the answer, but it's looking an awful lot 
like *any* hacky way to do it will have glitch issues.  I think in order 
to get rid of all issues (even with simple cuts), it would have to get rid 
of all issues for even complex cuts (open GOPs, frame-accurate edits, PTS 
adjustments).  Problems include:

- Frame accurate edits: Need to resolve references both within the current 
GOP, but probably also outside the current GOP.  That would allow for 
open GOPs, but require decoding the stream enough to find *all* references 
to frames that would disappear.  I don't know how many frames ahead or 
behind the MPEG2 spec allows to be referenced.

- Missing references: Could be resolved by either including all referenced 
frames (from above), but flagging them 'Don't display,' or by reencoding 
the dependent frames all into I-frames.  Given that the primary use for 
all of this is commercial cutting (not nonlinear/effects editing), I think 
the latter would be a good way to go.  There's really only about 10 cuts 
per hour, so have a few dozen more I-frames in an hour's worth of video 
won't increase the size much.  Although it's technically not "lossless," I 
don't think that lossless makes a whole lot of sense around cutpoints. 
Besides, I've heard that some decoders don't even honor the "Don't 
display" flag.

- PTS adjustments:  Even with the above edits, audio and video could get 
out of sync.  I've already dealt with captures that glitch midway through 
and change the A/V offset with the PTS mechanism.  Mplex can't deal with a 
changing offset, so I had to run the whole mess through avidemux (and it's 
indexing capability).  Basically after the first edit, all PTS's in the 
stream will have to be adjusted.  The replex tool already has the 
capability of doing a lot of this.  It might be a good place to start (by 
trying to add in cutting reencoding capability).

 	What a mess!.... :)

-Cory

*************************************************************************
* Cory Papenfuss							*
* Electrical Engineering candidate Ph.D. graduate student               *
* Virginia Polytechnic Institute and State University 			*
*************************************************************************