[mythtv-users] mythcutprojectx.sh and trunk (was: mythtranscode cutlist argument?)

Christopher Meredith chmeredith at gmail.com
Sat Jul 10 22:02:38 UTC 2010


On Sat, Jul 10, 2010 at 4:14 PM, John Pilkington <J.Pilk at tesco.net> wrote:
> On 10/07/10 20:21, John Pilkington wrote:
>>
>> On 10/07/10 19:41, Christopher Meredith wrote:
>>>
>>> On Sat, Jul 10, 2010 at 1:31 PM, John Pilkington<J.Pilk at tesco.net> wrote:
>>>>
>>>> On 10/07/10 18:31, Christopher Meredith wrote:
>>>>>
>>>>> On Sat, Jul 10, 2010 at 11:09 AM, John Pilkington<J.Pilk at tesco.net>
>>>>> wrote:
>>>>>>
>>>>>> On 10/07/10 16:06, John Pilkington wrote:
>>>>>>>
>>>>>>> On 10/07/10 14:55, Christopher Meredith wrote:
>>>>>>> <snip>
>>>>>>>
>>>>>>>> I played around with mythcutprojectx.sh some more and there are some
>>>>>>>> additional problems I'm finding. First of all, it seems "hardcoded"
>>>>>>>> for only mpeg2 audio even though many DVB and firewire recordings
>>>>>>>> will
>>>>>>>> have ac3 audio. Line 143 is the remuxing line and it assumes the
>>>>>>>> audio
>>>>>>>> track will be mpeg2. If you use it on a file with ac3 audio, it dies
>>>>>>>> at this stage.
>>>>>>>
>>>>>>> Yes: I'm dealing with dvb recordings in the UK, so I haven't tried
>>>>>>> it on
>>>>>>> ac3 audio. Sorry!
>>>>>>>>
>>>>>>>> Perhaps the more important problem is that it seems to be working in
>>>>>>>> reverse. In both tests I ran, it cut out the program, leaving
>>>>>>>> ONLY the
>>>>>>>> commercials. I've seen other threads on trouble with lossless
>>>>>>>> commercial cutting in trunk (I'm running trunk, by the way) and I
>>>>>>>> suspect this is related.
>>>>>>>>
>>>>>>>> I think I can probably modify the script to account for ac3 audio as
>>>>>>>> well as hacking together a way to autodetect the video and audio
>>>>>>>> stream IDs. But the inverse cutting is really fatal.
>>>>>>>
>>>>>>> That is new(ish) in trunk - not seen in 0.23-fixes, currently at
>>>>>>> 25138
>>>>>>> from ATrpms.
>>>>>>>
>>>>>>> John P
>>>>>>
>>>>>> Here are the annotated first few lines of output for a
>>>>>> single-section cut
>>>>>> from the middle of a recording:
>>>>>>
>>>>>> ionice -c3 ./mythcutprojectx 1009_20100710015800.mpg 0xc9 0x191
>>>>>> list0
>>>>>> 4253 104868 # cut-in, EOF
>>>>>>
>>>>>> list1
>>>>>> 0 92713 # Start of recording, cut-out
>>>>>>
>>>>>> list
>>>>>> 4253 92713 104868 cut-in, cut-out, EOF
>>>>>>
>>>>>> CollectionPanel.CutMode=0
>>>>>> 115486708 # cut-in, bytes. Start of output
>>>>>> 2060170552 # cut-out, bytes. End of output, 58m 57s
>>>>>> 2337428816 # EOF, bytes
>>>>>>
>>>>>> Reading GUI-Switch...
>>>>>> Reading Help Switch...
>>>>>>
>>>>>> A comparison may be useful
>>>>>>
>>>>>> HTH
>>>>>
>>>>> I pasted the entire output on pastebin. This is a 30-minute program
>>>>> with the lead-in and lead-out cut, as well as three commercial breaks
>>>>> in the middle.
>>>>>
>>>>> http://pastebin.com/eNnpDxiC
>>>>>
>>>>> What I imagine is the relevant part is this:
>>>>> list0
>>>>> 1836 18207 36778 54307
>>>>>
>>>>> list1
>>>>> 13700 29554 50077 55801
>>>>>
>>>>> list
>>>>> 1836 13700 18207 29554 36778 50077 54307 55801
>>>>>
>>>>> CollectionPanel.CutMode=0
>>>>> 0<<<<<<<<<<<<<<<<<<<<<<<<<<
>>>>> 77005740
>>>>> 565834504
>>>>> 769433616
>>>>> 1210499852
>>>>> 1502240132
>>>>> 2039828200
>>>>> 2225616380
>>>>> 2298227056
>>>>>
>>>>> Reading GUI-Switch...
>>>>> Reading Help Switch...
>>>>> _______________________________________________
>>>>>
>>>>
>>>> It looks as if the arrowed 0 is the cause of the inversion. Until(?)
>>>> trunk
>>>> is fixed the easiest hack would be to insert a zero in this position if
>>>> there isn't one, and remove it if there is.
>>>
>>> I'm guessing the relevant block would be this one:
>>>
>>>
>>> echo "CollectionPanel.CutMode=0"> cutlist$$ ;
>>> for i in $list1 ;
>>> do
>>> if [ $i = "0" ]
>>> then
>>> list=`echo "select mark from recordedmarkup
>>> where chanid=$chanid and starttime='$starttime' and type in (0,1)
>>> order by mark;" |
>>> mysql -N -u root -p$PASSWD mythconverg | tail -n +2 `
>>> # tail -n +2 drops the initial zero.
>>> else
>>> echo "0">> cutlist$$
>>> list=`echo "select mark from recordedmarkup
>>> where chanid=$chanid and starttime='$starttime' and type in (0,1)
>>> order by mark;" |
>>> mysql -N -u root -p$PASSWD mythconverg `
>>> fi
>>> # use only the first element of list1, as a switch.
>>> break
>>> done
>>>
>>> Would the change best be accomplished by negating one or more of the
>>> conditionals or by inserting a new function?
>>>
>>> BTW, does anyone know whether the inversion bug has been reported? I
>>> thinks it's throwing off mythcommflag too. I also wonder if it's
>>> related to other problems like missing dates in mythweb and the bug
>>> where freqid values are deleted when modifying channel info in
>>> mythweb.
>>
>> I'm afraid I'm not really fluent in bash, but in fact the suggested
>> change just reverses the effect of all the rather contorted logic and
>> inelegant coding above. My tentative suggestion would be to replace the
>> section you have quoted with:
>>
>>  > echo "CollectionPanel.CutMode=0"> cutlist$$ ;
>>  > list=`echo "select mark from recordedmarkup
>>  > where chanid=$chanid and starttime='$starttime' and type in (0,1)
>>  > order by mark;" |
>>  > mysql -N -u root -p$PASSWD mythconverg `
>>  >
>>
>> followed as before by
>> # find the key frame (mark type 9) right before each cut mark,
>> etc
>>
>> but no guarantees given, other suggestions welcome.
>>
>> Ticket #8546 is probably relevant
>>
>>
>
> I tried running this modified version in 0.23-fixes and now have a
> collection of adverts - is this called Superbowl mode?  - so it looks as if
> it did invert the effect of the cutlist and might be appropriate for use
> with the current version of trunk.  Worryingly, though, it seems not to have
> properly cleared the cutlist afterwards.  Now to restore the original
> recording, regenerate the cutlist and run the original script.
>
> Incidentally, someone reported 'ionice: execvp failed: Permission denied'.
>  I got that before setting the 'trunkified' script to be executable.

John,

This did indeed work very nicely. Only problem is that mplex choked on it:


   INFO: [mplex] mplex version 1.9.0 (2.2.7 $Date: 2006/02/01 22:23:01 $)
   INFO: [mplex] File /mnt/sdd/tempcut6247.m2v looks like an MPEG Video stream.
   INFO: [mplex] File /mnt/sdd/tempcut6247.ac3 looks like an AC3 Audio stream.
   INFO: [mplex] Video stream 0: profile 9 selected - ignoring
non-standard options!
   INFO: [mplex] Found 1 audio streams and 1 video streams
   INFO: [mplex] Selecting generic DVD output profile (PROVISIONAL)
   INFO: [mplex] Multiplexing video program stream!
   INFO: [mplex] Scanning for header info: Video stream e0
(/mnt/sdd/tempcut6247.m2v)
   INFO: [mplex] VIDEO STREAM: e0
   INFO: [mplex] Frame width     : 1920
   INFO: [mplex] Frame height    : 1080
   INFO: [mplex] Aspect ratio    : 16:9 display
   INFO: [mplex] Picture rate    : 29.970 frames/sec
   INFO: [mplex] Bit rate        : 9000000 bits/sec
   INFO: [mplex] Vbv buffer size : 999424 bytes
   INFO: [mplex] CSPF            : 0
   INFO: [mplex] Scanning for header info: AC3 Audio stream 00
(/mnt/sdd/tempcut6247.ac3)
   INFO: [mplex] AC3 frame size = 1536
   INFO: [mplex] AC3 AUDIO STREAM:
   INFO: [mplex] Bit rate       :    49152 bytes/sec (384 kbit/sec)
   INFO: [mplex] Frequency      :     48000 Hz
   INFO: [mplex] SYSTEMS/PROGRAM stream:
   INFO: [mplex] rough-guess multiplexed stream data rate    : 9587000
   INFO: [mplex] target data-rate specified               : 10080000
   INFO: [mplex] Setting specified specified data rate: 10080000
   INFO: [mplex] Run-in delay = 9009 Video delay = 9009 Audio delay = 12012
   INFO: [mplex] New sequence commences...
   INFO: [mplex] Video e0: buf=      0 frame=000000 sector=00000000
   INFO: [mplex] Audio bd: buf=      0 frame=000000 sector=00000000
++ WARN: [mplex] Stream e0: data will arrive too late sent(SCR)=183296
required(DTS)=183183
++ WARN: [mplex] Video e0: buf=  60740 frame=000058 sector=00001184
++ WARN: [mplex] Audio bd: buf=    573 frame=000060 sector=00000046
++ WARN: [mplex] Stream e0: data will arrive too late sent(SCR)=227181
required(DTS)=225225
++ WARN: [mplex] Video e0: buf=   2025 frame=000072 sector=00001472
++ WARN: [mplex] Audio bd: buf=   1725 frame=000076 sector=00000058
++ WARN: [mplex] Stream e0: data will arrive too late sent(SCR)=271067
required(DTS)=264264
++ WARN: [mplex] Video e0: buf=   2025 frame=000085 sector=00001762
++ WARN: [mplex] Audio bd: buf=      0 frame=000089 sector=00000068
++ WARN: [mplex] Stream e0: data will arrive too late sent(SCR)=314953
required(DTS)=300300
++ WARN: [mplex] Video e0: buf=   2025 frame=000097 sector=00002053
++ WARN: [mplex] Audio bd: buf=      0 frame=000101 sector=00000077
++ WARN: [mplex] Stream e0: data will arrive too late sent(SCR)=358838
required(DTS)=336336
++ WARN: [mplex] Video e0: buf=   2025 frame=000109 sector=00002343
++ WARN: [mplex] Audio bd: buf=      0 frame=000114 sector=00000087
++ WARN: [mplex] Stream e0: data will arrive too late sent(SCR)=402724
required(DTS)=369369
++ WARN: [mplex] Video e0: buf=   2025 frame=000120 sector=00002634
++ WARN: [mplex] Audio bd: buf=      0 frame=000125 sector=00000096
++ WARN: [mplex] Stream e0: data will arrive too late sent(SCR)=446610
required(DTS)=408408
++ WARN: [mplex] Video e0: buf=   2025 frame=000133 sector=00002924
++ WARN: [mplex] Audio bd: buf=      0 frame=000139 sector=00000106
++ WARN: [mplex] Stream e0: data will arrive too late sent(SCR)=490496
required(DTS)=444444
++ WARN: [mplex] Video e0: buf=   2025 frame=000146 sector=00003214
++ WARN: [mplex] Audio bd: buf=      0 frame=000152 sector=00000116
++ WARN: [mplex] Stream e0: data will arrive too late sent(SCR)=534381
required(DTS)=489489
++ WARN: [mplex] Video e0: buf=   2025 frame=000160 sector=00003503
++ WARN: [mplex] Audio bd: buf=      0 frame=000166 sector=00000127
++ WARN: [mplex] Stream e0: data will arrive too late sent(SCR)=578267
required(DTS)=525525
++ WARN: [mplex] Video e0: buf=   2025 frame=000172 sector=00003793
++ WARN: [mplex] Audio bd: buf=      0 frame=000179 sector=00000137
++ WARN: [mplex] Stream e0: data will arrive too late sent(SCR)=622153
required(DTS)=561561
++ WARN: [mplex] Video e0: buf=   2025 frame=000184 sector=00004084
++ WARN: [mplex] Audio bd: buf=      0 frame=000191 sector=00000146
**ERROR: [mplex] Too many frame drops -exiting

I tried changing the argument for the -f flag but it made no
different. However, mkvmerge was able to mux the streams into a
Matroska container and the resulting file is perfect. For my purposes,
this will be fine because commercial cutting is only a first step
before transcoding the file with ffmpeg or Handbrake. I don't reinsert
the clipped show back into the myth database.

For what it's worth, if I create an mkv then use this command:

$ ffmpeg -i matroska.mkv -acodec copy -vcodec copy mpeg.mpg

I get a myth-compliant mpeg2 file with no problems. Of course, mplex
is the more elegant way to get there but this isn't the mjpeg-tools
list. :)

In short, thanks very much for your help. Once I cobble together some
bash to automagically detect the stream IDs, this will work great in
my user job.

P.S. The ionice problem I reported earlier was just a smokescreen for
the fact that I did not have mplex on my system. Oops.


More information about the mythtv-users mailing list