[mythtv-users] MythArchive 0.27 mythburn.py patch for cutlist support (HD-PVR)
Will Dormann
wdormann at gmail.com
Tue Mar 4 02:27:37 UTC 2014
Hi folks,
I've finally gotten around to fixing mythburn.py in 0.27 to support
cutlists on HD-PVR recordings. My prior patch
<http://www.mythtv.org/pipermail/mythtv-users/2012-October/341879.html>
did an extra (redundant) pass of transcoding to MPEG2 before proceeding,
but it seems that this isn't required anymore. But commercial cutting
still didn't work right out of the box. If you want, go right to the
patch. If you're interested about what it does and how I got there,
read on...
I had to change how the cutlist was obtained, as it didn't seem to get
it right off the bat. It now calls mythutil --getcutlist to get the
cutlist and parse it for passing to the mythtranscode commandline.
The first problem I ran into was that mythtranscode would fail if the
cutlist excluded any part of the beginning of the recording. In my
short example, if mythburn.py did:
mythtranscode --mpeg2 --infile
"/var/lib/mytharchive/temp/work/1/2251_20140302230641.mpg" --outfile
"/var/lib/mytharchive/temp/work/1/newfile3.mpg" --honorcutlist "0-254
992-1209"
mythreplex would:
STARTING DEMUX
Video: aspect ratio: 16:9
starting with video PTS:
0:00:00.8938
Can't find all required streams
Please check if audio and video have standard IDs (0xc0 or 0xe0)
Depending on how many frames were cut from the beginning (e.g. 0-10
instead), I might end up with something like:
STARTING DEMUX
Video: aspect ratio: 16:9
starting with video PTS:
0:00:00.6269
starting audio PTS:
0:00:26.4240
Wrong audio frame size: 1438
ringbuffer overflow 250<2016 629145
video ring buffer overrun error
However, if I removed the cutlist from the beginning (0-254):
mythtranscode --mpeg2 --infile
"/var/lib/mytharchive/temp/work/1/2251_20140302230641.mpg" --outfile
"/var/lib/mytharchive/temp/work/1/newfile3.mpg" --honorcutlist "992-1209"
mythreplex would succeed. This seems like a bug, but I didn't want to
dig into mythreplex. So on to projectx.
Projectx couldn't find the recording details via:
rec = DB.searchRecorded(chanid=chanid, starttime=starttime).next()
The chanid and starttime variables seemed OK and matched what's in my
recorded database. But the above line fails with a StopIteration error.
So if finding the recording via the channel and time fails, I'm falling
back to finding it via the recording filename, which was already stored
in streaminfo_orig.xml.
Now projectx fails because the "-set ExternPanel.appendPidToFileName=1"
option is not valid with "recent" projectx versions. This also means
that the filenames produced by projectx aren't what mythburn.py is
expecting. There's also some ".mv2" extensions in there, which should
be ".m2v"
The last thing I noticed was that despite the hd-pvr recording in AC3
already (and potentially in 5.1), mythburn.py was re-encoding the audio
down to stereo. I made a custom encoding profile
<http://www.mythtv.org/wiki/MythArchive#How_do_I_add_my_own_encoding_parameters_to_the_encoding_profiles.3F>
where I made a copy of "HQ" and renamed it. And in the existing HQ
profile I changed the "-acodec" option to "copy" and removed the "-b:a"
and "-ac" options. Note that if not all of your recordings are in AC3,
you may have DVD compatibility issues. My older recordings are in AAC
because of this bug: <https://code.mythtv.org/trac/ticket/2077>
mythburn.py could probably be modified to use lossless conversion only
when AC3 is detected, but I didn't feel like getting into that.
Finally, I ended up swapping out occurrences of "mythffmpeg" with
"ffmpeg". mythffmpeg hung up on an ATSC recording, but the ffmpeg that
comes with Mythbuntu 12.04 seems to work fine. I suppose changing the
mythffmpeg symlink would suffice, but I wanted to keep all modifications
to just mythburn.py.
That's about it. The patch is a bit hacky, but it works for me.
-WD
-------------- next part --------------
--- mythburn.py.orig 2014-03-03 14:13:52.493777354 -0500
+++ mythburn.py 2014-03-03 21:15:53.812732838 -0500
@@ -87,6 +87,7 @@
import getopt
import traceback
import signal
+import subprocess
import xml.dom.minidom
from PIL import Image
from PIL import ImageDraw
@@ -793,7 +794,7 @@
thumbList = ','.join(thumbList)
if getthumbnails==True:
- extractVideoFrames( os.path.join(getItemTempPath(itemnum),"stream.mv2"),
+ extractVideoFrames( os.path.join(getItemTempPath(itemnum),"stream.m2v"),
os.path.join(getItemTempPath(itemnum),"chapter-%1.jpg"), thumbList)
return chapters
@@ -1562,7 +1563,7 @@
def encodeAudio(format, sourcefile, destinationfile, deletesourceafterencode):
write( "Encoding audio to "+format)
if format == "ac3":
- cmd = "mythffmpeg -v 0 -y "
+ cmd = "ffmpeg -v 0 -y "
if cpuCount > 1:
cmd += "-threads %d " % cpuCount
@@ -1571,7 +1572,7 @@
result = runCommand(cmd)
if result != 0:
- fatalError("Failed while running mythffmpeg to re-encode the audio to ac3\n"
+ fatalError("Failed while running ffmpeg to re-encode the audio to ac3\n"
"Command was %s" % cmd)
else:
fatalError("Unknown encodeAudio format " + format)
@@ -1686,7 +1687,7 @@
def getStreamInformation(filename, xmlFilename, lenMethod):
"""create a stream.xml file for filename"""
- command = "mytharchivehelper -q -q --getfileinfo --infile %s --outfile %s --method %d" % (quoteCmdArg(filename), quoteCmdArg(xmlFilename), lenMethod)
+ command = "mytharchivehelper -q --getfileinfo --infile %s --outfile %s --method %d" % (quoteCmdArg(filename), quoteCmdArg(xmlFilename), lenMethod)
result = runCommand(command)
@@ -1744,8 +1745,15 @@
write("Using cutlist: %s" % cutlist_s)
if (localfile != ""):
+ write("Getting cutlist...")
+ result = subprocess.Popen(['mythutil', '-q', '--getcutlist', '--video', localfile], stdout=subprocess.PIPE).communicate()[0]
+ write("%s" % result)
+ cutlist = string.replace(result,"Cutlist: ", "")
+ cutlist = string.replace(cutlist,"\n", "")
+ splitcutlist = string.split(cutlist,',')
+ cutlist = string.join(splitcutlist,' ')
if usecutlist == True:
- command = "mythtranscode --mpeg2 --honorcutlist %s --infile %s --outfile %s" % (cutlist_s, quoteCmdArg(localfile), quoteCmdArg(destination))
+ command = "mythtranscode --mpeg2 --infile %s --outfile %s --honorcutlist \"%s\"" % (quoteCmdArg(localfile), quoteCmdArg(destination), cutlist)
else:
command = "mythtranscode --mpeg2 --infile %s --outfile %s" % (quoteCmdArg(localfile), quoteCmdArg(destination))
else:
@@ -1754,6 +1762,7 @@
else:
command = "mythtranscode --mpeg2 --chanid %s --starttime %s --outfile %s" % (chanid, starttime, quoteCmdArg(destination))
+ write("Running command %s" % str(command))
result = runCommand(command)
if (result != 0):
@@ -1769,8 +1778,23 @@
def generateProjectXCutlist(chanid, starttime, folder):
"""generate cutlist_x.txt for ProjectX"""
-
- rec = DB.searchRecorded(chanid=chanid, starttime=starttime).next()
+
+ try:
+ rec = DB.searchRecorded(chanid=chanid, starttime=starttime).next()
+ except StopIteration:
+ try:
+ write("Cannot find recording on channel %s at %s. Falling back to filename search." % (chanid, starttime))
+ # Fall back to finding recording by filename
+ streaminfofile = os.path.join(folder, 'streaminfo_orig.xml')
+ streaminfoDOM = xml.dom.minidom.parse(streaminfofile)
+ fileNodes = streaminfoDOM.getElementsByTagName("file")
+ fileNode = fileNodes[0]
+ if fileNode.hasAttribute("filename"):
+ filename = fileNode.attributes["filename"].value
+ rec = DB.searchRecorded(basename=os.path.basename(filename)).next()
+ except StopIteration:
+ fatalError("Failed to get recording details from the DB for %s" % filename)
+
starttime = rec.starttime.utcisoformat()
cutlist = rec.markup.getcutlist()
@@ -1803,9 +1827,11 @@
# Use Project-X to cut commercials and/or demux an mpeg2 file
def runProjectX(chanid, starttime, folder, usecutlist, file):
+ write("running projectx...")
"""Use Project-X to cut commercials and demux an mpeg2 file"""
if usecutlist:
+ write('Using cutlist')
if generateProjectXCutlist(chanid, starttime, folder) == False:
write("Failed to generate Project-X cutlist.")
return False
@@ -1814,7 +1840,8 @@
write("Error: input file doesn't exist on local filesystem")
return False
- command = quoteCmdArg(path_projectx[0]) + " %s -id '%s' -set ExternPanel.appendPidToFileName=1 -out %s -name stream" % (quoteCmdArg(file), getStreamList(folder), quoteCmdArg(folder))
+ write('Invoking projectx...')
+ command = quoteCmdArg(path_projectx[0]) + " %s -id '%s' -out %s -name stream" % (quoteCmdArg(file), getStreamList(folder), quoteCmdArg(folder))
if usecutlist == True:
command += " -cut %s" % quoteCmdArg(os.path.join(folder, "cutlist_x.txt"))
write(command)
@@ -1831,46 +1858,11 @@
if addSubtitles:
subtitles = selectSubtitleStream(folder)
- videoID_hex = "0x%x" % video[VIDEO_ID]
- if audio1[AUDIO_ID] != -1:
- audio1ID_hex = "0x%x" % audio1[AUDIO_ID]
- else:
- audio1ID_hex = ""
- if audio2[AUDIO_ID] != -1:
- audio2ID_hex = "0x%x" % audio2[AUDIO_ID]
- else:
- audio2ID_hex = ""
- if addSubtitles and subtitles[SUBTITLE_ID] != -1:
- subtitlesID_hex = "0x%x" % subtitles[SUBTITLE_ID]
- else:
- subtitlesID_hex = ""
-
-
- files = os.listdir(folder)
- for file in files:
- if file[0:9] == "stream{0x": # don't rename files that have already been renamed
- PID = file[7:13]
- SubID = file[19:23]
- if PID == videoID_hex or SubID == videoID_hex:
- os.rename(os.path.join(folder, file), os.path.join(folder, "stream.mv2"))
- elif PID == audio1ID_hex or SubID == audio1ID_hex:
- os.rename(os.path.join(folder, file), os.path.join(folder, "stream0." + file[-3:]))
- elif PID == audio2ID_hex or SubID == audio2ID_hex:
- os.rename(os.path.join(folder, file), os.path.join(folder, "stream1." + file[-3:]))
- elif PID == subtitlesID_hex or SubID == subtitlesID_hex:
- if file[-3:] == "sup":
- os.rename(os.path.join(folder, file), os.path.join(folder, "stream.sup"))
- else:
- os.rename(os.path.join(folder, file), os.path.join(folder, "stream.sup.IFO"))
-
-
- # Fallback if assignment and renaming by ID failed
-
files = os.listdir(folder)
for file in files:
- if file[0:9] == "stream{0x": # don't rename files that have already been renamed
- if not os.path.exists(os.path.join(folder, "stream.mv2")) and file[-3:] == "m2v":
- os.rename(os.path.join(folder, file), os.path.join(folder, "stream.mv2"))
+ if file[0:7] == "stream.": # don't rename files that have already been renamed
+ if not os.path.exists(os.path.join(folder, "stream.m2v")) and file[-3:] == "m2v":
+ os.rename(os.path.join(folder, file), os.path.join(folder, "stream.m2v"))
elif not (os.path.exists(os.path.join(folder, "stream0.ac3")) or os.path.exists(os.path.join(folder, "stream0.mp2"))) and file[-3:] == "ac3":
os.rename(os.path.join(folder, file), os.path.join(folder, "stream0.ac3"))
elif not (os.path.exists(os.path.join(folder, "stream0.ac3")) or os.path.exists(os.path.join(folder, "stream0.mp2"))) and file[-3:] == "mp2":
@@ -2011,13 +2003,13 @@
# Re-encodes a file to mpeg2
def encodeVideoToMPEG2(source, destvideofile, video, audio1, audio2, aspectratio, profile):
- """Encodes an unknown video source file eg. AVI to MPEG2 video and AC3 audio, use mythffmpeg"""
+ """Encodes an unknown video source file eg. AVI to MPEG2 video and AC3 audio, use ffmpeg"""
profileNode = findEncodingProfile(profile)
passes = int(getText(profileNode.getElementsByTagName("passes")[0]))
- command = "mythffmpeg"
+ command = "ffmpeg"
if cpuCount > 1:
command += " -threads %d" % cpuCount
@@ -2077,7 +2069,7 @@
write(command)
result = runCommand(command)
if result!=0:
- fatalError("Failed while running mythffmpeg to re-encode video.\n"
+ fatalError("Failed while running ffmpeg to re-encode video.\n"
"Command was %s" % command)
else:
@@ -2089,7 +2081,7 @@
result = runCommand(pass1)
if result!=0:
- fatalError("Failed while running mythffmpeg (Pass 1) to re-encode video.\n"
+ fatalError("Failed while running ffmpeg (Pass 1) to re-encode video.\n"
"Command was %s" % command)
if os.path.exists(destvideofile):
@@ -2101,13 +2093,13 @@
result = runCommand(pass2)
if result!=0:
- fatalError("Failed while running mythffmpeg (Pass 2) to re-encode video.\n"
+ fatalError("Failed while running ffmpeg (Pass 2) to re-encode video.\n"
"Command was %s" % command)
#############################################################
# Re-encodes a nuv file to mpeg2 optionally removing commercials
def encodeNuvToMPEG2(chanid, starttime, mediafile, destvideofile, folder, profile, usecutlist):
- """Encodes a nuv video source file to MPEG2 video and AC3 audio, using mythtranscode & mythffmpeg"""
+ """Encodes a nuv video source file to MPEG2 video and AC3 audio, using mythtranscode & ffmpeg"""
# make sure mythtranscode hasn't left some stale fifos hanging around
if ((doesFileExist(os.path.join(folder, "audout")) or doesFileExist(os.path.join(folder, "vidout")))):
@@ -2193,7 +2185,7 @@
samplerate, channels = getAudioParams(folder)
videores, fps, aspectratio = getVideoParams(folder)
- command = "mythffmpeg -y "
+ command = "ffmpeg -y "
if cpuCount > 1:
command += "-threads %d " % cpuCount
@@ -2221,11 +2213,11 @@
if (not(doesFileExist(os.path.join(folder, "audout")) and doesFileExist(os.path.join(folder, "vidout")))):
fatalError("Waited too long for mythtranscode to create the fifos - giving up!!")
- write("Running mythffmpeg")
+ write("Running ffmpeg")
result = runCommand(command)
if result != 0:
os.kill(PID, signal.SIGKILL)
- fatalError("Failed while running mythffmpeg to re-encode video.\n"
+ fatalError("Failed while running ffmpeg to re-encode video.\n"
"Command was %s" % command)
#############################################################
@@ -2513,7 +2505,7 @@
#Generate a temp folder name for this file
folder=getItemTempPath(filecount)
#Process this file
- file=os.path.join(folder,"stream.mv2")
+ file=os.path.join(folder,"stream.m2v")
#Get size of vobfile in MBytes
totalvideosize+=os.path.getsize(file)
@@ -2554,7 +2546,7 @@
filecount+=1
folder=getItemTempPath(filecount)
progduration=getLengthOfVideo(filecount)
- file=os.path.join(folder,"stream.mv2")
+ file=os.path.join(folder,"stream.m2v")
progvsize=os.path.getsize(file)
progvbitrate=progvsize/progduration
if progvbitrate>rate :
@@ -2598,7 +2590,7 @@
for node in files:
filecount+=1
folder=getItemTempPath(filecount)
- file=os.path.join(folder,"stream.mv2")
+ file=os.path.join(folder,"stream.m2v")
vsize+=os.path.getsize(file)
duration+=getLengthOfVideo(filecount)
@@ -2634,7 +2626,7 @@
for node in files:
filecount+=1
folder=getItemTempPath(filecount)
- file=os.path.join(folder,"stream.mv2")
+ file=os.path.join(folder,"stream.m2v")
progvsize=os.path.getsize(file)
progduration=getLengthOfVideo(filecount)
progvbitrate=progvsize/progduration
@@ -2643,9 +2635,9 @@
scalefactor=1.0+(fudge_requant*float(progvbitrate-vrate)/float(vrate))
if scalefactor>3.0 :
write( "Large shrink factor. You may not like the result! ")
- runM2VRequantiser(os.path.join(getItemTempPath(filecount),"stream.mv2"),os.path.join(getItemTempPath(filecount),"stream.small.mv2"),scalefactor)
- os.remove(os.path.join(getItemTempPath(filecount),"stream.mv2"))
- os.rename(os.path.join(getItemTempPath(filecount),"stream.small.mv2"),os.path.join(getItemTempPath(filecount),"stream.mv2"))
+ runM2VRequantiser(os.path.join(getItemTempPath(filecount),"stream.m2v"),os.path.join(getItemTempPath(filecount),"stream.small.m2v"),scalefactor)
+ os.remove(os.path.join(getItemTempPath(filecount),"stream.m2v"))
+ os.rename(os.path.join(getItemTempPath(filecount),"stream.small.m2v"),os.path.join(getItemTempPath(filecount),"stream.m2v"))
else:
write( "Unpackaged total %.2f Mb. About %.0f Mb will be unused." % ((allfiles/mega),(mv2space-totalvideosize)/mega))
@@ -3241,7 +3233,7 @@
if node.nodeName=="graphic":
if node.attributes["filename"].value == "%movie":
#This is a movie preview item so we need to generate the thumbnails
- inputfile = os.path.join(getItemTempPath(videoitem),"stream.mv2")
+ inputfile = os.path.join(getItemTempPath(videoitem),"stream.m2v")
outputfile = os.path.join(previewfolder, "preview-i%d-t%%1-f%%2.jpg" % itemonthispage)
width = getScaledAttribute(node, "w")
height = getScaledAttribute(node, "h")
@@ -4543,8 +4535,8 @@
#do we need to re-encode the file to make it DVD compliant?
if not isFileOkayForDVD(file, folder):
if getFileType(folder) == 'nuv':
- #file is a nuv file which mythffmpeg has problems reading so use mythtranscode to pass
- #the video and audio streams to mythffmpeg to do the reencode
+ #file is a nuv file which ffmpeg has problems reading so use mythtranscode to pass
+ #the video and audio streams to ffmpeg to do the reencode
#we need to re-encode the file, make sure we get the right video/audio streams
#would be good if we could also split the file at the same time
@@ -4649,9 +4641,9 @@
if usebookmark == True and os.path.exists(previewImage):
copy(previewImage, titleImage)
else:
- extractVideoFrame(os.path.join(folder, "stream.mv2"), titleImage, thumboffset)
+ extractVideoFrame(os.path.join(folder, "stream.m2v"), titleImage, thumboffset)
else:
- extractVideoFrame(os.path.join(folder, "stream.mv2"), titleImage, thumboffset)
+ extractVideoFrame(os.path.join(folder, "stream.m2v"), titleImage, thumboffset)
write( "*************************************************************")
write( "Finished processing '%s'" % file.attributes["filename"].value)
@@ -4697,8 +4689,8 @@
#do we need to re-encode the file to make it DVD compliant?
if not isFileOkayForDVD(file, folder):
if getFileType(folder) == 'nuv':
- #file is a nuv file which mythffmpeg has problems reading so use mythtranscode to pass
- #the video and audio streams to mythffmpeg to do the reencode
+ #file is a nuv file which ffmpeg has problems reading so use mythtranscode to pass
+ #the video and audio streams to ffmpeg to do the reencode
#we need to re-encode the file, make sure we get the right video/audio streams
#would be good if we could also split the file at the same time
@@ -4818,9 +4810,9 @@
if usebookmark == True and os.path.exists(previewImage):
copy(previewImage, titleImage)
else:
- extractVideoFrame(os.path.join(folder, "stream.mv2"), titleImage, thumboffset)
+ extractVideoFrame(os.path.join(folder, "stream.m2v"), titleImage, thumboffset)
else:
- extractVideoFrame(os.path.join(folder, "stream.mv2"), titleImage, thumboffset)
+ extractVideoFrame(os.path.join(folder, "stream.m2v"), titleImage, thumboffset)
write( "*************************************************************")
write( "Finished processing file '%s'" % file.attributes["filename"].value)
@@ -4991,7 +4983,7 @@
#Multiplex this file
#(This also removes non-required audio feeds inside mpeg streams
#(through re-multiplexing) we only take 1 video and 1 or 2 audio streams)
- pid=multiplexMPEGStream(os.path.join(folder,'stream.mv2'),
+ pid=multiplexMPEGStream(os.path.join(folder,'stream.m2v'),
os.path.join(folder,'stream0'),
os.path.join(folder,'stream1'),
os.path.join(folder,'final.vob'),
@@ -5006,8 +4998,8 @@
for node in files:
filecount+=1
folder=getItemTempPath(filecount)
- if os.path.exists(os.path.join(folder, "stream.mv2")):
- os.remove(os.path.join(folder,'stream.mv2'))
+ if os.path.exists(os.path.join(folder, "stream.m2v")):
+ os.remove(os.path.join(folder,'stream.m2v'))
if os.path.exists(os.path.join(folder, "stream0.mp2")):
os.remove(os.path.join(folder,'stream0.mp2'))
if os.path.exists(os.path.join(folder, "stream1.mp2")):
More information about the mythtv-users
mailing list