[mythtv-users] Parallel transcoding and subtitles from DVB-S in Europe

belcampo belcampo at zonnet.nl
Fri May 1 10:04:49 UTC 2009


I've created a script to do this. I would like to receive comment on 
this, make it more portable, and better suggestions how to solve my 'To 
do's'

--------------

# script for parallel transcoding, on different machines, and extraction 
of srt/subtitle files from DVB-S Teletext streams in Europe
# prerequisites are ProjectX, Java runtime, MP4Box, ffmpeg, as much 
cpu-cores you can get
# network share, with the same name, available to all participants
# at least 1 ssh-server and the rest neads ssh-client, to login without 
passwords, after generating the required keys
# following small script to execute commands on remote-clients by their 
station name
# #!/bin/sh
# ssh `basename $0` $*

# To Do
# mv the resulting mp4 to $mythdir change mpg to mp4 in myth's database 
and clear the cutlist, eventually use mythrename to 'beautify' the name
# For BBC channels I've to add some map instructions to get the right 
audio-channel, but requires some if then, which I don't know of yet

pal=25
ntsc=30000/1001
mythdir=/spvfs/tv
workdir=/pvfs2/scratch
# cropdetection can be automated, but I've too many situations where 
this doesn't work, so I do it by hand
#crop=88
# to get start time of the process
touch /$workdir/begin
# The encoding arguments
sdparams=" -an -vcodec libx264 -b 1672k -deinterlace -flags +loop -coder 
1 -refs 2 -deblockalpha 0 -deblockbeta 0  -partitions +parti4x4 
-partitions +partp8x8 -partitions +parti8x8 -partitions +partb8x8 
-me_method umh -subq 7 -me_range 24 -cmp +chroma -bf 0 -b_strategy 1 -g 
250 -keyint_min 25 -sc_threshold 40 -maxrate 2500k -qmin 16 -qmax 32 -f 
rawvideo "
# Get amount of cores
cores=6
# remove workfiles from previous run
rm -rf $workdir/*.h264 $workdir/*.mpa $workdir/*.mpv $workdir/*.txt 
$workdir/cutl* $workdir/*.sup* $workdir/*.mp? $workdir/*.aac
# Go to working directory of myth
cd $mythdir
# Definition of the calculate function
c () { echo "scale=0;$*" | bc -l; }
# Inputvariable of the file to work on
nameoffile=$1
# Transform myth-cutlist to ProjectX cutlist format
mythcommflag --getcutlist -f $nameoffile | grep Cutlist | cut -d ":" -f 
2 | cut -d " " -f 2 | sed 's/,/-/g' | sed 's/^0-//g' | sed 's/-/\n/g' | 
sed 'x' > $workdir/cutlist$nameoffile
# Run ProjectX with certain ini-file to get TXT-page for subtitles
# I have several 'X.ini' files depending on what page the subtitles are
# On files starting with 50 or 83 it's 888.ini
# starting with 11 it's 893.ini
# starting with 29 or it's 150.ini
# the rest does't have subtitles that I'm interested in so there is a 
blank X.ini for this case
# have to find a way to automaticaly choose the right file, don't know 
enough about scripting yet to create an if then loop
projectx $nameoffile -ini ~/888.ini -out $workdir -demux -cut 
$workdir/cutlist$nameoffilenameoffile=`echo $nameoffile | cut -d "." -f 1`
# Rename the *[###].srt file to nameoffile.srt so that the name is 
identical to the resulting mp4 file
mv $workdir/*888*.srt $workdir/$nameoffile.srt
# Store the amount of frames in frames.txt, but first remove ^M from it
mediainfo -f $workdir/$nameoffile.mpv | grep "Frame count" | cut -d ":" 
-f 2 | cut -d " " -f 2 | dos2unix > frames.txt
# Store integers of frames.txt in variable framec
framec=$(grep [0-9] frames.txt)
echo $framec
# Calculate the duration in seconds from the amount of frames
timea=$(c $framec/$pal)
echo $timea
# Calculate minutes from the outcome of the seconds calculation
time=$(c $timea/60)
echo $time
# Calculate the amount minutes each core will process
ss0=$(c $time/$cores)
echo $ss0
# add 1 so the last core gets a little less to do, it gets time for 
audio-processing
ss1=$(c $ss0+1)
echo $ss1
# Calculate the starttime in seconds for the 2nd core
ss=$(c $ss1*60)
echo $ss
# Calculate the amount of frames each core except the last one has to 
process
vframes=$(c $ss*pal)
echo $vframes
# The actual ffmpeg commands. The first has no starttime, it starts from 
the beginning, the last one has no amount of frames; it does the rest.
mm03 ffmpeg                -vframes $vframes -i $workdir/$nameoffile.mpv 
-croptop $crop -cropbottom $crop $sdparams $workdir/1.h264  > /dev/null 
2>&1 &
mm03 ffmpeg -ss $ss        -vframes $vframes -i $workdir/$nameoffile.mpv 
-croptop $crop -cropbottom $crop $sdparams $workdir/2.h264  > /dev/null 
2>&1 &
mm04 ffmpeg -ss $(c $ss*2) -vframes $vframes -i $workdir/$nameoffile.mpv 
-croptop $crop -cropbottom $crop $sdparams $workdir/3.h264  > /dev/null 
2>&1 &
mm04 ffmpeg -ss $(c $ss*3) -vframes $vframes -i $workdir/$nameoffile.mpv 
-croptop $crop -cropbottom $crop $sdparams $workdir/4.h264  > /dev/null 
2>&1 &
mm05 ffmpeg -ss $(c $ss*4) -vframes $vframes -i $workdir/$nameoffile.mpv 
-croptop $crop -cropbottom $crop $sdparams $workdir/5.h264  > /dev/null 
2>&1 &
mm05 ffmpeg -ss $(c $ss*5) -i $workdir/$nameoffile.mpv -croptop $crop 
-cropbottom $crop $sdparams $workdir/6.h264
mm05 ffmpeg -i $workdir/$nameoffile.mpa -vn -acodec libfaac -ab 128k -ar 
48000 -ac 2 -f rawvideo $workdir/$nameoffile.aac
touch $workdir/end
# I have to know when MP4Box can start muxing, all transcoding has to be 
finished first
# I've created a files named 'begin' and 'end', the difference in time 
has to be >=0
# Don't know yet how to create the if then
#ready=${`stat -c%Y $workdir/end` - `stat -c%Y $workdir/begin` | bc}
# therefore I create a sleep to be sure everything is ready
sleep 120
MP4Box -tmp $workdir -add $workdir/1.h264 -cat $workdir/2.h264 -cat 
$workdir/3.h264 -cat $workdir/4.h264 -cat $workdir/5.h264 -cat 
$workdir/6.h264 -add $workdir/$nameoffile.aac $workdir/$nameoffile.mp4
touch $workdir/end2
# It took
echo `stat -c%Y $workdir/end2` - `stat -c%Y $workdir/begin` | bc
-----------------

Henk Schoneveld


More information about the mythtv-users mailing list