Difference between revisions of "Mythcommflag-wrapper"

From MythTV Official Wiki
Jump to: navigation, search
m (add potential improvements)
(Edit script variables: Info added from trac ticket.)
(28 intermediate revisions by 7 users not shown)
Line 4: Line 4:
 
|long=A simple bash script based on [[Silence-detect.sh]] that wraps mythcommflag and can be used manually (via 'Begin Advert Detection' menu option in recording browser) to use silence detection for commflagging for certain UK TV channels.
 
|long=A simple bash script based on [[Silence-detect.sh]] that wraps mythcommflag and can be used manually (via 'Begin Advert Detection' menu option in recording browser) to use silence detection for commflagging for certain UK TV channels.
 
|category=Scripts
 
|category=Scripts
|S24=yes}}
+
|S24=yes|S26=yes}}
  
=== Rename the existing mythcommflag binary ===
+
=== Prerequisites ===
  
Mythbackend runs the commflag differently according to the configured name (i.e <code>JobQueueCommFlagCommand</code> in the <code>settings</code> table). If it's not '<code>mythcommflag</code>', then it always runs the configured commflagger with no parameters (i.e. in "flag all uncommflagged recordings"). Therefore it's necessary for this script to replace and wrap <code>mythcommflag</code>. e.g.
+
This script needs <code>mytharchivehelper</code> which may be included in the MythArchive package ( No longer needed for 0.26 ).Ensure that it is installed on your machine. It also needs [[http://www.ffmpeg.org/<code>ffmpeg</code>]] and [[http://mp3splt.sourceforge.net/<code>mp3splt</code>]]. It's likely that these are packaged for your distro, either directly, or by a third-party repository (e.g. ATrpms for Fedora).
 
 
<code><pre>
 
mv /usr/bin/mythcommflag /usr/bin/mythcommflag.orig
 
</pre></code>
 
  
 
=== Install wrapper script in place of mythcommflag ===
 
=== Install wrapper script in place of mythcommflag ===
  
Install this script as /usr/bin/mythcommflag (or wherever your original mythcommflag binary was located)
+
Install this script to somewhere accessible by the backend.  In mythtv-setup, the command used to run for commercial detection can be modified.  The '''''mythcommflag''''' option is a special phrase, telling the jobqueue to run
 +
  ${PREFIX}/bin/mythcommflag -j %JOBID% -V %VERBOSELEVEL%
 +
If you define any string other than '''''mythcommflag''''', the command options must be explicitly defined, such as
 +
  /home/mythtv/bin/mythcommflag-wrapper.sh -j %JOBID% -V 0
  
 
{{Code box|mythcommflag-wrapper.sh|
 
{{Code box|mythcommflag-wrapper.sh|
 
<pre>
 
<pre>
#!/bin/sh
+
#!/bin/bash
 +
 
  
 
# edit/tune these #
 
# edit/tune these #
# DB username and password
 
# TODO AJB20110130 get these from $HOME/.mythtv/mysql.txt
 
MYTHUSER="mythtv"
 
MYTHPASS="mythtv"
 
# root of MythTV recordings
 
# TODO AJB20110130 generate this automatically from the storagegroups table
 
RECORDINGSROOT="/srv/myth"
 
 
# Allow ad breaks to be upto 400s long by coalescing non-silence
 
# Allow ad breaks to be upto 400s long by coalescing non-silence
 
MAXCOMMBREAKSECS=400
 
MAXCOMMBREAKSECS=400
Line 39: Line 32:
 
# Log file
 
# Log file
 
LOGFILE="/var/log/mythtv/mythcommflag-wrapper"
 
LOGFILE="/var/log/mythtv/mythcommflag-wrapper"
 +
# Copy commercial skiplist to cutlist automatically? 1=Enabled
 +
COPYTOCUTLIST=1
 +
 +
MYTHCFG=""
 +
# mythbackend in Fedora packages has $HOME=/etc/mythtv
 +
if [ -e $HOME/mysql.txt ]; then
 +
MYTHCFG="$HOME/mysql.txt"
 +
fi
 +
 +
 +
if [ -e $HOME/.mythtv/mysql.txt ]; then
 +
MYTHCFG="$HOME/.mythtv/mysql.txt"
 +
fi
 +
if [ "$MYTHCFG" = "" ]; then
 +
echo >>$LOGFILE "No mysql.txt found in $HOME or $HOME/.mythtv - exiting!"
 +
exit 1
 +
fi
 +
 +
MAH=`which mytharchivehelper`
 +
if [ "$MAH" = "" ]; then
 +
        echo "mytharchivehelper not found in $PATH - exiting!"
 +
        exit 1
 +
fi
 +
 +
# DB username and password
 +
MYTHHOST=`grep DBHostName <$MYTHCFG | awk -F= '{print $2}'`
 +
MYTHUSER=`grep DBUserName <$MYTHCFG | awk -F= '{print $2}'`
 +
MYTHPASS=`grep DBPassword <$MYTHCFG | awk -F= '{print $2}'`
 +
MYTHDB=`grep DBName <$MYTHCFG | awk -F= '{print $2}'`
 +
#echo >>$LOGFILE "[$MYTHHOST] [$MYTHUSER] [$MYTHPASS] [$MYTHDB]"
 +
 +
# root of MythTV recordings
 +
RECORDINGSROOT=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select dirname from storagegroup;" $MYTHDB | tail -n +2 | sort | uniq | tr '\n' ' '`
 +
  
 
echo >>$LOGFILE "$0 run with [$*] at `date` by `whoami`"
 
echo >>$LOGFILE "$0 run with [$*] at `date` by `whoami`"
 +
echo >>$LOGFILE "RECORDINGSROOT=$RECORDINGSROOT"
 +
#export >>$LOGFILE
 +
 +
#/usr/bin/mythcommflag run with [-j 584 -V 8213] at Sat Jan 29 17:26:58 GMT 2011 by root
 +
#select recorded.basename from recorded join jobqueue where jobqueue.id=584 and jobqueue.chanid=recorded.chanid and jobqueue.starttime=recorded.starttime;
  
 
silence_detect() {
 
silence_detect() {
Line 49: Line 81:
 
# get frame count, less 3 seconds (75 frames) fudge factor
 
# get frame count, less 3 seconds (75 frames) fudge factor
 
let FRAMES=`mytharchivehelper -i $filename $TMPDIR/streaminfo.xml 1 2>&1 | grep " frames = " | awk -F"= " '{print $2}'`-${ENDFUDGEFRAMES}
 
let FRAMES=`mytharchivehelper -i $filename $TMPDIR/streaminfo.xml 1 2>&1 | grep " frames = " | awk -F"= " '{print $2}'`-${ENDFUDGEFRAMES}
 +
echo >>$LOGFILE "total frames = $FRAMES (+/- 75)"
  
 
# From <http://www.mythtv.org/wiki/Silence-detect.sh>
 
# From <http://www.mythtv.org/wiki/Silence-detect.sh>
 +
 +
# AJB20110129 Have already checked for pre-existing cutlist, and this check doesn't work properly after --clearcutlist anyway
 +
# [[ "Cutlist: "  == `mythcommflag --getcutlist -f $filename  |grep Cutlist` ]] \
 +
#     || { echo already has cutlist && exit 1; }
  
 
cd $TMPDIR
 
cd $TMPDIR
Line 62: Line 99:
 
      {finish=$2} else {print int(start*25+1)"-"int(finish*25-25);
 
      {finish=$2} else {print int(start*25+1)"-"int(finish*25-25);
 
      start=$1; finish=$2;}}END{print int(start*25+1)"-"'$FRAMES'}'`
 
      start=$1; finish=$2;}}END{print int(start*25+1)"-"'$FRAMES'}'`
 +
echo >>$LOGFILE "silence-detect has generated cutlist: $CUTLIST"
 +
 +
echo >>$LOGFILE "silence-detect(): CHANID=$CHANID, STARTTIME=$STARTTIME, FRAMES=$FRAMES"
 +
CUTLIST=`tail --lines=+3 mp3splt.log|sort -g |\
 +
awk 'BEGIN{start=0;ORS="\n"}{if($2-start<'$MAXCOMMBREAKSECS') {finish=$2} else {print "INSERT INTO recordedmarkup (chanid,starttime,mark,type) VALUES ('$CHANID',\"'"$STARTTIME"'\","int(start*25+1)",4);\nINSERT INTO recordedmarkup (chanid,starttime,mark,type) VALUES ('$CHANID',\"'"$STARTTIME"'\","int(finish*25-25)",5);"; start=$1; finish=$2;}}END{print "INSERT INTO recordedmarkup (chanid,starttime,mark,type) VALUES ('$CHANID',\"'"$STARTTIME"'\","int(start*25+1)",4);\nINSERT INTO recordedmarkup (chanid,starttime,mark,type) VALUES ('$CHANID',\"'"$STARTTIME"'\",'$FRAMES',5);"}' | sed 's/\"/'\''/g'`
 +
RC=$?
 +
# echo >>$LOGFILE "silence-detect SQLgen returned with $RC, generated SQL skipist: $CUTLIST"
  
echo >>$LOGFILE "silence-detect has generated cutlist: $CUTLIST"
+
# CUTLIST="USE $MYTHDB; $CUTLIST"
  
 
rm -rf $TMPDIR
 
rm -rf $TMPDIR
Line 71: Line 115:
 
if [ $# -eq 0 ]; then
 
if [ $# -eq 0 ]; then
 
# run with no parameters, flag every unflagged recording
 
# run with no parameters, flag every unflagged recording
exec mythcommflag.orig
+
exec mythcommflag
 
exit $?
 
exit $?
 
else
 
else
Line 81: Line 125:
 
else
 
else
 
# we're being used in some other way, run the real mythcommmflag
 
# we're being used in some other way, run the real mythcommmflag
echo >>$LOGFILE "running mythcommflag.orig $*"
+
echo >>$LOGFILE "running mythcommflag $*"
exec mythcommflag.orig $*
+
exec mythcommflag $*
 
exit $?
 
exit $?
 
fi
 
fi
 
echo >>$LOGFILE "running job $JOB"
 
echo >>$LOGFILE "running job $JOB"
HASCUTLIST=`mysql -u${MYTHUSER} -p${MYTHPASS} -e "use mythconverg; select recorded.cutlist from recorded join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=recorded.chanid and jobqueue.starttime=recorded.starttime;" | tail -n +2`
+
HASCUTLIST=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select recorded.cutlist from recorded join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=recorded.chanid and jobqueue.starttime=recorded.starttime;" $MYTHDB | tail -n +2`
 
if [ "$HASCUTLIST" = "1" ]; then
 
if [ "$HASCUTLIST" = "1" ]; then
echo "program already has (manual?) cutlist, exiting"
+
echo >>$LOGFILE "program already has (manual?) cutlist, exiting"
 
exit 0
 
exit 0
 
fi
 
fi
CALLSIGN=`mysql -u${MYTHUSER} -p${MYTHPASS} -e "use mythconverg; select channel.callsign from channel join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=channel.chanid;" | tail -n +2`
+
CALLSIGN=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select channel.callsign from channel join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=channel.chanid;" $MYTHDB | tail -n +2`
CHANID=`mysql -u${MYTHUSER} -p${MYTHPASS} -e "use mythconverg; select chanid from jobqueue where jobqueue.id=$JOB;" | tail -n +2`
+
CHANID=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select chanid from jobqueue where jobqueue.id=$JOB;" $MYTHDB | tail -n +2`
STARTTIME=`mysql -u${MYTHUSER} -p${MYTHPASS} -e "use mythconverg; select starttime from jobqueue where jobqueue.id=$JOB;" | tail -n +2`
+
STARTTIME=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select starttime from jobqueue where jobqueue.id=$JOB;" $MYTHDB | tail -n +2`
 
echo >>$LOGFILE "channel callsign is $CALLSIGN"
 
echo >>$LOGFILE "channel callsign is $CALLSIGN"
 
echo >>$LOGFILE "chanid=$CHANID STARTTIME=$STARTTIME"
 
echo >>$LOGFILE "chanid=$CHANID STARTTIME=$STARTTIME"
BASENAME=`mysql -u${MYTHUSER} -p${MYTHPASS} -e "use mythconverg; select recorded.basename from recorded join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=recorded.chanid and jobqueue.starttime=recorded.starttime;" | tail -n +2`
+
BASENAME=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select recorded.basename from recorded join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=recorded.chanid and jobqueue.starttime=recorded.starttime;" $MYTHDB | tail -n +2`
 
echo >>$LOGFILE "basename is $BASENAME"
 
echo >>$LOGFILE "basename is $BASENAME"
 
FILENAME=`ionice -c3 nice find ${RECORDINGSROOT} -name $BASENAME`
 
FILENAME=`ionice -c3 nice find ${RECORDINGSROOT} -name $BASENAME`
 
echo >>$LOGFILE "filename is $FILENAME"
 
echo >>$LOGFILE "filename is $FILENAME"
  
if [ "$CALLSIGN" = "Film4" -o "$CALLSIGN" = "ITV1" -o "$CALLSIGN" = "ITV1 +1" -o "$CALLSIGN" = "ITV2" -o "$CALLSIGN" = "ITV2 +1" -o "$CALLSIGN" = "ITV3" -o "$CALLSIGN" = "ITV4" -o "$CALLSIGN" = "Dave" -o "$CALLSIGN" = "Dave ja vu" ]; then
+
#safe list
 +
# if [ "$CALLSIGN" = "Film4" -o "$CALLSIGN" = "ITV1" -o "$CALLSIGN" = "ITV1 +1" -o "$CALLSIGN" = "ITV2" -o "$CALLSIGN" = "ITV2 +1" -o "$CALLSIGN" = "ITV3" -o "$CALLSIGN" = "ITV4" -o "$CALLSIGN" = "Dave" -o "$CALLSIGN" = "Dave ja vu" ]; then
 +
# Channel 4 channels seem to work too, but sometimes last cut is too long?
 +
# if [ "$CALLSIGN" = "Channel 4" -o "$CALLSIGN" = "Channel 4+1" -o "$CALLSIGN" = "More 4" -o "$CALLSIGN" = "E4" -o "$CALLSIGN" = "E4+1" -o "$CALLSIGN" = "Film4" -o "$CALLSIGN" = "ITV1" -o "$CALLSIGN" = "ITV1 +1" -o "$CALLSIGN" = "ITV2" -o "$CALLSIGN" = "ITV2 +1" -o "$CALLSIGN" = "ITV3" -o "$CALLSIGN" = "ITV4" -o "$CALLSIGN" = "Dave" -o "$CALLSIGN" = "Dave ja vu" ]; then
 +
# if [ "$CALLSIGN" = "FIVE USA" -o "$CALLSIGN" = "FIVER" -o "$CALLSIGN" = "FIVE" -o "$CALLSIGN" = "Channel 4" -o "$CALLSIGN" = "Channel 4+1" -o "$CALLSIGN" = "More 4" -o "$CALLSIGN" = "E4" -o "$CALLSIGN" = "E4+1" -o "$CALLSIGN" = "Film4" -o "$CALLSIGN" = "ITV1" -o "$CALLSIGN" = "ITV1 +1" -o "$CALLSIGN" = "ITV2" -o "$CALLSIGN" = "ITV2 +1" -o "$CALLSIGN" = "ITV3" -o "$CALLSIGN" = "ITV4" -o "$CALLSIGN" = "Dave" -o "$CALLSIGN" = "Dave ja vu" ]; then
 +
# Fails on FIVER and cuts almost entire recording. Works for other FIVE channels with caveat that they include news bulletins which are't cut
 +
if [ "$CALLSIGN" = "FIVE USA" -o "$CALLSIGN" = "FIVE" -o "$CALLSIGN" = "Channel 4" -o "$CALLSIGN" = "Channel 4+1" -o "$CALLSIGN" = "More 4" -o "$CALLSIGN" = "E4" -o "$CALLSIGN" = "E4+1" -o "$CALLSIGN" = "Film4" -o "$CALLSIGN" = "ITV1" -o "$CALLSIGN" = "ITV1 +1" -o "$CALLSIGN" = "ITV2" -o "$CALLSIGN" = "ITV2 +1" -o "$CALLSIGN" = "ITV3" -o "$CALLSIGN" = "ITV4" -o "$CALLSIGN" = "Dave" -o "$CALLSIGN" = "Dave ja vu" ]; then
 
echo >>$LOGFILE "Callsign in whitelist - will run silence_detect"
 
echo >>$LOGFILE "Callsign in whitelist - will run silence_detect"
mysql -u${MYTHUSER} -p${MYTHPASS} -e "use mythconverg; update recorded set commflagged=2 where chanid=$CHANID and starttime='${STARTTIME}';"
+
mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update recorded set commflagged=2 where chanid=$CHANID and starttime='${STARTTIME}';" $MYTHDB
 
CUTLIST=""
 
CUTLIST=""
 
echo >>$LOGFILE "silence_detect $FILENAME"
 
echo >>$LOGFILE "silence_detect $FILENAME"
 
silence_detect $FILENAME
 
silence_detect $FILENAME
 
echo >>$LOGFILE "silect_detect() set CUTLIST to $CUTLIST"
 
echo >>$LOGFILE "silect_detect() set CUTLIST to $CUTLIST"
mythcommflag --setcutlist $CUTLIST -f $FILENAME
+
let BREAKS=`echo "$CUTLIST"|wc -l`/2
 +
echo >>$LOGFILE "$BREAKS break(s) found."
 +
# mythcommflag --setcutlist $CUTLIST -f $FILENAME
 +
mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "$CUTLIST" $MYTHDB
 
RC=$?
 
RC=$?
echo "mythcommflag --setcutlist returned $RC"
+
# echo "mythcommflag --setcutlist returned $RC"
 +
echo >>$LOGFILE "mysql setskiplist returned $RC"
 
if [ $RC -eq 0 ]; then
 
if [ $RC -eq 0 ]; then
mysql -u${MYTHUSER} -p${MYTHPASS} -e "use mythconverg; update recorded set commflagged=1 where chanid=$CHANID and starttime='${STARTTIME}';"
+
mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update recorded set commflagged=1 where chanid=$CHANID and starttime='${STARTTIME}';" ${MYTHDB}
 +
mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update jobqueue set status=272, comment='Finished, $BREAKS break(s) found.' where id=$JOB;" ${MYTHDB}
 +
if [ $COPYTOCUTLIST -eq 1 ]; then
 +
mythcommflag --gencutlist -f $FILENAME
 +
RC=$?
 +
if [ $RC -eq 0 ]; then
 +
echo >>$LOGFILE "mythcommflag --gencutlist successfully copied skip list to cut list"
 +
else
 +
echo >>$LOGFILE "mythcommflag --gencutlist failed, returned $RC"
 +
fi
 +
fi
 
else
 
else
 
echo >>$LOGFILE "mythcommflag failed; returned $RC"
 
echo >>$LOGFILE "mythcommflag failed; returned $RC"
mysql -u${MYTHUSER} -p${MYTHPASS} -e "use mythconverg; update recorded set commflagged=0 where chanid=$CHANID and starttime='${STARTTIME}';"
+
mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update recorded set commflagged=0 where chanid=$CHANID and starttime='${STARTTIME}';" ${MYTHDB}
 
fi
 
fi
 
else
 
else
 
# Not a whitelisted channel for silence_detect
 
# Not a whitelisted channel for silence_detect
echo >>$LOGFILE "won't run silence-detect, running mythcommflag.orig $*"
+
echo >>$LOGFILE "won't run silence-detect, running mythcommflag $*"
exec mythcommflag.orig $*
+
exec mythcommflag $*
 
exit $?
 
exit $?
 
fi
 
fi
 
fi
 
fi
 +
echo >>$LOGFILE
 +
 
</pre>
 
</pre>
 
}}
 
}}
  
 +
 +
 +
 +
Version using mythutil for 0.25+ and it no longer needs mythexport
 +
 +
{{Code box|mythcommflag-wrapper.sh|
 +
<pre>
 +
#!/bin/bash
 +
 +
# Version using mythutil for 0.25+ as it no longer needs mythexport: Tuned detect slightly, updated whitelist to include Freesat/Freeview +1 channels. Supports UK Freeview/Freesat HD Channels as they have MPEG audio channels. ( Channels with AC3 Only are not supported. ) Changed ffmpeg to avconv, as ffmpeg is deprecated. and ffmpeg -i occasionally chose the Audio Descriptive track, which is nearly all silence.
 +
 +
# edit/tune these #
 +
# Allow ad breaks to be upto 400s long by coalescing non-silence
 +
MAXCOMMBREAKSECS=400
 +
# -80dB and minimum of 0.12s to be considered 'silent'
 +
#MP3SPLT_OPTS="th=-70,min=0.15"
 +
MP3SPLT_OPTS="th=-80,min=0.12"
 +
# Log file
 +
LOGFILE="/var/log/mythtv/mythcommflag-wrapper"
 +
# Copy commercial skiplist to cutlist automatically? 1=Enabled
 +
#COPYTOCUTLIST=1
 +
 +
MYTHCFG=""
 +
# mythbackend in Fedora packages has $HOME=/etc/mythtv
 +
if [ -e $HOME/mysql.txt ]; then
 +
MYTHCFG="$HOME/mysql.txt"
 +
fi
 +
 +
 +
if [ -e $HOME/.mythtv/mysql.txt ]; then
 +
MYTHCFG="$HOME/.mythtv/mysql.txt"
 +
fi
 +
if [ "$MYTHCFG" = "" ]; then
 +
echo >>$LOGFILE "No mysql.txt found in $HOME or $HOME/.mythtv - exiting!"
 +
exit 1
 +
fi
 +
 +
# DB username and password
 +
MYTHHOST=`grep DBHostName <$MYTHCFG | awk -F= '{print $2}'`
 +
MYTHUSER=`grep DBUserName <$MYTHCFG | awk -F= '{print $2}'`
 +
MYTHPASS=`grep DBPassword <$MYTHCFG | awk -F= '{print $2}'`
 +
MYTHDB=`grep DBName <$MYTHCFG | awk -F= '{print $2}'`
 +
#echo >>$LOGFILE "[$MYTHHOST] [$MYTHUSER] [$MYTHPASS] [$MYTHDB]"
 +
 +
# root of MythTV recordings
 +
RECORDINGSROOT=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select dirname from storagegroup;" $MYTHDB | tail -n +2 | sort | uniq | tr '\n' ' '`
 +
 +
 +
echo >>$LOGFILE "$0 run with [$*] at `date` by `whoami`"
 +
echo >>$LOGFILE "RECORDINGSROOT=$RECORDINGSROOT"
 +
 +
# From <http://www.mythtv.org/wiki/Silence-detect.sh>
 +
 +
silence_detect() {
 +
local filename=$1
 +
 +
TMPDIR=`mktemp -d /tmp/mythcommflag.XXXXXX` || exit 1
 +
 +
cd $TMPDIR
 +
touch `basename $filename`.touch
 +
ionice -c3 nice avconv -i $filename -acodec copy sound.mp3
 +
ionice -c3 nice mp3splt -s -p $MP3SPLT_OPTS sound.mp3
 +
 +
CUTLIST=`tail --lines=+3 mp3splt.log|sort -g |\
 +
      awk 'BEGIN{start=0;ORS=","}{if($2-start<'$MAXCOMMBREAKSECS')
 +
      {finish=$2} else {print int(start*25+1)"-"int(finish*25-25);
 +
      start=$1; finish=$2;}}END{print int(start*25+1)"-"9999999}'`
 +
echo >>$LOGFILE "silence-detect has generated cutlist: $CUTLIST"
 +
 +
echo >>$LOGFILE "silence-detect(): CHANID=$CHANID, STARTTIME=$STARTTIME"
 +
 +
rm -rf $TMPDIR
 +
}
 +
 +
 +
if [ $# -eq 0 ]; then
 +
# run with no parameters, flag every unflagged recording
 +
exec mythcommflag
 +
exit $?
 +
else
 +
if [ $# -eq 4 -a "$1" = "-j" -a "$3" = "-V" ]; then
 +
# this is a manual flag job
 +
# TODO AJB20110129 generate a filelist from recorded table where commflagged=0 and cutlist=0
 +
# then run silence_detect over each filename
 +
JOB=$2
 +
else
 +
# we're being used in some other way, run the real mythcommmflag
 +
echo >>$LOGFILE "running mythcommflag $*"
 +
exec mythcommflag $*
 +
exit $?
 +
fi
 +
echo >>$LOGFILE "running job $JOB"
 +
HASCUTLIST=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select recorded.cutlist from recorded join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=recorded.chanid and jobqueue.starttime=recorded.starttime;" $MYTHDB | tail -n +2`
 +
if [ "$HASCUTLIST" = "1" ]; then
 +
echo >>$LOGFILE "program already has (manual?) cutlist, exiting"
 +
exit 0
 +
fi
 +
CALLSIGN=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select channel.callsign from channel join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=channel.chanid;" $MYTHDB | tail -n +2`
 +
CHANID=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select chanid from jobqueue where jobqueue.id=$JOB;" $MYTHDB | tail -n +2`
 +
STARTTIME=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select starttime from jobqueue where jobqueue.id=$JOB;" $MYTHDB | tail -n +2`
 +
echo >>$LOGFILE "channel callsign is $CALLSIGN"
 +
echo >>$LOGFILE "chanid=$CHANID STARTTIME=$STARTTIME"
 +
BASENAME=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select recorded.basename from recorded join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=recorded.chanid and jobqueue.starttime=recorded.starttime;" $MYTHDB | tail -n +2`
 +
echo >>$LOGFILE "basename is $BASENAME"
 +
FILENAME=`ionice -c3 nice find ${RECORDINGSROOT} -name $BASENAME`
 +
echo >>$LOGFILE "filename is $FILENAME"
 +
 +
 +
# make lower case and remove white space to account for minor
 +
# presentation changes between transmissions and over time callsigns
 +
# on Freesat and freeview differ e.g. "FIVE *" and "Five*". Also
 +
# removed is the +1 suffix from the criteria, if the primary channel
 +
# is allowable the +1 variant should also be included in white list.
 +
lowerCALLSIGN="$(echo $CALLSIGN | tr '[A-Z]' '[a-z]') | tr -d '+1' | tr -d '[:space:]'"
 +
 +
case $lowerCALLSIGN in # the following list should be in lower case and without any white space and no need to specify "+1" variants
 +
# E4/channel 4 treats trailers as program. The silence markers are
 +
# only either side of actual adverts. E4 often has trailers before and
 +
# after adverts, which will not be cut, and appear within the wanted show.
 +
# Works for other FIVE channels with caveat that they include news
 +
# bulletins which are't cut
 +
"five"|"5"|"fiveusa"|"5usa"|"channel4"|"channel4hd"|"more4"|"e4"|"film4"|"4seven"|"itv1"|"itv1hd"|"itv2"|"itv3"|"itv4"|"dave"|"davejavu")
 +
echo >>$LOGFILE "Callsign in whitelist - will run silence_detect"
 +
mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update recorded set commflagged=2 where chanid=$CHANID and starttime='${STARTTIME}';" $MYTHDB
 +
CUTLIST=""
 +
echo >>$LOGFILE "silence_detect $FILENAME"
 +
silence_detect $FILENAME
 +
echo >>$LOGFILE "silence_detect() set CUTLIST to $CUTLIST"
 +
let BREAKS=`grep -o "-" <<< "$CUTLIST" | wc -l`
 +
echo >>$LOGFILE "$BREAKS break(s) found."
 +
                mythutil --setskiplist $CUTLIST --chanid="$CHANID" --starttime="${STARTTIME}"
 +
RC=$?
 +
echo >>$LOGFILE "mythutil setskiplist returned $RC"
 +
if [ $RC -eq 0 ]; then
 +
mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update recorded set commflagged=1 where chanid=$CHANID and starttime='${STARTTIME}';" ${MYTHDB}
 +
mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update jobqueue set status=272, comment='Finished, $BREAKS break(s) found.' where id=$JOB;" ${MYTHDB}
 +
if [ $COPYTOCUTLIST -eq 1 ]; then
 +
mythutil --gencutlist --chanid="$CHANID" --starttime="${STARTTIME}"
 +
RC=$?
 +
if [ $RC -eq 0 ]; then
 +
echo >>$LOGFILE "mythutil --gencutlist successfully copied skip list to cut list"
 +
else
 +
echo >>$LOGFILE "mythutil --gencutlist failed, returned $RC"
 +
fi
 +
fi
 +
else
 +
echo >>$LOGFILE "mythcommflag failed; returned $RC"
 +
mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update recorded set commflagged=0 where chanid=$CHANID and starttime='${STARTTIME}';" ${MYTHDB}
 +
fi
 +
;;
 +
*)
 +
# Not a whitelisted channel for silence_detect
 +
echo >>$LOGFILE "won't run silence-detect, running mythcommflag $*"
 +
exec mythcommflag $*
 +
exit $?
 +
;;
 +
esac
 +
fi
 +
echo >>$LOGFILE
 +
</pre>
 +
}}
 +
 +
Info from trac Ticket [http://code.mythtv.org/trac/ticket/11936 http://code.mythtv.org/trac/ticket/11936]:-
 +
<pre style="white-space: pre-wrap;
 +
white-space: -moz-pre-wrap;
 +
white-space: -pre-wrap;
 +
white-space: -o-pre-wrap;
 +
word-wrap: break-word;">
 +
 +
In the later version of mythcommflag-wrapper.sh (ver 0.25+) there is an  error in the lowerCALLSIGN. The ')' is too early and should be before the  final '"'
 +
</pre>
 +
 +
I'm no coder. I couldn't possibly comment.
 +
[[User:Stevegoodey|Stevegoodey]] 11:40, 12 November 2013 (UTC)
 
=== Edit script variables ===
 
=== Edit script variables ===
  
Line 133: Line 373:
 
=== Enable commflagging ===
 
=== Enable commflagging ===
  
Check that commflagging is allowed in mythtv-setup (i.e. the <code>JobAllowCommFlag</code> setting should be 1) and that the commflag command (i.e. <code>JobQueueCommFlagCommand</code>) is <code>mythcommflag</code>.
+
Check that commflagging is allowed in mythtv-setup (i.e. the <code>JobAllowCommFlag</code> setting should be 1).
  
 
=== Test ===
 
=== Test ===
  
 
In the 'Watch Recordings' browser, select a recording that was from a channel included in the script's whitelist (Film4, Dave, Dave ja vu, ITV1-4, ITV1 +1, ITV2 +1), hit menu, then Jobs, then 'Begin Advert Detection'). Within a few minutes, that recording should be commflagged appropriately.
 
In the 'Watch Recordings' browser, select a recording that was from a channel included in the script's whitelist (Film4, Dave, Dave ja vu, ITV1-4, ITV1 +1, ITV2 +1), hit menu, then Jobs, then 'Begin Advert Detection'). Within a few minutes, that recording should be commflagged appropriately.
[[Category:Scripts]]
 
  
=== Improvements ===
+
=== Future Planned Improvements ===
 +
 
 +
<strike>Fill in the TODOs.</strike> Done, apart from automatic "flag all unflagged recordings" mode. Maybe when it can be trusted to not go haywire...
 +
 
 +
<strike>Expand the channel whitelist ("Five" tested once and failed, others???)</strike>
  
Fill in the TODOs.
+
More sanity checks of <code>mysql</code> client exit codes and parsed variables.
 +
 
 +
Refine <code>$CUTLIST</code> generation so that we print <code>$start</code> instead of <code>($finish*25-25)</code> if <code>($finish*25-25) < ($finish*25+1)</code>.
 +
 
 +
Per-channel profiles instead of whitelisting.
 +
 
 +
Experiment with other mp3splt options for other channels. Contributions here particularly welcome!
  
 
Merge this into the real mythcommflag. Will probably need a new column in the <code>channels</code> table to signify which commflagging method should be used, and that in turn will require some work to the channel-editor UI of <code>mythtv-setup</code>.
 
Merge this into the real mythcommflag. Will probably need a new column in the <code>channels</code> table to signify which commflagging method should be used, and that in turn will require some work to the channel-editor UI of <code>mythtv-setup</code>.
 +
 +
[[Category:User Job Scripts]]

Revision as of 11:40, 12 November 2013


Author Cowbutt
Description A simple bash script based on Silence-detect.sh that wraps mythcommflag and can be used manually (via 'Begin Advert Detection' menu option in recording browser) to use silence detection for commflagging for certain UK TV channels.
Supports Version24.png  Version26.png 


Prerequisites

This script needs mytharchivehelper which may be included in the MythArchive package ( No longer needed for 0.26 ).Ensure that it is installed on your machine. It also needs [ffmpeg] and [mp3splt]. It's likely that these are packaged for your distro, either directly, or by a third-party repository (e.g. ATrpms for Fedora).

Install wrapper script in place of mythcommflag

Install this script to somewhere accessible by the backend. In mythtv-setup, the command used to run for commercial detection can be modified. The mythcommflag option is a special phrase, telling the jobqueue to run

 ${PREFIX}/bin/mythcommflag -j %JOBID% -V %VERBOSELEVEL%

If you define any string other than mythcommflag, the command options must be explicitly defined, such as

 /home/mythtv/bin/mythcommflag-wrapper.sh -j %JOBID% -V 0


Script.png mythcommflag-wrapper.sh

#!/bin/bash


# edit/tune these #
# Allow ad breaks to be upto 400s long by coalescing non-silence
MAXCOMMBREAKSECS=400
# mytharchivehelper isn't frame-accurate, and mythcommflag won't add final cut pair if the end of the cut is greater
# than the actual final frame, so subtract $ENDFUDGEFRAMES from the value mytharchivehelper returns
ENDFUDGEFRAMES=75
# -70dB and minimum of 0.15s to be considered 'silent'
MP3SPLT_OPTS="th=-70,min=0.15"
# Log file
LOGFILE="/var/log/mythtv/mythcommflag-wrapper"
# Copy commercial skiplist to cutlist automatically? 1=Enabled
COPYTOCUTLIST=1

MYTHCFG=""
# mythbackend in Fedora packages has $HOME=/etc/mythtv
if [ -e $HOME/mysql.txt ]; then
	MYTHCFG="$HOME/mysql.txt"
fi


if [ -e $HOME/.mythtv/mysql.txt ]; then
	MYTHCFG="$HOME/.mythtv/mysql.txt"
fi
if [ "$MYTHCFG" = "" ]; then
	echo >>$LOGFILE "No mysql.txt found in $HOME or $HOME/.mythtv - exiting!"
	exit 1
fi

MAH=`which mytharchivehelper` 
if [ "$MAH" = "" ]; then
        echo "mytharchivehelper not found in $PATH - exiting!"
        exit 1
fi

# DB username and password
MYTHHOST=`grep DBHostName <$MYTHCFG | awk -F= '{print $2}'`
MYTHUSER=`grep DBUserName <$MYTHCFG | awk -F= '{print $2}'`
MYTHPASS=`grep DBPassword <$MYTHCFG | awk -F= '{print $2}'`
MYTHDB=`grep DBName <$MYTHCFG | awk -F= '{print $2}'`
#echo >>$LOGFILE "[$MYTHHOST] [$MYTHUSER] [$MYTHPASS] [$MYTHDB]"

# root of MythTV recordings
RECORDINGSROOT=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select dirname from storagegroup;" $MYTHDB | tail -n +2 | sort | uniq | tr '\n' ' '`


echo >>$LOGFILE "$0 run with [$*] at `date` by `whoami`"
echo >>$LOGFILE "RECORDINGSROOT=$RECORDINGSROOT"
#export >>$LOGFILE

#/usr/bin/mythcommflag run with [-j 584 -V 8213] at Sat Jan 29 17:26:58 GMT 2011 by root
#select recorded.basename from recorded join jobqueue where jobqueue.id=584 and jobqueue.chanid=recorded.chanid and jobqueue.starttime=recorded.starttime;

silence_detect() {
		local filename=$1
		
		TMPDIR=`mktemp -d /tmp/mythcommflag.XXXXXX` || exit 1		
		
		# get frame count, less 3 seconds (75 frames) fudge factor
		let FRAMES=`mytharchivehelper -i $filename $TMPDIR/streaminfo.xml 1 2>&1 | grep " frames = " | awk -F"= " '{print $2}'`-${ENDFUDGEFRAMES}
		echo >>$LOGFILE "total frames = $FRAMES (+/- 75)"		

		# From <http://www.mythtv.org/wiki/Silence-detect.sh>

# AJB20110129 Have already checked for pre-existing cutlist, and this check doesn't work properly after --clearcutlist anyway
#		[[ "Cutlist: "  == `mythcommflag --getcutlist -f $filename  |grep Cutlist` ]] \
#		    || { echo already has cutlist && exit 1; }

		cd $TMPDIR
		touch `basename $filename`.touch
		ionice -c3 nice ffmpeg -i $filename -acodec copy sound.mp3
		ionice -c3 nice mp3splt -s -p $MP3SPLT_OPTS sound.mp3

		#AJB20110129 add '$FRAMES' because mythcommflag now requires *pairs* of frames
		CUTLIST=`tail --lines=+3 mp3splt.log|sort -g |\
		       awk 'BEGIN{start=0;ORS=","}{if($2-start<'$MAXCOMMBREAKSECS')
		       {finish=$2} else {print int(start*25+1)"-"int(finish*25-25);
		       start=$1; finish=$2;}}END{print int(start*25+1)"-"'$FRAMES'}'`
		echo >>$LOGFILE "silence-detect has generated cutlist: $CUTLIST"

		echo >>$LOGFILE "silence-detect(): CHANID=$CHANID, STARTTIME=$STARTTIME, FRAMES=$FRAMES"		
		CUTLIST=`tail --lines=+3 mp3splt.log|sort -g |\
		awk 'BEGIN{start=0;ORS="\n"}{if($2-start<'$MAXCOMMBREAKSECS') {finish=$2} else {print "INSERT INTO recordedmarkup (chanid,starttime,mark,type) VALUES ('$CHANID',\"'"$STARTTIME"'\","int(start*25+1)",4);\nINSERT INTO recordedmarkup (chanid,starttime,mark,type) VALUES ('$CHANID',\"'"$STARTTIME"'\","int(finish*25-25)",5);"; start=$1; finish=$2;}}END{print "INSERT INTO recordedmarkup (chanid,starttime,mark,type) VALUES ('$CHANID',\"'"$STARTTIME"'\","int(start*25+1)",4);\nINSERT INTO recordedmarkup (chanid,starttime,mark,type) VALUES ('$CHANID',\"'"$STARTTIME"'\",'$FRAMES',5);"}' | sed 's/\"/'\''/g'`
		RC=$?
#		echo >>$LOGFILE "silence-detect SQLgen returned with $RC, generated SQL skipist: $CUTLIST"		

#		CUTLIST="USE $MYTHDB; $CUTLIST"

		rm -rf $TMPDIR
}		


if [ $# -eq 0 ]; then
	# run with no parameters, flag every unflagged recording
	exec mythcommflag
	exit $?
else
	if [ $# -eq 4 -a "$1" = "-j" -a "$3" = "-V" ]; then
		# this is a manual flag job
		# TODO AJB20110129 generate a filelist from recorded table where commflagged=0 and cutlist=0
		# then run silence_detect over each filename
		JOB=$2
	else
		# we're being used in some other way, run the real mythcommmflag
		echo >>$LOGFILE "running mythcommflag $*"
		exec mythcommflag $*
		exit $?
	fi
	echo >>$LOGFILE "running job $JOB"
	HASCUTLIST=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select recorded.cutlist from recorded join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=recorded.chanid and jobqueue.starttime=recorded.starttime;" $MYTHDB | tail -n +2`	
	if [ "$HASCUTLIST" = "1" ]; then
		echo >>$LOGFILE "program already has (manual?) cutlist, exiting"
		exit 0
	fi
	CALLSIGN=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select channel.callsign from channel join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=channel.chanid;" $MYTHDB | tail -n +2`
	CHANID=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select chanid from jobqueue where jobqueue.id=$JOB;" $MYTHDB | tail -n +2`	
	STARTTIME=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select starttime from jobqueue where jobqueue.id=$JOB;" $MYTHDB | tail -n +2`
	echo >>$LOGFILE "channel callsign is $CALLSIGN"
	echo >>$LOGFILE "chanid=$CHANID STARTTIME=$STARTTIME"
	BASENAME=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select recorded.basename from recorded join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=recorded.chanid and jobqueue.starttime=recorded.starttime;" $MYTHDB | tail -n +2`	
	echo >>$LOGFILE "basename is $BASENAME"
	FILENAME=`ionice -c3 nice find ${RECORDINGSROOT} -name $BASENAME`
	echo >>$LOGFILE "filename is $FILENAME"	

#safe list
#	if [ "$CALLSIGN" = "Film4" -o "$CALLSIGN" = "ITV1" -o "$CALLSIGN" = "ITV1 +1" -o "$CALLSIGN" = "ITV2" -o "$CALLSIGN" = "ITV2 +1" -o "$CALLSIGN" = "ITV3" -o "$CALLSIGN" = "ITV4" -o "$CALLSIGN" = "Dave" -o "$CALLSIGN" = "Dave ja vu" ]; then
# Channel 4 channels seem to work too, but sometimes last cut is too long?
#	if [ "$CALLSIGN" = "Channel 4" -o "$CALLSIGN" = "Channel 4+1" -o "$CALLSIGN" = "More 4" -o "$CALLSIGN" = "E4" -o "$CALLSIGN" = "E4+1" -o "$CALLSIGN" = "Film4" -o "$CALLSIGN" = "ITV1" -o "$CALLSIGN" = "ITV1 +1" -o "$CALLSIGN" = "ITV2" -o "$CALLSIGN" = "ITV2 +1" -o "$CALLSIGN" = "ITV3" -o "$CALLSIGN" = "ITV4" -o "$CALLSIGN" = "Dave" -o "$CALLSIGN" = "Dave ja vu" ]; then
#	if [ "$CALLSIGN" = "FIVE USA" -o "$CALLSIGN" = "FIVER" -o "$CALLSIGN" = "FIVE" -o "$CALLSIGN" = "Channel 4" -o "$CALLSIGN" = "Channel 4+1" -o "$CALLSIGN" = "More 4" -o "$CALLSIGN" = "E4" -o "$CALLSIGN" = "E4+1" -o "$CALLSIGN" = "Film4" -o "$CALLSIGN" = "ITV1" -o "$CALLSIGN" = "ITV1 +1" -o "$CALLSIGN" = "ITV2" -o "$CALLSIGN" = "ITV2 +1" -o "$CALLSIGN" = "ITV3" -o "$CALLSIGN" = "ITV4" -o "$CALLSIGN" = "Dave" -o "$CALLSIGN" = "Dave ja vu" ]; then	
# Fails on FIVER and cuts almost entire recording. Works for other FIVE channels with caveat that they include news bulletins which are't cut
	if [ "$CALLSIGN" = "FIVE USA" -o "$CALLSIGN" = "FIVE" -o "$CALLSIGN" = "Channel 4" -o "$CALLSIGN" = "Channel 4+1" -o "$CALLSIGN" = "More 4" -o "$CALLSIGN" = "E4" -o "$CALLSIGN" = "E4+1" -o "$CALLSIGN" = "Film4" -o "$CALLSIGN" = "ITV1" -o "$CALLSIGN" = "ITV1 +1" -o "$CALLSIGN" = "ITV2" -o "$CALLSIGN" = "ITV2 +1" -o "$CALLSIGN" = "ITV3" -o "$CALLSIGN" = "ITV4" -o "$CALLSIGN" = "Dave" -o "$CALLSIGN" = "Dave ja vu" ]; then	
		echo >>$LOGFILE "Callsign in whitelist - will run silence_detect"		
		mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update recorded set commflagged=2 where chanid=$CHANID and starttime='${STARTTIME}';" $MYTHDB
		CUTLIST=""
		echo >>$LOGFILE "silence_detect $FILENAME"
		silence_detect $FILENAME
		echo >>$LOGFILE "silect_detect() set CUTLIST to $CUTLIST"
		let BREAKS=`echo "$CUTLIST"|wc -l`/2
		echo >>$LOGFILE "$BREAKS break(s) found."
#		mythcommflag --setcutlist $CUTLIST -f $FILENAME
		mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "$CUTLIST" $MYTHDB
		RC=$?
#		echo "mythcommflag --setcutlist returned $RC"
		echo >>$LOGFILE "mysql setskiplist returned $RC"
		if [ $RC -eq 0 ]; then
			mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update recorded set commflagged=1 where chanid=$CHANID and starttime='${STARTTIME}';" ${MYTHDB}
			mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update jobqueue set status=272, comment='Finished, $BREAKS break(s) found.' where id=$JOB;" ${MYTHDB}			
			if [ $COPYTOCUTLIST -eq 1 ]; then
				mythcommflag --gencutlist -f $FILENAME
				RC=$?
				if [ $RC -eq 0 ]; then
					echo >>$LOGFILE "mythcommflag --gencutlist successfully copied skip list to cut list"
				else
					echo >>$LOGFILE "mythcommflag --gencutlist failed, returned $RC"		
				fi
			fi			
		else
			echo >>$LOGFILE "mythcommflag failed; returned $RC"
			mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update recorded set commflagged=0 where chanid=$CHANID and starttime='${STARTTIME}';" ${MYTHDB}
		fi		
	else
		# Not a whitelisted channel for silence_detect
		echo >>$LOGFILE "won't run silence-detect, running mythcommflag $*"
		exec mythcommflag $*
		exit $?
	fi		
fi
echo >>$LOGFILE



Version using mythutil for 0.25+ and it no longer needs mythexport


Script.png mythcommflag-wrapper.sh

#!/bin/bash

# Version using mythutil for 0.25+ as it no longer needs mythexport: Tuned detect slightly, updated whitelist to include Freesat/Freeview +1 channels. Supports UK Freeview/Freesat HD Channels as they have MPEG audio channels. ( Channels with AC3 Only are not supported. ) Changed ffmpeg to avconv, as ffmpeg is deprecated. and ffmpeg -i occasionally chose the Audio Descriptive track, which is nearly all silence.
 
# edit/tune these #
# Allow ad breaks to be upto 400s long by coalescing non-silence
MAXCOMMBREAKSECS=400
# -80dB and minimum of 0.12s to be considered 'silent'
#MP3SPLT_OPTS="th=-70,min=0.15"
MP3SPLT_OPTS="th=-80,min=0.12"
# Log file
LOGFILE="/var/log/mythtv/mythcommflag-wrapper"
# Copy commercial skiplist to cutlist automatically? 1=Enabled
#COPYTOCUTLIST=1

MYTHCFG=""
# mythbackend in Fedora packages has $HOME=/etc/mythtv
if [ -e $HOME/mysql.txt ]; then
	MYTHCFG="$HOME/mysql.txt"
fi


if [ -e $HOME/.mythtv/mysql.txt ]; then
	MYTHCFG="$HOME/.mythtv/mysql.txt"
fi
if [ "$MYTHCFG" = "" ]; then
	echo >>$LOGFILE "No mysql.txt found in $HOME or $HOME/.mythtv - exiting!"
	exit 1
fi

# DB username and password
MYTHHOST=`grep DBHostName <$MYTHCFG | awk -F= '{print $2}'`
MYTHUSER=`grep DBUserName <$MYTHCFG | awk -F= '{print $2}'`
MYTHPASS=`grep DBPassword <$MYTHCFG | awk -F= '{print $2}'`
MYTHDB=`grep DBName <$MYTHCFG | awk -F= '{print $2}'`
#echo >>$LOGFILE "[$MYTHHOST] [$MYTHUSER] [$MYTHPASS] [$MYTHDB]"

# root of MythTV recordings
RECORDINGSROOT=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select dirname from storagegroup;" $MYTHDB | tail -n +2 | sort | uniq | tr '\n' ' '`


echo >>$LOGFILE "$0 run with [$*] at `date` by `whoami`"
echo >>$LOGFILE "RECORDINGSROOT=$RECORDINGSROOT"

# From <http://www.mythtv.org/wiki/Silence-detect.sh>

silence_detect() {
		local filename=$1
		
		TMPDIR=`mktemp -d /tmp/mythcommflag.XXXXXX` || exit 1		

		cd $TMPDIR
		touch `basename $filename`.touch
		ionice -c3 nice avconv -i $filename -acodec copy sound.mp3
		ionice -c3 nice mp3splt -s -p $MP3SPLT_OPTS sound.mp3

		CUTLIST=`tail --lines=+3 mp3splt.log|sort -g |\
		       awk 'BEGIN{start=0;ORS=","}{if($2-start<'$MAXCOMMBREAKSECS')
		       {finish=$2} else {print int(start*25+1)"-"int(finish*25-25);
		       start=$1; finish=$2;}}END{print int(start*25+1)"-"9999999}'`
		echo >>$LOGFILE "silence-detect has generated cutlist: $CUTLIST"

		echo >>$LOGFILE "silence-detect(): CHANID=$CHANID, STARTTIME=$STARTTIME"		
		
		rm -rf $TMPDIR
}		


if [ $# -eq 0 ]; then
	# run with no parameters, flag every unflagged recording
	exec mythcommflag
	exit $?
else
	if [ $# -eq 4 -a "$1" = "-j" -a "$3" = "-V" ]; then
		# this is a manual flag job
		# TODO AJB20110129 generate a filelist from recorded table where commflagged=0 and cutlist=0
		# then run silence_detect over each filename
		JOB=$2
	else
		# we're being used in some other way, run the real mythcommmflag
		echo >>$LOGFILE "running mythcommflag $*"
		exec mythcommflag $*
		exit $?
	fi
	echo >>$LOGFILE "running job $JOB"
	HASCUTLIST=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select recorded.cutlist from recorded join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=recorded.chanid and jobqueue.starttime=recorded.starttime;" $MYTHDB | tail -n +2`	
	if [ "$HASCUTLIST" = "1" ]; then
		echo >>$LOGFILE "program already has (manual?) cutlist, exiting"
		exit 0
	fi
	CALLSIGN=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select channel.callsign from channel join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=channel.chanid;" $MYTHDB | tail -n +2`
	CHANID=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select chanid from jobqueue where jobqueue.id=$JOB;" $MYTHDB | tail -n +2`	
	STARTTIME=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select starttime from jobqueue where jobqueue.id=$JOB;" $MYTHDB | tail -n +2`
	echo >>$LOGFILE "channel callsign is $CALLSIGN"
	echo >>$LOGFILE "chanid=$CHANID STARTTIME=$STARTTIME"
	BASENAME=`mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "select recorded.basename from recorded join jobqueue where jobqueue.id=$JOB and jobqueue.chanid=recorded.chanid and jobqueue.starttime=recorded.starttime;" $MYTHDB | tail -n +2`	
	echo >>$LOGFILE "basename is $BASENAME"
	FILENAME=`ionice -c3 nice find ${RECORDINGSROOT} -name $BASENAME`
	echo >>$LOGFILE "filename is $FILENAME"	


	# make lower case and remove white space to account for minor 
	# presentation changes between transmissions and over time callsigns
	# on Freesat and freeview differ e.g. "FIVE *" and "Five*". Also 
	# removed is the +1 suffix from the criteria, if the primary channel
	# is allowable the +1 variant should also be included in white list.
	lowerCALLSIGN="$(echo $CALLSIGN | tr '[A-Z]' '[a-z]') | tr -d '+1' | tr -d '[:space:]'"

	case $lowerCALLSIGN in # the following list should be in lower case and without any white space and no need to specify "+1" variants
	# E4/channel 4 treats trailers as program. The silence markers are 
	# only either side of actual adverts. E4 often has trailers before and
	# after adverts, which will not be cut, and appear within the wanted show. 
	# Works for other FIVE channels with caveat that they include news 
	# bulletins which are't cut
	"five"|"5"|"fiveusa"|"5usa"|"channel4"|"channel4hd"|"more4"|"e4"|"film4"|"4seven"|"itv1"|"itv1hd"|"itv2"|"itv3"|"itv4"|"dave"|"davejavu")
		echo >>$LOGFILE "Callsign in whitelist - will run silence_detect"		
		mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update recorded set commflagged=2 where chanid=$CHANID and starttime='${STARTTIME}';" $MYTHDB
		CUTLIST=""
		echo >>$LOGFILE "silence_detect $FILENAME"
		silence_detect $FILENAME
		echo >>$LOGFILE "silence_detect() set CUTLIST to $CUTLIST"
		let BREAKS=`grep -o "-" <<< "$CUTLIST" | wc -l`
		echo >>$LOGFILE "$BREAKS break(s) found."
                mythutil --setskiplist $CUTLIST --chanid="$CHANID" --starttime="${STARTTIME}"
		RC=$?
		echo >>$LOGFILE "mythutil setskiplist returned $RC"
		if [ $RC -eq 0 ]; then
			mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update recorded set commflagged=1 where chanid=$CHANID and starttime='${STARTTIME}';" ${MYTHDB}
			mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update jobqueue set status=272, comment='Finished, $BREAKS break(s) found.' where id=$JOB;" ${MYTHDB}			
			if [ $COPYTOCUTLIST -eq 1 ]; then
				mythutil --gencutlist --chanid="$CHANID" --starttime="${STARTTIME}"
				RC=$?
				if [ $RC -eq 0 ]; then
					echo >>$LOGFILE "mythutil --gencutlist successfully copied skip list to cut list"
				else
					echo >>$LOGFILE "mythutil --gencutlist failed, returned $RC"		
				fi
			fi			
		else
			echo >>$LOGFILE "mythcommflag failed; returned $RC"
			mysql -h${MYTHHOST} -u${MYTHUSER} -p${MYTHPASS} -e "update recorded set commflagged=0 where chanid=$CHANID and starttime='${STARTTIME}';" ${MYTHDB}
		fi
		;;		
	*)
		# Not a whitelisted channel for silence_detect
		echo >>$LOGFILE "won't run silence-detect, running mythcommflag $*"
		exec mythcommflag $*
		exit $?
		;;
	esac		
fi
echo >>$LOGFILE

Info from trac Ticket http://code.mythtv.org/trac/ticket/11936:-


In the later version of mythcommflag-wrapper.sh (ver 0.25+) there is an  error in the lowerCALLSIGN. The ')' is too early and should be before the  final '"'

I'm no coder. I couldn't possibly comment. Stevegoodey 11:40, 12 November 2013 (UTC)

Edit script variables

Edit the environment variables in the wrapper script according to your local installation.

Enable commflagging

Check that commflagging is allowed in mythtv-setup (i.e. the JobAllowCommFlag setting should be 1).

Test

In the 'Watch Recordings' browser, select a recording that was from a channel included in the script's whitelist (Film4, Dave, Dave ja vu, ITV1-4, ITV1 +1, ITV2 +1), hit menu, then Jobs, then 'Begin Advert Detection'). Within a few minutes, that recording should be commflagged appropriately.

Future Planned Improvements

Fill in the TODOs. Done, apart from automatic "flag all unflagged recordings" mode. Maybe when it can be trusted to not go haywire...

Expand the channel whitelist ("Five" tested once and failed, others???)

More sanity checks of mysql client exit codes and parsed variables.

Refine $CUTLIST generation so that we print $start instead of ($finish*25-25) if ($finish*25-25) < ($finish*25+1).

Per-channel profiles instead of whitelisting.

Experiment with other mp3splt options for other channels. Contributions here particularly welcome!

Merge this into the real mythcommflag. Will probably need a new column in the channels table to signify which commflagging method should be used, and that in turn will require some work to the channel-editor UI of mythtv-setup.