Difference between revisions of "Script - RemoveCommercials"

From MythTV Official Wiki
Jump to: navigation, search
(Fix a couple of problems and inconsistencies pointed out by Jack Perveiler)
 
(20 intermediate revisions by 12 users not shown)
Line 1: Line 1:
The follow script can be setup as a User Job to automatically flag and remove commercials from a recording.
+
{{Warning box|This behavior is not supported within the mythcommflag/mythtranscode programs themselves as the commercial detector is not considered sufficiently accurate to permanently remove sections of video without a manual confirmation.}}
 +
 
 +
The follow script can be setup as a User Job to automatically flag and remove commercials from a recording. This script code was updated November 3rd, 2014 to work with MythTV 0.27.
 +
 
 +
{{Code box|remove_commercials.sh|
 +
<pre>
 +
#!/bin/sh
 +
 
 +
##### Commercial Remover Script for MythTV, updated November 3rd, 2014
 +
###
 +
##      Version Changes to mythutil and mythcommflag tools under MythTV 0.25
 +
##        necessitate different commands within the script. (Under MythTV < 0.25 you could
 +
##        just point mythcommflag at the recordings in the file system and get the job done, now
 +
##        under MythTV 0.25 you have to use mythutil to generate the cutlist and mythcommflag to
 +
##        flag commercials, AND point both at recordings based on %CHANID% and %STARTTIME%.  Under
 +
##        MythTV 0.25 mythtranscode can also be pointed at %CHANID% and %STARTTIME% but since it
 +
##        currently supports the older infile /outfile method as well (and I prefer how this
 +
##        script creates old/new files, it is the method used.
 +
##
 +
##      Since this script requires the User Job to pass additional arguments under MythTV 0.25,
 +
##      use following User Job syntax for userjobs under all versions:
 +
##
 +
##        'removecommercials.sh %DIR% %FILE% %CHANID% %STARTTIMEUTC%'
 +
##
 +
##      Credits: Zwhite, Ricks03, Waxrat @ www.mythtv.org/wiki
 +
##      Fix for MythTV-027 Xxsj
 +
###
 +
#####
 +
VIDEODIR=$1 #MythTV-024 %DIR%
 +
FILENAME=$2 #MythTV-024 %FILE%
 +
CHANID=$3
 +
STARTTIME=$4
 +
# Sanity checking, to make sure everything is in order.
 +
if [ -z "$VIDEODIR" -o -z "$FILENAME" -o -z "$CHANID" -o -z "$STARTTIME" ]; then
 +
        echo "Usage: $0 <VideoDirectory> <FileName> <CHANID> <STARTTIME>"
 +
        exit 5
 +
fi
 +
if [ ! -f "$VIDEODIR/$FILENAME" ]; then
 +
        echo "File does not exist: $VIDEODIR/$FILENAME"
 +
        exit 6
 +
fi
 +
# The meat of the script. Flag commercials, copy the flagged commercials to
 +
# the cutlist, and transcode the video to remove the commercials from the
 +
# file.
 +
 
 +
 
 +
echo "Flagging Commercials..."
 +
##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version <= 0.24
 +
#mythcommflag -f $VIDEODIR/$FILENAME
 +
 
 +
##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version >= 0.25
 +
mythcommflag --chanid $CHANID --starttime $STARTTIME
 +
ERROR=$?
 +
if [ $ERROR -gt 126 ]; then
 +
        echo "Commercial flagging failed for ${FILENAME} with error $ERROR"
 +
        exit $ERROR
 +
fi
 +
 
 +
 
 +
echo "Generating cutlist..."
 +
##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version <= 0.24
 +
#mythcommflag --gencutlist -f $VIDEODIR/$FILENAME
 +
 
 +
##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version >= 0.25
 +
mythutil --gencutlist --chanid $CHANID --starttime $STARTTIME
 +
ERROR=$?
 +
if [ $ERROR -ne 0 ]; then
 +
        echo "Copying cutlist failed for ${FILENAME} with error $ERROR"
 +
        exit $ERROR
 +
fi
 +
 
 +
 
 +
echo "Transcoding..."
 +
mythtranscode --honorcutlist --showprogress --chanid $CHANID --starttime $STARTTIME -o $VIDEODIR/$FILENAME.tmp
 +
ERROR=$?
 +
if [ $ERROR -ne 0 ]; then
 +
        echo "Transcoding failed for ${FILENAME} with error $ERROR"
 +
        exit $ERROR
 +
fi
 +
mv $VIDEODIR/$FILENAME $VIDEODIR/$FILENAME.old
 +
mv $VIDEODIR/$FILENAME.tmp $VIDEODIR/$FILENAME
 +
 
 +
 
 +
echo "Rebuilding..."
 +
##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version <= 0.24
 +
#mythcommflag -f $VIDEODIR/${FILENAME} --rebuild
 +
 
 +
##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version >= 0.25
 +
mythcommflag --chanid $CHANID --starttime $STARTTIME --rebuild
 +
ERROR=$?
 +
if [ $ERROR -ne 0 ]; then
 +
        echo "Rebuilding seek list failed for ${FILENAME} with error $ERROR"
 +
        exit $ERROR
 +
fi
 +
 
 +
 
 +
echo "Clearing cutlist..."
 +
##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version <= 0.24
 +
#mythcommflag --clearcutlist -f $VIDEODIR/$FILENAME
 +
 
 +
##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version >= 0.25
 +
mythutil --clearcutlist --chanid $CHANID --starttime $STARTTIME
 +
ERROR=$?
 +
if [ $ERROR -eq 0 ]; then
 +
        # Fix the database entry for the file
 +
##### - Note: MAY need to add mysql mythconverg DB credentials '--user=<blah> --password=<blah>'
 +
##      to the 'mysql mythconverg' command below.)
 +
        # Fix the database entry for the file
 +
        cat << EOF | mysql mythconverg
 +
UPDATE
 +
        recorded
 +
SET
 +
        cutlist = 0,
 +
        filesize = $(ls -l $VIDEODIR/$FILENAME | awk '{print $5}')
 +
WHERE
 +
        basename = '$FILENAME';
 +
EOF
 +
        exit 0
 +
else
 +
        echo "Clearing cutlist failed for ${FILENAME} with error $ERROR"
 +
        rm -f $VIDEODIR/$FILENAME.tmp
 +
        exit $ERROR
 +
fi
 +
</pre>
 +
}}
 +
 
 +
See the discussion tab for tips on using the mpeg2 lossless transcoding.
 +
 
 +
If you use this script and have commercial auto-skip set then during playback you will notice that MythTV skips over parts of your program. This is because the information about where commercials are located is still available. Adding this query after the UPDATE and before the EOF will remove the commercial points and allow things to run smoothly.
 +
 
 +
DELETE FROM
 +
        `recordedmarkup`
 +
WHERE
 +
        CONCAT( chanid, starttime ) IN (
 +
                SELECT
 +
                        CONCAT( chanid, starttime )
 +
                FROM
 +
                        recorded
 +
                WHERE
 +
                        basename = '$FILENAME'
 +
      );
  
#!/bin/sh
 
# removecommercials.sh
 
#
 
# Copyright 2006 Zach White <zwhite-mythtv@darkstar.frop.org>
 
#
 
# Permission to use, copy, modify, and distribute this software and
 
# its documentation for any purpose and without fee is hereby granted,
 
# provided that the above copyright notice appear in all copies and
 
# that both that copyright notice and this permission notice appear in
 
# supporting documentation.
 
#
 
# THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS
 
# SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
 
# FITNESS, IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
 
# SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER
 
# RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF
 
# CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
 
# CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 
#
 
VIDEODIR=$1
 
FILENAME=$2
 
# Sanity checking, to make sure everything is in order.
 
if [ -z "$VIDEODIR" -o -z "$FILENAME" ]; then
 
        echo "Usage: $0 <VideoDirectory> <FileName>"
 
        exit 5
 
fi
 
if [ ! -f $VIDEODIR/$FILENAME ]; then
 
        echo "File does not exist: $VIDEODIR/$FILENAME"
 
        exit 6
 
fi
 
# The meat of the script. Flag commercials, copy the flagged commercials to
 
# the cutlist, and transcode the video to remove the commercials from the
 
# file.
 
mythcommflag -f $VIDEODIR/$FILENAME
 
ERROR=$?
 
if [ $ERROR -gt 126 ]; then
 
        echo "Commercial flagging failed for ${FILENAME} with error $ERROR"
 
        exit 1
 
fi
 
mythcommflag --gencutlist -f $VIDEODIR/$FILENAME
 
ERROR=$?
 
if [ $ERROR -ne 0 ]; then
 
        echo "Copying cutlist failed for ${FILENAME} with error $ERROR"
 
        exit 2
 
fi
 
mythtranscode --honorcutlist --showprogress -i $VIDEODIR/$FILENAME -o $VIDEODIR/$FILENAME.tmp
 
ERROR=$?
 
if [ $ERROR -ne 0 ]; then
 
        echo "Transcoding failed for ${FILENAME} with error $ERROR"
 
        exit 3
 
fi
 
mythcommflag -f $VIDEODIR/${FILENAME}.tmp --rebuild
 
ERROR=$?
 
if [ $ERROR -ne 0 ]; then
 
        echo "Rebuilding seek list failed for ${FILENAME} with error $ERROR"
 
        exit 4
 
fi
 
mythcommflag --clearcutlist -f $VIDEODIR/$FILENAME
 
ERROR=$?
 
if [ $ERROR -eq 0 ]; then
 
        mv $VIDEODIR/$FILENAME $VIDEODIR/$FILENAME.old
 
        mv $VIDEODIR/$FILENAME.tmp $VIDEODIR/$FILENAME
 
        # Fix the database entry for the file
 
        cat << EOF | mysql mythconverg
 
UPDATE
 
        recorded
 
SET
 
        cutlist = 0,
 
        filesize = $(ls -l $VIDEODIR/$FILENAME | awk '{print $5}')
 
WHERE
 
        basename = '$FILENAME';
 
EOF
 
        exit 0
 
else
 
        echo "Clearing cutlist failed for ${FILENAME} with error $ERROR"
 
        rm $VIDEODIR/$FILENAME.tmp
 
        exit 5
 
fi
 
  
 
Please note that this script will save the original file as ${FILENAME}.old. You should periodically remove these files to save space. If you want to remove them automatically, you could use the following command:
 
Please note that this script will save the original file as ${FILENAME}.old. You should periodically remove these files to save space. If you want to remove them automatically, you could use the following command:
Line 86: Line 148:
 
Place this command into a script in /etc/cron.daily or into crontab. This will remove all backup files that are older than 2 days.
 
Place this command into a script in /etc/cron.daily or into crontab. This will remove all backup files that are older than 2 days.
  
Note:
+
[[Category:User_Job_Scripts]]
 
+
[[Category:Transcode_Scripts]]
I update this script as I make changes to my own system. You should [[?title=Script_-_RemoveCommercials&action=watch|watch]] this article so you're notified of changes as they occur. If you do not have a wiki account and do not wish to create one you should check back periodically to get changes.
 

Latest revision as of 05:25, 4 November 2014

Warning.png Warning: This behavior is not supported within the mythcommflag/mythtranscode programs themselves as the commercial detector is not considered sufficiently accurate to permanently remove sections of video without a manual confirmation.

The follow script can be setup as a User Job to automatically flag and remove commercials from a recording. This script code was updated November 3rd, 2014 to work with MythTV 0.27.


Script.png remove_commercials.sh

#!/bin/sh

##### Commercial Remover Script for MythTV, updated November 3rd, 2014
###
##       Version Changes to mythutil and mythcommflag tools under MythTV 0.25
##        necessitate different commands within the script. (Under MythTV < 0.25 you could
##        just point mythcommflag at the recordings in the file system and get the job done, now
##        under MythTV 0.25 you have to use mythutil to generate the cutlist and mythcommflag to
##        flag commercials, AND point both at recordings based on %CHANID% and %STARTTIME%.  Under
##        MythTV 0.25 mythtranscode can also be pointed at %CHANID% and %STARTTIME% but since it
##        currently supports the older infile /outfile method as well (and I prefer how this
##        script creates old/new files, it is the method used.
##
##      Since this script requires the User Job to pass additional arguments under MythTV 0.25,
##       use following User Job syntax for userjobs under all versions:
##
##        'removecommercials.sh %DIR% %FILE% %CHANID% %STARTTIMEUTC%'
##
##      Credits: Zwhite, Ricks03, Waxrat @ www.mythtv.org/wiki
##      Fix for MythTV-027 Xxsj
###
#####
VIDEODIR=$1 #MythTV-024 %DIR%
FILENAME=$2 #MythTV-024 %FILE%
CHANID=$3
STARTTIME=$4
# Sanity checking, to make sure everything is in order.
if [ -z "$VIDEODIR" -o -z "$FILENAME" -o -z "$CHANID" -o -z "$STARTTIME" ]; then
        echo "Usage: $0 <VideoDirectory> <FileName> <CHANID> <STARTTIME>"
        exit 5
fi
if [ ! -f "$VIDEODIR/$FILENAME" ]; then
        echo "File does not exist: $VIDEODIR/$FILENAME"
        exit 6
fi
# The meat of the script. Flag commercials, copy the flagged commercials to
# the cutlist, and transcode the video to remove the commercials from the
# file.


echo "Flagging Commercials..."
##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version <= 0.24
#mythcommflag -f $VIDEODIR/$FILENAME

##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version >= 0.25
mythcommflag --chanid $CHANID --starttime $STARTTIME
ERROR=$?
if [ $ERROR -gt 126 ]; then
        echo "Commercial flagging failed for ${FILENAME} with error $ERROR"
        exit $ERROR
fi


echo "Generating cutlist..."
##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version <= 0.24
#mythcommflag --gencutlist -f $VIDEODIR/$FILENAME

##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version >= 0.25
mythutil --gencutlist --chanid $CHANID --starttime $STARTTIME
ERROR=$?
if [ $ERROR -ne 0 ]; then
        echo "Copying cutlist failed for ${FILENAME} with error $ERROR"
        exit $ERROR
fi


echo "Transcoding..."
mythtranscode --honorcutlist --showprogress --chanid $CHANID --starttime $STARTTIME -o $VIDEODIR/$FILENAME.tmp
ERROR=$?
if [ $ERROR -ne 0 ]; then
        echo "Transcoding failed for ${FILENAME} with error $ERROR"
        exit $ERROR
fi
mv $VIDEODIR/$FILENAME $VIDEODIR/$FILENAME.old
mv $VIDEODIR/$FILENAME.tmp $VIDEODIR/$FILENAME


echo "Rebuilding..."
##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version <= 0.24
#mythcommflag -f $VIDEODIR/${FILENAME} --rebuild

##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version >= 0.25
mythcommflag --chanid $CHANID --starttime $STARTTIME --rebuild
ERROR=$?
if [ $ERROR -ne 0 ]; then
        echo "Rebuilding seek list failed for ${FILENAME} with error $ERROR"
        exit $ERROR
fi


echo "Clearing cutlist..."
##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version <= 0.24
#mythcommflag --clearcutlist -f $VIDEODIR/$FILENAME

##### - UNCOMMENT FOLLOWING LINE IF RUNNING MythTV Version >= 0.25
mythutil --clearcutlist --chanid $CHANID --starttime $STARTTIME
ERROR=$?
if [ $ERROR -eq 0 ]; then
        # Fix the database entry for the file
##### - Note: MAY need to add mysql mythconverg DB credentials '--user=<blah> --password=<blah>'
##       to the 'mysql mythconverg' command below.)
        # Fix the database entry for the file
        cat << EOF | mysql mythconverg
UPDATE
        recorded
SET
        cutlist = 0,
        filesize = $(ls -l $VIDEODIR/$FILENAME | awk '{print $5}')
WHERE
        basename = '$FILENAME';
EOF
        exit 0
else
        echo "Clearing cutlist failed for ${FILENAME} with error $ERROR"
        rm -f $VIDEODIR/$FILENAME.tmp
        exit $ERROR
fi

See the discussion tab for tips on using the mpeg2 lossless transcoding.

If you use this script and have commercial auto-skip set then during playback you will notice that MythTV skips over parts of your program. This is because the information about where commercials are located is still available. Adding this query after the UPDATE and before the EOF will remove the commercial points and allow things to run smoothly.

DELETE FROM 
       `recordedmarkup`
WHERE 
       CONCAT( chanid, starttime ) IN (
               SELECT 
                       CONCAT( chanid, starttime )
               FROM 
                       recorded
               WHERE 
                       basename = '$FILENAME'
      );


Please note that this script will save the original file as ${FILENAME}.old. You should periodically remove these files to save space. If you want to remove them automatically, you could use the following command:

find /var/video -name \*.old -ctime 2 -exec rm {} \;

Place this command into a script in /etc/cron.daily or into crontab. This will remove all backup files that are older than 2 days.