Difference between revisions of "Mythcommflag"

From MythTV Official Wiki
Jump to: navigation, search
(Added AMD Athlon II X4 600e Processor to Script Results.)
(Note current fault with experimental D2 methods)
(15 intermediate revisions by 7 users not shown)
Line 1: Line 1:
'''Mythcommflag''' is the internal process used by the backend to flag commercials in MythTV recordings. There are various options that can be set on how commercials are detected. See the [[User Manual:Utilities and Setup#Commercial Skip|front end configuration manual]] for details.
+
'''Mythcommflag''' is the internal process used by the backend to [[commercial detection|flag commercials]] in MythTV recordings. There are various options that can be set on how commercials are detected. See the [[User Manual:Utilities and Setup#Commercial Skip|front end configuration manual]] for details.
  
 
__TOC__
 
__TOC__
Line 5: Line 5:
 
<pre>
 
<pre>
 
$ mythcommflag --help
 
$ mythcommflag --help
Valid Options are:
+
Misc. Options:
-c OR --chanid chanid       Flag recording with given channel ID
+
--chanid                   Specify chanid of recording to operate on.
-s OR --starttime starttime  Flag recording with given starttime
+
--dry-run                  Don't actually queue operation, just list what would
-f OR --file filename        Flag recording with specific filename
+
                            be done
--video filename            Rebuild the seektable for a video (non-recording) file
+
-O OR --override-setting    Override a single setting defined by a key=value
--sleep                      Give up some CPU time after processing each frame
+
                            pair.
--rebuild                    Do not flag commercials, just rebuild seektable
+
--override-settings-file    Define a file of key=value pairs to be loaded for
--gencutlist                Copy the commercial skip list to the cutlist
+
                            setting overrides.
--clearcutlist              Clear the cutlist
+
--queue                     Insert flagging job into the JobQueue, rather than
--setcutlist CUTLIST        Set a new cutlist.  CUTLIST is of the form:
+
                            running flagging in the foreground.
                            #-#[,#-#]...  (ie, 1-100,1520-3012,4091-5094
+
-h OR --help OR --usage    Display this help printout, or give detailed
--getcutlist                Display the current cutlist
+
                            information of selected option.
-v or --verbose debug-level  Use '-v help' for level info
+
--version                  Display version information.
--queue                     Insert flagging job into the JobQueue rather than
+
--starttime                Specify start time of recording to operate on.
                            running flagging in the foreground
+
--quiet                      Turn OFF display (also causes the program to
+
                            sleep a little every frame so it doesn't hog CPU)
+
                            takes precedence over --blanks if given first)
+
--blanks                    Show list of blank frames if already in database
+
                            (takes precedence over --quiet if given first)
+
--all                        Re-run commercial flagging for all recorded
+
                            programs using current detection method.
+
--allstart YYYYMMDDHHMMSS    when using --all, only flag programs starting
+
                            after the 'allstart' date (default = Jan 1, 1970).
+
--allend YYYYMMDDHHMMSS      when using --all, only flag programs ending
+
                            before the 'allend' date (default is now).
+
--force                      Force flagging of a video even if mythcommflag
+
                            thinks it is already in use by another instance.
+
--hogcpu                    Do not nice the flagging process.
+
                            WARNING: This will consume all free CPU time.
+
-h OR --help                This text
+
  
Note: both --chanid and --starttime must be used together
+
Commflagging Options:
      if either is used.
+
--method                    Commercial flagging method[s] to employ:
 +
                            off, blank, scene, blankscene, logo, all, d2,
 +
                            d2_logo, d2_blank, d2_scene, d2_all
 +
</pre>
 +
--method does not currently accept text values. See http://code.mythtv.org/trac/ticket/10793 and http://code.mythtv.org/trac/ticket/10989
  
If no command line arguments are specified, all
+
Instead use integers:
unflagged videos will be flagged.
+
0=off, 1=blank, 2=scene, 4=logo
 +
 
 +
Combine methods by adding: ie blankscene=3=1+2, all=7=1+2+4
 +
 
 +
Add 256 for the experimental d2 methods eg d2_logo=260=256+4. One developer commented that the d2 methods are slower, but no better.
 +
 
 +
Note that when --method accepts text values, d2_all does not include the d2_scene method - found to be unhelpful, especially in the US. So d2_all=261, and not 263
 +
 
 +
The experimental methods appear to have a current fault. An infinite loop, causing a failure to process the job queue and using 100% CPU. See http://code.mythtv.org/trac/ticket/11141
 +
 
 +
<pre>
 +
--outputmethod              Format of output written to outputfile, essentials,
 +
                            full.
 
</pre>
 
</pre>
 +
--outputmethod option is currently ignored and defaults to essentials. See http://code.mythtv.org/trac/ticket/10793 and http://code.mythtv.org/trac/ticket/10989
 +
<pre>
 +
--rebuild                  Do not flag commercials, just rebuild the seektable.
  
 +
Advanced Options:
 +
--force                    Force operation, even if program appears to be in
 +
                            use.
 +
--outputfile                File to write commercial flagging output [debug].
 +
 +
Input Options:
 +
-f OR --file                Specify file to operate on.
 +
--video                    Rebuild the seek table for a video (non-recording)
 +
                            file.
 +
 +
Logging Options:
 +
--loglevel                  Set the logging level.  All log messages at lower
 +
                            levels will be discarded.
 +
                            In descending order: emerg, alert, crit, err,
 +
                            warning, notice, info, debug
 +
                            defaults to info
 +
--logpath                  Writes logging messages to a file in the directory
 +
                            logpath with filenames in the format:
 +
                            applicationName.date.pid.log.
 +
                            This is typically used in combination with --daemon,
 +
                            and if used in combination with --pidfile, this can
 +
                            be used with log rotators, using the HUP call to
 +
                            inform MythTV to reload the file
 +
--nodblog                  Disable database logging.
 +
--noprogress                Don't print progress on stdout.
 +
-q OR --quiet              Don't log to the console (-q).  Don't log anywhere
 +
                            (-q -q)
 +
--syslog                    Set the syslog logging facility.
 +
                            Set to "none" to disable, defaults to none.
 +
-v OR --verbose            Specify log filtering. Use '-v help' for level info.
 +
 +
</pre>
  
 
== Choosing a Recording ==
 
== Choosing a Recording ==
 
Two methods are available to choose which recording to work on. The first is to specify the channelid and start time,
 
Two methods are available to choose which recording to work on. The first is to specify the channelid and start time,
  
  -c 1067 -s 20060424183000
+
  --chanid 1067 --starttime 20060424183000
 
Note the start time format is in YYYYMMDDHHMMSS.
 
Note the start time format is in YYYYMMDDHHMMSS.
  
 
The other method is to specify the file,
 
The other method is to specify the file,
 
  -f /MythTV/tv/1856_20060425200000.mpg
 
  -f /MythTV/tv/1856_20060425200000.mpg
 
  
 
== Rebuild Seektable ==
 
== Rebuild Seektable ==
Line 67: Line 100:
 
--video filename            Rebuild the seektable for a video (non-recording) file
 
--video filename            Rebuild the seektable for a video (non-recording) file
 
--rebuild                    Do not flag commercials, just rebuild seektable
 
--rebuild                    Do not flag commercials, just rebuild seektable
--gencutlist                Copy the commercial skip list to the cutlist
 
--clearcutlist              Clear the cutlist
 
--setcutlist CUTLIST        Set a new cutlist.
 
--getcutlist                Display the current cutlist
 
--blanks                    Show list of blank frames if already in database
 
 
</pre>
 
</pre>
 
 
== Unimplemented Options ==
 
 
These options appear to be unimplemented or have been removed (as of [http://svn.mythtv.org/svn/branches/release-0-20-fixes/mythtv/programs/mythcommflag/main.cpp .20-fixes]):
 
 
<pre>
 
--blanks                    Show list of blank frames if already in database
 
</pre>
 
 
 
== Undocumented Options ==
 
 
These options exist in the code (as of [http://svn.mythtv.org/svn/branches/release-0-20-fixes/mythtv/programs/mythcommflag/main.cpp .20-fixes]), but are not documented in the help message. They are probably for internal use by MythTV.
 
 
; -O/--override-setting SETTING=VAL[,SETTING=VAL...] : override a setting loaded from the DB, where SETTING can be a variable like CommercialSkipMethod, JobQueueCPU, etc.
 
 
; --method N : specify [[Commflagging|commercial detection]] method N, where N is one of the values of the SkipTypes enumerated constant declared in [http://svn.mythtv.org/svn/branches/release-0-20-fixes/mythtv/programs/mythcommflag/CommDetector.h CommDetector.h]
 
 
; -j N : run the job identified by N
 
 
; --dontwritetodb : don't write the commercial break list or the status flags to the database
 
 
; --onlydumpdb : dumps the existing commercial break list (perhaps the replacement for --blanks)
 
 
; --outputfile FILE : write commercial break list (either just generated or loaded from the DB via --onlydumpdb) to FILE
 
  
 
== Examples of Mythcommflag Jobs Speed ==
 
== Examples of Mythcommflag Jobs Speed ==
Line 183: Line 185:
 
| AMD Athlon(tm) 64 X2 Dual Core Processor 5200+ || 2700.000 MHz || 2.6.37.6-0.7-desktop || Medium || mpeg2video / mpg || 1920x1080 || 18083 kb/s || 76 fps
 
| AMD Athlon(tm) 64 X2 Dual Core Processor 5200+ || 2700.000 MHz || 2.6.37.6-0.7-desktop || Medium || mpeg2video / mpg || 1920x1080 || 18083 kb/s || 76 fps
 
|-
 
|-
[[Category:Utilities]]
+
| Intel(R) Core(TM)2 Duo CPU    T6670  @ 2.20GHz || 2201.000 MHz || 2.6.38-8-generic-pae || Medium || mpeg2video / mpg || 1280x720 || 15843 kb/s || 94 fps
 +
|-
 +
| Intel(R) Core(TM)2 Duo CPU    T6670  @ 2.20GHz || 2201.000 MHz || 2.6.38-8-generic-pae || Medium || mpeg2video / mpg || 1920x1080 || 16713 kb/s || 60 fps
 +
|-
 +
| Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz || 4801.000 MHz || 2.6.38-13-generic || Medium || h264, / mp4 || 1920x1088 || 8862 kb/s || 97 fps
 +
|-
 +
| Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz || 4801.000 MHz || 2.6.38-13-generic || Medium || h264, / mp4 || 1280x720 || 2822 kb/s || 366 fps
 +
|-
 +
| Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz || 4801.000 MHz || 2.6.38-13-generic || Medium || h264, / mp4 || 512x464 || 1159 kb/s || 1203 fps
 +
|-
 +
| Intel(R) Atom(TM) CPU  330  @ 1.60GHz || 1599.885 MHz || 2.6.38-8-generic || Medium || mpeg2video, / mpg || 1920x1080 || 11370 kb/s || 38 fps
 +
|-
 +
| Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz || 1600.000 MHz || 3.2.0-26-generic || Medium || mpeg2video / mpg || 1280x720 || 15843 kb/s || 1180 fps
 +
|-
 +
| Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz || 1600.000 MHz || 3.2.0-26-generic || Medium || mpeg2video / mpg || 1920x1080 || 15870 kb/s || 528 fps
 +
|-
 +
 
 +
 
 +
 
 +
[[Category:MythTV_Software]]

Revision as of 20:20, 4 October 2012

Mythcommflag is the internal process used by the backend to flag commercials in MythTV recordings. There are various options that can be set on how commercials are detected. See the front end configuration manual for details.

Usage

$ mythcommflag --help
Misc. Options:
--chanid                    Specify chanid of recording to operate on.
--dry-run                   Don't actually queue operation, just list what would
                            be done
-O OR --override-setting    Override a single setting defined by a key=value
                            pair.
--override-settings-file    Define a file of key=value pairs to be loaded for
                            setting overrides.
--queue                     Insert flagging job into the JobQueue, rather than
                            running flagging in the foreground.
-h OR --help OR --usage     Display this help printout, or give detailed
                            information of selected option.
--version                   Display version information.
--starttime                 Specify start time of recording to operate on.

Commflagging Options:
--method                    Commercial flagging method[s] to employ:
                            off, blank, scene, blankscene, logo, all, d2,
                            d2_logo, d2_blank, d2_scene, d2_all

--method does not currently accept text values. See http://code.mythtv.org/trac/ticket/10793 and http://code.mythtv.org/trac/ticket/10989

Instead use integers: 0=off, 1=blank, 2=scene, 4=logo

Combine methods by adding: ie blankscene=3=1+2, all=7=1+2+4

Add 256 for the experimental d2 methods eg d2_logo=260=256+4. One developer commented that the d2 methods are slower, but no better.

Note that when --method accepts text values, d2_all does not include the d2_scene method - found to be unhelpful, especially in the US. So d2_all=261, and not 263

The experimental methods appear to have a current fault. An infinite loop, causing a failure to process the job queue and using 100% CPU. See http://code.mythtv.org/trac/ticket/11141

--outputmethod              Format of output written to outputfile, essentials,
                            full.

--outputmethod option is currently ignored and defaults to essentials. See http://code.mythtv.org/trac/ticket/10793 and http://code.mythtv.org/trac/ticket/10989

--rebuild                   Do not flag commercials, just rebuild the seektable.

Advanced Options:
--force                     Force operation, even if program appears to be in
                            use.
--outputfile                File to write commercial flagging output [debug].

Input Options:
-f OR --file                Specify file to operate on.
--video                     Rebuild the seek table for a video (non-recording)
                            file.

Logging Options:
--loglevel                  Set the logging level.  All log messages at lower
                            levels will be discarded.
                            In descending order: emerg, alert, crit, err,
                            warning, notice, info, debug
                            defaults to info
--logpath                   Writes logging messages to a file in the directory
                            logpath with filenames in the format:
                            applicationName.date.pid.log.
                            This is typically used in combination with --daemon,
                            and if used in combination with --pidfile, this can
                            be used with log rotators, using the HUP call to
                            inform MythTV to reload the file
--nodblog                   Disable database logging.
--noprogress                Don't print progress on stdout.
-q OR --quiet               Don't log to the console (-q).  Don't log anywhere
                            (-q -q)
--syslog                    Set the syslog logging facility.
                            Set to "none" to disable, defaults to none.
-v OR --verbose             Specify log filtering. Use '-v help' for level info.

Choosing a Recording

Two methods are available to choose which recording to work on. The first is to specify the channelid and start time,

--chanid 1067 --starttime 20060424183000

Note the start time format is in YYYYMMDDHHMMSS.

The other method is to specify the file,

-f /MythTV/tv/1856_20060425200000.mpg

Rebuild Seektable

Many times the seektable may be bad, or you may do some editing of a recording outside of Myth. In order for Myth to properly FF, RW and seek it needs to have an up to date seektable. If this issue occurrs, you can use mythcommflag as one of the possible tools to repair the seektable.

Important.png Note: After rebuilding the seektable, any cutlist you have will no longer be properly synced and should be recreated.

Non Flagging Operations

The following options are mutually exclusive with commercial flagging, and if specified they will cause mythcommflag to perform an action other than commercial flagging.

--video filename             Rebuild the seektable for a video (non-recording) file
--rebuild                    Do not flag commercials, just rebuild seektable

Examples of Mythcommflag Jobs Speed

Script

Run this script to add entries to the examples.

#!/bin/bash
#
# Script to generate wiki formatted mythcommflag report.
#
# Usage:
# 1. Download and save this script to "mythcommflag.report.sh" 
# 2. Run mythcommflag -f mythtv_recording_of_your_choice.mpg
#  - After logo detection the fps for commercial flagging will be printed out.
#    Change FPS below to your FPS that is reported (eg FPS="50 fps") and save this script.
# 3. Run "sh mythcommflag.report.sh mythtv_recording_of_your_choice.mpg"
#  - Cut and paste output and add to this website (you must first create an account to edit this page).
#

if [ $# -ne 1 ]
then
 echo Usage: $0 mythtv_recording_of_your_choice.mpg
 exit 1
fi

CPUNAME=`cat /proc/cpuinfo | grep "model name" | sed -e 's/model name\t: //' | head -n 1 | sed -e 's/\t/ /'`
CPUMHZ=`cat /proc/cpuinfo | grep "cpu MHz" | sed -e 's/cpu MHz//' | sed 's/ *//g' | sed 's/\t*://' | head -n 1`
KERNEL=`uname -r`
CPUSETTING='Medium'
FPS='not specified'
# RESOLUTION=`ffmpeg -i "$1" 2>&1 | grep Video | head -n 1 | awk '{ print $6 }' | sed -e 's/,//'`
RESOLUTION=`ffmpeg -i "$1" 2>&1 | grep Video | head -n 1 | sed 's/[^,]*,[^,]*, *\([^ ]*\).*/\1/'`
# FORMAT=`ffmpeg -i "$1" 2>&1 | grep Video | head -n 1 | awk '{ print $4 }' | sed -e 's/,//'`
FORMAT=`ffmpeg -i "$1" 2>&1 | grep Video | head -n 1 | sed 's/^.*Video: *\([^ ]*\).*/\1/'`
BITRATE=`ffmpeg -i "$1" 2>&1 | grep bitrate: | awk '{ print $6 }'`
EXTENSION=`basename "$1" | awk -F . '{print $NF}' | tr 'A-Z' 'a-z'`

echo "|-" > mcfreport.txt
echo "| $CPUNAME || $CPUMHZ MHz || $KERNEL || $CPUSETTING || $FORMAT / $EXTENSION || $RESOLUTION || $BITRATE kb/s || $FPS" >> mcfreport.txt
echo "|-" >> mcfreport.txt

echo "Thank you!  Please paste the following into the wiki page at: http://www.mythtv.org/wiki/Mythcommflag#Examples_of_Mythcommflag_Jobs_Speed"
cat mcfreport.txt

Script Results

Processor CPU Speed Kernel version CPU Setting Video Codec / Container Resolution Bitrate FPS
AMD Athlon MP 1200 Mhz Low 57 fps
AMD Sempron 3100+ 1800 Mhz Low 60 fps
AMD Sempron 3100+ 1800 Mhz Medium 208 fps
AMD Dual-Core 4850e 2500 Mhz Medium 380 fps
AMD Athlon(tm) II X4 600e Processor 2200.000 MHz 3.1.0-1.2-desktop Medium mpeg2video / mpg 1920x1080 16172 kb/s 95 fps
AMD Athlon II X4 635 2900 Mhz Medium 720x480 460 fps
AMD Athlon II X4 635 2900 Mhz 2.6.32-24-generic Medium h264 / mpg 1280x720 3926 kb/s 125 fps
AMD Athlon II X4 635 2900 MHz 2.6.32-24-generic Medium h264 / mpg 1920x1080 9840 kb/s 37 fps
Intel Core2 Duo 6300 1860 MHz 2.6.32-25-generic Medium mpeg2video / mpg 720x480 (NTSC) 5188 kb/s 340 fps
Intel Core2 Duo 6300 1860 MHz 2.6.32-25-generic Medium mpeg2video / mpg 1920x1080 (ATSC) 17990 kb/s 72 fps
Intel Pentium 4 2594 MHz 2.6.35-22-generic-pae Medium mpeg2video / mpg 480x480 (NTSC) 5192 kb/s 310 fps
AMD Athlon(tm) 64 X2 Dual Core Processor 5000+ 2599.923 MHz 2.6.35.9-64.fc14.x86_64 Medium mpeg2video / mpg 720x480 5221 kb/s 498 fps
AMD Athlon(tm) 64 X2 Dual Core Processor 5200+ 2700.000 MHz 2.6.37.6-0.7-desktop Medium mpeg2video / mpg 1920x1080 18083 kb/s 76 fps
Intel(R) Core(TM)2 Duo CPU T6670 @ 2.20GHz 2201.000 MHz 2.6.38-8-generic-pae Medium mpeg2video / mpg 1280x720 15843 kb/s 94 fps
Intel(R) Core(TM)2 Duo CPU T6670 @ 2.20GHz 2201.000 MHz 2.6.38-8-generic-pae Medium mpeg2video / mpg 1920x1080 16713 kb/s 60 fps
Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz 4801.000 MHz 2.6.38-13-generic Medium h264, / mp4 1920x1088 8862 kb/s 97 fps
Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz 4801.000 MHz 2.6.38-13-generic Medium h264, / mp4 1280x720 2822 kb/s 366 fps
Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz 4801.000 MHz 2.6.38-13-generic Medium h264, / mp4 512x464 1159 kb/s 1203 fps
Intel(R) Atom(TM) CPU 330 @ 1.60GHz 1599.885 MHz 2.6.38-8-generic Medium mpeg2video, / mpg 1920x1080 11370 kb/s 38 fps
Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz 1600.000 MHz 3.2.0-26-generic Medium mpeg2video / mpg 1280x720 15843 kb/s 1180 fps
Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz 1600.000 MHz 3.2.0-26-generic Medium mpeg2video / mpg 1920x1080 15870 kb/s 528 fps