Difference between revisions of "Removing Commercials"

From MythTV Official Wiki
Jump to: navigation, search
(Important Note:)
m (change link to renamed page)
 
(16 intermediate revisions by 11 users not shown)
Line 1: Line 1:
== Introduction ==
+
{{User_Manual_TOC}}
  
MythTV does not automatically remove commercials.  The system will flag commercials and skip them during playback.  It can also [[transcode]] recordings between [[MPEG-2]] and [[MPEG-4]] or [[lossless transcode]] MPEG-2 and remove a cutlist from the program.  Since a list of commercial flags is not a cutlist, it is not removed automatically.  This document describes how to remove commercials permanently from the source material.
+
MythTV does not automatically remove commercials.  The system will flag commercials and optionally skip them during playback.  It can also [[transcode]] recordings between [[MPEG-2]] and [[MPEG-4]] or [[lossless transcode]] MPEG-2 and remove a cut list from the program.  Since a list of commercial flags is not a cut list, it is not removed automatically.  This document describes how to remove commercials permanently from the source material
 
+
=== Important Note: ===
+
 
{{Note box
 
{{Note box
 
|msg=You do not need to do anything on this page if all you are doing is watching recorded shows using Myth TV, then deleting them or letting them auto-expire. You will not save very much space. What the instructions on this page are useful for is if you're storing or archiving your recordings, or are watching your recordings outside of MythTV, or want to watch "HDTV" programming on a slower machine in standard definition.
 
|msg=You do not need to do anything on this page if all you are doing is watching recorded shows using Myth TV, then deleting them or letting them auto-expire. You will not save very much space. What the instructions on this page are useful for is if you're storing or archiving your recordings, or are watching your recordings outside of MythTV, or want to watch "HDTV" programming on a slower machine in standard definition.
 
}}
 
}}
  
If you just want to skip commercials from the front end, see [[Commercial Flagging]].
+
 
 +
If you just want to skip commercials from the front end, see [[Commercial detection]].
  
 
== Removing commercials using mythfrontend ==
 
== Removing commercials using mythfrontend ==
Line 14: Line 13:
 
===Editing Recordings===
 
===Editing Recordings===
  
The Editor allows you to remove any commercials that the may not have been detected by the system during recording. You might also want to remove the lead in prior to a show or the overrun at the end of a show. The process of editing is often referred to as creating a Cut List. This is because MythTV does not actually cut the Recording. Instead it creates a collection of in and out points in the database that is associated with that recording.
+
The recording editor allows you to mark areas of the recording to skip over during playback. Once you edit a recording, you can transcode the recording to permanently remove the cut areas from the recording file.
 
+
To Edit a Recording press [[Image:remoteMenu.png]] to bring up the On Screen Menu. Select Edit Recording.
+
 
+
Hit Z to load the current commercial cutlist, if the commercial flagging job had already been run.
+
 
+
[[Image:EditMenu.jpg]]
+
  
The Timeline Editor will now appear. If it's barely visible, you might want to switch the OSD theme to retro, which has a colorful timeline. OSD themes are set separately from the MythTV theme in the Frontend "Playback" setup.
+
If you would like the transcoder to permanently remove any portions of the recording--including commercials or the lead in prior to a show or the overrun at the end of a show--you must first [[Editing_Recordings|edit the recording]].
 
+
[[Image:EditTimeline.jpg]]
+
 
+
You can now move along the time line with [[Image:remoteLeft.png]] [[Image:remoteRight.png]] by the amount indicated at the bottom right of the screen Default is 1 Sec
+
 
+
Use [[Image:remoteUp.png]] [[Image:remoteDown.png]] to change the amount to move. This varies between 0.5 Secs to 10Mins. You also have the option of 1 frame at a time, 1 keyframe at a time and finally a cutpoint at a time (jumps from cutpoint to cutpoint).
+
 
+
To insert a cutpoint press [[Image:remoteSelect.png]]. The Cutpoint menu will appear
+
 
+
[[Image:EditCutpoint.jpg]]
+
 
+
Use [[Image:remoteSelect.png]] to choose between Delete before this point or after this point. Just add the cutpoints that you need. The system will remember these the next time that you bring the recording up. Red areas along the Timeline will be deleted (Skipped). Green areas will be played.
+
 
+
If you now transcode this recording the system will take the Cutlist and produce an edited Recording ready to be exported.
+
  
 
===Transcoding Recordings ===
 
===Transcoding Recordings ===
Line 52: Line 31:
  
 
now select either [[RTjpeg]] or [[MPEG-4]] (selecting any of the hardware encoders will result in the transcoder not working). set the other parameters as you'd like. For best performance, you should match the audio to the 'Default' profile. If you plan to only use the manual transcoder, you can choose to set the video to be the same as the default profile too (as long as it is [[MPEG-4]] or [[RTjpeg]])
 
now select either [[RTjpeg]] or [[MPEG-4]] (selecting any of the hardware encoders will result in the transcoder not working). set the other parameters as you'd like. For best performance, you should match the audio to the 'Default' profile. If you plan to only use the manual transcoder, you can choose to set the video to be the same as the default profile too (as long as it is [[MPEG-4]] or [[RTjpeg]])
 +
 +
{{Note box|Even though MythTV 0.21 allows you to change the names of the recording profiles, don't do it.  You can only edit the existing profiles.  This is a feature that was partially implemented but hasn't been finished.}}
 +
 +
{{Note box|When transcoding fails with in /var/log/mythtv/mythbackendlog messages like "No video information found! Transcode Errored: Program: Autodetect (exit status 255, job status was Errored", you should try to change the Default Transcoding (in Utilities/Setup -> Setup -> TV Settings -> General) from AutoDetect to for example "High Quality".}}
 +
  
 
The transcoder can be used in two ways:
 
The transcoder can be used in two ways:
Line 80: Line 64:
 
You must have already flagged commercials for this option to work.
 
You must have already flagged commercials for this option to work.
  
Use mythcommflag to create a cutlist by using
+
Use mythcommflag to create a cut list by using
  
 
  mythcommflag --gencutlist
 
  mythcommflag --gencutlist
  
Then use mythtranscode to transcode the recording using the cutlist with
+
Then use mythtranscode to transcode the recording using the cut list with
  
 
  mythtranscode --honorcutlist
 
  mythtranscode --honorcutlist
Line 126: Line 110:
  
 
=== Transcode error - Transcode failed with status: 247 ===
 
=== Transcode error - Transcode failed with status: 247 ===
If it throws an error at the transcoding step check your transcoding prefs. I had to change mine from RJeg to MPEG in the transcoding option in the Recording Prefs.
+
If it throws an error at the transcoding step check your transcoding prefs. I had to change mine from RTjpeg to MPEG in the transcoding option in the Recording Prefs.
 
From the Myth Frontend:
 
From the Myth Frontend:
 
# Utilities/Setup
 
# Utilities/Setup
Line 134: Line 118:
 
# Transcoders
 
# Transcoders
 
# Edit all "Autodetect" profiles (two in my case), on the second screen of each profile is a "codec" option, change this to "MPEG-4" from "RTjpeg".
 
# Edit all "Autodetect" profiles (two in my case), on the second screen of each profile is a "codec" option, change this to "MPEG-4" from "RTjpeg".
 +
 +
=== MythTransCode runs OK, but doesn't cut commercials ===
 +
 +
What I noticed is that (even from the command line) mythtranscode fails to 'honor the cut list' when used with the "-i <filename>" syntax, but when that is switched to "-c <channel> -s <datetimestamp>" everything suddenly works fine. Don't know why, but it helped me to make things work. 
 +
 +
I edited the above script to include the channel and datetimestamp in the mythtranscode command.  To make the changes, replace lines 4-9 with
 +
 +
  CHANID=$3
 +
  STARTTIME=$4
 +
  # Sanity checking, to make sure everything is in order.
 +
  if [ -z "$VIDEODIR" -o -z "$FILENAME" -o -z "$CHANID" -o -z "$STARTTIME" ]; then
 +
      echo "Usage: $0 <VideoDirectory> <FileName> <CHANID> <STARTTIME>"
 +
 +
and line 30 with
 +
 +
  mythtranscode --honorcutlist --showprogress -c $CHANID -s $STARTTIME -o $VIDEODIR/$FILENAME.tmp
 +
 +
Lastly, you need to change the command in your user job entry.  It should read
 +
 +
removecommercials %DIR% %FILE% %CHANID% %STARTTIME%
 +
 +
=== MythTransCode command line option changes ===
 +
As of 10/2012, (mythtranscode version: fixes/0.25 [v0.25.2-15-g46cab93]) mythtranscode seems to no longer recognize command line options "-c" and "-s".
 +
Instead, use --chanid (rather than -c) and --starttime (rather than -s)
  
 
=== Database update fails after transcoding completes ===
 
=== Database update fails after transcoding completes ===

Latest revision as of 17:12, 20 March 2013

MythTV does not automatically remove commercials. The system will flag commercials and optionally skip them during playback. It can also transcode recordings between MPEG-2 and MPEG-4 or lossless transcode MPEG-2 and remove a cut list from the program. Since a list of commercial flags is not a cut list, it is not removed automatically. This document describes how to remove commercials permanently from the source material

Important.png Note: You do not need to do anything on this page if all you are doing is watching recorded shows using Myth TV, then deleting them or letting them auto-expire. You will not save very much space. What the instructions on this page are useful for is if you're storing or archiving your recordings, or are watching your recordings outside of MythTV, or want to watch "HDTV" programming on a slower machine in standard definition.


If you just want to skip commercials from the front end, see Commercial detection.

Removing commercials using mythfrontend

Editing Recordings

The recording editor allows you to mark areas of the recording to skip over during playback. Once you edit a recording, you can transcode the recording to permanently remove the cut areas from the recording file.

If you would like the transcoder to permanently remove any portions of the recording--including commercials or the lead in prior to a show or the overrun at the end of a show--you must first edit the recording.

Transcoding Recordings

The transcoder re-encodes files from one MythTV format to another. There are two reasons to transcode

  1. Allow users to create MPEG-4 streams to save space.
  2. Physically cut the commercials or video segments from the recordings.


Important.png Note: This is not used for creating MPEGs for burning to DVD as you will get audio video sync issues, especially with DVB recordings. You need to use MythArchive for this or manually with Project X and DVD Author.

Use the Recordings Profiles to change how the system transcodes your recordings

mythfrontend -> utilities/setup -> TV Settings --> recording Profiles -> Transcoders.

now select either RTjpeg or MPEG-4 (selecting any of the hardware encoders will result in the transcoder not working). set the other parameters as you'd like. For best performance, you should match the audio to the 'Default' profile. If you plan to only use the manual transcoder, you can choose to set the video to be the same as the default profile too (as long as it is MPEG-4 or RTjpeg)


Important.png Note: Even though MythTV 0.21 allows you to change the names of the recording profiles, don't do it. You can only edit the existing profiles. This is a feature that was partially implemented but hasn't been finished.


Important.png Note: When transcoding fails with in /var/log/mythtv/mythbackendlog messages like "No video information found! Transcode Errored: Program: Autodetect (exit status 255, job status was Errored", you should try to change the Default Transcoding (in Utilities/Setup -> Setup -> TV Settings -> General) from AutoDetect to for example "High Quality".


The transcoder can be used in two ways:

  1. Automatically re-encode every file once it has completed recording.
  2. After marking commercials, the transcoder can be run manually to delete the commercials from the file (thus further saving space).

The two methods are independent. The first is an automatic process that can be enabled/disabled through the setup program; the second is a manual method which must be invoked on each recording individually.

You can transcode a recording that has already been transcoded, for example to cut a further section with no loss in quality provided you are going from and to the same format.

To enable automatic transcoding, do the following: start the setup program under the host-specific settings: set the Transcoder Auto-run checkbox, or in newer versions of the program, you may select automatic transcoding from any of the recording profiles you have set up.

Everything should now be setup properly. If you elected to use the Auto-Run feature, the transcoder will automatically launch after each recording is complete. The transcoder thread runs at a low priority, so it should not impact any critical tasks or other recordings.

If you want to manually transcode a program, simply press RemoteMenu.png while watching a recording and select Transcode (you should have already finished marking all commercials and creating your Cut List with the Recording editor).

RecordingTranscode.jpg

If you change your mind press RemoteMenu.png and select Stop Transcoding.

Once the transcode is complete, mythbackend will replace the old file with the new as soon as it is no longer in use.

Removing commercials from the command-line

You can also remove commercials using the command-line utilities mythcommflag and mythtranscode.

You must have already flagged commercials for this option to work.

Use mythcommflag to create a cut list by using

mythcommflag --gencutlist

Then use mythtranscode to transcode the recording using the cut list with

mythtranscode --honorcutlist

You will need the additional standard options to both programs to choose the proper program based on filename or channel and time. Look at the --help output from each program for details on using these options

Automatically removing commercials

In the 0.20 release of MythTV, you can specify a custom command-line option for both the commercial flagging job and the transcoding job. Using this function, you could create a wrapper script for transcoding after commercial flagging that would use the command-line options outlined above.

One such way to implement this is to setup a User Job that uses this script. Save that script as /usr/local/bin/removecommercials and ensure that the script is executable by running:

chmod 755 /usr/local/bin/removecommercials

Set up a user job that runs the following:

removecommercials %DIR% %FILE%

You can do this by following the directions found on the User Jobs page.

Name the user job "Commercial Removal". You can now remove commercials by highlighting the recording in the recording list, hitting the right arrow, selecting "Job Options", and then selecting "Begin Commercial Removal". You may also set up your recordings to automatically run this script after a recording finishes. For more information on doing this, see the User Manual.

Update All Recordings

If you want to update all your recordings to automatically remove commercials, you could use the following SQL statement:

mysql -u root mythconverg;

or if you have a password on the root mysql account

mysql -u root -p mythconverg;

Then:

UPDATE record SET autotranscode = 0, autocommflag = 0, autouserjob1 = 1, 
autouserjob2 = 0, autouserjob3 = 0, autouserjob4 =0 WHERE autocommflag = 1;

After That:

exit;

This SQL statement assumes you've set up the job as User Job 1, and will only update those recordings already set to flag commercials.

Troubleshooting

Transcode error - Transcode failed with status: 247

If it throws an error at the transcoding step check your transcoding prefs. I had to change mine from RTjpeg to MPEG in the transcoding option in the Recording Prefs. From the Myth Frontend:

  1. Utilities/Setup
  2. Setup
  3. TV Settings
  4. Recording Profiles
  5. Transcoders
  6. Edit all "Autodetect" profiles (two in my case), on the second screen of each profile is a "codec" option, change this to "MPEG-4" from "RTjpeg".

MythTransCode runs OK, but doesn't cut commercials

What I noticed is that (even from the command line) mythtranscode fails to 'honor the cut list' when used with the "-i <filename>" syntax, but when that is switched to "-c <channel> -s <datetimestamp>" everything suddenly works fine. Don't know why, but it helped me to make things work.

I edited the above script to include the channel and datetimestamp in the mythtranscode command. To make the changes, replace lines 4-9 with

 CHANID=$3
 STARTTIME=$4
 # Sanity checking, to make sure everything is in order.
 if [ -z "$VIDEODIR" -o -z "$FILENAME" -o -z "$CHANID" -o -z "$STARTTIME" ]; then
      echo "Usage: $0 <VideoDirectory> <FileName> <CHANID> <STARTTIME>"

and line 30 with

 mythtranscode --honorcutlist --showprogress -c $CHANID -s $STARTTIME -o $VIDEODIR/$FILENAME.tmp

Lastly, you need to change the command in your user job entry. It should read

removecommercials %DIR% %FILE% %CHANID% %STARTTIME%

MythTransCode command line option changes

As of 10/2012, (mythtranscode version: fixes/0.25 [v0.25.2-15-g46cab93]) mythtranscode seems to no longer recognize command line options "-c" and "-s". Instead, use --chanid (rather than -c) and --starttime (rather than -s)

Database update fails after transcoding completes

Also if it gets past the transcoding and then crashes by saying user @ localhost does not have privileges. The mythbackend.log says something like "Access denied for user 'mythtv'@'localhost'". I had to run backend as root (I don't know how to make my user account have the privileges).

Note from manicmike: An easy way to give the user permissions is to 1) Edit* the sudoers file as root with "visudo" and add the entry <the mythtv user> ALL=NOPASSWD:/usr/local/bin/removecommercials (obviously, change the angle brackets bit to the user running mythtv) then save and quit. 2) change "removecommercials" in your user job to "sudo removecommercials". This will run the job as root while you're still the user.

  • in vi, press "i" to go into insert mode. Make the changes and exit using the sequence Esc : x (hit Esc, type a colon, type an x, hit enter).

Running the removecommercials script as system root probably only works when the MySQL root user has a blank password; leaving the MySQL root user without a password is a fairly basic security no-no. Another option would be to create a user-level configuration file for the mythtv user. The configuration file contains the database, user name and password to be used automatically by MySQL when scripts running as 'mythtv' make database calls.

  • add the file '.my.cnf' to the mythtv home directory
  • add the database options to the file, like so:
[client]
user=mythtv
password=<replace_with_mythtv_dbpassword>
database=mythconverg
  • Only the mythtv system user needs read permissions. Note: MySQL will not use this password if the file is world-writable.
-rw-rw---- 1 mythtv mythtv /home/mythtv/.my.cnf
  • For debugging, you can add the command 'my_print_defaults' to the shell script to print the configuration options read and used by MySQL.
my_print_defaults client mysql