Difference between revisions of "Removing Commercials"

From MythTV Official Wiki
Jump to: navigation, search
(note change to mythtranscode: --chanid (rather than -c) & --starttime (rather than -s))
(27 intermediate revisions by 18 users not shown)
Line 1: Line 1:
== Introduction ==
+
{{User_Manual_TOC}}
  
MythTV does not automatically remove commercials.  The system will flag commercials and skip them during playback.  It can also [[transcode]] recordings between [[MPEG-2]] and [[MPEG-4]] or [[lossless transcode]] MPEG-2 and remove a cutlist from the program.  Since a list of commercial flags is not a cutlist, it is not removed automatically.  This document describes how to remove commercials permanently from the source material.
+
MythTV does not automatically remove commercials.  The system will flag commercials and optionally skip them during playback.  It can also [[transcode]] recordings between [[MPEG-2]] and [[MPEG-4]] or [[lossless transcode]] MPEG-2 and remove a cut list from the program.  Since a list of commercial flags is not a cut list, it is not removed automatically.  This document describes how to remove commercials permanently from the source material
 +
{{Note box
 +
|msg=You do not need to do anything on this page if all you are doing is watching recorded shows using Myth TV, then deleting them or letting them auto-expire. You will not save very much space. What the instructions on this page are useful for is if you're storing or archiving your recordings, or are watching your recordings outside of MythTV, or want to watch "HDTV" programming on a slower machine in standard definition.
 +
}}
  
=== Important Note: ===
 
 
You do not need to do anything on this page if all you are doing is watching recorded shows using Myth TV, then deleting them or letting them auto-expire. You will not save very much space. What the instructions on this page are useful for is if you're storing or archiving your recordings, or are watching your recordings outside of Myth TV.
 
  
 
If you just want to skip commercials from the front end, see [[Commercial Flagging]].
 
If you just want to skip commercials from the front end, see [[Commercial Flagging]].
Line 11: Line 11:
 
== Removing commercials using mythfrontend ==
 
== Removing commercials using mythfrontend ==
  
[[Image:EditMenu.jpg|right]]
+
===Editing Recordings===
You can remove commercials that have been flagged by [[User_Manual:Daily_Use#Editing_Recordings|editing the recording]] using [[mythfrontend]].  Once you have brought up the timeline editor (press '''E''') you can press '''Z''' to load the flagged commercials as a cutlist in the editor.  Once the cutpoints are created, you can manipulate them normally in the editor to get them exactly where you want them.  It is uncommon for the commercial detection to be perfect all the time so some tweaking may be necessary.
+
  
After editing is complete, you can choose '''Begin Transcoding''' from the [[OSD]] menu or you can also choose to Begin Transcoding from the INFO menu in the Watch Recordings screen under Job Options. This will insert a transcode job into the [[Job Queue]] which will convert the recording based on your transcode settings in the [[User_Manual:Utilities_and_Setup#Recording_Profiles|recording profiles]] setup.
+
The recording editor allows you to mark areas of the recording to skip over during playback.  Once you edit a recording, you can transcode the recording to permanently remove the cut areas from the recording file.
<br clear=right>
+
 
 +
If you would like the transcoder to permanently remove any portions of the recording--including commercials or the lead in prior to a show or the overrun at the end of a show--you must first [[Editing_Recordings|edit the recording]].
 +
 
 +
===Transcoding Recordings ===
 +
 
 +
The transcoder re-encodes files from one MythTV format to another. There are two reasons to transcode
 +
 
 +
# Allow users to create [[MPEG-4]] streams to save space.
 +
# Physically cut the commercials or video segments from the recordings.
 +
 
 +
{{Note box|This is not used for creating MPEGs for burning to DVD as you will get audio video sync issues, especially with DVB recordings. You need to use [[MythArchive]] for this or manually with Project X and DVD Author.}}
 +
 
 +
Use the Recordings Profiles to change how the system transcodes your recordings
 +
 
 +
''mythfrontend -> utilities/setup -> TV Settings --> recording Profiles -> Transcoders.''  
 +
 
 +
now select either [[RTjpeg]] or [[MPEG-4]] (selecting any of the hardware encoders will result in the transcoder not working). set the other parameters as you'd like. For best performance, you should match the audio to the 'Default' profile. If you plan to only use the manual transcoder, you can choose to set the video to be the same as the default profile too (as long as it is [[MPEG-4]] or [[RTjpeg]])
 +
 
 +
{{Note box|Even though MythTV 0.21 allows you to change the names of the recording profiles, don't do it.  You can only edit the existing profiles.  This is a feature that was partially implemented but hasn't been finished.}}
 +
 
 +
{{Note box|When transcoding fails with in /var/log/mythtv/mythbackendlog messages like "No video information found! Transcode Errored: Program: Autodetect (exit status 255, job status was Errored", you should try to change the Default Transcoding (in Utilities/Setup -> Setup -> TV Settings -> General) from AutoDetect to for example "High Quality".}}
 +
 
 +
 
 +
The transcoder can be used in two ways:
 +
 
 +
# Automatically re-encode every file once it has completed recording.
 +
# After marking commercials, the transcoder can be run manually to delete the commercials from the file (thus further saving space).
 +
 
 +
The two methods are independent. The first is an automatic process that can be enabled/disabled through the setup program; the second is a manual method which must be invoked on each recording individually.
 +
 
 +
You can transcode a recording that has already been transcoded, for example to cut a further section with no loss in quality provided you are going from and to the same format.
 +
 
 +
To enable automatic transcoding, do the following: start the setup program under the host-specific settings: set the Transcoder Auto-run checkbox, or in newer versions of the program, you may select automatic transcoding from any of the recording profiles you have set up.
 +
 
 +
Everything should now be setup properly. If you elected to use the Auto-Run feature, the transcoder will automatically launch after each recording is complete. The transcoder thread runs at a low priority, so it should not impact any critical tasks or other recordings.
 +
 
 +
If you want to manually transcode a program, simply press [[Image:RemoteMenu.png]] while watching a recording and select Transcode (you should have already finished marking all commercials and creating your Cut List with the Recording editor).
 +
 
 +
[[Image:RecordingTranscode.jpg]]
 +
 
 +
If you change your mind press [[Image:RemoteMenu.png]] and select Stop Transcoding.
 +
 
 +
Once the transcode is complete, mythbackend will replace the old file with the new as soon as it is no longer in use.
  
 
== Removing commercials from the command-line ==
 
== Removing commercials from the command-line ==
Line 23: Line 64:
 
You must have already flagged commercials for this option to work.
 
You must have already flagged commercials for this option to work.
  
Use mythcommflag to create a cutlist by using
+
Use mythcommflag to create a cut list by using
  
 
  mythcommflag --gencutlist
 
  mythcommflag --gencutlist
  
Then use mythtranscode to transcode the recording using the cutlist with
+
Then use mythtranscode to transcode the recording using the cut list with
  
 
  mythtranscode --honorcutlist
 
  mythtranscode --honorcutlist
Line 53: Line 94:
 
If you want to update all your recordings to automatically remove commercials, you could use the following SQL statement:
 
If you want to update all your recordings to automatically remove commercials, you could use the following SQL statement:
 
  mysql -u root mythconverg;
 
  mysql -u root mythconverg;
 +
 +
or if you have a password on the root mysql account
 +
mysql -u root -p mythconverg;
  
 
Then:
 
Then:
Line 65: Line 109:
 
==Troubleshooting==
 
==Troubleshooting==
  
If it trows an error at the transcoding step check your transcoding prefs. I had to change mine from RJeg to MPEG in the transcoding option in the Recording Prefs.
+
=== Transcode error - Transcode failed with status: 247 ===
 +
If it throws an error at the transcoding step check your transcoding prefs. I had to change mine from RTjpeg to MPEG in the transcoding option in the Recording Prefs.
 +
From the Myth Frontend:
 +
# Utilities/Setup
 +
# Setup
 +
# TV Settings
 +
# Recording Profiles
 +
# Transcoders
 +
# Edit all "Autodetect" profiles (two in my case), on the second screen of each profile is a "codec" option, change this to "MPEG-4" from "RTjpeg".
 +
 
 +
=== MythTransCode runs OK, but doesn't cut commercials ===
 +
 
 +
What I noticed is that (even from the command line) mythtranscode fails to 'honor the cut list' when used with the "-i <filename>" syntax, but when that is switched to "-c <channel> -s <datetimestamp>" everything suddenly works fine. Don't know why, but it helped me to make things work. 
 +
 
 +
I edited the above script to include the channel and datetimestamp in the mythtranscode command.  To make the changes, replace lines 4-9 with
 +
 
 +
  CHANID=$3
 +
  STARTTIME=$4
 +
  # Sanity checking, to make sure everything is in order.
 +
  if [ -z "$VIDEODIR" -o -z "$FILENAME" -o -z "$CHANID" -o -z "$STARTTIME" ]; then
 +
      echo "Usage: $0 <VideoDirectory> <FileName> <CHANID> <STARTTIME>"
 +
 
 +
and line 30 with
 +
 
 +
  mythtranscode --honorcutlist --showprogress -c $CHANID -s $STARTTIME -o $VIDEODIR/$FILENAME.tmp
 +
 
 +
Lastly, you need to change the command in your user job entry.  It should read
 +
 
 +
removecommercials %DIR% %FILE% %CHANID% %STARTTIME%
 +
 
 +
=== MythTransCode command line option changes ===
 +
As of 10/2012, (mythtranscode version: fixes/0.25 [v0.25.2-15-g46cab93]) mythtranscode seems to no longer recognize command line options "-c" and "-s".
 +
Instead, use --chanid (rather than -c) and --starttime (rather than -s)
 +
 
 +
=== Database update fails after transcoding completes ===
 +
Also if it gets past the transcoding and then crashes by saying user @ localhost does not have privileges. The mythbackend.log says something like "Access denied for user 'mythtv'@'localhost'".
 +
I had to run backend as root (I don't know how to make my user account have the privileges).
 +
 
 +
Note from manicmike: An easy way to give the user permissions is to 1) Edit* the sudoers file as root with "visudo" and add the entry <the mythtv user>  ALL=NOPASSWD:/usr/local/bin/removecommercials (obviously, change the angle brackets bit to the user running mythtv) then save and quit. 2) change "removecommercials" in your user job to "sudo removecommercials". This will run the job as root while you're still the user.
 +
* in vi, press "i" to go into insert mode. Make the changes and exit using the sequence Esc : x (hit Esc, type a colon, type an x, hit enter).
  
Also if it gets past the transcoding and then crashes by saying user @ localhost does not have privileges. I had to run backend as root (I don't know how to make my user account have the privileges).  
+
Running the removecommercials script as system root probably only works when the MySQL root user has a blank password; leaving the MySQL root user without a password is a fairly basic security no-no. Another option would be to create a user-level configuration file for the mythtv user. The configuration file contains the database, user name and password to be used automatically by MySQL when scripts running as 'mythtv' make database calls.
 +
* add the file '.my.cnf' to the mythtv home directory
 +
* add the database options to the file, like so:
 +
[client]
 +
user=mythtv
 +
password=<replace_with_mythtv_dbpassword>
 +
database=mythconverg
 +
* Only the mythtv system user needs read permissions. Note: MySQL will not use this password if the file is world-writable.
 +
-rw-rw---- 1 mythtv mythtv /home/mythtv/.my.cnf
 +
* For debugging, you can add the command 'my_print_defaults' to the shell script to print the configuration options read and used by MySQL.
 +
my_print_defaults client mysql
  
 
[[Category:HOWTO]]
 
[[Category:HOWTO]]

Revision as of 17:29, 21 October 2012

MythTV does not automatically remove commercials. The system will flag commercials and optionally skip them during playback. It can also transcode recordings between MPEG-2 and MPEG-4 or lossless transcode MPEG-2 and remove a cut list from the program. Since a list of commercial flags is not a cut list, it is not removed automatically. This document describes how to remove commercials permanently from the source material

Important.png Note: You do not need to do anything on this page if all you are doing is watching recorded shows using Myth TV, then deleting them or letting them auto-expire. You will not save very much space. What the instructions on this page are useful for is if you're storing or archiving your recordings, or are watching your recordings outside of MythTV, or want to watch "HDTV" programming on a slower machine in standard definition.


If you just want to skip commercials from the front end, see Commercial Flagging.

Removing commercials using mythfrontend

Editing Recordings

The recording editor allows you to mark areas of the recording to skip over during playback. Once you edit a recording, you can transcode the recording to permanently remove the cut areas from the recording file.

If you would like the transcoder to permanently remove any portions of the recording--including commercials or the lead in prior to a show or the overrun at the end of a show--you must first edit the recording.

Transcoding Recordings

The transcoder re-encodes files from one MythTV format to another. There are two reasons to transcode

  1. Allow users to create MPEG-4 streams to save space.
  2. Physically cut the commercials or video segments from the recordings.


Important.png Note: This is not used for creating MPEGs for burning to DVD as you will get audio video sync issues, especially with DVB recordings. You need to use MythArchive for this or manually with Project X and DVD Author.

Use the Recordings Profiles to change how the system transcodes your recordings

mythfrontend -> utilities/setup -> TV Settings --> recording Profiles -> Transcoders.

now select either RTjpeg or MPEG-4 (selecting any of the hardware encoders will result in the transcoder not working). set the other parameters as you'd like. For best performance, you should match the audio to the 'Default' profile. If you plan to only use the manual transcoder, you can choose to set the video to be the same as the default profile too (as long as it is MPEG-4 or RTjpeg)


Important.png Note: Even though MythTV 0.21 allows you to change the names of the recording profiles, don't do it. You can only edit the existing profiles. This is a feature that was partially implemented but hasn't been finished.


Important.png Note: When transcoding fails with in /var/log/mythtv/mythbackendlog messages like "No video information found! Transcode Errored: Program: Autodetect (exit status 255, job status was Errored", you should try to change the Default Transcoding (in Utilities/Setup -> Setup -> TV Settings -> General) from AutoDetect to for example "High Quality".


The transcoder can be used in two ways:

  1. Automatically re-encode every file once it has completed recording.
  2. After marking commercials, the transcoder can be run manually to delete the commercials from the file (thus further saving space).

The two methods are independent. The first is an automatic process that can be enabled/disabled through the setup program; the second is a manual method which must be invoked on each recording individually.

You can transcode a recording that has already been transcoded, for example to cut a further section with no loss in quality provided you are going from and to the same format.

To enable automatic transcoding, do the following: start the setup program under the host-specific settings: set the Transcoder Auto-run checkbox, or in newer versions of the program, you may select automatic transcoding from any of the recording profiles you have set up.

Everything should now be setup properly. If you elected to use the Auto-Run feature, the transcoder will automatically launch after each recording is complete. The transcoder thread runs at a low priority, so it should not impact any critical tasks or other recordings.

If you want to manually transcode a program, simply press RemoteMenu.png while watching a recording and select Transcode (you should have already finished marking all commercials and creating your Cut List with the Recording editor).

RecordingTranscode.jpg

If you change your mind press RemoteMenu.png and select Stop Transcoding.

Once the transcode is complete, mythbackend will replace the old file with the new as soon as it is no longer in use.

Removing commercials from the command-line

You can also remove commercials using the command-line utilities mythcommflag and mythtranscode.

You must have already flagged commercials for this option to work.

Use mythcommflag to create a cut list by using

mythcommflag --gencutlist

Then use mythtranscode to transcode the recording using the cut list with

mythtranscode --honorcutlist

You will need the additional standard options to both programs to choose the proper program based on filename or channel and time. Look at the --help output from each program for details on using these options

Automatically removing commercials

In the 0.20 release of MythTV, you can specify a custom command-line option for both the commercial flagging job and the transcoding job. Using this function, you could create a wrapper script for transcoding after commercial flagging that would use the command-line options outlined above.

One such way to implement this is to setup a User Job that uses this script. Save that script as /usr/local/bin/removecommercials and ensure that the script is executable by running:

chmod 755 /usr/local/bin/removecommercials

Set up a user job that runs the following:

removecommercials %DIR% %FILE%

You can do this by following the directions found on the User Jobs page.

Name the user job "Commercial Removal". You can now remove commercials by highlighting the recording in the recording list, hitting the right arrow, selecting "Job Options", and then selecting "Begin Commercial Removal". You may also set up your recordings to automatically run this script after a recording finishes. For more information on doing this, see the User Manual.

Update All Recordings

If you want to update all your recordings to automatically remove commercials, you could use the following SQL statement:

mysql -u root mythconverg;

or if you have a password on the root mysql account

mysql -u root -p mythconverg;

Then:

UPDATE record SET autotranscode = 0, autocommflag = 0, autouserjob1 = 1, 
autouserjob2 = 0, autouserjob3 = 0, autouserjob4 =0 WHERE autocommflag = 1;

After That:

exit;

This SQL statement assumes you've set up the job as User Job 1, and will only update those recordings already set to flag commercials.

Troubleshooting

Transcode error - Transcode failed with status: 247

If it throws an error at the transcoding step check your transcoding prefs. I had to change mine from RTjpeg to MPEG in the transcoding option in the Recording Prefs. From the Myth Frontend:

  1. Utilities/Setup
  2. Setup
  3. TV Settings
  4. Recording Profiles
  5. Transcoders
  6. Edit all "Autodetect" profiles (two in my case), on the second screen of each profile is a "codec" option, change this to "MPEG-4" from "RTjpeg".

MythTransCode runs OK, but doesn't cut commercials

What I noticed is that (even from the command line) mythtranscode fails to 'honor the cut list' when used with the "-i <filename>" syntax, but when that is switched to "-c <channel> -s <datetimestamp>" everything suddenly works fine. Don't know why, but it helped me to make things work.

I edited the above script to include the channel and datetimestamp in the mythtranscode command. To make the changes, replace lines 4-9 with

 CHANID=$3
 STARTTIME=$4
 # Sanity checking, to make sure everything is in order.
 if [ -z "$VIDEODIR" -o -z "$FILENAME" -o -z "$CHANID" -o -z "$STARTTIME" ]; then
      echo "Usage: $0 <VideoDirectory> <FileName> <CHANID> <STARTTIME>"

and line 30 with

 mythtranscode --honorcutlist --showprogress -c $CHANID -s $STARTTIME -o $VIDEODIR/$FILENAME.tmp

Lastly, you need to change the command in your user job entry. It should read

removecommercials %DIR% %FILE% %CHANID% %STARTTIME%

MythTransCode command line option changes

As of 10/2012, (mythtranscode version: fixes/0.25 [v0.25.2-15-g46cab93]) mythtranscode seems to no longer recognize command line options "-c" and "-s". Instead, use --chanid (rather than -c) and --starttime (rather than -s)

Database update fails after transcoding completes

Also if it gets past the transcoding and then crashes by saying user @ localhost does not have privileges. The mythbackend.log says something like "Access denied for user 'mythtv'@'localhost'". I had to run backend as root (I don't know how to make my user account have the privileges).

Note from manicmike: An easy way to give the user permissions is to 1) Edit* the sudoers file as root with "visudo" and add the entry <the mythtv user> ALL=NOPASSWD:/usr/local/bin/removecommercials (obviously, change the angle brackets bit to the user running mythtv) then save and quit. 2) change "removecommercials" in your user job to "sudo removecommercials". This will run the job as root while you're still the user.

  • in vi, press "i" to go into insert mode. Make the changes and exit using the sequence Esc : x (hit Esc, type a colon, type an x, hit enter).

Running the removecommercials script as system root probably only works when the MySQL root user has a blank password; leaving the MySQL root user without a password is a fairly basic security no-no. Another option would be to create a user-level configuration file for the mythtv user. The configuration file contains the database, user name and password to be used automatically by MySQL when scripts running as 'mythtv' make database calls.

  • add the file '.my.cnf' to the mythtv home directory
  • add the database options to the file, like so:
[client]
user=mythtv
password=<replace_with_mythtv_dbpassword>
database=mythconverg
  • Only the mythtv system user needs read permissions. Note: MySQL will not use this password if the file is world-writable.
-rw-rw---- 1 mythtv mythtv /home/mythtv/.my.cnf
  • For debugging, you can add the command 'my_print_defaults' to the shell script to print the configuration options read and used by MySQL.
my_print_defaults client mysql