Simple Filter to Block Unwanted SchedulesDirect Programs - by Rating

This is a seamless, MythTV-upgrade-friendly way to filter out certain SchedulesDirect program content by removing it from the stream of XML data being fed into MythTV.

Background

A script gets called instead of Mythfilldatabase. The script uses an xmltv command to download the SchedulesDirect XML and transforms the XML, stripping out programs with specified USA "Advisories" like 'Brief Nudity' and 'Graphic Violence' using an XSLT file. The resulting filtered XML file is then fed into MythTV, preventing MythTV from knowing that such program content even exists - it can't record or present a listing of what it doesn't know about.

Intent

Make the MythTV programming content better suited to a family-friendly environment.

Requirements

• You should already know how and be able to log into your database from the mysql text console (or have installed phpMyAdmin) with permissions to administer your MythTv tables. See Category:MySQL for details and answers about the database itself.
• Download, build, and install the latest XMLTV from https://sourceforge.net/projects/xmltv/files/xmltv/
• sudo apt-get install xsltproc
• A SchedulesDirect account with a lineup already configured and working with MythTV

Warnings & Caveats

Warning: Backup your database before trying any of this.

Note: It is important to read this entire document before actually starting to alter your own system.

This is the section where I make scary statements and relieve myself of responsibility for anything that might go wrong and of the responsibility to teach anyone how to use MySQL and write/edit safe scripts.

I categorize this as being for advanced users only because if done incorrectly, your MythTV database could be destroyed. If you are confused about what I'm talking about in describing what to do or how the tools work don't just keep plugging along. If you are lost with what is described, please read up on basic database and scripting concepts and how to use the mysql text console or phpMyAdmin.

Note: You have backed everything up first, right? :)

Implementation

Copy and save the following file into your MythTV $HOME directory, to a file named mythcleanerfilldb.sh

 #!/bin/sh
 
 export PATH=/usr/local/bin:$PATH
 
 rm -f /tmp/dd.xml
 tv_grab_na_dd --days 14 --dd-data /tmp/dd.xml --download-only
 
 if [ -s /tmp/dd.xml ]
 then
   xsltproc $HOME/dd.xslt /tmp/dd.xml > /tmp/mythtv.xml
 
   if [ -s /tmp/mythtv.xml ]
   then
     mythfilldatabase --dd-file --xmlfile /tmp/mythtv.xml --sourceid 1 --lineupid PC:48504 --offset 0 --do-channel-updates
   else
     echo "xsltproc generated an empty or non-existent file for MythTV!"
   fi
 else
   echo "tv_grab_na_dd generated an empty or non-existent file for MythTV!"
 fi

In a moment, you'll need to edit the file you just saved, to potentially change the sourceid and very likely the zip code in the mythfilldatabase command line. You can get the values you need from your mythconverge database table named videosource. The sourceid column gives the value for the --sourceid argument in the script. The lineupid column gives the value for the --lineupid argument in the script. Now edit your script and save the change(s).

Copy and save the following file into your MythTV $HOME directory, to a file named dd.xslt

 <?xml version="1.0" encoding="utf-8"?>
 <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:tv="urn:TMSWebServices">
 <xsl:output method="xml" encoding="utf-8"/>
 
 <xsl:template match="node()|@*">
     <xsl:copy>
       <xsl:apply-templates select="node()|@*"/>
     </xsl:copy>
 </xsl:template>
  <xsl:template match="tv:program[descendant::tv:advisory[text() = 'Nudity' or text() = 'Brief Nudity' or text() = 'Strong Sexual Content' or text() = 'Graphic Language' or text() = 'Graphic Violence' or text() = 'Rape']]"/>
 </xsl:stylesheet>

You may wish to edit the dd.xslt file you just saved, if you want to relax or tighten the filtering. For a guide on other advisory names to use, see https://en.wikipedia.org/wiki/Pay_television_content_descriptors

As noted in the Requirements section above, assure you have downloaded, built, and install a modern version of XMLTV, following the README file in the archive. Also install xsltproc as noted above.

Configuration and Quick Test

Now you need to do a one-time configuration of tv_grab_na_dd so it correctly accesses your SchedulesDirect account and lineup. The configuration will be stored in a .xmltv folder in your $HOME account. Run this command

 tv_grab_na_dd --configure

It will ask for timezone, SchedulesDirect username, SchedulesDirect password, and will confirm the lineup to get - which in the default script is PC:48504 - but you edited that to your needs, right?

Note: Assuming that went well, you should now have a $HOME/.xmltv/tv_grab_na_dd.conf file with your entries, and the channels it sees in your SchedulesDirect lineup.

Now comes the fun part. Run this command and see if XMLTV can get your SchedulesDirect XML lineup file:

 tv_grab_na_dd --days 14 --dd-data /tmp/dd.xml --download-only

You should now have a /tmp/dd.xml file full of TV program information.

Warning: Assure your MythTV database is backed up - the next step will overwrite your guide information and puts your database at risk, if anything goes horribly wrong.

Loading the Filtered Program Guide Manually

Now run the full script to confirm the entire process works. In your $HOME directory, run mythcleanerfilldb.sh

This is what a good run will pretty much look like

 ./mythcleanerfilldb.sh
 using config filename /home/myuser/.xmltv/tv_grab_na_dd.conf
 Fetching from Schedules Direct
   dd_data is in /tmp/dd.xml
   Fetched 15245 k/bytes in 10 seconds
 2016-07-20 21:56:54.029300 C  mythfilldatabase version: fixes/0.27 [v0.27-193-g8ee257c] www.mythtv.org
 2016-07-20 21:56:54.029318 C  Qt version: compile: 4.8.6, runtime: 4.8.6
 2016-07-20 21:56:54.029329 N  Enabled verbose msgs:  general
 2016-07-20 21:56:54.029340 N  Setting Log Level to LOG_INFO
 2016-07-20 21:56:54.040731 I  Bypassing grabbers, reading directly from file
 2016-07-20 21:56:54.040892 I  Added logging to the console
 2016-07-20 21:56:54.042085 I  Setup Interrupt handler
 2016-07-20 21:56:54.042120 I  Setup Terminated handler
 2016-07-20 21:56:54.042146 I  Setup Segmentation fault handler
 2016-07-20 21:56:54.042166 I  Setup Aborted handler
 2016-07-20 21:56:54.042189 I  Setup Bus error handler
 2016-07-20 21:56:54.042211 I  Setup Floating point exception handler
 2016-07-20 21:56:54.042233 I  Setup Illegal instruction handler
 2016-07-20 21:56:54.042262 I  Setup Real-time signal 0 handler
 2016-07-20 21:56:54.042342 N  Using runtime prefix = /usr
 2016-07-20 21:56:54.042372 N  Using configuration directory = /home/myuser/.mythtv
 2016-07-20 21:56:54.042531 I  Assumed character encoding: en_US.UTF-8
 2016-07-20 21:56:54.043110 I  Using localhost value of 192.168.1.185
 2016-07-20 21:56:54.061211 N  Setting QT default locale to en_US
 2016-07-20 21:56:54.061284 I  Current locale en_US
 2016-07-20 21:56:54.061331 N  Reading locale defaults from /usr/share/mythtv//locales/en_us.xml
 2016-07-20 21:56:54.068037 I  Loading en_us translation for module mythfrontend
 2016-07-20 21:56:54.069465 I  Current MythTV Schema Version (DBSchemaVer): 1317
 2016-07-20 21:56:54.072286 I  MythCoreContext: Connecting to backend server: 192.168.1.185:6543 (try 1 of 1)
 2016-07-20 21:56:54.073423 I  Using protocol version 77
 2016-07-20 21:56:54.073981 I  Opening blocking connection to master backend
 2016-07-20 21:56:54.075313 I  Retrieving datadirect data.
 2016-07-20 21:56:54.075345 I  Grabbing data for Thu Jul 21 2016 offset 0
 2016-07-20 21:56:54.075360 I  From 2016-07-21T00:00:00Z to 2016-07-22T00:00:00Z (UTC)
 2016-07-20 21:56:54.075375 I  DataDirect: Grabbing listing data
 2016-07-20 21:56:54.110655 I  New static DB connectionDataDirectCon
 2016-07-20 21:56:54.123225 I  DataDirect: Your subscription expires on Tue Jul 25 2017 5:40 AM
 2016-07-20 21:56:54.142269 I  New Client:  (#1)
 2016-07-20 21:56:54.150873 I  DataDirect: sourceid 1 has lineup type: LocalBroadcast
 2016-07-20 21:57:25.485523 I  Grab complete.  Actual data from 2016-07-20T00:00:01Z to 2016-08-03T00:00:01Z (UTC)
 2016-07-20 21:57:25.486136 I  Main temp tables populated.
 2016-07-20 21:57:25.486141 I  Updating MythTV channels.
 2016-07-20 21:57:25.872486 I  Channels updated.
 2016-07-20 21:57:26.571406 I  Clearing data for source.
 2016-07-20 21:57:26.571449 I  Clearing from 2016-07-19T20:00:01 to 2016-08-02T20:00:01 (localtime)
 2016-07-20 21:57:30.655991 I  Data for source cleared.
 2016-07-20 21:57:30.656000 I  Updating programs.
 2016-07-20 21:57:38.065970 I  Program table update complete.
 2016-07-20 21:57:38.066047 I  Adjusting program database end times.
 2016-07-20 21:57:38.275200 I      7 replacements made
 2016-07-20 21:57:38.275210 I  Marking generic episodes.
 2016-07-20 21:57:38.432492 I      Found 5563
 2016-07-20 21:57:38.432504 I  Extending non-unique programids with multiple parts.
 2016-07-20 21:57:38.454019 I      Found 0
 2016-07-20 21:57:38.454040 I  Fixing missing original airdates.
 2016-07-20 21:57:38.645030 I      Found 0 with programids
 2016-07-20 21:57:38.646129 I      Found 0 without programids
 2016-07-20 21:57:38.646138 I  Marking repeats.
 2016-07-20 21:57:39.008191 I      Found 15308
 2016-07-20 21:57:39.008202 I  Unmarking new episode rebroadcast repeats.
 2016-07-20 21:57:39.077096 I      Found 0
 2016-07-20 21:57:39.216122 I  Marking episode first showings.
 2016-07-20 21:57:39.481670 I      Found 8524
 2016-07-20 21:57:39.481679 I  Marking episode last showings.
 2016-07-20 21:57:39.750455 I      Found 8197
 2016-07-20 21:57:39.754638 I  
 ===============================================================
 | Attempting to contact the master backend for rescheduling.  |
 | If the master is not running, rescheduling will happen when |
 | the master backend is restarted.                            |
 ===============================================================
 2016-07-20 21:57:39.755339 N  mythfilldatabase run complete.
 2016-07-20 21:57:39.755410 I  Waiting for threads to exit.

Final Integration

In mythtv-setup, you likely have a screen that contains your mythfilldatabase command. Replace this entry with the full path to your mythcleanerfilldb.sh file. Example, /home/myuser/mythcleanerfilldb.sh

You may also have a crontab entry that runs mythfilldatabase. It's critical that you comment that line out, and replace the command with your new script path above. If you don't, MythTV will have its lineup replaced by an unfiltered one.