Difference between revisions of "Uk xmltv"

Revision as of 11:07, 12 February 2014

tv_grab_uk_rt is the only UK listings grabber directly supported by MythTV.

How to find out which channels are available and what their xmltv IDs are:

Note: The "xmltv" software module must have be installed in order to access the following features and programs.

XMLTV IDs look like hostnames, for example bbc2.bbc.co.uk

Use the command

tv_grab_uk_rt --list-channels

Noticed that timeshifted channels appear as plus-1.xyz and tsod.plus-1.xyz. The former is a separate channel from the data source and the latter is dynamically generated in the grabber. For better duplicate detection it is preferable to used the dynamic copies generated by the grabber as the source channels can have different data for the same programs.

Configuring tv_grab_uk_rt:

There are two ways to configure the grabber script.

NOT RECOMMENDED

The first, probably the most popular is to allow mythtv-setup to run the grabber configuration script from the 'Video Source Setup' page. This generates ~/.mythtv/<videosourcename>.xmltv

Please note that running the grabber config script from here has problems where it is not possible to scroll to the bottom of the screen. See #7962. As of 22nd June 2011 this has been fixed by Paul Harrison, thanks Paul.

RECOMMENDED

The second way to configure the grabber script is to run it directly

tv_grab_uk_rt --configure

By default this script generates ~/.xmltv/tv_grab_uk_rt.conf

The Process

1. In mythtv-setup, add a Video Source Setup using the menu option 3 - choose UK Radio Times as the grabber script - do not run configure - name this any source you like, but remember the name exactly - you'll need in later on
2. exit mythtv-setup but DO NOT run mythfilldatabase on exit
3. Run the grabber script by executing the following:

   tv_grab_uk_rt --configure

   1. Select output format - default recommended
   2. Select cache directory - default recommended
   3. Enable title processing? - default recommended
   4. Postcode - enter the first part of your postcode to get localised listings
   5. Choose platform - Sky/Virgin/Freeview/etc.
   6. Choose channels - you can answer yes/y/no/n/none/all - answer yes to process one at a time, however, if you want to select the majority, choose "all" then manually edit the file after the script finishes to remove the ones you don't want. NOTE: as at 06/11/09, the Community Channel causes errors in later processing if selected - i choose "all" then remove the Community Channel by manual edit after script has run.
   7. You should see a list of channel titles scroll by, then a return to the prompt

What to do when tv_grab_uk_rt has written its config file

If tv_grab_uk_rt --configure has been run outside of mythtv-setup, make sure to copy the resultant file to ~/.mythtv/videosourcename.xmltv (where videosourcename is the name of the video source you wish to bind to the channels you chose - chosen in Step 1 of the process above).

So, for a video source named "Test" the following command would need to be run:

cp ~/.xmltv/tv_grab_uk_rt.conf ~/.mythtv/Test.xmltv

The next step

The next step is to manually enter each and every single xmltv ID for every channel in MythTV's channel table. This can be done in the channel editor directly (laborious), in MythWeb's channel editor page (slightly less laborious) or in a MYSQL editor such as phpmyadmin (about as laborious as MythWeb's channel editor).

Refer to either the --list-channels option the grabber provides or /usr/share/xmltv/tv_grab_uk_rt/channel_ids to ensure the xmltv IDs entered into the channel table are correct.

This generally only needs to be done once, so when it's done, it's done!

Note: it'd be a great idea for somebody to collect channel lineups & configs along with regional data so that this process could be wizard-ised

Adding new channels so that the grabber picks them up

It isn't enough just to add a new channel's xmltvid in the channel table.

If the new xmltvid is known, either:

• Add the new xmltvid to ~/.mythtv/<videosourcename>.xmltv in the form 'channel xmltvid'
• Or re-run tv_grab_uk_rt --configure --config-file ~/.mythtv/<videosourcename>.xmltv

(where <videosourcename> is the name of the 'video source' you want the grabber to grab data for)

Troubleshooting

If mythfilldatabase running automatically isn't picking up guide data for the channels which have been configured, it's possible that there are two instances of the <videosourcename>.xmltv file - one in /home/mythtv and one somewhere else. Check for that and make sure that mythfilldatabase is always run as the correct user.

If you are unable to select the Radio Times grabber in mythtv-setup, it could be because Myth is timing out before it has managed to find all the grabbers. Thanks to Nickrout at http://ubuntuforums.org/showthread.php?t=1067288 for finding this. This was with the following command to delete the grabbers that were not needed:

rm `ls -1 tv_grab_* |grep -v "uk"`

An Alternative Approach for mixed transports (e.g. Sat AND Terrestrial)

So read all of the above first. This is a work in progress - and I think I can make it slicker, but it's here in case you might find it useful. This is for a setup in the UK with 1 x DVB Terrestrial and 1 x DVB Satellite (Fedora Core 5) The idea is that you grab the channels you want once from the Radio Times site. Then there are two "identical in all but name" Listing Sources setup in the main mythtv-setup program. One is tied to each card (they have different tuning info so this seemed best to me). You do have to manually enter the xmlids which is a (REAL) pain especially if like me you keep installing SYNs and/or rescanning, but I'll find a neat way of catching these properly (bit of SQL practice needed). Anyway, here's the process

• Setup your cards as normal, but setup two seperate UK RT sources (one for Satellite, one for Terrestrial). In the command line window when it asks what channels to get just select all for now.
• Turn off the mythfrontend setup options to run mythfilldatabase
• Create the following file as /root/.mythtv/mythfilldatabasecron, it's based on the one distributed - note there are a couple of diags in it (e.g. whoami) which you can comment out as you see fit. It should be fairly self explanatory.

#!/bin/bash
echo 'Running as user:'
whoami
echo 'Running TV Grabber'
# The following is to ensure tv_grab_uk_rt doesn't moan from crond or interactive
export PATH=$PATH:/usr/local/bin:/bin:/usr/bin:
export QTDIR=/usr/lib/qt3
# using config filename /root/.xmltv/tv_grab_uk_rt.conf
echo 'Choosing channels from tv_grab_uk_rt.conf.master'
/bin/grep -v \# /root/.xmltv/tv_grab_uk_rt.conf.master > /root/.xmltv/tv_grab_uk_rt.conf
echo 'Grabbing xml and saving'
# The following is one line
/bin/nice /usr/bin/tv_grab_uk_rt --config-file /root/.xmltv/tv_grab_uk_rt.conf --output /root/.xmltv/crondxmltv.xml
# The following line just helps you check if the xml file was actually created ok in the logs
ls -l /root/.xmltv/crondxmltv.xml
echo 'Running mythfilldatabase'
# For Satellite (Listings Source 1 on my system - see the --file X)
/bin/nice /usr/bin/mythfilldatabase --update --file 1 -1 /root/.xmltv/crondxmltv.xml
# For Terrestrial (Listings Source 2 on my system - see the --file X)
/bin/nice /usr/bin/mythfilldatabase --update --file 2 -1 /root/.xmltv/crondxmltv.xml
echo 'Checking stuff'
# Didn't really touch it after here.
# The minimum number of days you will allow before you
# want to receive a notification message
MIN_DAYS=3
 
# The e-mail address to send the warning message to
EMAIL=root
 
DB=mythconverg
SQL='SELECT TO_DAYS(max(endtime)) - TO_DAYS(now()) FROM program;'
 
DAYS_LEFT=`mysql --skip-column-names -B -D $DB --execute "$SQL"`
 
if [ $DAYS_LEFT -lt $MIN_DAYS ]; then
       mail -s "[MYTHTV] Only $DAYS_LEFT days of programs left!" $EMAIL <<-END
               Maybe you should check if mythfilldatabase is configured
               correctly or that your XMLTV service provider is still
               on-line?
       END
fi
 
echo 'All Done'

• cp /root/.xmltv/tv_grab_uk_rt.conf /root/.xmltv/tv_grab_uk_rt.conf.master
• open up /root/.xmltv/tv_grab_uk_rt.conf.master in your prefered text editor (vi? - hey anyone use joe?) and add a "#" at the begining of the line for any channel you don't want. This makes it real easy to add or remove channels from the grab (fewer channels=quicker grab).
• Realise that the script will recreate /root/.xmltv/tv_grab_uk_rt.conf every time it runs.
• echo '52 4 * * * root /root/.mythtv/mythfilldatabasecron >> /var/log/mythtv/mythfilldatabasecron.log' > /etc/crontab

This sets the script to be run at 4.52am every day and logs them as specified. You may want to ensure logrotate is configured, although there isn't too much to these logs.

• Oh yeah... don't forget to chmod a+x /root/.mythtv/mythfilldatabasecron
• You can also run it interactively.

Comments Welcomed

A Method to simplify inserting xmltvids

This is a lot quicker than doing it by hand, a scripting guru could I'm sure do this all in a script, but this is how I do it...

• Get the channels listing, this includes names and xmltvids, do this with...
• tv_grab_uk_rt --list-channels --output uk_channels.xml

• Transfer this file to your PC and open it in Excel (sorry) or OOo Calc
• Write a simple formula in the last column like this...
• ="update channel set xmltvid=*" & D2 & "* where callsign=*" & E2 & "*;"
• Expand that formula down to create all the statements for all the rows
• Copy the results into a text editor and add the top line as follows...
• Search and replace the * with "

USE mythconverg;

update channel set xmltvid="1.setanta.com" where callsign="Setanta Sport 1";
update channel set xmltvid="1.setanta.com" where callsign="Setanta Sports 1";
update channel set xmltvid="1.sports.sky.com" where callsign="Sky Sports 1";
update channel set xmltvid="1.sports.sky.com" where callsign="Sky Sports 1";
update channel set xmltvid="10.movies.sky.com" where callsign="Sky Movies 10";
... etc...

• Save the file as mysql_textfile.sql and transfer it back to mythtv

• Backup your DB

• Run the SQL commands in e.g. mysql -uroot -pwhatever < mysql_textfile.sql

That should save you some time. Tidy up in the Channel editor.

Script to do this...

The following script comes with all the standard disclaimers... It seems to work for me, and is pretty much an implementation of the above using the perl XMLTV module, and perl DBI mysql modules. I BELIEVE that both of these are in place by default, but you may need to add them. (Graham Wood)

#!/usr/bin/perl

use XMLTV;
use Data::Dumper;
use DBI;

my $DB='mythconverg';                                           # Local DB
# my $DB='mythconverg;host=dbhost.example.com;port=3306';       # Remote DB

# Login details;
my $DBuser='mythtv';
my $DBpass='mythtv';

my $DBH=DBI->connect("DBI:mysql:database=${DB}", $DBuser, $DBpass) 
        or die "Cannot Connect to DB: $!";
my $sth=$DBH->prepare("Update channel set xmltvid=? where callsign=?");

# Read the output from the Radio Times Lister
my $output=`/usr/bin/tv_grab_uk_rt --list-channels`;
my ($encoding, $credits, $ch, $progs) = @{XMLTV::parse($output)};

# Finally work through each name adding the xmltvid
foreach my $cycle ( sort keys %$ch)
{
        print $cycle . "\n";
        foreach my $cycle2 ( @{$$ch{$cycle}{'display-name'}})
        {
                print " $$cycle2[0]\n";
                $sth->execute($cycle, $$cycle2[0]);
        }
}

Another script to update xmltvids including icons

Get the channel list with tv_grab_uk_rt --list-channel > uk_channel_list.xml

Then run the following script with the above uk_channel_list.xml and redirect output to another file.

This file will need editing because not all the call signs match up.

Then apply to the database, eg

mysql -u mythtv -p < the_edited_file

The following script comes with all the standard disclaimers... It seems to work for me,

#!/usr/bin/perl


if ( $#ARGV != 0 ) {
  print "Usage ", $0, " xmltv_channel_list.xml\n";
  exit 1;
}

# use module
use XML::Simple;
use Data::Dumper;

# create object
$xml = new XML::Simple;

# read XML file
$data = $xml->XMLin( $ARGV[0] );

print "USE mythconverg;\n";

while ( ($key, $value) = each(%{$data -> {channel}}) ) {
    print "update channel set xmltvid=\"", $key, "\", icon=\"", $value -> {'icon'}->{src}, "\"  where callsign=\"",  $value -> {'display-name'}, "\";\n";
}

This is my first contribution, hope I have put this in the correct place.

Further Reading

This wiki also has a page with more information about managing UK TV channels.