Uk xmltv

From MythTV Official Wiki
Revision as of 11:22, 10 April 2008 by Rauxon (talk | contribs) (A Method to simplify inserting xmltvids)

Jump to: navigation, search

United Kingdom

tv_grab_uk_rt is the only UK listings grabber directly supported by MythTV.

Configuring tv_grab_uk_rt:

There are two ways to configure the grabber script.

The first, probably the most popular is to allow mythtv-setup to run the grabber configuration script from the 'Video Source Setup' page. Please note that running the grabber config script from here requires the user to switch focus back to the terminal session mythtv-setup was started from.

The second way to configure the grabber script is to run

tv_grab_uk_rt --configure


Assuming no .xmltv file already exists, the script will present the text

Enter the name of a channel, or '.' to finish selecting channels:

At the prompt, enter the name of a channel (or part of a channel name), e.g. BBC1

If more than one possibility for the channel is found, a 'menu' will appear allowing the selection to be refined:

Which channel to add?

0: BBC1

1: BBC1 East

2: BBC1 London & South East

3: BBC1 Midlands

4: BBC1 North

5: BBC1 North East

6: BBC1 North West

7: BBC1 Northern Ireland

8: BBC1 Scotland

9: BBC1 South

10: BBC1 South East

11: BBC1 South West

12: BBC1 Wales

13: BBC1 West

14: None of the above are what I wanted

Select one: [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14 (default=10)]

Sometimes choosing the right option can be a matter of pure guesswork, but it helps to know which BBC & ITV region you belong to.

When adding the channels is done, enter . and press RETURN.

The script will then write a <videosourcename>.xmltv file in ~/.mythtv or ~/.xmltv/tv_grab_uk_rt.conf

A note about newer versions of xmltv

The old method of channel entry has gone from xmltv in more recent versions (0.5.42 or so) and now entry takes the form of simple yes/no questions, e.g.

Add channel ABC1?  [yes,no,all,none (default=yes)]

Do not be tempted too answer 'all' unless ALL the channels offered by this grabber are required - otherwise empty and unusable channels will be created every time mythfilldatabase is run!

Answer every prompt with yes or no until the script stops. When the script is done, a .xmltv file will be written in either ~/.mythtv (videosourcename.xmltv) or ~/.xmltv/tv_grab_uk_rt.conf

How to find out which channels are available and what their xmltv IDs are:

Use the command

tv_grab_uk_rt --list-channels

What to do when tv_grab_uk_rt has written its config file

If tv_grab_uk_rt --configure has been run outside of mythtv-setup, make sure to copy the resultant file to ~/.mythtv/videosourcename.xmltv (where videosourcename is the name of the video source you wish to bind to the channels you chose).

The next step is to manually enter each and every single xmltv ID for every channel in MythTV's channel table. This can be done in the channel editor directly (laborious), in MythWeb's channel editor page (slightly less laborious) or in a MYSQL editor such as phpmyadmin (about as laborious as MythWeb's channel editor).

Refer to either the --list-channels option the grabber provides or /usr/share/xmltv/tv_grab_uk_rt/channel_ids to ensure the xmltv IDs entered into the channel table are correct.

This generally only needs to be done once, so when it's done, it's done!

Note: it'd be a great idea for somebody to collect channel lineups & configs along with regional data so that this process could be wizard-ised

Adding new channels so that the grabber picks them up

It isn't enough just to add a new channel's xmltvid in the channel table.

If the new xmltvid is known, either:

  • Add the new xmltvid to ~/.mythtv/<videosourcename>.xmltv in the form 'channel xmltvid'
  • Or re-run tv_grab_uk_rt --configure <videosourcename>.xmltv

(where <videosourcename> is the name of the 'video source' you want the grabber to grab data for)


If mythfilldatabase running automatically isn't picking up guide data for the channels which have been configured, it's possible that there are two instances of the <videosourcename>.xmltv file - one in /home/mythtv and one somewhere else. Check for that and make sure that mythfilldatabase is always run as the correct user.

An Alternative Approach for mixed transports (e.g. Sat AND Terrestrial)

So read all of the above first. This is a work in progress - and I think I can make it slicker, but it's here in case you might find it useful. This is for a setup in the UK with 1 x DVB Terrestrial and 1 x DVB Satellite (Fedora Core 5) The idea is that you grab the channels you want once from the Radio Times site. Then there are two "identical in all but name" Listing Sources setup in the main mythtv-setup program. One is tied to each card (they have different tuning info so this seemed best to me). You do have to manually enter the xmlids which is a (REAL) pain especially if like me you keep installing SYNs and/or rescanning, but I'll find a neat way of catching these properly (bit of SQL practice needed). Anyway, here's the process

  • Setup your cards as normal, but setup two seperate UK RT sources (one for Satellite, one for Terrestrial). In the command line window when it asks what channels to get just select all for now.
  • Turn off the mythfrontend setup options to run mythfilldatabase
  • Create the following file as /root/.mythtv/mythfilldatabasecron, it's based on the one distributed - note there are a couple of diags in it (e.g. whoami) which you can comment out as you see fit. It should be fairly self explanatory.

echo 'Running as user:'
echo 'Running TV Grabber'
# The following is to ensure tv_grab_uk_rt doesn't moan from crond or interactive
export PATH=$PATH:/usr/local/bin:/bin:/usr/bin:
export QTDIR=/usr/lib/qt3
# using config filename /root/.xmltv/tv_grab_uk_rt.conf
echo 'Choosing channels from tv_grab_uk_rt.conf.master'
/bin/grep -v \# /root/.xmltv/tv_grab_uk_rt.conf.master > /root/.xmltv/tv_grab_uk_rt.conf
echo 'Grabbing xml and saving'
# The following is one line
/bin/nice /usr/bin/tv_grab_uk_rt --config-file /root/.xmltv/tv_grab_uk_rt.conf --output /root/.xmltv/crondxmltv.xml
# The following line just helps you check if the xml file was actually created ok in the logs
ls -l /root/.xmltv/crondxmltv.xml
echo 'Running mythfilldatabase'
# For Satellite (Listings Source 1 on my system - see the --file X)
/bin/nice /usr/bin/mythfilldatabase --update --file 1 -1 /root/.xmltv/crondxmltv.xml
# For Terrestrial (Listings Source 2 on my system - see the --file X)
/bin/nice /usr/bin/mythfilldatabase --update --file 2 -1 /root/.xmltv/crondxmltv.xml
echo 'Checking stuff'
# Didn't really touch it after here.
# The minimum number of days you will allow before you
# want to receive a notification message
# The e-mail address to send the warning message to
SQL='SELECT TO_DAYS(max(endtime)) - TO_DAYS(now()) FROM program;'
DAYS_LEFT=`mysql --skip-column-names -B -D $DB --execute "$SQL"`
if [ $DAYS_LEFT -lt $MIN_DAYS ]; then
       mail -s "[MYTHTV] Only $DAYS_LEFT days of programs left!" $EMAIL <<-END
               Maybe you should check if mythfilldatabase is configured
               correctly or that your XMLTV service provider is still
echo 'All Done'

  • cp /root/.xmltv/tv_grab_uk_rt.conf /root/.xmltv/tv_grab_uk_rt.conf.master
  • open up /root/.xmltv/tv_grab_uk_rt.conf.master in your prefered text editor (vi? - hey anyone use joe?) and add a "#" at the begining of the line for any channel you don't want. This makes it real easy to add or remove channels from the grab (fewer channels=quicker grab).
  • Realise that the script will recreate /root/.xmltv/tv_grab_uk_rt.conf every time it runs.
  • echo '52 4 * * * root /root/.mythtv/mythfilldatabasecron >> /var/log/mythtv/mythfilldatabasecron.log' > /etc/crontab

This sets the script to be run at 4.52am every day and logs them as specified. You may want to ensure logrotate is configured, although there isn't too much to these logs.

  • Oh yeah... don't forget to chmod a+x /root/.mythtv/mythfilldatabasecron
  • You can also run it interactively.

Comments Welcomed

A Method to simplify inserting xmltvids

This is a lot quicker than doing it by hand, a scripting guru could I'm sure do this all in a script, but this is how I do it...

  • Get the channels listing, this includes names and xmltvids, do this with...
  • tv_grab_uk_rt --list-channels --output uk_channels.xml
  • Transfer this file to your PC and open it in Excel (sorry)
  • Write a simple formula in the last column like this...
  • ="update channel set xmltvid=*" & D2 & "* where callsign=*" & E2 & "*;"
  • Expand that formula down to create all the statements for all the rows
  • Copy the results into a text editor and add the top line as follows...
  • Search and replace the * with "

USE mythconverg;

update channel set xmltvid="" where callsign="Setanta Sport 1";
update channel set xmltvid="" where callsign="Setanta Sports 1";
update channel set xmltvid="" where callsign="Sky Sports 1";
update channel set xmltvid="" where callsign="Sky Sports 1";
update channel set xmltvid="" where callsign="Sky Movies 10";
... etc...

  • Save the file as mysql_textfile.sql and transfer it back to mythtv
  • Backup your DB
  • Run the SQL commands in e.g. mysql -uroot -pwhatever < mysql_textfile.sql

That should save you some time. Tidy up in the Channel editor.

Script to do this...

The following script comes with all the standard disclaimers... It seems to work for me, and is pretty much an implementation of the above using the perl XMLTV module, and perl DBI mysql modules. I BELIEVE that both of these are in place by default, but you may need to add them. (Graham Wood)


use XMLTV;
use Data::Dumper;
use DBI;

my $DB='mythconverg';                                           # Local DB
# my $DB='mythconverg;;port=3306';       # Remote DB

# Login details;
my $DBuser='mythtv';
my $DBpass='mythtv';

my $DBH=DBI->connect("DBI:mysql:database=${DB}", $DBuser, $DBpass) 
        or die "Cannot Connect to DB: $!";
my $sth=$DBH->prepare("Update channel set xmltvid=? where callsign=?");

# Read the output from the Radio Times Lister
my $output=`/usr/bin/tv_grab_uk_rt --list-channels`;
my ($encoding, $credits, $ch, $progs) = @{XMLTV::parse($output)};

# Finally work through each name adding the xmltvid
foreach my $cycle ( sort keys %$ch)
        print $cycle . "\n";
        foreach my $cycle2 ( @{$$ch{$cycle}{'display-name'}})
                print " $$cycle2[0]\n";
                $sth->execute($cycle, $$cycle2[0]);