Australian TV Listings

From MythTV Official Wiki
Revision as of 14:03, 10 September 2006 by WhyTey (talk | contribs) (tv_grab_au_reg)

Jump to: navigation, search

Guide Data Copyright In Australia

Australia doesn't have a service like Zap2It to provide TV program guide listings. Over the years, several people have written grabbers to scrape the data from web sites such as Yahoo, Nine-MSN, YourTV etc. These grabbers have frequently stopped working because of efforts from companies such as Nine-MSN to obfuscate the data. Each time, the workaround is trivial, but the continual changes are draining on the volunteers maintaining the grabbers.

It is unknown what purpose these companies have in trying to prevent their TV guides being used by their audience, possibly some misguided sense of copyright.

Sources Of Guide Data

There are a number of sources of data available with different grabbers.

Nine-MSN and Yahoo grabbers

http://www.whuffy.com/tv_grab_au/

Currently (10 Sept 2006) the Nine-MSN grabber is broken due to changes in the web site. It has previously been one of the best sources of data.

Ice TV

You can purchase the guide data from Ice TV with a subscription. The grabber to use is the same one from OzTiVo below.

OzTiVo

Alternatively, for those who want guide data for free, there is a source of data which is freely and openly available and designed to be downloaded for PVRs. This is provided by the nice folks at OzTiVo. and was originally designed for Aussie owners of TiVo boxes. However, because it outputs guide data in XMLTV format it can also be used by MythTV users. Have a look at the Australian XMLTV grabber page to get more details. In order to use this you will need to create a free account on their web site first.

tv_grab_au_reg

This script provided by the OzTiVo guys is a python script which does some clever things like:

  • store a username/password in a config file.
  • re-write XMLTV ids (useful if you're switching from a different data source).
  • force the addition of explicit timezones on timestamps (useful for people from some states).
  • filter out specific channels entirely.
  • grab some channel's data from tvguide.org.au and some from IceTV and merge the result into one XMLTV dataset (tvguide.org.au covers more channels than IceTV).
  • gives newbies who are looking for a 'tv_grab_au' script something to get hold of.
  • use standard XMLTV grabber script options.

xmlguide.pl

However, an alternative is to use the xmlguide.pl script which is a script built into the whole tvguide.org.au service. This script is very fast to run and because it's a "built-in" to the tvguide.org.au service you shouldn't need to worry about downloading a new version etc when they change things. When run, it queries the preferences associated with your username and password to decide which channels to download. It then extracts 7 days worth of data for those channels into XMLTV format which you can then load into MythTV using mythfilldatabase. The XMLTVIDs in this data probably won't match what you have in your MythTV channel setup, so this is what you need to do to get it to work:


  1. Put the commands below in a shell script (NB - the wget/sed command should be all one line)
  2. Change the sed commands to be suitable for your own XMLTV IDs
  3. Change the output file name to suit after the ">"

wget --input-file=/usr/local/bin/mythtv/tv_grab_au_tvg.conf --output-document=- | sed 
's/channel="SBS-NEWS"/channel="news\.sbs\.com\.au"/g ; s/channel="Seven-Syd"/channel="sydney\.seven\.com\.au"/g ; 
s/channel="Ten-NSW"/channel="sydney\.ten\.com\.au"/g ; s/channel="SBS-NSW"/channel="sydney\.sbs\.com\.au"/g ; 
s/channel="ABC2"/channel="abc2\.abc\.gov\.au"/g ; s/channel="Nine-Syd"/channel="sydney\.nine\.com\.au"/g ; 
s/channel="ABC-NSW"/channel="nsw\.abc\.gov\.au"/g' > /usr/local/bin/mythtv/xmltvSydney.xml

mythfilldatabase --file 1 7 /home/mythtv/.xmltv/tvlistings_tvg.xml

The url for wget is specified in the --input-file param as it needs to contain your tvguide.org.au username and password. Make this file rw------- by chmod 600 so that others can't see your account logon details. The file should have one line like this wherer UserName and password are the username and password for your tvguide.org.au account:

http://UserName:password@minnie.tuhs.org/tivo-bin/xmlguide.pl


An easy way to run this automatically is to run this from a daily cron job. That can be set up either in the crontab for a particular user, or /etc/crontab or /etc/cron.daily/. Whichever is best may depend on your linux distribution. One thing to note when running it from cron is that you should put a randomiser into the call so that it doesn't do it at exactly the same time each day. Eg:

0 1 * * * sleep $((RANDOM % 18000)) && <whatever the download command is>

This helps to reduce the peak loads put on the tvguide.org.au servers by running the script sometime between 1am and 6am every day.

see also XMLTV#Australia