Australian TV Listings

Guide Data Copyright In Australia

Australia doesn't have a service like Zap2It to provide TV program guide listings. Over the years, several people have written grabbers to scrape the data from web sites such as Yahoo, Nine-MSN, YourTV etc. These grabbers have frequently stopped working because of efforts from companies such as Nine-MSN to obfuscate the data. Each time, the workaround is trivial, but the continual changes are draining on the volunteers maintaining the grabbers.

It is unknown what purpose these companies have in trying to prevent their TV guides being used by their audience, possibly some misguided sense of copyright.

Sources Of Guide Data

There are a number of sources of data available with different grabbers. One list is maintained here.

Nine-MSN and Yahoo grabbers

http://www.whuffy.com/tv_grab_au/

Currently (10 Sept 2006) the Rex grabber is broken due to changes in the web site. It has previously been one of the best sources of data. While it doesn't get its data from NineMSN or Yahoo it is based on a grabber which used to. Rex 1.x is broken but the latest version at the link above does work at time of writing (17-Sep-06).

Ice TV

You can purchase the guide data from Ice TV with a subscription. The tv_grab_au_reg grabber will work with both IceTV and OzTiVo data sources.

OzTiVo

Alternatively, for those who want guide data for free, there is a source of data which is freely and openly available and designed to be downloaded for PVRs. This is provided by the nice folks at OzTiVo. and was originally designed for Aussie owners of TiVo boxes. However, because it outputs guide data in XMLTV format it can also be used by MythTV users. Have a look at the Australian XMLTV grabber page to get more details. In order to use this you will need to create a free account on their web site first.

tv_grab_au_reg

This script written to grab the OzTiVo data is a python script which does some clever things like:

• store a username/password in a config file.
• re-write XMLTV ids (useful if you're switching from a different data source).
• force the addition of explicit timezones on timestamps (useful for people from some states).
• filter out specific channels entirely.
• grab some channel's data from tvguide.org.au and some from IceTV and merge the result into one XMLTV dataset (tvguide.org.au covers more channels than IceTV).
• gives newbies who are looking for a 'tv_grab_au' script something to get hold of.
• use standard XMLTV grabber script options.

xmlguide.pl

However, an alternative is to use the xmlguide.pl script which is a script built into the whole tvguide.org.au service. This script is good for those who want to get their hands dirty, but be careful because if the tvguide.org.au interface changes you'll need to update it yourself. Users who go this route are requested to follow the tvguide.org.au mailing list to stay on top of any changes to the server.

When accessed, the server script queries the preferences associated with your username and password to decide which channels to download. It then extracts 7 days worth of data for those channels into XMLTV format which you can then load into MythTV using mythfilldatabase. Full details available at XMLGuideAPI. The XMLTVIDs in this data probably won't match what you have in your MythTV channel setup, so this is what you need to do to get it to work:

1. Put the commands below in a shell script (NB - the wget/sed command should be all one line)
2. Change the sed commands to be suitable for your own XMLTV IDs if required
3. or change the XMLTV IDs of your channels in myth
4. Change the output file name to suit after the ">"

wget --input-file=/usr/local/bin/mythtv/tv_grab_au_tvg.conf --output-document=- | sed 
's/channel="SBS-NEWS"/channel="news\.sbs\.com\.au"/g ; s/channel="Seven-Syd"/channel="sydney\.seven\.com\.au"/g ; 
s/channel="Ten-NSW"/channel="sydney\.ten\.com\.au"/g ; s/channel="SBS-NSW"/channel="sydney\.sbs\.com\.au"/g ; 
s/channel="ABC2"/channel="abc2\.abc\.gov\.au"/g ; s/channel="Nine-Syd"/channel="sydney\.nine\.com\.au"/g ; 
s/channel="ABC-NSW"/channel="nsw\.abc\.gov\.au"/g' > /usr/local/bin/mythtv/xmltvSydney.xml

mythfilldatabase --file 1 7 /usr/local/bin/mythtv/xmltvSydney.xml

The url for wget is specified in the --input-file param as it needs to contain your tvguide.org.au username and password. Make this file rw------- by chmod 600 so that others can't see your account logon details. The file should have one line like this wherer UserName and password are the username and password for your tvguide.org.au account:

http://UserName:password@minnie.tuhs.org/tivo-bin/xmlguide.pl

An easy way to run this automatically is to run this from a daily cron job. That can be set up either in the crontab for a particular user, or /etc/crontab or /etc/cron.daily/. Whichever is best may depend on your linux distribution. One thing to note when running it from cron is that you should put a randomiser into the call so that it doesn't do it at exactly the same time each day. Eg:

0 1 * * * sleep $((RANDOM % 18000)) && <whatever the download command is>

This helps to reduce the peak loads put on the tvguide.org.au servers by running the script sometime between 1am and 6am every day.

see also XMLTV#Australia