Difference between revisions of "Australian TV Listings"
(→See also) |
(→Grabbers Of Guide Data: Deleted dead grabber Rex, updated status of obsolete grabbers, removed Shepherd warning as per Talk page) |
||
| (41 intermediate revisions by 19 users not shown) | |||
| Line 1: | Line 1: | ||
| − | == Guide Data Copyright In Australia == | + | {{Sisterproject |project=Wikipedia |
| + | |image=Wikipedia-logo-en.png | ||
| + | |text=Wikipedia has an article on: | ||
| + | |link=[[Wikipedia:{{{1|Freeview_(Australia)}}}|{{{1|Freeview_(Australia)}}}]] | ||
| + | }} | ||
| + | |||
| + | == Electronic Service Guide Information == | ||
| + | |||
| + | Currently the best source of guide data is the [[EIT]] content being broadcast by the Free TV networks over DVB-T. | ||
| + | |||
| + | === Free TV Australia Operational Practice OP-44 === | ||
| + | [http://www.freetv.com.au/media/Engineering/OP44%20_Implementation_of_Electronic_Service_Guide_Information_%20Issue_3_September_2008.pdf Implementation Guide for the DVB Event Information Table (EITp/f)] provides good with interoperability with MythTV. | ||
| + | |||
| + | The status of the networks compliance is currently as follows: (Information based on Perth data) | ||
| + | |||
| + | {| border=1 cellspacing=0 cellpadding=5 | ||
| + | | Network | ||
| + | | event_name_char (Title) | ||
| + | | text_char (Episode) | ||
| + | | extended_event_descriptor (Description) | ||
| + | | content_descriptor (Categories) | ||
| + | |- | ||
| + | | ABC | ||
| + | | good | ||
| + | | contains description, gets truncated sometimes | ||
| + | | unused | ||
| + | | good | ||
| + | |- | ||
| + | | Seven | ||
| + | | good | ||
| + | | contains description, rpt info, closed captions info | ||
| + | | unused | ||
| + | | good | ||
| + | |- | ||
| + | | Nine | ||
| + | | good | ||
| + | | generally ok, contains extra flags (live, rpt, rating) at start, "Movie" if movie, sometimes a (truncated) duplicate of description | ||
| + | | ok, sometimes contains cast | ||
| + | | good | ||
| + | |- | ||
| + | | Ten | ||
| + | | ok (live events prefixed with LIVE:) | ||
| + | | contains description, some programs have just generic series description | ||
| + | | copyright declaration | ||
| + | | good | ||
| + | |- | ||
| + | | SBS | ||
| + | | good | ||
| + | | contains description | ||
| + | | unused | ||
| + | | good | ||
| + | |} | ||
| + | |||
| + | === Guide Data Copyright In Australia === | ||
Australia doesn't have a service like Zap2It to provide TV program guide listings. Over the years, several people have written grabbers to scrape the data from web sites such as Yahoo, Nine-MSN, YourTV etc. These grabbers have frequently stopped working because of efforts from companies such as Nine-MSN to obfuscate the data. Each time, the workaround is trivial, but the continual changes are draining on the volunteers maintaining the grabbers. | Australia doesn't have a service like Zap2It to provide TV program guide listings. Over the years, several people have written grabbers to scrape the data from web sites such as Yahoo, Nine-MSN, YourTV etc. These grabbers have frequently stopped working because of efforts from companies such as Nine-MSN to obfuscate the data. Each time, the workaround is trivial, but the continual changes are draining on the volunteers maintaining the grabbers. | ||
It is unknown what purpose these companies have in trying to prevent their TV guides being used by their audience, possibly some misguided sense of copyright. | It is unknown what purpose these companies have in trying to prevent their TV guides being used by their audience, possibly some misguided sense of copyright. | ||
| + | |||
| + | ==== Background: ==== | ||
| + | |||
| + | ===== Nine loses electronic program guide case ===== | ||
| + | |||
| + | http://www.smh.com.au/news/home-theatre/nine-loses-program-guide-case/2007/08/09/1186530493565.html | ||
| + | |||
| + | Also loses High Court Appeal (22/04/09): | ||
| + | |||
| + | http://www.goodgearguide.com.au/article/300251/icetv_wins_landmark_high_court_decision_versus_nine_network | ||
| + | |||
| + | ===== Free TV networks to broadcast industry electronic program guide ===== | ||
| + | |||
| + | http://www.freetv.com.au/media/News-Media_Release/2007-0304_Free_TV_Networks_to_Broadcast_Industry_Electronic_Program_Guide_141107.pdf | ||
== Sources Of Guide Data == | == Sources Of Guide Data == | ||
| Line 9: | Line 76: | ||
There are a number of sources of data available with different grabbers. One list is [http://www.cse.unsw.edu.au/~willu/xmltv/ maintained here]. | There are a number of sources of data available with different grabbers. One list is [http://www.cse.unsw.edu.au/~willu/xmltv/ maintained here]. | ||
| − | + | Also see [[XMLTV#Australia]]. | |
| − | + | === Nine-MSN, Yahoo And Other Site Scrappers === | |
| − | + | These sites have been the most detailed and accurate source of data but constantly break. | |
=== Ice TV === | === Ice TV === | ||
| − | You can purchase the guide data from [http://www.icetv.com.au Ice TV] with a subscription. The [ | + | You can purchase the guide data from [http://www.icetv.com.au Ice TV] with a AU$3 per week subscription. The [[#tv_grab_au_reg]] grabber will work with both IceTV and OzTiVo data sources. |
=== OzTiVo === | === OzTiVo === | ||
| − | Alternatively, for those who want guide data for free, there is a source of data which is freely and openly available and designed to be downloaded for PVRs. This is provided by the nice folks at [http://www. | + | Alternatively, for those who want guide data for free, there is a source of data which is freely and openly available and designed to be downloaded for PVRs. This is provided by the nice folks at [http://www.oztivo.net/ OzTiVo] and was originally designed for Aussie owners of TiVo boxes. However, because it outputs guide data in XMLTV format it can also be used by MythTV users. In order to use this you will need to create a free account on their web site first. Users who go this route are requested to follow the tvguide.org.au mailing list to stay on top of any changes to the service and help with corrections. The [[#tv_grab_au_reg]] grabber will work with both IceTV and OzTiVo data sources, while [[#xmlguide.pl]] will only work for OzTiVo. |
| + | |||
| + | === EPGStream === | ||
| + | http://epgstream.net/ provides a simple source of data (one http request). Mainly designed for MediaCenter users, they also have an xmltv interface. I wrote a script to download the xmltv to a temporary file, then upload it to the database using the mythfilldatabase --file option. It's online here as [[EPGStreamUpdateTvGuide]]. | ||
| + | |||
| + | ===EIT Scanning=== | ||
| + | As of January 1st 2008 all commercial free to air networks will broadcast EPG data for up to 7 days (No minimum specified). Previously the only broadcasted data was for 'now' and 'next'. Therefore there may not be a need to use any of the grabbers/site scrappers mentioned within this page for free to air TV only. | ||
| + | |||
| + | See this article for more information: [http://www.rapidtvnews.com/default.asp?sourceid=&smenu=1&twindow=Default&mad=No&sdetail=2360&wpage=&skeyword=&sidate=&ccat=&ccatm=&restate=&restatus=&reoption=&retype=&repmin=&repmax=&rebed=&rebath=&s Aussie nets agree EPG start,Rapid TV News] | ||
| + | |||
| + | == Grabbers Of Guide Data == | ||
| + | |||
| + | === tv_grab_oztivo === | ||
| + | Status: Possibly dead (web page no longer loads). | ||
| + | |||
| + | Downloads guide data from [http://www.oztivo.net/ OzTiVo] | ||
| + | * uses standard XMLTV grabber script options. | ||
| + | * re-writing of XMLTV ids. | ||
| + | * Changing timezones of channels data. | ||
| + | * Filtering out specific channels. | ||
| + | * Direct execution of mythfilldatabase on data. | ||
| + | * Converting OzTiVo categories to DataDirect categories. | ||
| + | * Setting HDTV flag on HD channels. | ||
| + | [http://web.aanet.com.au/auric/?q=node/14 Authors site] | ||
| − | + | === tv_grab_au_reg === | |
| − | This script | + | Status: Possibly dead (last updated 2007). |
| + | |||
| + | This python script grabs the [[#Ice TV]] and [[#OzTiVo]] data and does some clever things like: | ||
* store a username/password in a config file. | * store a username/password in a config file. | ||
* re-write XMLTV ids (useful if you're switching from a different data source). | * re-write XMLTV ids (useful if you're switching from a different data source). | ||
| − | * force the addition of explicit timezones on timestamps (useful for people from some states). | + | * force the addition of explicit timezones on timestamps (useful for people from some states and pay tv data). |
* filter out specific channels entirely. | * filter out specific channels entirely. | ||
* grab some channel's data from tvguide.org.au and some from IceTV and merge the result into one XMLTV dataset (tvguide.org.au covers more channels than IceTV). | * grab some channel's data from tvguide.org.au and some from IceTV and merge the result into one XMLTV dataset (tvguide.org.au covers more channels than IceTV). | ||
| Line 32: | Line 124: | ||
* use standard XMLTV grabber script options. | * use standard XMLTV grabber script options. | ||
| − | + | You can find the script plus some other useful info about Aussie grabbers at William Uther's site [http://www.cse.unsw.edu.au/~willu/xmltv/tv_grab_au_reg.html tv_grab_au_reg]. | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | + | === Shepherd === | |
| − | + | Status: Working as of February 2014. | |
| − | |||
| − | |||
| − | |||
| − | + | Shepherd aims to solve the traditional problems of tv_grab_au scripts by: | |
| − | + | # employing multiple TV guide grabbers and data sources and switching between them as necessary; and | |
| + | # auto-updating to eliminate the need for users to manually address grabber breakages | ||
| − | + | There are details here: [http://svn.whuffy.com/wiki]http://svn.whuffy.com/wiki | |
| − | + | Shepherd also includes the ability to update the channel icons displayed within mythTV. Examples of the icon-sets that can used are shown [http://svn.whuffy.com/index.fcgi/wiki/channel_icons here]. | |
| − | http:// | ||
| − | |||
| + | Please note that during setup, Shepherd offers the option of auto-configuring MythTV on your behalf. Users should be aware that agreeing will cause Shepherd to register itself directly with the MythTV database, which is not XMLTV-compliant behaviour. | ||
| − | + | == TV Icons == | |
| + | Inspired by FrogBoy's most excellent icon set, a number of icons exist for the free2air channels: | ||
| − | + | An example of FrogBoy's work follows | |
| − | + | [[Image:Au-nine.png]] | |
| − | + | [[Image:Au-sbs-hd.png]] | |
| − | |||
| − | + | Icons are available at: | |
| − | + | * http://ash.ms/projects/channels/ FrogBoy and T8 | |
Revision as of 19:27, 16 February 2014
Contents
Electronic Service Guide Information
Currently the best source of guide data is the EIT content being broadcast by the Free TV networks over DVB-T.
Free TV Australia Operational Practice OP-44
Implementation Guide for the DVB Event Information Table (EITp/f) provides good with interoperability with MythTV.
The status of the networks compliance is currently as follows: (Information based on Perth data)
| Network | event_name_char (Title) | text_char (Episode) | extended_event_descriptor (Description) | content_descriptor (Categories) |
| ABC | good | contains description, gets truncated sometimes | unused | good |
| Seven | good | contains description, rpt info, closed captions info | unused | good |
| Nine | good | generally ok, contains extra flags (live, rpt, rating) at start, "Movie" if movie, sometimes a (truncated) duplicate of description | ok, sometimes contains cast | good |
| Ten | ok (live events prefixed with LIVE:) | contains description, some programs have just generic series description | copyright declaration | good |
| SBS | good | contains description | unused | good |
Guide Data Copyright In Australia
Australia doesn't have a service like Zap2It to provide TV program guide listings. Over the years, several people have written grabbers to scrape the data from web sites such as Yahoo, Nine-MSN, YourTV etc. These grabbers have frequently stopped working because of efforts from companies such as Nine-MSN to obfuscate the data. Each time, the workaround is trivial, but the continual changes are draining on the volunteers maintaining the grabbers.
It is unknown what purpose these companies have in trying to prevent their TV guides being used by their audience, possibly some misguided sense of copyright.
Background:
Nine loses electronic program guide case
http://www.smh.com.au/news/home-theatre/nine-loses-program-guide-case/2007/08/09/1186530493565.html
Also loses High Court Appeal (22/04/09):
Free TV networks to broadcast industry electronic program guide
Sources Of Guide Data
There are a number of sources of data available with different grabbers. One list is maintained here.
Also see XMLTV#Australia.
Nine-MSN, Yahoo And Other Site Scrappers
These sites have been the most detailed and accurate source of data but constantly break.
Ice TV
You can purchase the guide data from Ice TV with a AU$3 per week subscription. The #tv_grab_au_reg grabber will work with both IceTV and OzTiVo data sources.
OzTiVo
Alternatively, for those who want guide data for free, there is a source of data which is freely and openly available and designed to be downloaded for PVRs. This is provided by the nice folks at OzTiVo and was originally designed for Aussie owners of TiVo boxes. However, because it outputs guide data in XMLTV format it can also be used by MythTV users. In order to use this you will need to create a free account on their web site first. Users who go this route are requested to follow the tvguide.org.au mailing list to stay on top of any changes to the service and help with corrections. The #tv_grab_au_reg grabber will work with both IceTV and OzTiVo data sources, while #xmlguide.pl will only work for OzTiVo.
EPGStream
http://epgstream.net/ provides a simple source of data (one http request). Mainly designed for MediaCenter users, they also have an xmltv interface. I wrote a script to download the xmltv to a temporary file, then upload it to the database using the mythfilldatabase --file option. It's online here as EPGStreamUpdateTvGuide.
EIT Scanning
As of January 1st 2008 all commercial free to air networks will broadcast EPG data for up to 7 days (No minimum specified). Previously the only broadcasted data was for 'now' and 'next'. Therefore there may not be a need to use any of the grabbers/site scrappers mentioned within this page for free to air TV only.
See this article for more information: Aussie nets agree EPG start,Rapid TV News
Grabbers Of Guide Data
tv_grab_oztivo
Status: Possibly dead (web page no longer loads).
Downloads guide data from OzTiVo
- uses standard XMLTV grabber script options.
- re-writing of XMLTV ids.
- Changing timezones of channels data.
- Filtering out specific channels.
- Direct execution of mythfilldatabase on data.
- Converting OzTiVo categories to DataDirect categories.
- Setting HDTV flag on HD channels.
tv_grab_au_reg
Status: Possibly dead (last updated 2007).
This python script grabs the #Ice TV and #OzTiVo data and does some clever things like:
- store a username/password in a config file.
- re-write XMLTV ids (useful if you're switching from a different data source).
- force the addition of explicit timezones on timestamps (useful for people from some states and pay tv data).
- filter out specific channels entirely.
- grab some channel's data from tvguide.org.au and some from IceTV and merge the result into one XMLTV dataset (tvguide.org.au covers more channels than IceTV).
- gives newbies who are looking for a 'tv_grab_au' script something to get hold of.
- use standard XMLTV grabber script options.
You can find the script plus some other useful info about Aussie grabbers at William Uther's site tv_grab_au_reg.
Shepherd
Status: Working as of February 2014.
Shepherd aims to solve the traditional problems of tv_grab_au scripts by:
- employing multiple TV guide grabbers and data sources and switching between them as necessary; and
- auto-updating to eliminate the need for users to manually address grabber breakages
There are details here: [1]http://svn.whuffy.com/wiki
Shepherd also includes the ability to update the channel icons displayed within mythTV. Examples of the icon-sets that can used are shown here.
Please note that during setup, Shepherd offers the option of auto-configuring MythTV on your behalf. Users should be aware that agreeing will cause Shepherd to register itself directly with the MythTV database, which is not XMLTV-compliant behaviour.
TV Icons
Inspired by FrogBoy's most excellent icon set, a number of icons exist for the free2air channels:
An example of FrogBoy's work follows
Icons are available at:
- http://ash.ms/projects/channels/ FrogBoy and T8
