Difference between revisions of "Australian TV Listings"

From MythTV Official Wiki
Jump to: navigation, search
m (→‎Shepherd)
 
(43 intermediate revisions by 21 users not shown)
Line 1: Line 1:
−
=== Guide Data Copyright In Australia ===
+
{{Sisterproject |project=Wikipedia
 +
|image=Wikipedia-logo-en.png
 +
|text=Wikipedia has an article on:
 +
|link=[[Wikipedia:{{{1|Freeview_(Australia)}}}|{{{1|Freeview_(Australia)}}}]]
 +
}}
  
−
Australia doesn't have a service like Zap2It to provide TV program guide listings. Over the years, several people have written grabbers to scrape the data from web sites such as Yahoo, Nine-MSN, YourTV etc. These grabbers have frequently stopped working because of efforts from companies such as Nine-MSN to obfuscate the data. Each time, the workaround is trivial, but the continual changes are draining on the volunteers maintaining the grabbers.
+
== Electronic Service Guide Information ==
  
−
It is unknown what purpose these companies have in trying to prevent their TV guides being used by their audience, possibly some misguided sense of copyright.
+
Currently the best source of guide data is the [[EIT]] content being broadcast by the Free TV networks over DVB-T.
  
−
== Sources Of Guide Data ==
+
=== Free TV Australia Operational Practice OP-44 ===
 +
[http://www.freetv.com.au/media/Engineering/OP44%20_Implementation_of_Electronic_Service_Guide_Information_%20Issue_3_September_2008.pdf  Implementation Guide for the DVB Event Information Table (EITp/f)]  provides good with interoperability with MythTV.
  
−
There are a number of sources of data available with different grabbers.  One list is [http://www.cse.unsw.edu.au/~willu/xmltv/ maintained here].
+
The status of the networks compliance is currently as follows: (Information based on Perth data)
  
−
Also see [[XMLTV#Australia]].
+
{| border=1 cellspacing=0 cellpadding=5
 +
| Network
 +
| event_name_char (Title)
 +
| text_char (Subtitle)
 +
| extended_event_descriptor (Description)
 +
| content_descriptor (Categories)
 +
|-
 +
| ABC
 +
| good
 +
| contains description, gets truncated sometimes
 +
| unused
 +
| good
 +
|-
 +
| Seven
 +
| good
 +
| contains description, rpt info, closed captions info
 +
| unused
 +
| good
 +
|-
 +
| Nine
 +
| good
 +
| generally ok, contains extra flags (live, rpt, rating) at start, "Movie" if movie, sometimes a (truncated) duplicate of description
 +
| ok, sometimes contains cast
 +
| good
 +
|-
 +
| Ten
 +
| ok (live events prefixed with LIVE:)
 +
| contains description, some programs have just generic series description
 +
| copyright declaration
 +
| good
 +
|-
 +
| SBS
 +
| good
 +
| contains description
 +
| unused
 +
| good
 +
|}
  
−
=== Nine-MSN, Yahoo And Other Site Scrappers ===
+
=== Guide Data Copyright In Australia ===
  
−
These sites have been the most detailed and accurate source of data but constantly break.
+
Australia doesn't have a service like Zap2It to provide TV program guide listings. Over the years, several people have written grabbers to scrape the data from web sites such as Yahoo, Nine-MSN, YourTV etc. These grabbers have frequently stopped working because of efforts from companies such as Nine-MSN to obfuscate the data. Each time, the workaround is trivial, but the continual changes are draining on the volunteers maintaining the grabbers.
  
−
=== Ice TV ===
+
It is unknown what purpose these companies have in trying to prevent their TV guides being used by their audience, possibly some misguided sense of copyright.
−
You can purchase the guide data from [http://www.icetv.com.au Ice TV] with a AU$3 per week subscription. The [[#tv_grab_au_reg]] grabber will work with both IceTV and OzTiVo data sources.
 
  
−
=== OzTiVo ===
+
==== Background: ====
−
Alternatively, for those who want guide data for free, there is a source of data which is freely and openly available and designed to be downloaded for PVRs. This is provided by the nice folks at [http://www.tvguide.org.au OzTiVo] and was originally designed for Aussie owners of TiVo boxes. However, because it outputs guide data in XMLTV format it can also be used by MythTV users. In order to use this you will need to create a free account on their web site first. Users who go this route are requested to follow the tvguide.org.au mailing list to stay on top of any changes to the service and help with corrections. The [[#tv_grab_au_reg]] grabber will work with both IceTV and OzTiVo data sources, while [[#xmlguide.pl]] will only work for OzTiVo.
 
  
−
== Grabbers Of Guide Data ==
+
===== Nine loses electronic program guide case =====
  
−
=== Site Scrapers ===
+
http://www.smh.com.au/news/home-theatre/nine-loses-program-guide-case/2007/08/09/1186530493565.html
  
−
http://www.whuffy.com/tv_grab_au/
+
Also loses High Court Appeal (22/04/09):
  
−
(10 Sept 2006) Rex 1.x grabber is broken due to changes in the web site.
+
http://www.goodgearguide.com.au/article/300251/icetv_wins_landmark_high_court_decision_versus_nine_network
  
−
(17 Sept 2006) Rex 1.x is broken but there is a later version at the link above which does work.
+
===== Free TV networks to broadcast industry electronic program guide =====
  
−
=== tv_grab_au_reg ===
+
http://www.freetv.com.au/media/News-Media_Release/2007-0304_Free_TV_Networks_to_Broadcast_Industry_Electronic_Program_Guide_141107.pdf
−
This python script grabs the [[#Ice TV]] and [[#OzTiVo]] data and does some clever things like:
 
  
−
* store a username/password in a config file.
+
== Sources Of Guide Data ==
−
* re-write XMLTV ids (useful if you're switching from a different data source).
 
−
* force the addition of explicit timezones on timestamps (useful for people from some states and pay tv data).
 
−
* filter out specific channels entirely.
 
−
* grab some channel's data from tvguide.org.au and some from IceTV and merge the result into one XMLTV dataset (tvguide.org.au covers more channels than IceTV).
 
−
* gives newbies who are looking for a 'tv_grab_au' script something to get hold of.
 
−
* use standard XMLTV grabber script options.
 
  
−
You can find the script plus some other useful info about Aussie grabbers at William Uther's site [http://www.cse.unsw.edu.au/~willu/xmltv/tv_grab_au_reg.html tv_grab_au_reg].
+
There are a number of sources of data available with different grabbers.  An old list is [http://www.cse.unsw.edu.au/~willu/xmltv/ maintained here].
  
−
=== xmlguide.pl ===
+
Also see [[XMLTV#Australia]].
−
An alternative is to use the xmlguide.pl cgi script on the [http://www.tvguide.org.au OzTiVo] web site.  This maybe a little more complicated setting up but hopefully if the API changes you should be covered automatically with no need to make any updates. Users who go this route are requested to follow the tvguide.org.au mailing list to stay on top of any changes to the service and help with corrections.
 
  
−
When accessed, the server script queries the preferences associated with your username and password to decide which channels to download. It then extracts up to 7 days worth of data for those channels into XMLTV format which you can then load into MythTV using mythfilldatabase. Full details available at [http://minnie.tuhs.org/twiki/bin/view/TVGuide/XMLGuideAPI XMLGuideAPI].
+
=== Nine-MSN, Yahoo And Other Site Scrappers ===
  
−
==== How To ====
+
These sites have been the most detailed and accurate source of data but constantly break.
  
−
Create an account on [http://www.tvguide.org.au OzTiVo].
+
=== Ice TV ===
 +
You can purchase the guide data from [http://www.icetv.com.au Ice TV] with a AU$7.99 per month subscription. The [[#tv_grab_au_reg]] grabber will work with both IceTV and OzTiVo data sources.
  
−
Create a file /home/mythtv/tv_grab_au_tvg.conf with one line like this where UserName and Password are the username and password for your tvguide.org.au account:
+
Note: While their guide data is very good, IceTV do not support MythTV for most of their advertised features. This includes "Free Apps for Remote Recording", "Smart Series Recording" and others.
  
−
<pre>
+
=== OzTiVo ===
−
http://UserName:Password@minnie.tuhs.org/tivo-bin/xmlguide.pl
+
Alternatively, for those who want guide data for free, there is a source of data which is freely and openly available and designed to be downloaded for PVRs. This is provided by the nice folks at [http://www.oztivo.net/ OzTiVo] and was originally designed for Aussie owners of TiVo boxes. However, because it outputs guide data in XMLTV format it can also be used by MythTV users. In order to use this you will need to create a free account on their web site first. Users who go this route are requested to follow the tvguide.org.au mailing list to stay on top of any changes to the service and help with corrections. The [[#tv_grab_au_reg]] grabber will work with both IceTV and OzTiVo data sources, while [[#xmlguide.pl]] will only work for OzTiVo.
−
</pre>
 
  
−
Make this file rw------- by chmod 600 so that other users can't access your account logon details. The url for wget is specified in the --input-file param to hide your OzTiVo username and password from process viewers (eg. ps).
+
=== EPGStream ===
 +
http://epgstream.net/ provides a simple source of data (one http request). Mainly designed for MediaCenter users, they also have an xmltv interface. I wrote a script to download the xmltv to a temporary file, then upload it to the database using the mythfilldatabase --file option. It's online here as [[EPGStreamUpdateTvGuide]].
  
−
To load the guide data into the MythTV database with these two commands or create a file /usr/local/bin/mfd with:
+
===EIT Scanning===
−
<pre>
+
As of January 1st 2008 all commercial free to air networks will broadcast EPG data for up to 7 days (No minimum specified).  Previously the only broadcasted data was for 'now' and 'next'. Therefore there may not be a need to use any of the grabbers/site scrappers mentioned within this page for free to air TV only.
−
wget --header="accept-encoding: gzip" --input-file=/home/mythtv/tv_grab_au_tvg.conf --output-document=- | gzip -d > /tmp/xmltv.xml
 
−
mythfilldatabase --file 1 7 /tmp/xmltv.xml
 
−
</pre>
 
  
−
You can change the output file name to suit after the ">".
+
See this article for more information: [http://www.rapidtvnews.com/default.asp?sourceid=&smenu=1&twindow=Default&mad=No&sdetail=2360&wpage=&skeyword=&sidate=&ccat=&ccatm=&restate=&restatus=&reoption=&retype=&repmin=&repmax=&rebed=&rebath=&s  Aussie nets agree EPG start,Rapid TV News]
  
−
An easy way to run this automatically is to run this from a daily cron job. That can be set up either in the crontab for a particular user, /etc/crontab, /etc/cron.d/ or /etc/cron.daily/. Whichever is best may depend on your linux distribution. One thing to note when running it from cron is that you should put a randomiser into the call so that it doesn't do it at exactly the same time each day. For example on [http://www.debian.org Debian] create a file /etc/cron.d/mfd with
+
== Grabbers Of Guide Data ==
  
−
<pre>
+
=== Shepherd ===
−
0 1 * * * sleep $((RANDOM % 18000)) && /usr/local/bin/mfd
+
Status: Working as of October 2020.
−
</pre>
 
  
−
This helps to reduce the peak loads put on the OzTiVo servers by running the script sometime between 1am and 6am every day.  Please note Vixie cron requires a '\' before '%' like this 'RANDOM \% 18000'.
+
Shepherd aims to solve the traditional problems of tv_grab_au scripts by:
 +
# employing multiple TV guide grabbers and data sources and switching between them as necessary; and
 +
# auto-updating to eliminate the need for users to manually address grabber breakages
  
−
The XMLTVIDs in this data probably won't match what you have in your MythTV channel setup, so there are a couple of options to get it to work.  
+
There are details here: [https://github.com/ShephedProject]https://github.com/ShephedProject. There is also a Google Groups mailing list at [https://groups.google.com/g/shepherd-list]https://groups.google.com/g/shepherd-list
  
−
* Change the XMLTV IDs of your channels in Myth itself.
+
Shepherd also includes the ability to update the channel icons displayed within mythTV.  Examples of the icon-sets that can used are shown [http://svn.whuffy.com/index.fcgi/wiki/channel_icons here].
−
* Use sed to translate XMLTVIDs
 
−
* Use xslt to translate XMLTVIDs.
 
  
−
===== Use sed to translate XMLTVIDs =====
+
Please note that during setup, Shepherd offers the option of auto-configuring MythTV on your behalf. Users should be aware that agreeing will cause Shepherd to register itself directly with the MythTV database, which is not XMLTV-compliant behaviour.
−
Change the /usr/local/bin/mfd file with the wget/sed commands all on one line.
 
−
Make changes for your XMLTV IDs as required.
 
  
−
<pre>
+
=== tv_grab_oztivo ===
−
wget --header="accept-encoding: gzip" --input-file=/home/mythtv/tv_grab_au_tvg.conf
+
Status: Possibly dead (web page no longer loads).
−
--output-document=- | gzip -d | sed 's/channel="SBS-NEWS"/channel="news\.sbs\.com\.au"/g ; s/channel="Seven-Syd"/channel="sydney\.seven\.com\.au"/g ;
 
−
s/channel="Ten-NSW"/channel="sydney\.ten\.com\.au"/g ;
 
−
s/channel="SBS-NSW"/channel="sydney\.sbs\.com\.au"/g ;
 
−
s/channel="ABC2"/channel="abc2\.abc\.gov\.au"/g ;
 
−
s/channel="Nine-Syd"/channel="sydney\.nine\.com\.au"/g ;
 
−
s/channel="ABC-NSW"/channel="nsw\.abc\.gov\.au"/g' > /tmp/xmltv.xml
 
−
mythfilldatabase --file 1 7 /tmp/xmltv.xml
 
−
</pre>
 
  
−
===== Use xslt to translate XMLTVIDs =====
+
Downloads guide data from [http://www.oztivo.net/ OzTiVo]
−
The trouble with the sed method is that you <u>could</u> substitute a string in a program description or elsewhere, rather than just in the channel ID. So to avoid that you can use xslt (extensible stylesheet language transformation) to ensure you only substitute that string when it's part of a channel setting.
+
* uses standard XMLTV grabber script options.
 +
* re-writing of XMLTV ids.
 +
* Changing timezones of channels data.
 +
* Filtering out specific channels.
 +
* Direct execution of mythfilldatabase on data.
 +
* Converting OzTiVo categories to DataDirect categories.
 +
* Setting HDTV flag on HD channels.
 +
[http://web.aanet.com.au/auric/?q=node/14 Authors site]
  
−
Make sure you have the xsltproc binary. (On Fedora Core this is provided by the libxslt rpm).
+
=== tv_grab_au_reg ===
 +
Status: Possibly dead (last updated 2007).
  
−
Change the /usr/local/bin/mfd file with the wget/xsltproc commands all on one line.
+
This python script grabs the [[#Ice TV]] and [[#OzTiVo]] data and does some clever things like:
  
−
<pre>
+
* store a username/password in a config file.
−
wget --header="accept-encoding: gzip" --input-file=/home/mythtv/tv_grab_au_tvg.conf
+
* re-write XMLTV ids (useful if you're switching from a different data source).
−
--output-document=- | gzip -d | xsltproc /home/mythtv/tv_grab_au_tvg.xsl - > /tmp/xmltv.xml
+
* force the addition of explicit timezones on timestamps (useful for people from some states and pay tv data).
−
mythfilldatabase --file 1 7 /tmp/xmltv.xml
+
* filter out specific channels entirely.
−
</pre>
+
* grab some channel's data from tvguide.org.au and some from IceTV and merge the result into one XMLTV dataset (tvguide.org.au covers more channels than IceTV).
 +
* gives newbies who are looking for a 'tv_grab_au' script something to get hold of.
 +
* use standard XMLTV grabber script options.
  
−
The file /home/mythtv/tv_grab_au_tvg.xsl looks like this (thanks go to Joshua King for providing this). If it all looks a bit scary to you, don't worry - all you have to do is modify the channel mappings near the beginning. (Note that this also has some untested code in it to adjust the time offset by 8 hours but the code is commented out).
+
You can find the script plus some other useful info about Aussie grabbers at William Uther's site [http://www.cse.unsw.edu.au/~willu/xmltv/tv_grab_au_reg.html tv_grab_au_reg].
−
<pre>
 
−
<?xml version="1.0" encoding="UTF-8"?>
 
−
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
 
−
<xsl:output method="xml" indent="yes"/>
 
−
 
 
−
<xsl:template name="mod-channel">
 
−
    <xsl:choose>
 
−
        <!-- Add channel mappings here -->
 
−
        <xsl:when test="string()='ABC-NSW'">nsw.abc.gov.au</xsl:when>
 
−
        <xsl:when test="string()='ABC2'">abc2.abc.gov.au</xsl:when>
 
−
        <xsl:when test="string()='Seven-Syd'">sydney.seven.com.au</xsl:when>
 
−
        <xsl:when test="string()='Nine-Syd'">sydney.nine.com.au</xsl:when>
 
−
        <xsl:when test="string()='Ten-NSW'">sydney.ten.com.au</xsl:when>
 
−
        <xsl:when test="string()='SBS-NSW'">sydney.sbs.com.au</xsl:when>
 
−
        <xsl:when test="string()='SBS-NEWS'">news.sbs.com.au</xsl:when>
 
−
 
 
−
        <!-- This ensures any unmatched channels don't become empty -->
 
−
        <xsl:otherwise><xsl:value-of select="." /></xsl:otherwise>
 
−
    </xsl:choose>
 
−
</xsl:template>
 
−
 
 
−
<xsl:template name="add-timezone">
 
−
    <!-- This emulates force-explicit-timezone -->
 
−
    <xsl:value-of select="concat(.,' +0000')" />
 
−
</xsl:template>
 
−
 
 
−
<xsl:template name="mod-timezone">
 
−
    <!-- Change timezone in file by + 8 hours 2 etc lines down -->
 
−
    <xsl:choose>
 
−
        <xsl:when test="substring(.,9,2) + 8 &gt;= 24">
 
−
            <!-- Need to increment day -->
 
−
            <xsl:choose>
 
−
                <xsl:when test="substring(.,7,2)=31 and substring(.,5,2)=12">
 
−
                    <!-- Need to increment year -->
 
−
                    <xsl:value-of select="substring(.,1,4) + 1" />0101<xsl:if test="substring(.,9,2) + 8 mod 24 &lt; 10">0</xsl:if><xsl:value-of select="substring(.,9,2) + 8 mod 24" /><xsl:value-of select="substring(.,11)" /><xsl:text> +0800</xsl:text>
 
−
                </xsl:when>
 
−
                <xsl:when test="(substring(.,7,2)=31 and (substring(.,5,2)=1 or substring(.,5,2)=3 or substring(.,5,2)=5 or substring(.,5,2)=7 or substring(.,5,2)=8 or substring(.,5,2)=10)) or (substring(.,7,2)=30 and (substring(.,5,2)=4 or substring(.,5,2)=6 or substring(.,5,2)=9 or substring(.,5,2)=11))">
 
−
                    <!-- Need to increment month -->
 
−
                    <xsl:value-of select="substring(.,1,4)" /><xsl:if test="substring(.,5,2) + 1 &lt; 10">0</xsl:if><xsl:value-of select="substring(.,5,2) + 1" />01<xsl:if test="substring(.,9,2) + 8 mod 24 &lt; 10">0</xsl:if><xsl:value-of select="substring(.,9,2) + 8 mod 24" /><xsl:value-of select="substring(.,11)" /><xsl:text> +0800</xsl:text>
 
−
                </xsl:when>
 
−
                <xsl:when test="(substring(.,7,2)=29 and (substring(.,5,2)=2)) or (substring(.,7,2)=28 and substring(.,5,2)=2 and (substring(.,1,4) mod 4!=0 or substring(.,1,4) mod 400=0))">
 
−
                    <!-- Need to increment month -->
 
−
                    <xsl:value-of select="substring(.,1,4)" />0301<xsl:if test="substring(.,9,2) + 8 mod 24 &lt; 10">0</xsl:if><xsl:value-of select="substring(.,9,2) + 8 mod 24" /><xsl:value-of select="substring(.,11)" /><xsl:text> +0800</xsl:text>
 
−
                </xsl:when>
 
−
                <xsl:otherwise>
 
−
                    <!-- Nothing to do but day and add -->
 
−
                    <xsl:value-of select="substring(.,1,6)" /><xsl:if test="substring(.,7,2) + 1 &lt; 10">0</xsl:if><xsl:value-of select="substring(.,7,2) + 1"/><xsl:if test="substring(.,9,2) + 8 mod 24 &lt; 10">0</xsl:if><xsl:value-of select="substring(.,9,2) + 8 mod 24" /><xsl:value-of select="substring(.,11)" /><xsl:text> +0800</xsl:text>
 
−
                </xsl:otherwise>
 
−
                    <xsl:value-of select="substring(.,1,6)" /><xsl:if test="substring(.,7,2) + 1 &lt; 10">0</xsl:if><xsl:value-of select="substring(.,7,2) + 1"/><xsl:if test="substring(.,9,2) + 8 mod 24 &lt; 10">0</xsl:if><xsl:value-of select="substring(.,9,2) + 8 mod 24" /><xsl:value-of select="substring(.,11)" /><xsl:text> +0800</xsl:text>
 
−
                </xsl:otherwise>
 
−
            </xsl:choose>
 
−
        </xsl:when>
 
−
        <xsl:otherwise>
 
−
            <!-- Nothing to do but add -->
 
−
            <xsl:value-of select="substring(.,1,8)" /><xsl:if test="substring(.,9,2) + 8 &lt; 10">0</xsl:if><xsl:value-of select="substring(.,9,2) + 8" /><xsl:value-of select="substring(.,11)" /><xsl:text> +0800</xsl:text>
 
−
        </xsl:otherwise>
 
−
  </xsl:choose>
 
−
</xsl:template>
 
−
 
 
−
<xsl:template name="copy">
 
−
    <xsl:element name="{name()}">
 
−
    <xsl:for-each select="@*">
 
−
        <xsl:attribute name="{name()}">
 
−
        <xsl:choose>
 
−
            <xsl:when test="name(..)='programme' and name()='channel'"><xsl:call-template name="mod-channel" /></xsl:when>
 
−
            <xsl:when test="name(..)='channel' and name()='id'"><xsl:call-template name="mod-channel" /></xsl:when>
 
−
 
 
−
<!-- Uncomment here to try out the time filters
 
−
            <xsl:when test="name(..)='programme' and name()='start'"><xsl:call-template name="mod-timezone" /></xsl:when>
 
−
            <xsl:when test="name(..)='programme' and name()='stop'"><xsl:call-template name="mod-timezone" /></xsl:when>
 
−
-->
 
−
            <xsl:otherwise><xsl:value-of select="." /></xsl:otherwise>
 
−
        </xsl:choose>
 
−
        </xsl:attribute>
 
−
    </xsl:for-each>
 
−
    <xsl:for-each select="*|text()">
 
−
        <xsl:choose>
 
−
            <xsl:when test="name()">
 
−
                <xsl:call-template name="copy" />
 
−
            </xsl:when>
 
−
            <xsl:otherwise>
 
−
                <xsl:value-of select="string()" />
 
−
            </xsl:otherwise>
 
−
        </xsl:choose>
 
−
    </xsl:for-each>
 
−
</xsl:element>
 
−
</xsl:template>
 
−
 
 
−
<xsl:template match="/">
 
−
    <xsl:for-each select="*">
 
−
        <xsl:call-template name="copy" />
 
−
    </xsl:for-each>
 
−
</xsl:template>
 
−
</xsl:stylesheet>
 
−
</pre>
 
−
 
 
−
== Running Automatically ==
 
−
 
 
−
Put an entry in your crontab something like this:
 
−
 
 
−
<pre>
 
−
0 1 * * * sleep $((RANDOM % 18000)) && <whatever the download command is>
 
−
</pre>
 
−
 
 
−
This will ensure that your grabber runs sometime between 1am and 7am. The reason for doing it like this is because if everyone does it this way the load on the servers will get smoothed out between these hours. So please be a good netizen and do it this way!
 
  
 
== TV Icons ==
 
== TV Icons ==
−
FrogBoy has a set of most excellent icons for use with australian free2air channels:
+
Inspired by FrogBoy's most excellent icon set, a number of icons exist for the free2air channels:
  
 +
An example of FrogBoy's work follows
 
[[Image:Au-nine.png]]
 
[[Image:Au-nine.png]]
 
[[Image:Au-sbs-hd.png]]
 
[[Image:Au-sbs-hd.png]]
  
−
They can be downloaded from:
+
Icons are available at:
−
SD: http://home.exetel.com.au/chorton/Files/mythtv/Australia_TV_Icons.tar
+
* http://ash.ms/projects/channels/ FrogBoy and T8
−
HD ('Note: currently missing channel 9 icon'): http://home.exetel.com.au/chorton/Files/mythtv/Australia_TV_Icons_HD.tar
 

Latest revision as of 03:34, 6 October 2020

Wikipedia-logo-en.png
Wikipedia has an article on:

Electronic Service Guide Information

Currently the best source of guide data is the EIT content being broadcast by the Free TV networks over DVB-T.

Free TV Australia Operational Practice OP-44

Implementation Guide for the DVB Event Information Table (EITp/f) provides good with interoperability with MythTV.

The status of the networks compliance is currently as follows: (Information based on Perth data)

Network event_name_char (Title) text_char (Subtitle) extended_event_descriptor (Description) content_descriptor (Categories)
ABC good contains description, gets truncated sometimes unused good
Seven good contains description, rpt info, closed captions info unused good
Nine good generally ok, contains extra flags (live, rpt, rating) at start, "Movie" if movie, sometimes a (truncated) duplicate of description ok, sometimes contains cast good
Ten ok (live events prefixed with LIVE:) contains description, some programs have just generic series description copyright declaration good
SBS good contains description unused good

Guide Data Copyright In Australia

Australia doesn't have a service like Zap2It to provide TV program guide listings. Over the years, several people have written grabbers to scrape the data from web sites such as Yahoo, Nine-MSN, YourTV etc. These grabbers have frequently stopped working because of efforts from companies such as Nine-MSN to obfuscate the data. Each time, the workaround is trivial, but the continual changes are draining on the volunteers maintaining the grabbers.

It is unknown what purpose these companies have in trying to prevent their TV guides being used by their audience, possibly some misguided sense of copyright.

Background:

Nine loses electronic program guide case

http://www.smh.com.au/news/home-theatre/nine-loses-program-guide-case/2007/08/09/1186530493565.html

Also loses High Court Appeal (22/04/09):

http://www.goodgearguide.com.au/article/300251/icetv_wins_landmark_high_court_decision_versus_nine_network

Free TV networks to broadcast industry electronic program guide

http://www.freetv.com.au/media/News-Media_Release/2007-0304_Free_TV_Networks_to_Broadcast_Industry_Electronic_Program_Guide_141107.pdf

Sources Of Guide Data

There are a number of sources of data available with different grabbers. An old list is maintained here.

Also see XMLTV#Australia.

Nine-MSN, Yahoo And Other Site Scrappers

These sites have been the most detailed and accurate source of data but constantly break.

Ice TV

You can purchase the guide data from Ice TV with a AU$7.99 per month subscription. The #tv_grab_au_reg grabber will work with both IceTV and OzTiVo data sources.

Note: While their guide data is very good, IceTV do not support MythTV for most of their advertised features. This includes "Free Apps for Remote Recording", "Smart Series Recording" and others.

OzTiVo

Alternatively, for those who want guide data for free, there is a source of data which is freely and openly available and designed to be downloaded for PVRs. This is provided by the nice folks at OzTiVo and was originally designed for Aussie owners of TiVo boxes. However, because it outputs guide data in XMLTV format it can also be used by MythTV users. In order to use this you will need to create a free account on their web site first. Users who go this route are requested to follow the tvguide.org.au mailing list to stay on top of any changes to the service and help with corrections. The #tv_grab_au_reg grabber will work with both IceTV and OzTiVo data sources, while #xmlguide.pl will only work for OzTiVo.

EPGStream

http://epgstream.net/ provides a simple source of data (one http request). Mainly designed for MediaCenter users, they also have an xmltv interface. I wrote a script to download the xmltv to a temporary file, then upload it to the database using the mythfilldatabase --file option. It's online here as EPGStreamUpdateTvGuide.

EIT Scanning

As of January 1st 2008 all commercial free to air networks will broadcast EPG data for up to 7 days (No minimum specified). Previously the only broadcasted data was for 'now' and 'next'. Therefore there may not be a need to use any of the grabbers/site scrappers mentioned within this page for free to air TV only.

See this article for more information: Aussie nets agree EPG start,Rapid TV News

Grabbers Of Guide Data

Shepherd

Status: Working as of October 2020.

Shepherd aims to solve the traditional problems of tv_grab_au scripts by:

  1. employing multiple TV guide grabbers and data sources and switching between them as necessary; and
  2. auto-updating to eliminate the need for users to manually address grabber breakages

There are details here: [1]https://github.com/ShephedProject. There is also a Google Groups mailing list at [2]https://groups.google.com/g/shepherd-list

Shepherd also includes the ability to update the channel icons displayed within mythTV. Examples of the icon-sets that can used are shown here.

Please note that during setup, Shepherd offers the option of auto-configuring MythTV on your behalf. Users should be aware that agreeing will cause Shepherd to register itself directly with the MythTV database, which is not XMLTV-compliant behaviour.

tv_grab_oztivo

Status: Possibly dead (web page no longer loads).

Downloads guide data from OzTiVo

  • uses standard XMLTV grabber script options.
  • re-writing of XMLTV ids.
  • Changing timezones of channels data.
  • Filtering out specific channels.
  • Direct execution of mythfilldatabase on data.
  • Converting OzTiVo categories to DataDirect categories.
  • Setting HDTV flag on HD channels.

Authors site

tv_grab_au_reg

Status: Possibly dead (last updated 2007).

This python script grabs the #Ice TV and #OzTiVo data and does some clever things like:

  • store a username/password in a config file.
  • re-write XMLTV ids (useful if you're switching from a different data source).
  • force the addition of explicit timezones on timestamps (useful for people from some states and pay tv data).
  • filter out specific channels entirely.
  • grab some channel's data from tvguide.org.au and some from IceTV and merge the result into one XMLTV dataset (tvguide.org.au covers more channels than IceTV).
  • gives newbies who are looking for a 'tv_grab_au' script something to get hold of.
  • use standard XMLTV grabber script options.

You can find the script plus some other useful info about Aussie grabbers at William Uther's site tv_grab_au_reg.

TV Icons

Inspired by FrogBoy's most excellent icon set, a number of icons exist for the free2air channels:

An example of FrogBoy's work follows Au-nine.png Au-sbs-hd.png

Icons are available at:

Retrieved from "http://www.mythtv.org/wiki?title=Australian_TV_Listings&oldid=65532"