Difference between revisions of "XMLTV"

From MythTV Official Wiki
Jump to: navigation, search
(Add instructions for upgrading from DD to XMLTV)
(201 intermediate revisions by 65 users not shown)
Line 1: Line 1:
Setting up XmlTV for use with MythTV is documented in the
+
{{User Manual TOC}}
[http://www.mythtv.org/docs/mythtv-HOWTO-5.html#ss5.3 MythTV HOWTO 5.3].
 
  
The purpose of this page is to cover specific issues that aren't necessarily covered in the official documentation. 
+
==Introduction==
  
== [[Data Direct]] ==
+
XMLTV is the name of a system which obtains and processes program guide information. If you are in North America you can use the [[Schedules Direct]] grabber which now supports XMLTV format as well as the old Data Direct format. Using the XMLTV format with Schedules Direct gets you more days of data than the old Data Direct option. All users in the USA and other countries need to install and configure XMLTV in order to obtain program guide listings.
  
With zap2it no longer providing listing information to XMLTV for North America (tv_grab_na), the only option is for the new Data Direct solution.  Supported fully in version 0.15 of MythTV, the 0.14 version of MythTV, for a short time, requires a fix to continue downloading listing information successfully.  More information is available in the [[Data Direct]].
+
==Installation==
 +
Install the xmltv package on your master backend server, using your distribution's package manager or package downloaded from XMLTV.
  
== Outside North America ==
+
The ubuntu packages may have an out of date version of xmltv. You may be able to get a later version as follows:
If you live outside the USA and Canada, there are some other sources of guide information
+
 
 +
Add to /etc/apt/sources.list:
 +
<pre>deb http://ftp.us.debian.org/debian sid main </pre>
 +
Run sudo apt-get update and install xmltv.
 +
After installing it, remove that entry from /etc/apt/sources.list.
 +
 
 +
Info on XMLTV is here: http://xmltv.org/wiki/. There are other download options described there, including a nightly snapshot.
 +
 
 +
The package consists of perl programs. They do not need to be compiled, so if you have installed the package you can download the nightly snapshot and overwrite any out of date member with the new one. A better approach may be to add any new or updated grabbers to a separate directory, which you place on the PATH before /usr/bin. All of the programs in the package are named starting with tv_ and the system will find them as long as they are on the PATH.
 +
 
 +
==Configuration==
 +
Configuration of XMLTV must be done on the system where mythfilldatabase will be run. Normally that will be your primary backend machine, but you can run it in other places if you prefer.
 +
 
 +
==MythTV Setup==
 +
XMLTV can be configured through the Video Sources Setup page in mythtv-setup. depending on the grabber, it will prompt for any information it requires. If it comes down to selecting channels one at a time manually that can be a challenge if there are many channels.
 +
 
 +
Some grabbers may not be able to be fully configured through the MythTV setup interface.
 +
 
 +
Logon or su to the user id that will run mythfilldatabase. In the case of mythbuntu installation that is user id mythtv. Run mythtv-setup from that user id. If you do not want to logon to that id or you cannot, you can run the mythtv-setup from another id on the same backend. After completing setup, copy or link the contents of the $HOME/.xmltv directory to the mythtv home directory. Also copy or link the $HOME/.mythtv/XXXX.xmltv files to the mythtv home directory. If copying or linking the files please make sure that the files and directories are read-write permitted by the mythtv user id.
 +
 
 +
==Command Line Setup==
 +
Additional configuration can be done from the command line. As long as you have selected your xmltv grabber in MythTV Setup you can perform some or all configuration via the command line.
 +
 
 +
Logon to the user that will run mythfilldatabase (normally mythtv). As with mythtv setup if you cannot or will not, you can use any user id and then copy or link directories and files as described above.
 +
 
 +
===List Grabbers===
 +
<pre>tv_find_grabbers</pre>
 +
This gives a listing of grabbers, corresponding to the list you see in MythTV Setup. here is a small sample:
 +
<pre>
 +
/usr/bin/tv_grab_sd_json|Schedules Direct JSON API
 +
/usr/bin/tv_grab_is|Iceland
 +
/usr/bin/tv_grab_es_laguiatv|Spain (laguiatv.com)
 +
/usr/bin/tv_grab_se_swedb|Sweden (swedb/tvsajten)
 +
/usr/bin/tv_grab_combiner|Combine data from several other grabbers
 +
/opt/mythtv/bin/tv_grab_zz_sdjson_sqlite|Multinational (Schedules Direct JSON web services with SQLite DB)
 +
</pre>
 +
Find in the list the grabber you selected in MythTV Setup, and get the name of the executable from the beginning of the line. Find out information about it with the --info option.
 +
<pre>tv_grab_sd_json --info</pre>
 +
That will give information on the usage of that grabber. Most options will be common to all grabbers. To use the --info option you need the perldoc package installed.
 +
 
 +
===Run the configuration===
 +
This is the same process that is run from MythTV Setup when you use the configure button in Video Source Setup.
 +
<pre>tv_grab_xx_xxxxxx --configure --config-file $HOME/.mythtv/YYYY.xmltv</pre>
 +
Use the name of the selected grabber. The value YYYY in the configuration file name corresponds to the Video Source Name. Use your video source name here. If your video source name includes spaces put quotes around it.
 +
 
 +
The grabber will prompt you required information. If your lineup includes a large number of channels and you do not subscribe to all of them, some grabbers will allow you to select the desired channels.
 +
 
 +
You should '''not''' include channels in your xmltv config which are not required. This places an unnecessarily high load on both the source websites and your own system.
 +
 
 +
==Schedules Direct Setup==
 +
[[Schedules Direct]] now supports two formats, Data Direct and XMLTV. In the USA the XMLTV format provides 20 days of data while the Data Direct format provides 13 days of data. Here are detailed steps for setting up to use the XMLTV format.
 +
 
 +
===Schedules Direct Grabbers===
 +
There are two grabbers for schedules direct. If you want to use Schedules Direct with XMLTV you should obtain the latest version, since there have been recent changes. Download the latest nightly snapshot, extract these two files and make sure they are in a directory on the path, ahead of /usr/bin, or else put them in /usr/bin:
 +
<pre>tv_grab_zz_sdjson_sqlite
 +
tv_grab_sd_json</pre>
 +
Run tv_find_grabbers and you should see these included in the list.
 +
<pre>/usr/bin/tv_grab_sd_json|Schedules Direct JSON API
 +
/usr/bin/tv_grab_zz_sdjson_sqlite|Multinational (Schedules Direct JSON web services with SQLite DB)</pre>
 +
You will need to pick one of these. The tv_grab_zz_sdjson_sqlite has more options and needs some command line setup that cannot be done from the mythtv-setup program. Both of them cache data in a cache file or sqllite database. This allows them to download a minimal amount of data when running. Only changes since the previous download are downloaded.
 +
 
 +
===Setup with tv_grab_zz_sdjson_sqlite===
 +
If you want the very latest version of this you can download it from [https://github.com/garybuhrmaster/tv_grab_zz_sdjson_sqlite github].
 +
 +
Run MythTV Setup and select ''Multinational (Schedules Direct JSON web services with SQLite DB)'' for your listings source.
 +
 
 +
From the command line
 +
<pre>tv_grab_zz_sdjson_sqlite --manage-lineups --config-file $HOME/.mythtv/SD.xmltv</pre>
 +
Replace SD with the name of the source you had set up in MythTV Setup.
 +
* Enter your Schedules Direct user and password when prompted.
 +
* Select option 4 ''Initialize/update the local database''.
 +
* Accept the default database name.
 +
* Accept the password hash question.
 +
* Add or delete lineups using the prompts.
 +
Do not go into channel selection yet. Run this:
 +
<pre>tv_grab_zz_sdjson_sqlite --configure --config-file $HOME/.mythtv/SD.xmltv</pre>
 +
* Supply the database name you selected above.
 +
* Accept defaults for the prompted questions.
 +
Run these:
 +
<pre>tv_grab_zz_sdjson_sqlite --days 0 --config-file $HOME/.mythtv/SD.xmltv
 +
tv_grab_zz_sdjson_sqlite --manage-lineups --config-file $HOME/.mythtv/SD.xmltv</pre>
 +
* If you get all channels in the lineup, then select the options that will make current and future channels selected.
 +
* If you only get a few of the channels you will need to select the appropriate channels. Select a default for existing and future channels, then use option 5 to select which channels you need. This goes though all channels one at a time. You have to complete this for every channel in the list. You cannot stop partway through.
 +
* If you need to do the channel selection again because of errors or channel changes, your previous values are remembered and used as defaults.
 +
 
 +
====Channel selection shortcut====
 +
The channel selection process with tv_grab_zz_sdjson_sqlite is tedious and error prone.
 +
An alternative method is as follows. First configure the grabber to select no channels.
 +
Install the perl database shell utility. I recommend you install this on one user id as a private library, and preferably do not use the mythtv user id.
 +
<pre>cpan App::cpanminus
 +
cpanm DBI::Shell</pre>
 +
List the tables in the sqllite database. If you did not use the default database name, enter the one that you used.  You may have to specify your database name with /home/mythtv instead of $HOME. In that case first make sure the file is writeable by your login user id.
 +
<pre>dbish dbi:SQLite:$HOME/.xmltv/SchedulesDirect.DB
 +
SELECT name FROM sqlite_master WHERE type='table';</pre>
 +
List the channels.
 +
<pre>dbish dbi:SQLite:$HOME/.xmltv/SchedulesDirect.DB
 +
select * from channels;</pre>
 +
You can copy the results from the screen, or pipe the results to a file. The value for ''selected'' is 0 if unselected or 1 if selected. Edit the file and produce an xml file like this to set the ones you want to select:
 +
<pre>update channels set selected = 1 where channum = 2;
 +
update channels set selected = 1 where channum = 3;
 +
...
 +
update channels set selected = 1 where channum = 983;
 +
update channels set selected = 1 where channum = 986;</pre>
 +
Update them with a command like this:
 +
<pre>dbish dbi:SQLite:$HOME/.xmltv/SchedulesDirect.DB < selected.sql</pre>
 +
 
 +
====Converting from DataDirect to XMLTV====
 +
There are several reasons to convert your Datadirect setup to use XMLTV.
 +
* You get three weeks of data instead of two.
 +
* You get icons for all channels.
 +
* Season and episode information are included in listings.
 +
 
 +
If you decide to do this, first complete the setup of tv_grab_zz_sdjson_sqlite described above.
 +
 
 +
When you switch, the xmltvid's for the existing channels will change. Using this grabber the xmltvid has the format <code>I11460.json.schedulesdirect.org</code>. Using DD the xmltvid's are a numeric value.
 +
 
 +
There are two options for the conversion:
 +
# Delete all your channels and let mythfilldatabase add them back. This option is not available in cases where you have to scan for channels using DVB or ATSC cards.
 +
# Manually update each channel xmltvid.
 +
 
 +
I recommend that you backup your database and make sure there are no recordings scheduled for the near future before starting.
 +
 
 +
=====Deleting and re-adding channels=====
 +
 
 +
This is only possible for card types that do not need scanning for channels.
 +
 
 +
Before attempting this bear in mind the following. If any channel's chanid changes, your existing programs will show the wrong channel that the program was recorded on.
 +
 
 +
The mechanisms in MythTV for adding channels use a format of sourceid and channel number to create a chanid wherever possible. So channel 2 on source id 2 will be chanid 2002, for example. If it cannot do that then it will allocate a chanid number which is unrelated to the channel number or source id.
 +
 
 +
In order to see the chanids you will have to look at the channel table in the database. See if your existing chanid numbers match the scheme described in the previous paragraph.
 +
 
 +
If the chanids match the scheme above you should be able to safely delete them and let mythfilldatabase add them.
 +
 
 +
You can delete all of the existing channels for each video source in mythtv-setup, channel editor. Select the Video source and click '''Delete'''.
 +
 
 +
To add all of the channels you have selected as well as listings, run mythfilldatabase. Default options will add channels and listings.
 +
 
 +
=====Manually Updating xmltvids=====
 +
 
 +
If you prefer not to delete your channels or you have a DVB type tuner you will have to update the xmltvids in all of them. Using this grabber the xmltvid has the format <code>Innnnn.json.schedulesdirect.org</code>. You will have to change all of your existing ids to this format. The easiest place to make this change is with mythweb channel editor, although you can use the mythtv-setup channel editor as well. You can get the list of channels with their new format xmltvids by running <code>tv_grab_zz_sdjson_sqlite --days 0 --config-file $HOME/.mythtv/SD.xmltv</code>, which is one step of the setup process (see above). If you did not capture the output you can run this again at any time, no harm will be done.
 +
 
 +
Once you have updated all of the xmltvid's, run mythfilldatabase. Your channel names and icons will be updated to match the values supplied by schedules direct.
 +
 
 +
In a tuner type that does not use scanning, such as a cable card tuner, any channels not found with matching xmltvids are added. If you have missed updating some xmltvids that could result in duplication of the channel.
 +
 
 +
===Setup with tv_grab_sd_json===
 +
Run MythTV Setup and select ''Schedules Direct JSON API'' for your listings source.
 +
 
 +
You can complete the setup using the ''Configure'' button in MythTV Setup. If you prefer to use the command line, run
 +
<pre>tv_grab_sd_json --configure --config-file $HOME/.mythtv/SD.xmltv</pre>
 +
Replace SD with the name of the source you had set up in MythTV Setup.
 +
* Enter your Schedules Direct user and password when prompted.
 +
* For channel id format you can select any of the supplied options. Option 0 is the same format as used by tv_grab_zz_sdjson_sqlite. Option 2 is the format used by Schedules Direct Data Direct.
 +
* Follow the prompts to complete setup.
 +
* When selecting channels the selection process allows you to skip remaining channels in the list and mark them all as yes or no.
 +
* If you need to do the channel selection again because of errors or channel changes, your previous values are lost. You have to do the entire selection again.
 +
 
 +
====Channel selection shortcut====
 +
The channel selection process with tv_grab_sd_json is tedious and error prone.
 +
An alternative method is as follows. First configure the grabber to select all channels.
 +
Edit the file <code>$HOME/.mythtv/SD.xmltv</code>.
 +
Replace SD with the name of the source you had set up in MythTV Setup. You will see a bunch of lines like this.
 +
<pre>channel=74348
 +
channel=11460</pre>
 +
When a channel is selected it has '''=''', when unselected it has '''!'''. You need to set the channels you want to use '''=''' and the others to '''!'''.
 +
 
 +
Get a list of channels by running
 +
<pre>tv_grab_sd_json --days 0 --config-file $HOME/.mythtv/SD.xmltv > list.xml</pre>
 +
Although this requests 0 days it actually produces 1 day of programs. At the start of the file before the programs is the channel listing. Delete the program schedules and keep the channel listing at the top of the file. The channel id and display names identify each channel by its XMLTV id and the number and name. Note that the channel id in this file may have extra characters appended depending on the format you chose during configure. For purpose of the configuration file you only need the 5 numeric digit of the channel id. Using your favorite editor and your provider's channel list update the channel lines in the configuration file to have '''=''' for ones you want and '''!''' for ones you do not.
 +
 
 +
== List of Grabbers ==
 +
{{note_box|Not all grabbers on this page are XMLTV compliant. Only fully compliant grabbers are supported by MythTV and we advise against the use of the rest.
 +
 
 +
If you know a grabber listed here does not conform entirely to both the xmltv grabber specifications ([http://wiki.xmltv.org/index.php/HowtoWriteAGrabber XMLTV grabber spec]) and XMLTV xml format ([http://wiki.xmltv.org/index.php/XMLTVFormat XMLTV Format]) then please mark it as unsupported. If in doubt XMLTV provide a script to validate a grabber ([http://wiki.xmltv.org/index.php/XmltvValidation XMLTV Validator])}}
 +
 
 +
There is also a list on the [http://wiki.xmltv.org/index.php/XMLTVProject XMLTV wiki].
 +
 
 +
=== EPG Collector ===
 +
[http://sourceforge.net/projects/epgcollector/ EPG Collector] is a .NET/Mono program for collecting the transmitted guide data from DVB or ATSC streams in various formats and creating an XMLTV file.
 +
 
 +
=== Multinational ===
 +
[[Schedules Direct]] supports countries in North America, South America, Caribbean and Europe for $25 (US) per year. For a list of supported countries see http://schedulesdirect.org/regions. See [[#Schedules_Direct_Setup]] above
  
 
=== Australia ===
 
=== Australia ===
http://www.mythtv.org/wiki/index.php?title=XmlTv&action=edit&section=2http://membled.com/work/apps/xmltv/
+
Refer to [[Australian TV Listings]] for further details.
# http://www.icetv.com.au (AU$3 per week subscription)
+
 
# http://immir.com/tv_grab_au (Free)
+
# [http://www.oztivo.net/ OzTivo Guide] (Free, requires registration)
# http://www.tvguide.org.au (Free, requires registration?)
+
# http://www.icetv.com.au (AU$99 per year subscription)
# [http://www.cse.unsw.edu.au/~willu/xmltv/index.html Australian XMLTV grabbers] (ice + tvguide)
+
# [http://svn.whuffy.com/wiki Shepherd] (Free: hybrid multi source)
 +
 
 +
See also this (largely historical) [http://www.cse.unsw.edu.au/~willu/xmltv/index.html list of Australian grabbers].
  
 
=== Belgium ===
 
=== Belgium ===
  
# http://users.skynet.be/jxmltv (very fast, written in Java)
+
The following is a list of grabbers for Belgium. It is presented only for historical purposes, because at the time of writing this (January 2011) none of them work.
 +
 
 +
# https://launchpad.net/belguide
 +
# http://users.skynet.be/jxmltv  
 
# http://pytvgrab.sourceforge.net/
 
# http://pytvgrab.sourceforge.net/
# http://membled.com/work/apps/xmltv/
 
  
=== Netherlands ===
+
=== Brazil ===
 +
 
 +
There are a working grabber, tv_grab_br_net a perl scripts (or any other Linux native solutions) for Brazilian TV programming. This grabber get data from Net [http://www.net.tv.br] cable TV. No open to air TV on this source, just paid tv programme. It is detected by mythtvsetup and you are prompted (on background windows) to make your choice about your city, line-up and channels. It provide both, simple and complete description for programs.
 +
 
 +
Other functional alternative is to use [http://www.tvxb.com TVxb], which isn't open source or even Linux, but works well with wine (at least as far as version 1.0085). 
 +
 
 +
TVxb is configurable for different countries via ini-files.  Check the following page for Brazilian templates.  As of today, there are some for NET (PR), and others for TVA as well.
 +
 
 +
http://www.tvxb.com/country/br-ini.html
 +
 
 +
For information on how to set it up with MythTV, check out [[TVxb]].
 +
 
 +
=== Croatia ===
 +
The grabber in the official XMLTV distribution works correctly. More information is available on http://www.gonix.net/.
 +
 
 +
=== Czech Republic ===
 +
 
 +
The XMLTV grabber for "Hungary/Romania" was extended to the Czech Republic.
 +
 
 +
Another way to get TV listings in Czech Republic is [[TVxb]].
 +
 
 +
Not public XMLTV file for czech and slovak channels (UPC channels included) can be send after request to [mailto:rfordinal@gmail.com rfordinal].
 +
 
 +
=== Denmark ===
 +
 
 +
In Denmark there is a group maintaining several grabbers. This group has created the following:
 +
 
 +
* A Wiki located at http://niels.dybdahl.dk/xmltvdk
 +
* A mailing list located at http://uk.groups.yahoo.com/group/xmltvdk very 10 minutes. Only title and descr
 +
* A sourceforge repository located at http://sourceforge.net/projects/xmltvdk
 +
* Another source of info: http://xmltv.se
 +
 
 +
Currently three grabbers are maintained:
 +
 
 +
* DR 2012 grabber written in perl (dr.dk)
 +
* An additional written in python (yousee.dk)
 +
* An universal grabber combining data from mutiple sources (tvtid.tv2.dk, yousee.dk, tv-guiden.dk, jubii.dk, ontv.dk, ahot.dk, dr.dk and swedb)
 +
 
 +
=== Finland ===
 +
The grabbers in the official XMLTV distribution work correctly.
 +
Alternate source: http://xmltv.se
 +
 
 +
=== France ===
 +
 
 +
The grabber in the upcoming XMLTV release works correctly. More information is available on [http://www.kazer.org/].
 +
 
 +
=== Germany/Austria ===
 +
# [http://wiki.xmltv.org/index.php/Europe epgData] offers data on a subscription basis, the grabber from the main xmltv distribution works.
 +
# [http://xmltv.se/ XMLTV.se] offers many of the main channels for personal use, the grabber from the main xmltv distribution works.
 +
# Some channels are available via [http://xmltv.spaetfruehstuecken.org/xmltv/ Egon Zappt] in the upcoming release of xmltv.
 +
# TV Movie can not give out their data without additional license, see [http://www.bucksch.org/1/projects/various/xmltv/ Ben Bucksch's log].
 +
 
 +
=== Greece ===
  
# http://visualization.tudelft.nl/~paul/grabber/
+
Try this link for Nova Greece: [http://sgcpm.com/epg/devilcosta.gz http://sgcpm.com/epg/devilcosta.gz], for example using mythfilldatabase. (Be aware that its just a dump of DVB-EIT using the service_id as channel_id which is not proper xmltv. Please consider writing a proper grabber for the file so it can be configured via the GUI or simply provide a proper guide feed with tools like [http://wiki.xmltv.org/index.php/NonameTV NonameTV].)
# http://www.addictivesoftware.net/index.php?option=content&task=view&id=3&Itemid=28
 
  
=== Sweden ===
+
=== India ===
 +
The grabber for What's-On-India has been added to the official XMLTV distribution.
  
# http://tv.swedb.se/
+
An XMLTV grabber for all channels in India can be found at [http://ravihtpc.wordpress.com/2010/01/09/india-epg-data-for-mythtv-xmltv-listings-grabber/ http://ravihtpc.wordpress.com/2010/01/09/india-epg-data-for-mythtv-xmltv-listings-grabber/].
  
 
=== Italy ===
 
=== Italy ===
# http://www.xiaprojects.com/www/prodotti/tv_grab_it/main.php (Haven't tested it)
+
The grabber in the official XMLTV distribution works correctly.
  
=== Austria/Germany ===
+
=== Hong Kong ===
The XMLTV Data usefull in Austria/ Germany you get with tv_grab_de_tvtoday. It  has the episodes of series all mixed up in he main title. The part Number ('Teil')  of the series are not always nicely structured and spill over to the title and subtitle field. This makes it quite impossible to record every sequence of a series (if you do not want to make a time based recording on a single channel). I decided to write a small mySQL script to update the programm data after performing the daily load of the xmltv file. This mySQL statment is batched after the daily mythfilldatabase entry in my CRON Script. The Statment looks for sequences That have the Word 'Teil' in it. This is the German word for 'Part'.
+
First thanks to http://www.akbkhome.com/blog.php/View/119/Mythtv_xmltv_importer_for_HK.html.
 +
 
 +
I modify his source code to get information from TVB new web site.
 +
 
 +
    download Tv.php http://mythtv-xmltvhk.googlecode.com/files/Tv.php
 +
    download hongkong.ini http://mythtv-xmltvhk.googlecode.com/files/hongkong.ini
 +
    build xml from tvb, atv website using following command, just entry
 +
          php ./Tv.php hongkong.ini > hongkong.xml
 +
    using 'myfilldatabase' to update mythtv schedule program, just entry
 +
          mythfilldatabase --file 1 hongkong.xml
 +
 
 +
    (* it is my first development on php, so it may be simplified *)
 +
[Last Update by hwkit (17/04/2010)]
 +
 
 +
Since TVB web site is changed, the program need to be updated. (My php skill is poor, so there are error when running, however it also can get schedule from TVB web site), I also change the program name called 'xmltvhk.php'. Please download it from google.
 +
     
 +
    download xmltvhk.php and copy it to /etc/xmltvhk (http://mythtv-xmltvhk.googlecode.com/files/xmltvhk.php)
 +
    download updated hongkong.ini and copy it to /etc/xmltvhk (http://mythtv-xmltvhk.googlecode.com/files/hongkong.ini)
 +
    download filltv and copy it to /etc/cron.weekly (make it executable (chmod +x /etc/cron.weekly/filltv) (http://mythtv-xmltvhk.googlecode.com/files/filltv)
 +
 
 +
    Source code
 +
    svn checkout http://mythtv-xmltvhk.googlecode.com/svn/trunk/ mythtv-xmltvhk-read-only
 +
 
 +
[Last Update by hwkit (07/07/2010)]
 +
 
 +
=== Netherlands ===
 +
Netherlands (Holland) specific info can be found here: [[Netherlands]]
 +
 
 +
=== New Zealand ===
 +
Sky TV threatened legal action to get the [http://reven.co.nz/post/whatever-happened-to-xmltvnz xmltv grabber for NZ taken down].
 +
You can still receive the transmitted guide in MHEG5 with tools like [[XMLTV#EPG_Collector|EPG Collector]].
  
 
=== Norway ===
 
=== Norway ===
 +
The grabber in the official XMLTV distribution works correctly.
 +
Another source of info: http://xmltv.se
 +
 +
=== Slovakia ===
 +
The XMLTV grabber for "Hungary/Romania" was extended to Slovakia.
 +
 +
=== South Africa ===
 +
See the [[South Africa MUG]] WIKI page for details.
 +
 +
=== Spain ===
 +
Both grabbers in the official XMLTV distribution work correctly.
 +
 +
=== Sweden ===
 +
The grabber in the official XMLTV distribution works correctly. More information is available on http://xmltv.se.
 +
 +
=== Switzerland ===
 +
The grabber in the official XMLTV distribution works correctly.
 +
 +
=== United Kingdom ===
 +
The grabbers in the official XMLTV distribution work correctly.
 +
A brief [[Uk_xmltv|guide to configuring tv_grab_uk_rt]] is available, and one for [[Atlas|Metabroadcast Atlas]].
 +
 +
=== Ukraine, Russia, Belarus ===
 +
# [http://www.free-x.de/xmltv/ tv_grab_ru] Converts JTV-formatted archive with about 200 channels to XMLTV. But this application does not show additional info about programmes.
 +
# [http://tipok.org.ua/downloads/media/mythtv/tv_grab_ua/tv_grab_ua tv_grab_ua] Downloads XMLTV-formatted data from sites like [http://www.teleguide.info/article1.html teleguide], [http://izbushka.kiev.ua/television/xmltv/ izbushka] or [http://www.star.poltava.ua/index.php?id=132 star.poltava.ua]. In addition it parses existing data and adds category/rating/credits into output XMLTV file.
 +
# [http://www.teleguide.info/download/new3/tvguide.zip teleguide.info] XMLTV-formatted zip archive with programmes inside.
  
<pre><nowiki>UPDATE IGNORE program
+
=== Latvia ===
    SET    chanid=chanid
 
          ,starttime=starttime
 
          ,subtitle= concat_ws(' ', substring_index(substring(title,locate('Teil', title)-6,4+20), ' ', -2), subtitle )
 
          ,title= left(title,locate(substring_index(substring(title,locate('Teil', title)-6,4+20), ' ', -2),title) -1)
 
    WHERE title like '%Teil%';</nowiki></pre>
 
  
 
[[Category:HOWTO]]
 
[[Category:HOWTO]]
 +
[[Category:EPG Providers]]
 +
[[Category:Post Installation Tasks|1300]]

Revision as of 20:09, 19 September 2017


Introduction

XMLTV is the name of a system which obtains and processes program guide information. If you are in North America you can use the Schedules Direct grabber which now supports XMLTV format as well as the old Data Direct format. Using the XMLTV format with Schedules Direct gets you more days of data than the old Data Direct option. All users in the USA and other countries need to install and configure XMLTV in order to obtain program guide listings.

Installation

Install the xmltv package on your master backend server, using your distribution's package manager or package downloaded from XMLTV.

The ubuntu packages may have an out of date version of xmltv. You may be able to get a later version as follows:

Add to /etc/apt/sources.list:

deb http://ftp.us.debian.org/debian sid main 

Run sudo apt-get update and install xmltv. After installing it, remove that entry from /etc/apt/sources.list.

Info on XMLTV is here: http://xmltv.org/wiki/. There are other download options described there, including a nightly snapshot.

The package consists of perl programs. They do not need to be compiled, so if you have installed the package you can download the nightly snapshot and overwrite any out of date member with the new one. A better approach may be to add any new or updated grabbers to a separate directory, which you place on the PATH before /usr/bin. All of the programs in the package are named starting with tv_ and the system will find them as long as they are on the PATH.

Configuration

Configuration of XMLTV must be done on the system where mythfilldatabase will be run. Normally that will be your primary backend machine, but you can run it in other places if you prefer.

MythTV Setup

XMLTV can be configured through the Video Sources Setup page in mythtv-setup. depending on the grabber, it will prompt for any information it requires. If it comes down to selecting channels one at a time manually that can be a challenge if there are many channels.

Some grabbers may not be able to be fully configured through the MythTV setup interface.

Logon or su to the user id that will run mythfilldatabase. In the case of mythbuntu installation that is user id mythtv. Run mythtv-setup from that user id. If you do not want to logon to that id or you cannot, you can run the mythtv-setup from another id on the same backend. After completing setup, copy or link the contents of the $HOME/.xmltv directory to the mythtv home directory. Also copy or link the $HOME/.mythtv/XXXX.xmltv files to the mythtv home directory. If copying or linking the files please make sure that the files and directories are read-write permitted by the mythtv user id.

Command Line Setup

Additional configuration can be done from the command line. As long as you have selected your xmltv grabber in MythTV Setup you can perform some or all configuration via the command line.

Logon to the user that will run mythfilldatabase (normally mythtv). As with mythtv setup if you cannot or will not, you can use any user id and then copy or link directories and files as described above.

List Grabbers

tv_find_grabbers

This gives a listing of grabbers, corresponding to the list you see in MythTV Setup. here is a small sample:

/usr/bin/tv_grab_sd_json|Schedules Direct JSON API
/usr/bin/tv_grab_is|Iceland
/usr/bin/tv_grab_es_laguiatv|Spain (laguiatv.com)
/usr/bin/tv_grab_se_swedb|Sweden (swedb/tvsajten)
/usr/bin/tv_grab_combiner|Combine data from several other grabbers
/opt/mythtv/bin/tv_grab_zz_sdjson_sqlite|Multinational (Schedules Direct JSON web services with SQLite DB)

Find in the list the grabber you selected in MythTV Setup, and get the name of the executable from the beginning of the line. Find out information about it with the --info option.

tv_grab_sd_json --info

That will give information on the usage of that grabber. Most options will be common to all grabbers. To use the --info option you need the perldoc package installed.

Run the configuration

This is the same process that is run from MythTV Setup when you use the configure button in Video Source Setup.

tv_grab_xx_xxxxxx --configure --config-file $HOME/.mythtv/YYYY.xmltv

Use the name of the selected grabber. The value YYYY in the configuration file name corresponds to the Video Source Name. Use your video source name here. If your video source name includes spaces put quotes around it.

The grabber will prompt you required information. If your lineup includes a large number of channels and you do not subscribe to all of them, some grabbers will allow you to select the desired channels.

You should not include channels in your xmltv config which are not required. This places an unnecessarily high load on both the source websites and your own system.

Schedules Direct Setup

Schedules Direct now supports two formats, Data Direct and XMLTV. In the USA the XMLTV format provides 20 days of data while the Data Direct format provides 13 days of data. Here are detailed steps for setting up to use the XMLTV format.

Schedules Direct Grabbers

There are two grabbers for schedules direct. If you want to use Schedules Direct with XMLTV you should obtain the latest version, since there have been recent changes. Download the latest nightly snapshot, extract these two files and make sure they are in a directory on the path, ahead of /usr/bin, or else put them in /usr/bin:

tv_grab_zz_sdjson_sqlite
tv_grab_sd_json

Run tv_find_grabbers and you should see these included in the list.

/usr/bin/tv_grab_sd_json|Schedules Direct JSON API
/usr/bin/tv_grab_zz_sdjson_sqlite|Multinational (Schedules Direct JSON web services with SQLite DB)

You will need to pick one of these. The tv_grab_zz_sdjson_sqlite has more options and needs some command line setup that cannot be done from the mythtv-setup program. Both of them cache data in a cache file or sqllite database. This allows them to download a minimal amount of data when running. Only changes since the previous download are downloaded.

Setup with tv_grab_zz_sdjson_sqlite

If you want the very latest version of this you can download it from github.

Run MythTV Setup and select Multinational (Schedules Direct JSON web services with SQLite DB) for your listings source.

From the command line

tv_grab_zz_sdjson_sqlite --manage-lineups --config-file $HOME/.mythtv/SD.xmltv

Replace SD with the name of the source you had set up in MythTV Setup.

  • Enter your Schedules Direct user and password when prompted.
  • Select option 4 Initialize/update the local database.
  • Accept the default database name.
  • Accept the password hash question.
  • Add or delete lineups using the prompts.

Do not go into channel selection yet. Run this:

tv_grab_zz_sdjson_sqlite --configure --config-file $HOME/.mythtv/SD.xmltv
  • Supply the database name you selected above.
  • Accept defaults for the prompted questions.

Run these:

tv_grab_zz_sdjson_sqlite --days 0 --config-file $HOME/.mythtv/SD.xmltv
tv_grab_zz_sdjson_sqlite --manage-lineups --config-file $HOME/.mythtv/SD.xmltv
  • If you get all channels in the lineup, then select the options that will make current and future channels selected.
  • If you only get a few of the channels you will need to select the appropriate channels. Select a default for existing and future channels, then use option 5 to select which channels you need. This goes though all channels one at a time. You have to complete this for every channel in the list. You cannot stop partway through.
  • If you need to do the channel selection again because of errors or channel changes, your previous values are remembered and used as defaults.

Channel selection shortcut

The channel selection process with tv_grab_zz_sdjson_sqlite is tedious and error prone. An alternative method is as follows. First configure the grabber to select no channels. Install the perl database shell utility. I recommend you install this on one user id as a private library, and preferably do not use the mythtv user id.

cpan App::cpanminus
cpanm DBI::Shell

List the tables in the sqllite database. If you did not use the default database name, enter the one that you used. You may have to specify your database name with /home/mythtv instead of $HOME. In that case first make sure the file is writeable by your login user id.

dbish dbi:SQLite:$HOME/.xmltv/SchedulesDirect.DB 
SELECT name FROM sqlite_master WHERE type='table';

List the channels.

dbish dbi:SQLite:$HOME/.xmltv/SchedulesDirect.DB 
select * from channels;

You can copy the results from the screen, or pipe the results to a file. The value for selected is 0 if unselected or 1 if selected. Edit the file and produce an xml file like this to set the ones you want to select:

update channels set selected = 1 where channum = 2;
update channels set selected = 1 where channum = 3;
...
update channels set selected = 1 where channum = 983;
update channels set selected = 1 where channum = 986;

Update them with a command like this:

dbish dbi:SQLite:$HOME/.xmltv/SchedulesDirect.DB < selected.sql

Converting from DataDirect to XMLTV

There are several reasons to convert your Datadirect setup to use XMLTV.

  • You get three weeks of data instead of two.
  • You get icons for all channels.
  • Season and episode information are included in listings.

If you decide to do this, first complete the setup of tv_grab_zz_sdjson_sqlite described above.

When you switch, the xmltvid's for the existing channels will change. Using this grabber the xmltvid has the format I11460.json.schedulesdirect.org. Using DD the xmltvid's are a numeric value.

There are two options for the conversion:

  1. Delete all your channels and let mythfilldatabase add them back. This option is not available in cases where you have to scan for channels using DVB or ATSC cards.
  2. Manually update each channel xmltvid.

I recommend that you backup your database and make sure there are no recordings scheduled for the near future before starting.

Deleting and re-adding channels

This is only possible for card types that do not need scanning for channels.

Before attempting this bear in mind the following. If any channel's chanid changes, your existing programs will show the wrong channel that the program was recorded on.

The mechanisms in MythTV for adding channels use a format of sourceid and channel number to create a chanid wherever possible. So channel 2 on source id 2 will be chanid 2002, for example. If it cannot do that then it will allocate a chanid number which is unrelated to the channel number or source id.

In order to see the chanids you will have to look at the channel table in the database. See if your existing chanid numbers match the scheme described in the previous paragraph.

If the chanids match the scheme above you should be able to safely delete them and let mythfilldatabase add them.

You can delete all of the existing channels for each video source in mythtv-setup, channel editor. Select the Video source and click Delete.

To add all of the channels you have selected as well as listings, run mythfilldatabase. Default options will add channels and listings.

Manually Updating xmltvids

If you prefer not to delete your channels or you have a DVB type tuner you will have to update the xmltvids in all of them. Using this grabber the xmltvid has the format Innnnn.json.schedulesdirect.org. You will have to change all of your existing ids to this format. The easiest place to make this change is with mythweb channel editor, although you can use the mythtv-setup channel editor as well. You can get the list of channels with their new format xmltvids by running tv_grab_zz_sdjson_sqlite --days 0 --config-file $HOME/.mythtv/SD.xmltv, which is one step of the setup process (see above). If you did not capture the output you can run this again at any time, no harm will be done.

Once you have updated all of the xmltvid's, run mythfilldatabase. Your channel names and icons will be updated to match the values supplied by schedules direct.

In a tuner type that does not use scanning, such as a cable card tuner, any channels not found with matching xmltvids are added. If you have missed updating some xmltvids that could result in duplication of the channel.

Setup with tv_grab_sd_json

Run MythTV Setup and select Schedules Direct JSON API for your listings source.

You can complete the setup using the Configure button in MythTV Setup. If you prefer to use the command line, run

tv_grab_sd_json --configure --config-file $HOME/.mythtv/SD.xmltv

Replace SD with the name of the source you had set up in MythTV Setup.

  • Enter your Schedules Direct user and password when prompted.
  • For channel id format you can select any of the supplied options. Option 0 is the same format as used by tv_grab_zz_sdjson_sqlite. Option 2 is the format used by Schedules Direct Data Direct.
  • Follow the prompts to complete setup.
  • When selecting channels the selection process allows you to skip remaining channels in the list and mark them all as yes or no.
  • If you need to do the channel selection again because of errors or channel changes, your previous values are lost. You have to do the entire selection again.

Channel selection shortcut

The channel selection process with tv_grab_sd_json is tedious and error prone. An alternative method is as follows. First configure the grabber to select all channels. Edit the file $HOME/.mythtv/SD.xmltv. Replace SD with the name of the source you had set up in MythTV Setup. You will see a bunch of lines like this.

channel=74348
channel=11460

When a channel is selected it has =, when unselected it has !. You need to set the channels you want to use = and the others to !.

Get a list of channels by running

tv_grab_sd_json --days 0 --config-file $HOME/.mythtv/SD.xmltv > list.xml

Although this requests 0 days it actually produces 1 day of programs. At the start of the file before the programs is the channel listing. Delete the program schedules and keep the channel listing at the top of the file. The channel id and display names identify each channel by its XMLTV id and the number and name. Note that the channel id in this file may have extra characters appended depending on the format you chose during configure. For purpose of the configuration file you only need the 5 numeric digit of the channel id. Using your favorite editor and your provider's channel list update the channel lines in the configuration file to have = for ones you want and ! for ones you do not.

List of Grabbers

Important.png Note: Not all grabbers on this page are XMLTV compliant. Only fully compliant grabbers are supported by MythTV and we advise against the use of the rest.

If you know a grabber listed here does not conform entirely to both the xmltv grabber specifications (XMLTV grabber spec) and XMLTV xml format (XMLTV Format) then please mark it as unsupported. If in doubt XMLTV provide a script to validate a grabber (XMLTV Validator)

There is also a list on the XMLTV wiki.

EPG Collector

EPG Collector is a .NET/Mono program for collecting the transmitted guide data from DVB or ATSC streams in various formats and creating an XMLTV file.

Multinational

Schedules Direct supports countries in North America, South America, Caribbean and Europe for $25 (US) per year. For a list of supported countries see http://schedulesdirect.org/regions. See #Schedules_Direct_Setup above

Australia

Refer to Australian TV Listings for further details.

  1. OzTivo Guide (Free, requires registration)
  2. http://www.icetv.com.au (AU$99 per year subscription)
  3. Shepherd (Free: hybrid multi source)

See also this (largely historical) list of Australian grabbers.

Belgium

The following is a list of grabbers for Belgium. It is presented only for historical purposes, because at the time of writing this (January 2011) none of them work.

  1. https://launchpad.net/belguide
  2. http://users.skynet.be/jxmltv
  3. http://pytvgrab.sourceforge.net/

Brazil

There are a working grabber, tv_grab_br_net a perl scripts (or any other Linux native solutions) for Brazilian TV programming. This grabber get data from Net [1] cable TV. No open to air TV on this source, just paid tv programme. It is detected by mythtvsetup and you are prompted (on background windows) to make your choice about your city, line-up and channels. It provide both, simple and complete description for programs.

Other functional alternative is to use TVxb, which isn't open source or even Linux, but works well with wine (at least as far as version 1.0085).

TVxb is configurable for different countries via ini-files. Check the following page for Brazilian templates. As of today, there are some for NET (PR), and others for TVA as well.

http://www.tvxb.com/country/br-ini.html

For information on how to set it up with MythTV, check out TVxb.

Croatia

The grabber in the official XMLTV distribution works correctly. More information is available on http://www.gonix.net/.

Czech Republic

The XMLTV grabber for "Hungary/Romania" was extended to the Czech Republic.

Another way to get TV listings in Czech Republic is TVxb.

Not public XMLTV file for czech and slovak channels (UPC channels included) can be send after request to rfordinal.

Denmark

In Denmark there is a group maintaining several grabbers. This group has created the following:

Currently three grabbers are maintained:

  • DR 2012 grabber written in perl (dr.dk)
  • An additional written in python (yousee.dk)
  • An universal grabber combining data from mutiple sources (tvtid.tv2.dk, yousee.dk, tv-guiden.dk, jubii.dk, ontv.dk, ahot.dk, dr.dk and swedb)

Finland

The grabbers in the official XMLTV distribution work correctly. Alternate source: http://xmltv.se

France

The grabber in the upcoming XMLTV release works correctly. More information is available on [2].

Germany/Austria

  1. epgData offers data on a subscription basis, the grabber from the main xmltv distribution works.
  2. XMLTV.se offers many of the main channels for personal use, the grabber from the main xmltv distribution works.
  3. Some channels are available via Egon Zappt in the upcoming release of xmltv.
  4. TV Movie can not give out their data without additional license, see Ben Bucksch's log.

Greece

Try this link for Nova Greece: http://sgcpm.com/epg/devilcosta.gz, for example using mythfilldatabase. (Be aware that its just a dump of DVB-EIT using the service_id as channel_id which is not proper xmltv. Please consider writing a proper grabber for the file so it can be configured via the GUI or simply provide a proper guide feed with tools like NonameTV.)

India

The grabber for What's-On-India has been added to the official XMLTV distribution.

An XMLTV grabber for all channels in India can be found at http://ravihtpc.wordpress.com/2010/01/09/india-epg-data-for-mythtv-xmltv-listings-grabber/.

Italy

The grabber in the official XMLTV distribution works correctly.

Hong Kong

First thanks to http://www.akbkhome.com/blog.php/View/119/Mythtv_xmltv_importer_for_HK.html.

I modify his source code to get information from TVB new web site.

   download Tv.php http://mythtv-xmltvhk.googlecode.com/files/Tv.php
   download hongkong.ini http://mythtv-xmltvhk.googlecode.com/files/hongkong.ini
   build xml from tvb, atv website using following command, just entry 
          php ./Tv.php hongkong.ini > hongkong.xml
   using 'myfilldatabase' to update mythtv schedule program, just entry
          mythfilldatabase --file 1 hongkong.xml
   (* it is my first development on php, so it may be simplified *)

[Last Update by hwkit (17/04/2010)]

Since TVB web site is changed, the program need to be updated. (My php skill is poor, so there are error when running, however it also can get schedule from TVB web site), I also change the program name called 'xmltvhk.php'. Please download it from google.

   download xmltvhk.php and copy it to /etc/xmltvhk (http://mythtv-xmltvhk.googlecode.com/files/xmltvhk.php)
   download updated hongkong.ini and copy it to /etc/xmltvhk (http://mythtv-xmltvhk.googlecode.com/files/hongkong.ini)
   download filltv  and copy it to /etc/cron.weekly (make it executable (chmod +x /etc/cron.weekly/filltv) (http://mythtv-xmltvhk.googlecode.com/files/filltv)
   Source code 
   svn checkout http://mythtv-xmltvhk.googlecode.com/svn/trunk/ mythtv-xmltvhk-read-only

[Last Update by hwkit (07/07/2010)]

Netherlands

Netherlands (Holland) specific info can be found here: Netherlands

New Zealand

Sky TV threatened legal action to get the xmltv grabber for NZ taken down. You can still receive the transmitted guide in MHEG5 with tools like EPG Collector.

Norway

The grabber in the official XMLTV distribution works correctly. Another source of info: http://xmltv.se

Slovakia

The XMLTV grabber for "Hungary/Romania" was extended to Slovakia.

South Africa

See the South Africa MUG WIKI page for details.

Spain

Both grabbers in the official XMLTV distribution work correctly.

Sweden

The grabber in the official XMLTV distribution works correctly. More information is available on http://xmltv.se.

Switzerland

The grabber in the official XMLTV distribution works correctly.

United Kingdom

The grabbers in the official XMLTV distribution work correctly. A brief guide to configuring tv_grab_uk_rt is available, and one for Metabroadcast Atlas.

Ukraine, Russia, Belarus

  1. tv_grab_ru Converts JTV-formatted archive with about 200 channels to XMLTV. But this application does not show additional info about programmes.
  2. tv_grab_ua Downloads XMLTV-formatted data from sites like teleguide, izbushka or star.poltava.ua. In addition it parses existing data and adds category/rating/credits into output XMLTV file.
  3. teleguide.info XMLTV-formatted zip archive with programmes inside.

Latvia