Difference between revisions of "XMLTV"

From MythTV Official Wiki
Jump to: navigation, search
m (Belgium: Date correction)
m (MythTV Setup: Typo)
(46 intermediate revisions by 13 users not shown)
Line 1: Line 1:
The purpose of this page is to cover specific issues that aren't necessarily covered in the official documentation. 
+
{{User Manual TOC}}
  
Info on XMLTV
+
==Introduction==
http://xmltv.org/wiki/
 
  
 +
XMLTV is the name of a system which obtains and processes program guide information. If you are in North America you can use the [[Schedules Direct]] grabber which now supports XMLTV format as well as the old Data Direct format. Using the XMLTV format with Schedules Direct gets you more days of data than the old Data Direct option. All users in the USA and other countries need to install and configure XMLTV in order to obtain program guide listings.
  
 +
==Installation==
 +
Install the xmltv package on your master backend server, using your distribution's package manager or package downloaded from XMLTV.
  
== Outside North America ==
+
The ubuntu packages may have an out of date version of xmltv. You may be able to get a later version as follows:
Generally, XMLTV is used by people living outside of North America.  North American users should instead see [[Schedules Direct]].
 
  
If you live outside the USA and Canada, there are some other sources of guide information.
+
Add to /etc/apt/sources.list:
 +
<pre>deb http://ftp.us.debian.org/debian sid main </pre>
 +
Run sudo apt-get update and install xmltv.
 +
After installing it, remove that entry from /etc/apt/sources.list.
  
=== Important Notes for all XMLTV users ===
+
Info on XMLTV is here: http://xmltv.org/wiki/. There are other download options described there, including a nightly snapshot.
  
{{Warning box|You should '''not''' include channels in your xmltv config which are not required. This places an unnecessarily high load on both the source websites and your own system.}}
+
The package consists of perl programs. They do not need to be compiled, so if you have installed the package you can download the nightly snapshot and overwrite any out of date member with the new one. A better approach may be to add any new or updated grabbers to a separate directory, which you place on the PATH before /usr/bin. All of the programs in the package are named starting with tv_ and the system will find them as long as they are on the PATH.
 +
 
 +
==Configuration==
 +
Configuration of XMLTV must be done on the system where mythfilldatabase will be run. Normally that will be your primary backend machine, but you can run it in other places if you prefer.
 +
 
 +
==MythTV Setup==
 +
XMLTV can be configured through the Video Sources Setup page in mythtv-setup. depending on the grabber, it will prompt for any information it requires. If it comes down to selecting channels one at a time manually that can be a challenge if there are many channels.
 +
 
 +
Some grabbers may not be able to be fully configured through the MythTV setup interface.
 +
 
 +
Logon or su to the user id that will run mythfilldatabase. In the case of mythbuntu installation that is user id mythtv. Run mythtv-setup from that user id. If you do not want to logon to that id or you cannot, you can run the mythtv-setup from another id on the same backend. After completing setup, copy or link the contents of the $HOME/.xmltv directory to the mythtv home directory. Also copy or link the $HOME/.mythtv/XXXX.xmltv files to the mythtv home directory. If copying or linking the files please make sure that the files and directories are read-write permitted by the mythtv user id.
 +
 
 +
==Command Line Setup==
 +
Additional configuration can be done from the command line. As long as you have selected your xmltv grabber in MythTV Setup you can perform some or all configuration via the command line.
 +
 
 +
Logon to the user that will run mythfilldatabase (normally mythtv). As with mythtv setup if you cannot or will not, you can use any user id and then copy or link directories and files as described above.
 +
 
 +
===List Grabbers===
 +
<pre>tv_find_grabbers</pre>
 +
This gives a listing of grabbers, corresponding to the list you see in MythTV Setup. here is a small sample:
 +
<pre>
 +
/usr/bin/tv_grab_sd_json|Schedules Direct JSON API
 +
/usr/bin/tv_grab_is|Iceland
 +
/usr/bin/tv_grab_es_laguiatv|Spain (laguiatv.com)
 +
/usr/bin/tv_grab_se_swedb|Sweden (swedb/tvsajten)
 +
/usr/bin/tv_grab_combiner|Combine data from several other grabbers
 +
/opt/mythtv/bin/tv_grab_zz_sdjson_sqlite|Multinational (Schedules Direct JSON web services with SQLite DB)
 +
</pre>
 +
Find in the list the grabber you selected in MythTV Setup, and get the name of the executable from the beginning of the line. Find out information about it with the --info option.
 +
<pre>tv_grab_sd_json --info</pre>
 +
That will give information on the usage of that grabber. Most options will be common to all grabbers. To use the --info option you need the perldoc package installed.
 +
 
 +
===Run the configuration===
 +
This is the same process that is run from MythTV Setup when you use the configure button in Video Source Setup.
 +
<pre>tv_grab_xx_xxxxxx --configure --config-file $HOME/.mythtv/YYYY.xmltv</pre>
 +
Use the name of the selected grabber. The value YYYY in the configuration file name corresponds to the Video Source Name. Use your video source name here. If your video source name includes spaces put quotes around it.
 +
 
 +
The grabber will prompt you required information. If your lineup includes a large number of channels and you do not subscribe to all of them, some grabbers will allow you to select the desired channels.
 +
 
 +
You should '''not''' include channels in your xmltv config which are not required. This places an unnecessarily high load on both the source websites and your own system.
 +
 
 +
==Schedules Direct Setup==
 +
[[Schedules Direct]] now supports two formats, Data Direct and XMLTV. In the USA the XMLTV format provides 20 days of data while the Data Direct format provides 13 days of data. Here are detailed steps for setting up to use the XMLTV format.
 +
 
 +
===Schedules Direct Grabbers===
 +
There are two grabbers for schedules direct. If you want to use Schedules Direct with XMLTV you should obtain the latest version, since there have been recent changes. Download the latest nightly snapshot, extract these two files and make sure they are in a directory on the path, ahead of /usr/bin, or else put them in /usr/bin:
 +
<pre>tv_grab_zz_sdjson_sqlite
 +
tv_grab_sd_json</pre>
 +
Run tv_find_grabbers and you should see these included in the list.
 +
<pre>/usr/bin/tv_grab_sd_json|Schedules Direct JSON API
 +
/usr/bin/tv_grab_zz_sdjson_sqlite|Multinational (Schedules Direct JSON web services with SQLite DB)</pre>
 +
You will need to pick one of these. The tv_grab_zz_sdjson_sqlite has more options and needs some command line setup that cannot be done from the mythtv-setup program. Both of them cache data in a cache file or sqllite database. This allows them to download a minimal amount of data when running. Only changes since the previous download are downloaded.
 +
 
 +
===Setup with tv_grab_zz_sdjson_sqlite===
 +
Run MythTV Setup and select ''Multinational (Schedules Direct JSON web services with SQLite DB)'' for your listings source.
 +
 
 +
From the command line
 +
<pre>tv_grab_zz_sdjson_sqlite --manage-lineups --config-file $HOME/.mythtv/SD.xmltv</pre>
 +
Replace SD with the name of the source you had set up in MythTV Setup.
 +
* Enter your Schedules Direct user and password when prompted.
 +
* Select option 4 ''Initialize/update the local database''.
 +
* Accept the default database name.
 +
* Accept the password hash question.
 +
* Add or delete lineups using the prompts.
 +
Do not go into channel selection yet. Run this:
 +
<pre>tv_grab_zz_sdjson_sqlite --configure --config-file $HOME/.mythtv/SD.xmltv</pre>
 +
* Supply the database name you selected above.
 +
* Accept defaults for the prompted questions.
 +
Run these:
 +
<pre>tv_grab_zz_sdjson_sqlite --days 0 --config-file $HOME/.mythtv/SD.xmltv
 +
tv_grab_zz_sdjson_sqlite --manage-lineups --config-file $HOME/.mythtv/SD.xmltv</pre>
 +
* If you get all channels in the lineup, then select the options that will make current and future channels selected.
 +
* If you only get a few of the channels you will need to select the appropriate channels. Select a default for existing and future channels, then use option 5 to select which channels you need. This goes though all channels one at a time. You have to complete this for every channel in the list. You cannot stop partway through.
 +
* If you need to do the channel selection again because of errors or channel changes, your previous values are remembered and used as defaults.
 +
 
 +
====Channel selection shortcut====
 +
The channel selection process with tv_grab_zz_sdjson_sqlite is tedious and error prone.
 +
An alternative method is as follows. First configure the grabber to select no channels.
 +
Install the perl database shell utility. I recommend you install this on one user id as a private library, and preferably do not use the mythtv user id.
 +
<pre>cpan App::cpanminus
 +
cpanm DBI::Shell</pre>
 +
List the tables in the sqllite database. If you did not use the default database name, enter the one that you used.  You may have to specify your database name with /home/mythtv instead of $HOME. In that case first make sure the file is writeable by your login user id.
 +
<pre>dbish dbi:SQLite:$HOME/.xmltv/SchedulesDirect.DB
 +
SELECT name FROM sqlite_master WHERE type='table';</pre>
 +
List the channels.
 +
<pre>dbish dbi:SQLite:$HOME/.xmltv/SchedulesDirect.DB
 +
select * from channels;</pre>
 +
You can copy the results from the screen, or pipe the results to a file. The value for ''selected'' is 0 if unselected or 1 if selected. Edit the file and produce an xml file like this to set the ones you want to select:
 +
<pre>update channels set selected = 1 where channum = 2;
 +
update channels set selected = 1 where channum = 3;
 +
...
 +
update channels set selected = 1 where channum = 983;
 +
update channels set selected = 1 where channum = 986;</pre>
 +
Update them with a command like this:
 +
<pre>dbish dbi:SQLite:$HOME/.xmltv/SchedulesDirect.DB < selected.sql</pre>
 +
 
 +
===Setup with tv_grab_sd_json===
 +
Run MythTV Setup and select ''Schedules Direct JSON API'' for your listings source.
 +
 
 +
You can complete the setup using the ''Configure'' button in MythTV Setup. If you prefer to use the command line, run
 +
<pre>tv_grab_zz_sdjson_sqlite --configure --config-file $HOME/.mythtv/SD.xmltv</pre>
 +
Replace SD with the name of the source you had set up in MythTV Setup.
 +
* Enter your Schedules Direct user and password when prompted.
 +
* For channel id format you can select any of the supplied options. Option 0 is the same format as used by tv_grab_zz_sdjson_sqlite. Option 2 is the format used by Schedules Direct Data Direct.
 +
* Follow the prompts to complete setup.
 +
* When selecting channels the selection process allows you to skip remaining channels in the list and mark them all as yes or no.
 +
* If you need to do the channel selection again because of errors or channel changes, your previous values are lost. You have to do the entire selection again.
 +
 
 +
====Channel selection shortcut====
 +
The channel selection process with tv_grab_sd_json is tedious and error prone.
 +
An alternative method is as follows. First configure the grabber to select all channels.
 +
Edit the file <code>$HOME/.mythtv/SD.xmltv</code>.
 +
Replace SD with the name of the source you had set up in MythTV Setup. You will see a bunch of lines like this.
 +
<pre>channel=74348
 +
channel=11460</pre>
 +
When a channel is selected it has '''=''', when unselected it has '''!'''. You need to set the channels you want to use '''=''' and the others to '''!'''.
 +
 
 +
Get a list of channels by running
 +
<pre>tv_grab_sd_json --days 0 --config-file $HOME/.mythtv/SD.xmltv > list.xml</pre>
 +
Although this requests 0 days it actually produces 1 day of programs. At the start of the file before the programs is the channel listing. Delete the program schedules and keep the channel listing at the top of the file. The channel id and display names identify each channel by its XMLTV id and the number and name. Note that the channel id in this file may have extra characters appended depending on the format you chose during configure. For purpose of the configuration file you only need the 5 numeric digit of the channel id. Using your favorite editor and your provider's channel list update the channel lines in the configuration file to have '''=''' for ones you want and '''!''' for ones you do not.
 +
 
 +
== List of Grabbers ==
 +
{{note_box|Not all grabbers on this page are XMLTV compliant. Only fully compliant grabbers are supported by MythTV and we advise against the use of the rest.
 +
 
 +
If you know a grabber listed here does not conform entirely to both the xmltv grabber specifications ([http://wiki.xmltv.org/index.php/HowtoWriteAGrabber XMLTV grabber spec]) and XMLTV xml format ([http://wiki.xmltv.org/index.php/XMLTVFormat XMLTV Format]) then please mark it as unsupported. If in doubt XMLTV provide a script to validate a grabber ([http://wiki.xmltv.org/index.php/XmltvValidation XMLTV Validator])}}
 +
 
 +
There is also a list on the [http://wiki.xmltv.org/index.php/XMLTVProject XMLTV wiki].
 +
 
 +
=== EPG Collector ===
 +
[http://sourceforge.net/projects/epgcollector/ EPG Collector] is a .NET/Mono program for collecting the transmitted guide data from DVB or ATSC streams in various formats and creating an XMLTV file.
 +
 
 +
=== Multinational ===
 +
[[Schedules Direct]] supports countries in North America, South America, Caribbean and Europe for $25 (US) per year. For a list of supported countries see http://schedulesdirect.org/regions. See [[#Schedules_Direct_Setup]] above
  
 
=== Australia ===
 
=== Australia ===
 
Refer to [[Australian TV Listings]] for further details.
 
Refer to [[Australian TV Listings]] for further details.
  
# [http://minnie.tuhs.org/tivo-bin/tvguide.pl OzTivo Guide] (Free, requires registration)
+
# [http://www.oztivo.net/ OzTivo Guide] (Free, requires registration)
# http://www.icetv.com.au (AU$3 per week subscription)
+
# http://www.icetv.com.au (AU$99 per year subscription)
# [http://www.cse.unsw.edu.au/~willu/xmltv/index.html Australian XMLTV grabbers] (ice + tvguide)
+
# [http://svn.whuffy.com/wiki Shepherd] (Free: hybrid multi source)
# [http://svn.whuffy.com/wiki Shepherd ] (Free : Ice,OzTivo hybrid multi source )
+
 
# http://immir.com/tv_grab_au (Free) (may be inactive/broken now)
+
See also this (largely historical) [http://www.cse.unsw.edu.au/~willu/xmltv/index.html list of Australian grabbers].
# http://www.onlinetractorparts.com.au/rohbags/xmltvau/ (not confirmed if working)
 
  
 
=== Belgium ===
 
=== Belgium ===
Line 32: Line 167:
 
# http://users.skynet.be/jxmltv  
 
# http://users.skynet.be/jxmltv  
 
# http://pytvgrab.sourceforge.net/
 
# http://pytvgrab.sourceforge.net/
# http://membled.com/work/apps/xmltv/
 
  
 
=== Brazil ===
 
=== Brazil ===
Line 47: Line 181:
  
 
=== Croatia ===
 
=== Croatia ===
 +
The grabber in the official XMLTV distribution works correctly. More information is available on http://www.gonix.net/.
  
The grabber for Croatia tv_grab_hr is using the goNIX service as its backend.
+
=== Czech Republic ===
 
 
More information is available on http://www.gonix.net/
 
 
 
  
=== Czech Republic ===
+
The XMLTV grabber for "Hungary/Romania" was extended to the Czech Republic.
  
The only way to get TV listings in Czech Republic is [[TVxb]], there is no grabber for XMLTV.
+
Another way to get TV listings in Czech Republic is [[TVxb]].
  
 
Not public XMLTV file for czech and slovak channels (UPC channels included) can be send after request to [mailto:rfordinal@gmail.com rfordinal].
 
Not public XMLTV file for czech and slovak channels (UPC channels included) can be send after request to [mailto:rfordinal@gmail.com rfordinal].
Line 66: Line 198:
 
* A mailing list located at http://uk.groups.yahoo.com/group/xmltvdk very 10 minutes. Only title and descr
 
* A mailing list located at http://uk.groups.yahoo.com/group/xmltvdk very 10 minutes. Only title and descr
 
* A sourceforge repository located at http://sourceforge.net/projects/xmltvdk
 
* A sourceforge repository located at http://sourceforge.net/projects/xmltvdk
 +
* Another source of info: http://xmltv.se
  
Currently seven grabbers are maintained:  
+
Currently three grabbers are maintained:
 
 
* The original (tv.tv2.dk),
 
* An additional written in perl (dr.dk)
 
* Five written in Python (tdc.dk, tv-guiden.dk, ahot.dk, jubii.dk, ontv.dk).
 
 
 
The group also maintains the following:
 
 
 
* A merger written in Python to combine XMLTV information from multiple sources, to obtain the highest possible quality (merge end time of last show from tdc.dk with episode numbers from tv2.dk and long descriptions from tv-guiden.dk etc.).
 
* A "combined" grabber, which configures each of the other grabbers and runs them in parallel and then merge the output. That combined grabber does also support the swedish grabber (swedb).
 
  
The mailing list is used to inform users when there are updates or problems with the current grabbers.
+
* DR 2012 grabber written in perl (dr.dk)
 +
* An additional written in python (yousee.dk)
 +
* An universal grabber combining data from mutiple sources (tvtid.tv2.dk, yousee.dk, tv-guiden.dk, jubii.dk, ontv.dk, ahot.dk, dr.dk and swedb)
  
 
=== Finland ===
 
=== Finland ===
Finnish XMLTV data can be acquired with tv_grab_fi. The script parses www.telkku.com web site in order to get EPG entries.
+
The grabbers in the official XMLTV distribution work correctly.
 
+
Alternate source: http://xmltv.se
Latest version of the tv_grab_fi can be downloaded from: <br>
 
http://xmltv.cvs.sourceforge.net/*checkout*/xmltv/xmltv/grab/fi/tv_grab_fi <br>
 
Due frequent site updates of the www.telkku.com user should always have the latest version of the grabber. <br>
 
Add latest version of tv_grab_fi somewhere where mythtv can find it for example in debian system to /usr/bin/. Also remember to assure that you don't have any older versions installed.
 
 
 
To use tv_grab_fi you need to first run in ~/.mythtv
 
tv_grab_fi --configure --config-file dvb.xmltv <br>
 
This will ask which channels does the user want to use grabber for. <br>
 
Once you have generated dvb.xmltv it should contain lines like this 'channel 0 YLE1'. Channels with '#' in front of them will not be added.
 
 
 
Now we need to tell XMLTV ID's to mythtv. In dvb.xmltv take the number after word "channel" and add ".telkku.com" after that and you'll have the ID. For example if the channel has line of 'channel 62 YLE24' the ID is 62.telkku.com. <br>
 
Start Mythtv setup <br>
 
mythtv-setup <br>
 
Go to: 'channel editor' -> "channel that you want to use grabber for" -> XMLTV ID: xx.telkku.com (Where 'xx' is the number in dvb.xmltv) -> Next -> Ready.
 
 
 
To update program guide do
 
mythfilldatabase
 
If that doesn't add EPG entries then do:
 
mythfilldatabase --refresh-all
 
 
 
Another way to get XMLTV ID's is to do
 
tv_grab_fi --config-file dvb.xmltv --days 0 --output channels.tmp
 
In channels.tmp find line that has '<channel id="...>' in it. In place of "..." you will have the XMLTV ID. For example if the line is: "<channel id="2.telkku.com">" then the XMLTV ID is 2.telkku.com.
 
  
 
=== France ===
 
=== France ===
  
This web site [http://www.kazer.org/] provides free and very complete XMLTV listings as a downloadable XML file. You will need to run mythfilldatabase with the --file argument to use the downloaded xml.
+
The grabber in the upcoming XMLTV release works correctly. More information is available on [http://www.kazer.org/].
  
 
=== Germany/Austria ===
 
=== Germany/Austria ===
====TV Movie====
+
# [http://wiki.xmltv.org/index.php/Europe epgData] offers data on a subscription basis, the grabber from the main xmltv distribution works.
The TV Movie paper has changed its EPG data publishment in January 2009, breaking the [http://mythtv.linux-dude.de old script].
+
# [http://xmltv.se/ XMLTV.se] offers many of the main channels for personal use, the grabber from the main xmltv distribution works.
 
+
# Some channels are available via [http://xmltv.spaetfruehstuecken.org/xmltv/ Egon Zappt] in the upcoming release of xmltv.
A script for the new format can be found [http://swolter.sdf1.org/software/article-tvmovie.html here].
+
# TV Movie can not give out their data without additional license, see [http://www.bucksch.org/1/projects/various/xmltv/ Ben Bucksch's log].
This grabber is very fast (runs in a few seconds) and offers up to 8 days of program data for all
 
free-TV stations, including ratings, comments and actor lists.
 
 
 
====XMLTV.info====
 
You can download the TV program as XMLTV file from [http://xmltv.info xmltv.info]. There is also a fast grabber available to automatically fetch the file.
 
  
 
=== Greece ===
 
=== Greece ===
  
This web site [http://www.xmltv.eu/] provides free and very complete XMLTV listings as a downloadable XML file. You will need to run mythfilldatabase with the --file argument to use the downloaded xml.
+
Try this link for Nova Greece: [http://sgcpm.com/epg/devilcosta.gz http://sgcpm.com/epg/devilcosta.gz], for example using mythfilldatabase. (Be aware that its just a dump of DVB-EIT using the service_id as channel_id which is not proper xmltv. Please consider writing a proper grabber for the file so it can be configured via the GUI or simply provide a proper guide feed with tools like [http://wiki.xmltv.org/index.php/NonameTV NonameTV].)
  
 
=== India ===
 
=== India ===
 +
The grabber for What's-On-India has been added to the official XMLTV distribution.
 +
 
An XMLTV grabber for all channels in India can be found at [http://ravihtpc.wordpress.com/2010/01/09/india-epg-data-for-mythtv-xmltv-listings-grabber/ http://ravihtpc.wordpress.com/2010/01/09/india-epg-data-for-mythtv-xmltv-listings-grabber/].
 
An XMLTV grabber for all channels in India can be found at [http://ravihtpc.wordpress.com/2010/01/09/india-epg-data-for-mythtv-xmltv-listings-grabber/ http://ravihtpc.wordpress.com/2010/01/09/india-epg-data-for-mythtv-xmltv-listings-grabber/].
  
Line 162: Line 262:
  
 
=== New Zealand ===
 
=== New Zealand ===
A c# application to scrape sites can be found here, runs fine using mono on linux
+
Sky TV threatened legal action to get the [http://reven.co.nz/post/whatever-happened-to-xmltvnz xmltv grabber for NZ taken down].
# http://www.reven.co.nz/
+
You can still receive the transmitted guide in MHEG5 with tools like [[XMLTV#EPG_Collector|EPG Collector]].
  
 
=== Norway ===
 
=== Norway ===
 +
The grabber in the official XMLTV distribution works correctly.
 +
Another source of info: http://xmltv.se
  
To use the XMLTV Data for Norway, you will have to use:
+
=== Slovakia ===
tv_grab_no --configure --config-file=tv_grab_no.xmltv
+
The XMLTV grabber for "Hungary/Romania" was extended to Slovakia.
It will then ask you what channels you would like.
 
Next copy the file to your ''.mythtv'' directory.
 
 
 
Next, you have to get the tags for each channel.. So let's fetch the data and save it to a tmp file.
 
tv_grab_no --config-file=tv_grab_no.xmltv --list-channels --output=channels.tmp
 
 
 
Open that file and write down each tag for each channel. Look for lines like:
 
<channel id="nrk1.nrk.no">
 
 
 
As you will see in the file: '''nrk1.nrk.no''' is for '''Nrk1'''.
 
 
 
So now when you run mythtv-setup --> channel editor, in order to capture '''Nrk1''', you need to set the '''XMLTV ID''' to '''nrk1.nrk.no'''.
 
 
 
Do this for all the channels. When you are finally finished with doing all that, just run
 
mythfilldatabase
 
 
 
When its finished, start up mythfrontend.
 
  
 
=== South Africa ===
 
=== South Africa ===
 
 
See the [[South Africa MUG]] WIKI page for details.
 
See the [[South Africa MUG]] WIKI page for details.
  
 
=== Spain ===
 
=== Spain ===
 
+
Both grabbers in the official XMLTV distribution work correctly.
The included parser works (www.elpais.es/parrillatv/index.html). Although it has been found some inconsistencies with the real programation.
 
 
 
One alternative, abeit not been as complete: http://www.miguiatv.com/todos-los-canales.html
 
  
 
=== Sweden ===
 
=== Sweden ===
 
+
The grabber in the official XMLTV distribution works correctly. More information is available on http://xmltv.se.
# http://tv.swedb.se/
 
  
 
=== Switzerland ===
 
=== Switzerland ===
 
+
The grabber in the official XMLTV distribution works correctly.
# [[XMLTV_CH]]: Info on available grabbers and icon listings.
 
# [http://xmltv.info xmltv.info] offers an XMLTV file for download which includes the TV programm for a number of Swiss channels.
 
  
 
=== United Kingdom ===
 
=== United Kingdom ===
 
+
The grabbers in the official XMLTV distribution work correctly.
# [[Uk_xmltv]]
+
A brief [[Uk_xmltv|guide to configuring tv_grab_uk_rt]] is available, and one for [[Atlas|Metabroadcast Atlas]].
 
 
A brief guide to configuring tv_grab_uk_rt is available at the link above.
 
  
 
=== Ukraine, Russia, Belarus ===
 
=== Ukraine, Russia, Belarus ===
Line 218: Line 294:
  
 
=== Latvia ===
 
=== Latvia ===
 
http://lvtv.devlat.eu/xmltv.xml XMLTV-formatted EPG for channels received in Latvia. Updated every 10 minutes.
 
Contains LV national free-view EPG for current week (mon - sun), + other pay cable and SAT channels. Titles and descriptions only in original languages. No categories or other data.
 
 
Found on http://alvils.latvietis.com/index.cgi?action=5&id=168.
 
 
'''Free-view DVB-T channels''':
 
*LTV1
 
*LTV7
 
*TV3
 
*LNT
 
*TV5
 
 
'''Non-free-view channels''':
 
REN TV Baltic, TV6, TV3+, NTV Mir, 1. Baltijas kanals, RTR Planeta, TV1000 Krievu kino, TV Center International, RTVi, TV1000, Hallmark Channel,Eurosport, Cartoon Network, Discovery Channel, Extreme Sports Channel, VH1, ESPN Classic, MCM, PRO7, VIASAT Explorer, Discovery World, Dom Kino, NASN, Mezzo, Discovery Science, Boomerang,VIASAT Sport Baltic,Discovery Travel and Living, VIASAT History, Eurosport 2, TV1000 East, TV1000 Premium, TV1000 Action, BCC World, Playhouse Disney, Toon Disney, E!, Private Spice, Playboy, Travel
 
 
MythBuntu usage: (in sudo mythtv accounts "crontab -e" add 2 records to update EPG every hour) 
 
<pre>
 
0 * * * * wget --connect-timeout=30  -O /home/mythtv/lv_xmltv.xml -T 30 --no-cache --user-agent=Mozilla/4.0 "http://lvtv.devlat.eu/xmltv.xml"; mythfilldatabase --file 1 /home/mythtv/lv_xmltv.xml
 
</pre>
 
''Hint: don't forget configure proxy in wget, if you using one. Try both commands from shell first.''
 
  
 
[[Category:HOWTO]]
 
[[Category:HOWTO]]
 
[[Category:EPG Providers]]
 
[[Category:EPG Providers]]
 +
[[Category:Post Installation Tasks|1300]]

Revision as of 19:24, 19 August 2016


Introduction

XMLTV is the name of a system which obtains and processes program guide information. If you are in North America you can use the Schedules Direct grabber which now supports XMLTV format as well as the old Data Direct format. Using the XMLTV format with Schedules Direct gets you more days of data than the old Data Direct option. All users in the USA and other countries need to install and configure XMLTV in order to obtain program guide listings.

Installation

Install the xmltv package on your master backend server, using your distribution's package manager or package downloaded from XMLTV.

The ubuntu packages may have an out of date version of xmltv. You may be able to get a later version as follows:

Add to /etc/apt/sources.list:

deb http://ftp.us.debian.org/debian sid main 

Run sudo apt-get update and install xmltv. After installing it, remove that entry from /etc/apt/sources.list.

Info on XMLTV is here: http://xmltv.org/wiki/. There are other download options described there, including a nightly snapshot.

The package consists of perl programs. They do not need to be compiled, so if you have installed the package you can download the nightly snapshot and overwrite any out of date member with the new one. A better approach may be to add any new or updated grabbers to a separate directory, which you place on the PATH before /usr/bin. All of the programs in the package are named starting with tv_ and the system will find them as long as they are on the PATH.

Configuration

Configuration of XMLTV must be done on the system where mythfilldatabase will be run. Normally that will be your primary backend machine, but you can run it in other places if you prefer.

MythTV Setup

XMLTV can be configured through the Video Sources Setup page in mythtv-setup. depending on the grabber, it will prompt for any information it requires. If it comes down to selecting channels one at a time manually that can be a challenge if there are many channels.

Some grabbers may not be able to be fully configured through the MythTV setup interface.

Logon or su to the user id that will run mythfilldatabase. In the case of mythbuntu installation that is user id mythtv. Run mythtv-setup from that user id. If you do not want to logon to that id or you cannot, you can run the mythtv-setup from another id on the same backend. After completing setup, copy or link the contents of the $HOME/.xmltv directory to the mythtv home directory. Also copy or link the $HOME/.mythtv/XXXX.xmltv files to the mythtv home directory. If copying or linking the files please make sure that the files and directories are read-write permitted by the mythtv user id.

Command Line Setup

Additional configuration can be done from the command line. As long as you have selected your xmltv grabber in MythTV Setup you can perform some or all configuration via the command line.

Logon to the user that will run mythfilldatabase (normally mythtv). As with mythtv setup if you cannot or will not, you can use any user id and then copy or link directories and files as described above.

List Grabbers

tv_find_grabbers

This gives a listing of grabbers, corresponding to the list you see in MythTV Setup. here is a small sample:

/usr/bin/tv_grab_sd_json|Schedules Direct JSON API
/usr/bin/tv_grab_is|Iceland
/usr/bin/tv_grab_es_laguiatv|Spain (laguiatv.com)
/usr/bin/tv_grab_se_swedb|Sweden (swedb/tvsajten)
/usr/bin/tv_grab_combiner|Combine data from several other grabbers
/opt/mythtv/bin/tv_grab_zz_sdjson_sqlite|Multinational (Schedules Direct JSON web services with SQLite DB)

Find in the list the grabber you selected in MythTV Setup, and get the name of the executable from the beginning of the line. Find out information about it with the --info option.

tv_grab_sd_json --info

That will give information on the usage of that grabber. Most options will be common to all grabbers. To use the --info option you need the perldoc package installed.

Run the configuration

This is the same process that is run from MythTV Setup when you use the configure button in Video Source Setup.

tv_grab_xx_xxxxxx --configure --config-file $HOME/.mythtv/YYYY.xmltv

Use the name of the selected grabber. The value YYYY in the configuration file name corresponds to the Video Source Name. Use your video source name here. If your video source name includes spaces put quotes around it.

The grabber will prompt you required information. If your lineup includes a large number of channels and you do not subscribe to all of them, some grabbers will allow you to select the desired channels.

You should not include channels in your xmltv config which are not required. This places an unnecessarily high load on both the source websites and your own system.

Schedules Direct Setup

Schedules Direct now supports two formats, Data Direct and XMLTV. In the USA the XMLTV format provides 20 days of data while the Data Direct format provides 13 days of data. Here are detailed steps for setting up to use the XMLTV format.

Schedules Direct Grabbers

There are two grabbers for schedules direct. If you want to use Schedules Direct with XMLTV you should obtain the latest version, since there have been recent changes. Download the latest nightly snapshot, extract these two files and make sure they are in a directory on the path, ahead of /usr/bin, or else put them in /usr/bin:

tv_grab_zz_sdjson_sqlite
tv_grab_sd_json

Run tv_find_grabbers and you should see these included in the list.

/usr/bin/tv_grab_sd_json|Schedules Direct JSON API
/usr/bin/tv_grab_zz_sdjson_sqlite|Multinational (Schedules Direct JSON web services with SQLite DB)

You will need to pick one of these. The tv_grab_zz_sdjson_sqlite has more options and needs some command line setup that cannot be done from the mythtv-setup program. Both of them cache data in a cache file or sqllite database. This allows them to download a minimal amount of data when running. Only changes since the previous download are downloaded.

Setup with tv_grab_zz_sdjson_sqlite

Run MythTV Setup and select Multinational (Schedules Direct JSON web services with SQLite DB) for your listings source.

From the command line

tv_grab_zz_sdjson_sqlite --manage-lineups --config-file $HOME/.mythtv/SD.xmltv

Replace SD with the name of the source you had set up in MythTV Setup.

  • Enter your Schedules Direct user and password when prompted.
  • Select option 4 Initialize/update the local database.
  • Accept the default database name.
  • Accept the password hash question.
  • Add or delete lineups using the prompts.

Do not go into channel selection yet. Run this:

tv_grab_zz_sdjson_sqlite --configure --config-file $HOME/.mythtv/SD.xmltv
  • Supply the database name you selected above.
  • Accept defaults for the prompted questions.

Run these:

tv_grab_zz_sdjson_sqlite --days 0 --config-file $HOME/.mythtv/SD.xmltv
tv_grab_zz_sdjson_sqlite --manage-lineups --config-file $HOME/.mythtv/SD.xmltv
  • If you get all channels in the lineup, then select the options that will make current and future channels selected.
  • If you only get a few of the channels you will need to select the appropriate channels. Select a default for existing and future channels, then use option 5 to select which channels you need. This goes though all channels one at a time. You have to complete this for every channel in the list. You cannot stop partway through.
  • If you need to do the channel selection again because of errors or channel changes, your previous values are remembered and used as defaults.

Channel selection shortcut

The channel selection process with tv_grab_zz_sdjson_sqlite is tedious and error prone. An alternative method is as follows. First configure the grabber to select no channels. Install the perl database shell utility. I recommend you install this on one user id as a private library, and preferably do not use the mythtv user id.

cpan App::cpanminus
cpanm DBI::Shell

List the tables in the sqllite database. If you did not use the default database name, enter the one that you used. You may have to specify your database name with /home/mythtv instead of $HOME. In that case first make sure the file is writeable by your login user id.

dbish dbi:SQLite:$HOME/.xmltv/SchedulesDirect.DB 
SELECT name FROM sqlite_master WHERE type='table';

List the channels.

dbish dbi:SQLite:$HOME/.xmltv/SchedulesDirect.DB 
select * from channels;

You can copy the results from the screen, or pipe the results to a file. The value for selected is 0 if unselected or 1 if selected. Edit the file and produce an xml file like this to set the ones you want to select:

update channels set selected = 1 where channum = 2;
update channels set selected = 1 where channum = 3;
...
update channels set selected = 1 where channum = 983;
update channels set selected = 1 where channum = 986;

Update them with a command like this:

dbish dbi:SQLite:$HOME/.xmltv/SchedulesDirect.DB < selected.sql

Setup with tv_grab_sd_json

Run MythTV Setup and select Schedules Direct JSON API for your listings source.

You can complete the setup using the Configure button in MythTV Setup. If you prefer to use the command line, run

tv_grab_zz_sdjson_sqlite --configure --config-file $HOME/.mythtv/SD.xmltv

Replace SD with the name of the source you had set up in MythTV Setup.

  • Enter your Schedules Direct user and password when prompted.
  • For channel id format you can select any of the supplied options. Option 0 is the same format as used by tv_grab_zz_sdjson_sqlite. Option 2 is the format used by Schedules Direct Data Direct.
  • Follow the prompts to complete setup.
  • When selecting channels the selection process allows you to skip remaining channels in the list and mark them all as yes or no.
  • If you need to do the channel selection again because of errors or channel changes, your previous values are lost. You have to do the entire selection again.

Channel selection shortcut

The channel selection process with tv_grab_sd_json is tedious and error prone. An alternative method is as follows. First configure the grabber to select all channels. Edit the file $HOME/.mythtv/SD.xmltv. Replace SD with the name of the source you had set up in MythTV Setup. You will see a bunch of lines like this.

channel=74348
channel=11460

When a channel is selected it has =, when unselected it has !. You need to set the channels you want to use = and the others to !.

Get a list of channels by running

tv_grab_sd_json --days 0 --config-file $HOME/.mythtv/SD.xmltv > list.xml

Although this requests 0 days it actually produces 1 day of programs. At the start of the file before the programs is the channel listing. Delete the program schedules and keep the channel listing at the top of the file. The channel id and display names identify each channel by its XMLTV id and the number and name. Note that the channel id in this file may have extra characters appended depending on the format you chose during configure. For purpose of the configuration file you only need the 5 numeric digit of the channel id. Using your favorite editor and your provider's channel list update the channel lines in the configuration file to have = for ones you want and ! for ones you do not.

List of Grabbers

Important.png Note: Not all grabbers on this page are XMLTV compliant. Only fully compliant grabbers are supported by MythTV and we advise against the use of the rest.

If you know a grabber listed here does not conform entirely to both the xmltv grabber specifications (XMLTV grabber spec) and XMLTV xml format (XMLTV Format) then please mark it as unsupported. If in doubt XMLTV provide a script to validate a grabber (XMLTV Validator)

There is also a list on the XMLTV wiki.

EPG Collector

EPG Collector is a .NET/Mono program for collecting the transmitted guide data from DVB or ATSC streams in various formats and creating an XMLTV file.

Multinational

Schedules Direct supports countries in North America, South America, Caribbean and Europe for $25 (US) per year. For a list of supported countries see http://schedulesdirect.org/regions. See #Schedules_Direct_Setup above

Australia

Refer to Australian TV Listings for further details.

  1. OzTivo Guide (Free, requires registration)
  2. http://www.icetv.com.au (AU$99 per year subscription)
  3. Shepherd (Free: hybrid multi source)

See also this (largely historical) list of Australian grabbers.

Belgium

The following is a list of grabbers for Belgium. It is presented only for historical purposes, because at the time of writing this (January 2011) none of them work.

  1. https://launchpad.net/belguide
  2. http://users.skynet.be/jxmltv
  3. http://pytvgrab.sourceforge.net/

Brazil

There are a working grabber, tv_grab_br_net a perl scripts (or any other Linux native solutions) for Brazilian TV programming. This grabber get data from Net [1] cable TV. No open to air TV on this source, just paid tv programme. It is detected by mythtvsetup and you are prompted (on background windows) to make your choice about your city, line-up and channels. It provide both, simple and complete description for programs.

Other functional alternative is to use TVxb, which isn't open source or even Linux, but works well with wine (at least as far as version 1.0085).

TVxb is configurable for different countries via ini-files. Check the following page for Brazilian templates. As of today, there are some for NET (PR), and others for TVA as well.

http://www.tvxb.com/country/br-ini.html

For information on how to set it up with MythTV, check out TVxb.

Croatia

The grabber in the official XMLTV distribution works correctly. More information is available on http://www.gonix.net/.

Czech Republic

The XMLTV grabber for "Hungary/Romania" was extended to the Czech Republic.

Another way to get TV listings in Czech Republic is TVxb.

Not public XMLTV file for czech and slovak channels (UPC channels included) can be send after request to rfordinal.

Denmark

In Denmark there is a group maintaining several grabbers. This group has created the following:

Currently three grabbers are maintained:

  • DR 2012 grabber written in perl (dr.dk)
  • An additional written in python (yousee.dk)
  • An universal grabber combining data from mutiple sources (tvtid.tv2.dk, yousee.dk, tv-guiden.dk, jubii.dk, ontv.dk, ahot.dk, dr.dk and swedb)

Finland

The grabbers in the official XMLTV distribution work correctly. Alternate source: http://xmltv.se

France

The grabber in the upcoming XMLTV release works correctly. More information is available on [2].

Germany/Austria

  1. epgData offers data on a subscription basis, the grabber from the main xmltv distribution works.
  2. XMLTV.se offers many of the main channels for personal use, the grabber from the main xmltv distribution works.
  3. Some channels are available via Egon Zappt in the upcoming release of xmltv.
  4. TV Movie can not give out their data without additional license, see Ben Bucksch's log.

Greece

Try this link for Nova Greece: http://sgcpm.com/epg/devilcosta.gz, for example using mythfilldatabase. (Be aware that its just a dump of DVB-EIT using the service_id as channel_id which is not proper xmltv. Please consider writing a proper grabber for the file so it can be configured via the GUI or simply provide a proper guide feed with tools like NonameTV.)

India

The grabber for What's-On-India has been added to the official XMLTV distribution.

An XMLTV grabber for all channels in India can be found at http://ravihtpc.wordpress.com/2010/01/09/india-epg-data-for-mythtv-xmltv-listings-grabber/.

Italy

The grabber in the official XMLTV distribution works correctly.

Hong Kong

First thanks to http://www.akbkhome.com/blog.php/View/119/Mythtv_xmltv_importer_for_HK.html.

I modify his source code to get information from TVB new web site.

   download Tv.php http://mythtv-xmltvhk.googlecode.com/files/Tv.php
   download hongkong.ini http://mythtv-xmltvhk.googlecode.com/files/hongkong.ini
   build xml from tvb, atv website using following command, just entry 
          php ./Tv.php hongkong.ini > hongkong.xml
   using 'myfilldatabase' to update mythtv schedule program, just entry
          mythfilldatabase --file 1 hongkong.xml
   (* it is my first development on php, so it may be simplified *)

[Last Update by hwkit (17/04/2010)]

Since TVB web site is changed, the program need to be updated. (My php skill is poor, so there are error when running, however it also can get schedule from TVB web site), I also change the program name called 'xmltvhk.php'. Please download it from google.

   download xmltvhk.php and copy it to /etc/xmltvhk (http://mythtv-xmltvhk.googlecode.com/files/xmltvhk.php)
   download updated hongkong.ini and copy it to /etc/xmltvhk (http://mythtv-xmltvhk.googlecode.com/files/hongkong.ini)
   download filltv  and copy it to /etc/cron.weekly (make it executable (chmod +x /etc/cron.weekly/filltv) (http://mythtv-xmltvhk.googlecode.com/files/filltv)
   Source code 
   svn checkout http://mythtv-xmltvhk.googlecode.com/svn/trunk/ mythtv-xmltvhk-read-only

[Last Update by hwkit (07/07/2010)]

Netherlands

Netherlands (Holland) specific info can be found here: Netherlands

New Zealand

Sky TV threatened legal action to get the xmltv grabber for NZ taken down. You can still receive the transmitted guide in MHEG5 with tools like EPG Collector.

Norway

The grabber in the official XMLTV distribution works correctly. Another source of info: http://xmltv.se

Slovakia

The XMLTV grabber for "Hungary/Romania" was extended to Slovakia.

South Africa

See the South Africa MUG WIKI page for details.

Spain

Both grabbers in the official XMLTV distribution work correctly.

Sweden

The grabber in the official XMLTV distribution works correctly. More information is available on http://xmltv.se.

Switzerland

The grabber in the official XMLTV distribution works correctly.

United Kingdom

The grabbers in the official XMLTV distribution work correctly. A brief guide to configuring tv_grab_uk_rt is available, and one for Metabroadcast Atlas.

Ukraine, Russia, Belarus

  1. tv_grab_ru Converts JTV-formatted archive with about 200 channels to XMLTV. But this application does not show additional info about programmes.
  2. tv_grab_ua Downloads XMLTV-formatted data from sites like teleguide, izbushka or star.poltava.ua. In addition it parses existing data and adds category/rating/credits into output XMLTV file.
  3. teleguide.info XMLTV-formatted zip archive with programmes inside.

Latvia