Difference between revisions of "XMLTV"

From MythTV Official Wiki
Jump to: navigation, search
(Ukraine, Russia, Belarus: add tv_grab_ua xmltv parser)
(Setup with tv_grab_zz_sdjson)
 
(105 intermediate revisions by 32 users not shown)
Line 1: Line 1:
Setting up XmlTV for use with MythTV is documented in the
+
{{User Manual TOC}}
[http://www.mythtv.org/docs/mythtv-HOWTO-5.html#ss5.2 MythTV HOWTO 5.2].
 
  
The purpose of this page is to cover specific issues that aren't necessarily covered in the official documentation. 
+
=Introduction=
  
Info on XMLTV
+
XMLTV is the name of a system which obtains and processes program guide information. If you are in North America you can use the [[Schedules Direct]] grabber which now supports XMLTV format as well as the old Data Direct format ('''until v31 and above'''.) Using the XMLTV format with Schedules Direct gets you more days of data than the old Data Direct option. All users in the USA and other countries need to install and configure XMLTV in order to obtain program guide listings.
http://xmltv.org/wiki/
 
  
 +
=Installation=
 +
Install the xmltv package on your master backend server, using your distribution's package manager.
  
 +
The ubuntu packages and others e.g. Debian, Raspbian are likely to have out of date version of xmltv. You can get the latest version as follows in a terminal session:
  
== Outside North America ==
+
<pre>
If you live outside the USA and Canada, there are some other sources of guide information
+
sudo apt update
 +
sudo apt install git make xmltv perl-doc
 +
# Note: xmltv gets the required perl modules
  
=== Import Notes for all XMLTV users ===
+
mkdir -p build
 +
cd build
 +
git clone https://github.com/XMLTV/xmltv.git
 +
cd xmltv
 +
perl Makefile.PL
 +
make
 +
make test
 +
sudo make install
 +
</pre>
  
{{Warning box|You should '''not''' include channels in your xmltv config which are not required. This places an unnecessarily high load on both the source websites and your own system.}}
+
The latest version is installed in /usr/local/bin, which will be used in preference to the original versions in /usr/bin/
  
=== Australia ===
+
use tv_find_grabbers to list all grabbers, including location
Refer to [[Australian TV Listings]] for further details.
+
 
 +
Info on XMLTV is here: https://github.com/XMLTV/xmltv.
 +
 
 +
=Configuration=
 +
Configuration of XMLTV must be done on the system where mythfilldatabase will be run. Normally that will be your primary backend machine, but you can run it in other places if you prefer.
 +
 
 +
=MythTV Setup=
 +
 
 +
If user mythtv is allowed to login and start the window manager, XMLTV can be configured through the Video Sources Setup page in mythtv-setup.
 +
Otherwise, follow the steps below and do the configuration from the command line.
 +
Depending on the grabber, it will prompt for any information it requires. If it comes down to selecting channels one at a time manually that can be a challenge if there are many channels.
 +
 
 +
{{Tip box|Setup of XMLTV via command line is less challenging and more consistent with XMLTV instructions/usage.}}
 +
 
 +
 
 +
{{Warning box|It seems users are missing the step that says run the setup '''as the user that will run mythfilldatabase''' (which is frequently user mythtv.)}}
 +
 
 +
Some grabbers may not be able to be fully configured through the MythTV setup interface.
 +
 
 +
The general steps are:
 +
# Logon or su to the user that will run mythfilldatabase.
 +
#* In the case of mythbuntu installation that is the user id mythtv.
 +
# Run mythtv-setup using that user id.
 +
#* If you do not want to logon to that id or you cannot, you can run the mythtv-setup from another id on the same backend.
 +
# Setup Myth to use XMLTV following the instructions at [[Setup Video Sources]].
 +
The next steps are optional, and not recommended. By running all programs as user ''mythtv'', as described above, duplicating
 +
the involved directories and files is not required.
 +
 
 +
  4. After completing setup, copy or link the contents of the $HOME/.xmltv directory to the mythtv home directory
 +
* The configuration file in <code>$HOME/.xmltv</code> '''MUST''' be renamed to the name supplied to <code>mythtv-setup</code> Video Source name. Example: Assuming you named the video source as '''xmltv''', the configuration file name must be named <code>$HOME/.mythtv/<b>xmltv</b>.xmltv</code>.
 +
  5. If copying or linking the files please make sure that the files and directories are read-write permitted by the mythtv user id.
 +
 
 +
=Command Line Setup=
 +
Additional configuration can be done from the command line. As long as you have selected your xmltv grabber in MythTV Setup you can perform some or all configuration via the command line.
 +
 
 +
'''Logon to the user that will run mythfilldatabase (normally mythtv).''' As with mythtv setup if you cannot or will not, you can use any user id and then copy or link directories and files as described above.
 +
 
 +
==List Grabbers==
 +
<pre>tv_find_grabbers</pre>
 +
This gives a listing of grabbers, corresponding to the list you see in MythTV Setup. here is a small sample:
 +
<pre>
 +
/usr/bin/tv_grab_zz_sdjson|Schedules Direct JSON API
 +
/usr/bin/tv_grab_is|Iceland
 +
/usr/bin/tv_grab_es_laguiatv|Spain (laguiatv.com)
 +
/usr/bin/tv_grab_se_swedb|Sweden (swedb/tvsajten)
 +
/usr/bin/tv_grab_combiner|Combine data from several other grabbers
 +
/opt/mythtv/bin/tv_grab_zz_sdjson_sqlite|Multinational (Schedules Direct JSON web services with SQLite DB)
 +
</pre>
 +
Find in the list the grabber you selected in MythTV Setup, and get the name of the executable from the beginning of the line. Find out information about it with the --info option.
 +
<pre>tv_grab_zz_sdjson --info</pre>
 +
That will give information on the usage of that grabber. Most options will be common to all grabbers. To use the --info option you need the perl-doc package installed.
 +
 
 +
==Run the configuration==
 +
This is the same process that is run from MythTV Setup when you use the configure button in Video Source Setup.
 +
<pre>tv_grab_xx_xxxxxx --configure --config-file $HOME/.mythtv/YYYY.xmltv</pre>
 +
Use the name of the selected grabber. The value YYYY in the configuration file name corresponds to the Video Source Name. Use your video source name here. If your video source name includes spaces put quotes around it.
 +
 
 +
The grabber will prompt you required information. If your lineup includes a large number of channels and you do not subscribe to all of them, some grabbers will allow you to select the desired channels.
 +
 
 +
You should '''not''' include channels in your xmltv config which are not required. This places an unnecessarily high load on both the source websites and your own system.
 +
 
 +
For more information on editing channels see the Channel Editor [https://www.mythtv.org/wiki/Channel_Editor]
 +
 
 +
=Schedules Direct Setup=
 +
[[Schedules Direct]] now uses XMLTV only. In the USA the XMLTV format provides 18 days of data.
 +
 
 +
For a list of countries/regions supported see  https://www.schedulesdirect.org/regions
 +
Up to 4 different lineups are supported at any one time.
 +
 
 +
Here are detailed steps for setting up to use the XMLTV format. In the following, <pre>$HOME/.mythtv/SD.xmltv</pre>
 +
is used as an example. That works for a user logged in (super used to) mythtv. Another example: <pre>~mythtv/.mythtv/SD.xmltv</pre>
 +
eliminated the confusion of the $HOME value. Users on systems that have a mythtv user that has no home directory/shell should
 +
simply use the full path to the .xmltv file. the ''sdjson'' grabbers don't understand $MYTHCONFDIR.
 +
 
 +
Be sure to read: [[#MythTV_Setup]].
 +
{{Warning}}Beginning in v31, the Data Direct format is no longer supported. Also, the mythfilldatabase --dd-grab-all switch is no longer valid. Be sure to remove it with mythtv-setup (General -> Program Schedule Downloading Options -> Guide Data Arguements).
 +
 
 +
==Types of Schedules Direct Grabbers==
 +
As XMLTV-from-Schedules Direct is supported using JSON, there are two grabbers that can be used for downloading from Schedules Direct:
 +
* tv_grab_zz_sdjson_sqlite - a JSON grabber that utilizes a SQL Lite database-based configuration file
 +
* tv_grab_zz_sdjson - a JSON grabber that utilizes a text-based configuration file
  
# [http://minnie.tuhs.org/tivo-bin/tvguide.pl OzTivo Guide] (Free, requires registration)
+
If you run <code>tv_find_grabbers</code> you should see these included in the list:
# http://www.icetv.com.au (AU$3 per week subscription)
+
  tv_find_grabbers | grep Schedules
# [http://www.cse.unsw.edu.au/~willu/xmltv/index.html Australian XMLTV grabbers] (ice + tvguide)
+
  /usr/bin/tv_grab_zz_sdjson|Schedules Direct JSON API
# [http://svn.whuffy.com/wiki Shepherd ] (Free : Ice,OzTivo hybrid multi source )
+
  /usr/bin/tv_grab_zz_sdjson_sqlite|Multinational (Schedules Direct JSON web services with SQLite DB)
# http://immir.com/tv_grab_au (Free) (may be inactive/broken now)
 
# http://www.onlinetractorparts.com.au/rohbags/xmltvau/ (not confirmed if working)
 
  
=== Belgium ===
+
{{Tip box|You should obtain the latest version, since there have been recent changes. When you download the a more recent version, you must ensure your copies of the programs are found before the version provided for by your Linux distro - do this by placing ensuring you either replace the versions directly in <code>/usr/bin</code>, or adjusting <code>PATH</code> so that your downloaded version is found before the existing version in <code>/usr/bin</code>.}}
  
# https://launchpad.net/belguide
+
Use one or the other: The <code>tv_grab_zz_sdjson_sqlite</code> has more options and needs some command line setup that cannot be done from the <code>mythtv-setup</code> program. Both of them cache data allowing them to download a minimal amount of data when running. Only changes since the previous download are downloaded.
# http://users.skynet.be/jxmltv
 
# http://pytvgrab.sourceforge.net/
 
# http://membled.com/work/apps/xmltv/
 
# http://mc2xml.110mb.com/ : this one downloads its information from Microsoft. It is intended for MS Windows media center edition. ATM in Belgium it works better than xmltv but you might want to change channel names since they're in the xml file in pretty non-human parseable names. This script works pretty well:
 
  
./mc2xml -c be -g postcode -o xmltv.xml
+
==Setup with tv_grab_zz_sdjson_sqlite==
echo Chan namen veranderen ...
+
If you want the very latest version of this you can download it from [https://github.com/garybuhrmaster/tv_grab_zz_sdjson_sqlite github].
cat replacements | while read line
 
do
 
        sed -e "s/$line/g" -i xmltv.xml
 
done
 
mythfilldatabase --update --refresh-all --file 1 xmltv.xml
 
 
   
 
   
replacements contains lines like:
+
Run MythTV Setup and select ''Multinational (Schedules Direct JSON web services with SQLite DB)'' for your listings source.
 +
 
 +
From the command line
 +
<pre>tv_grab_zz_sdjson_sqlite --manage-lineups --config-file $HOME/.mythtv/SD.xmltv</pre>
 +
Replace SD with the name of the source you had set up in MythTV Setup.
 +
* Enter your Schedules Direct user and password when prompted.
 +
* Select the ''Initialize/update the local database'' option.
 +
* Accept the default database name.
 +
* Accept the password hash question.
 +
* Add or delete lineups using the prompts.
 +
Do not go into channel selection yet. Run this:
 +
<pre>tv_grab_zz_sdjson_sqlite --configure --config-file $HOME/.mythtv/SD.xmltv</pre>
 +
* Supply the database name you selected above.
 +
* Accept defaults for the prompted questions.
 +
Run these:
 +
<pre>tv_grab_zz_sdjson_sqlite --days 0 --config-file $HOME/.mythtv/SD.xmltv
 +
tv_grab_zz_sdjson_sqlite --manage-lineups --config-file $HOME/.mythtv/SD.xmltv</pre>
 +
* If you get all channels in the lineup, then select the options that will make current and future channels selected.
 +
* If you only get a few of the channels you will need to select the appropriate channels. Select a default for existing and future channels, then use option 5 to select which channels you need. This goes though all channels one at a time. You have to complete this for every channel in the list. You cannot stop partway through.
 +
* If you need to do the channel selection again because of errors or channel changes, your previous values are remembered and used as defaults.
 +
 
 +
===Channel selection shortcut===
 +
The channel selection process with tv_grab_zz_sdjson_sqlite is tedious and error prone. Here is a quick and dirty python script to make it easier:
 +
<pre>
 +
#!/usr/bin/python
  
I1.750821.microsoft.com/Een
+
# -*- coding: utf-8 -*-
I2.750822.microsoft.com/Ketnet-Canvas
+
#---------------------------
 +
# Name: sdjson-sqlite-select
 +
#---------------------------
 +
__title__  = "sdjson-sqlite-select"
 +
__version__ = "v0.1.0"
  
The first time I imported channels, I cut out every channel from the xml file (not so difficult) that I can't receive (-> cleaner guide in mythtv).
+
'''
 +
List and update selected status of available callsigns
 +
'''
  
You can also get your XML file directly from http://mythportal.be/XML-tv-gids
 
  
=== Brazil ===
+
import time
 +
import argparse
 +
from argparse import RawDescriptionHelpFormatter
 +
import os
 +
import sys
 +
import logging
 +
import traceback
  
There are a working grabber, tv_grab_br_net a perl scripts (or any other Linux native solutions) for Brazilian TV programming. This grabber get data from Net [http://www.net.tv.br] cable TV. No open to air TV on this source, just paid tv programme. It is detected by mythtvsetup and you are prompted (on background windows) to make your choice about your city, line-up and channels. It provide both, simple and complete description for programs.
+
import pymysql
 +
import sqlite3
 +
import json
 +
 
 +
 
 +
def process_command_line():
 +
    '''All command line processing is done here.'''
 +
 
 +
    appname = os.path.basename(sys.argv[0])
 +
 
 +
    examples = (('\nExamples:\n'
 +
                '  Extract all callsigns from /home/mythtv/SchedulesDirect.DB '
 +
                'and place them in /tmp/callsigns:\n'
 +
                '    {0} --list --database /home/mythtv/SchedulesDirect.DB '
 +
                '--file /tmp/callsigns\n'
 +
                '\n'
 +
                '  Set selected status in /home/mythtv/SchedulesDirect.DB '
 +
                'from callsigns in /tmp/callsigns:\n'
 +
                '    {0} --set --database /home/mythtv/SchedulesDirect.DB '
 +
                '--file /tmp/callsigns\n'
 +
                '\n'
 +
                'The callsigns file will list all the callsigns. Any callsign '
 +
                'with a leading explamation point (aka bang: "!") is/will be '
 +
                'deselected.\n\n'
 +
                'When callsigns are listed to the file, The callsigns will be '
 +
                'sorted, with selected callsigns at the bottom of the file'
 +
                '\n\n'
 +
                'Use --list to generate file, then edit the file to add "!" '
 +
                'in front of any callsigns you do not want downloaded. Use '
 +
                '--set to apply those changed to the database.')
 +
                .format(appname)
 +
                )
 +
 
 +
    parser = argparse.ArgumentParser(description='List and update selected '
 +
                                    'status of available callsigns.',
 +
                                    formatter_class=RawDescriptionHelpFormatter,
 +
                                    epilog='{}'.format(examples))
 +
 
 +
    parser.add_argument('--debug', action='store_true',
 +
                        help='turn on debug messages (%(default)s)')
 +
 
 +
    parser.add_argument('--version', action='version',
 +
                        version='%(prog)s 0.1.0')
 +
 
 +
    parser.add_argument('--file', type=str, required=True,
 +
                            metavar='<filename>',
 +
                            help='File for callsigns')
 +
 
 +
    parser.add_argument('--database', type=str, required=True,
 +
                            metavar='<filename>',
 +
                            help='File for callsigns')
 +
 
 +
    parser.add_argument('--set', action='store_true',
 +
                        help='Set selected status in SchedulesDirect.DB '
 +
                        'from <file>')
 +
 
 +
    parser.add_argument('--list', action='store_true',
 +
                        help='List available callsigns into <file>')
 +
 
 +
    return vars(parser.parse_args())
 +
 
 +
 
 +
 
 +
def list_callsigns(args):
 +
    db = sqlite3.connect(args['database'])
 +
    cursor = db.cursor()
 +
 
 +
    channels = {}
 +
 
 +
    for row in cursor.execute('SELECT station,selected from channels'):
 +
        channels[row[0]] = bool(row[1])
 +
 
 +
    result = []
 +
    for row in cursor.execute('SELECT details from stations'):
 +
        data = json.loads(row[0])
 +
        v = '' if channels[data['stationID']] else '!'
 +
        result.append('{}{}'.format(v, data['callsign']))
 +
 
 +
    result.sort()
 +
    with open(args['file'], 'w') as f:
 +
        for callsign in result:
 +
            f.write('{}\n'.format(callsign))
 +
       
 +
    return True
 +
 
 +
def set_selected(args):
 +
    db = sqlite3.connect(args['database'])
 +
    cursor = db.cursor()
 +
 
 +
    changed = 0
 +
 
 +
    channels = {}
 +
    with open(args['file'], 'r') as file:
 +
        for callsign in file:
 +
            if callsign[0] == '!':
 +
                selected = False
 +
                callsign = callsign[1:]
 +
            else:
 +
                selected = True
 +
            channels[callsign.strip()] = selected
 +
 
 +
    stations = {}
 +
    for row in cursor.execute('SELECT details from stations'):
 +
        data = json.loads(row[0])
 +
        stations[data['callsign']] = data['stationID']
 +
        if args['debug']:
 +
            print('callsign {} = station {}'
 +
                  .format(data['callsign'], data['stationID']))
 +
 
 +
    for chan in channels.keys():
 +
        if args['debug']:
 +
            print('Checking {}'.format(chan))
 +
        st = (stations[chan], )
 +
        for row in cursor.execute('SELECT selected from channels '
 +
                                  'where station=?', st):
 +
            if bool(row[0]) != channels[chan]:
 +
                print('{}: selected {} -> {}'.format(chan, bool(row[0]),
 +
                                                    channels[chan]))
 +
                st = (channels[chan], stations[chan], )
 +
                cursor.execute('UPDATE channels SET selected=? WHERE '
 +
                              'station=?', st)
 +
                db.commit()
 +
                changed += 1
 +
 
 +
    print('Modified {} channels'.format(changed))
 +
    return True
 +
 
 +
   
 +
def main():
 +
    args = process_command_line()
 +
 
 +
    opts = dict()
 +
 
 +
    if args['set']:
 +
        return set_selected(args)
 +
    elif args['list']:
 +
        return list_callsigns(args)
 +
 
 +
    return False
 +
 
 +
 
 +
if __name__ == '__main__':
 +
    if main():
 +
        sys.exit(0)
 +
    else:
 +
        sys.exit(1)
 +
</pre>
 +
Use --help to get examples on how to run it.
 +
 
 +
An alternative method is as follows. First configure the grabber to select no channels.
 +
Install sqlite3.
 +
List the tables in the sqlite database. If you did not use the default database name, enter the one that you used.  You may have to specify your database name with /home/mythtv instead of $HOME. In that case first make sure the file is writeable by your login user id.
 +
<pre>sqlite3 $HOME/.xmltv/SchedulesDirect.DB
 +
SELECT name FROM sqlite_master WHERE type='table';</pre>
 +
List the channels.
 +
<pre>sqlite3 $HOME/.xmltv/SchedulesDirect.DB
 +
select * from channels;</pre>
 +
You can copy the results from the screen, or pipe the results to a file. The value for ''selected'' is 0 if unselected or 1 if selected. Edit the file and create a text file like this to set the ones you want to select:
 +
<pre>update channels set selected = 1 where channum = 2;
 +
update channels set selected = 1 where channum = 3;
 +
...
 +
update channels set selected = 1 where channum = 983;
 +
update channels set selected = 1 where channum = 986;</pre>
 +
Update them with a command like this:
 +
<pre>sqlite3 $HOME/.xmltv/SchedulesDirect.DB < selected.sql</pre>
 +
 
 +
===Converting from DataDirect to XMLTV===
 +
There are several reasons to convert your Datadirect setup to use XMLTV.
 +
* You get 18 days of data instead of 14.
 +
* You get icons for all channels.
 +
* Season and episode information are included in listings.
 +
 
 +
If you decide to do this, first complete the setup of tv_grab_zz_sdjson_sqlite described above.
 +
 
 +
When you switch, the xmltvid's for the existing channels will change. Using this grabber the xmltvid has the format <code>I11460.json.schedulesdirect.org</code>. Using DD the xmltvid's are a numeric value.
 +
 
 +
There are two options for the conversion:
 +
# Delete all your channels and let mythfilldatabase add them back. This option is not available in cases where you have to scan for channels using DVB or ATSC cards.
 +
# Manually update each channel xmltvid.
 +
 
 +
I recommend that you backup your database and make sure there are no recordings scheduled for the near future before starting.
 +
 
 +
====Deleting and re-adding channels====
 +
 
 +
This is only possible for card types that do not need scanning for channels.
 +
 
 +
Before attempting this bear in mind the following. If any channel's chanid changes, your existing programs will show the wrong channel that the program was recorded on.
 +
 
 +
The mechanisms in MythTV for adding channels use a format of sourceid and channel number to create a chanid wherever possible. So channel 2 on source id 2 will be chanid 2002, for example. If it cannot do that then it will allocate a chanid number which is unrelated to the channel number or source id.
 +
 
 +
In order to see the chanids you will have to look at the channel table in the database. See if your existing chanid numbers match the scheme described in the previous paragraph.
 +
 
 +
If the chanids match the scheme above you should be able to safely delete them and let mythfilldatabase add them.
 +
 
 +
You can delete all of the existing channels for each video source in mythtv-setup, channel editor. Select the Video source and click '''Delete'''.
 +
 
 +
To add all of the channels you have selected as well as listings, run mythfilldatabase. Default options will add channels and listings.
 +
 
 +
====Manually Updating xmltvids====
 +
 
 +
If you prefer not to delete your channels or you have a DVB type tuner you will have to update the xmltvids in all of them. Using this grabber the xmltvid has the format <code>Innnnn.json.schedulesdirect.org</code>. You will have to change all of your existing ids to this format. The easiest place to make this change is with mythweb channel editor, although you can use the mythtv-setup channel editor as well. You can get the list of channels with their new format xmltvids by running <code>tv_grab_zz_sdjson_sqlite --days 0 --config-file $HOME/.mythtv/SD.xmltv</code>, which is one step of the setup process (see above). If you did not capture the output you can run this again at any time, no harm will be done.
  
Other functional alternative is to use [http://www.tvxb.com TVxb], which isn't open source or even Linux, but works well with wine (at least as far as version 1.0085).
+
Once you have updated all of the xmltvid's, run mythfilldatabase. Your channel names and icons will be updated to match the values supplied by schedules direct.
  
TVxb is configurable for different countries via ini-files.  Check the following page for Brazilian templates.  As of today, there are some for NET (PR), and others for TVA as well.
+
In a tuner type that does not use scanning, such as a cable card tuner, any channels not found with matching xmltvids are added. If you have missed updating some xmltvids that could result in duplication of the channel.
  
http://www.tvxb.com/country/br-ini.html
+
==Setup with tv_grab_zz_sdjson==
 +
Run MythTV Setup and select ''Schedules Direct JSON API'' for your listings source.
  
For information on how to set it up with MythTV, check out [[TVxb]].
+
You can complete the setup using the ''Configure'' button in MythTV Setup. If you prefer to use the command line, run
 +
<pre>tv_grab_zz_sdjson --configure --config-file $HOME/.mythtv/SD.xmltv</pre>
 +
Select the channel option: Select mode: [lineups,channels (default=lineups)] channels
 +
Replace SD with the name of the source you had set up in MythTV Setup.
 +
* Enter your Schedules Direct user and password when prompted.
 +
* For channel id format you can select any of the supplied options. Option 0 is the same format as used by tv_grab_zz_sdjson_sqlite. Option 2 is the format used by Schedules Direct Data Direct.
 +
* Follow the prompts to complete setup.
 +
* When selecting channels the selection process allows you to skip remaining channels in the list and mark them all as yes or no.
 +
* If you need to do the channel selection again because of errors or channel changes, your previous values are lost. You have to do the entire selection again.
  
=== Croatia ===
+
===Channel selection shortcut===
 +
The channel selection process with tv_grab_zz_sdjson is tedious and error prone.
 +
An alternative method is as follows. First configure the grabber to select all channels using: <pre>tv_grab_zz_sdjson --configure --config-file $HOME/.mythtv/SD.xmltv</pre>.
 +
Edit the file <code>$HOME/.mythtv/SD.xmltv</code> (replace SD with the name of the source you had set up in MythTV Setup). You will see a bunch of lines like this.
 +
<pre>channel=74348
 +
channel=11460</pre>
 +
When a channel is selected it has '''=''', when unselected it has '''!'''. You need to set the channels you want to use '''=''' and the others to '''!'''.
  
The grabber for Croatia tv_grab_hr is using the goNIX service as its backend.
+
Get a list of channels by running
 +
<pre>tv_grab_zz_sdjson --days 0 --config-file $HOME/.mythtv/SD.xmltv > list.xml</pre>
 +
Although this requests 0 days it actually produces 1 day of programs. At the start of the file before the programs is the channel listing. Delete the program schedules and keep the channel listing at the top of the file. The channel id and display names identify each channel by its XMLTV id and the number and name. Note that the channel id in this file may have extra characters appended depending on the format you chose during configure. For purpose of the configuration file you only need the 5 numeric digit of the channel id. Using your favorite editor and your provider's channel list update the channel lines in the configuration file to have '''=''' for ones you want and '''!''' for ones you do not.
  
More information is available on http://www.gonix.net/
+
= List of Grabbers =
 +
{{note_box|Not all grabbers on this page are XMLTV compliant. Only fully compliant grabbers are supported by MythTV and we advise against the use of the rest.
  
 +
If you know a grabber listed here does not conform entirely to both the xmltv grabber specifications ([http://wiki.xmltv.org/index.php/HowtoWriteAGrabber XMLTV grabber spec]) and XMLTV xml format ([http://wiki.xmltv.org/index.php/XMLTVFormat XMLTV Format]) then please mark it as unsupported. If in doubt XMLTV provide a script to validate a grabber ([http://wiki.xmltv.org/index.php/XmltvValidation XMLTV Validator])}}
  
=== Czech Republic ===
+
There is also a list on the [http://wiki.xmltv.org/index.php/XMLTVProject XMLTV wiki].
  
The only way to get TV listings in Czech Republic is [[TVxb]], there is no grabber for xmltv.
+
== EPG Collector ==
 +
[http://sourceforge.net/projects/epgcollector/ EPG Collector] is a .NET/Mono program for collecting the transmitted guide data from DVB or ATSC streams in various formats and creating an XMLTV file.
  
Not public xmltv file for czech and slovak channels (UPC channels included) can be send after request to [mailto:rfordinal@gmail.com rfordinal].
+
== Multinational ==
 +
[[Schedules Direct]] supports countries in North America, South America, Caribbean and Europe for $25 (US) per year. For a list of supported countries see http://schedulesdirect.org/regions. See [[#Schedules_Direct_Setup]] above
  
=== Denmark ===
+
== Australia ==
 +
Refer to [[Australian TV Listings]] for further details.
  
In Denmark there is a group maintaining several grabbers. This group has created the following:
+
# [http://www.oztivo.net/ OzTivo Guide] (Free, requires registration)
 +
# http://www.icetv.com.au (AU$99 per year subscription)
 +
# [https://github.com/ShephedProject/shepherd Shepherd] (Free: hybrid multi source)
  
* A Wiki located at http://niels.dybdahl.dk/xmltvdk
+
See also this (largely historical) [http://www.cse.unsw.edu.au/~willu/xmltv/index.html list of Australian grabbers].
* A mailing list located at http://uk.groups.yahoo.com/group/xmltvdk
 
* A sourceforge repository located at http://sourceforge.net/projects/xmltvdk
 
  
Currently seven grabbers are maintained:
+
== Belgium ==
  
* The original (tv.tv2.dk),  
+
The following is a list of grabbers for Belgium. It is presented only for historical purposes, because at the time of writing this (January 2011) none of them work.
* An additional written in perl (dr.dk)
 
* Five written in Python (tdc.dk, tv-guiden.dk, ahot.dk, jubii.dk, ontv.dk).  
 
  
The group also maintains the following:
+
# https://launchpad.net/belguide
 +
# http://users.skynet.be/jxmltv
 +
# http://pytvgrab.sourceforge.net/
  
* A merger written in Python to combine xmltv information from multiple sources, to obtain the highest possible quality (merge end time of last show from tdc.dk with episode numbers from tv2.dk and long descriptions from tv-guiden.dk etc.).
+
== Brazil ==
* A "combined" grabber, which configures each of the other grabbers and runs them in parallel and then merge the output. That combined grabber does also support the swedish grabber (swedb).
 
  
The mailing list is used to inform users when there are updates or problems with the current grabbers.
+
There are a working grabber, tv_grab_br_net a perl scripts (or any other Linux native solutions) for Brazilian TV programming. This grabber get data from Net [http://www.net.tv.br] cable TV. No open to air TV on this source, just paid tv programme. It is detected by mythtvsetup and you are prompted (on background windows) to make your choice about your city, line-up and channels. It provide both, simple and complete description for programs.
  
=== Finland ===
+
Other functional alternative is to use [http://www.tvxb.com TVxb], which isn't open source or even Linux, but works well with wine (at least as far as version 1.0085).
Finnish XMLTV data can be acquired with tv_grab_fi. The script parses www.telkku.com web site in order to get EPG entries.
 
  
Latest version of the tv_grab_fi can be downloaded from: <br>
+
TVxb is configurable for different countries via ini-files. Check the following page for Brazilian templates. As of today, there are some for NET (PR), and others for TVA as well.
  http://xmltv.cvs.sourceforge.net/*checkout*/xmltv/xmltv/grab/fi/tv_grab_fi <br>
 
Due frequent site updates of the www.telkku.com user should always have the latest version of the grabber. <br>
 
Add latest version of tv_grab_fi somewhere where mythtv can find it for example in debian system to /usr/bin/. Also remember to assure that you don't have any older versions installed.  
 
  
To use tv_grab_fi you need to first run in ~/.mythtv
+
http://www.tvxb.com/country/br-ini.html
tv_grab_fi --configure --config-file dvb.xmltv <br>
 
This will ask which channels does the user want to use grabber for. <br>
 
Once you have generated dvb.xmltv it should contain lines like this 'channel 0 YLE1'. Channels with '#' in front of them will not be added.
 
  
Now we need to tell XMLTV ID's to mythtv. In dvb.xmltv take the number after word "channel" and add ".telkku.com" after that and you'll have the ID. For example if the channel has line of 'channel 62 YLE24' the ID is 62.telkku.com. <br>
+
For information on how to set it up with MythTV, check out [[TVxb]].
Start Mythtv setup <br>
 
mythtv-setup <br>
 
Go to: 'channel editor' -> "channel that you want to use grabber for" -> XMLTV ID: xx.telkku.com (Where 'xx' is the number in dvb.xmltv) -> Next -> Ready.
 
  
To update program guide do
+
== Croatia ==
mythfilldatabase
+
The grabber in the official XMLTV distribution works correctly. More information is available on http://www.gonix.net/.
If that doesn't add EPG entries then do:
 
mythfilldatabase --refresh-all
 
  
Another way to get XMLTV ID's is to do
+
== Czech Republic ==
tv_grab_fi --config-file dvb.xmltv --days 0 --output channels.tmp
 
In channels.tmp find line that has '<channel id="...>' in it. In place of "..." you will have the XMLTV ID. For example if the line is: "<channel id="2.telkku.com">" then the XMLTV ID is 2.telkku.com.
 
  
=== France ===
+
The XMLTV grabber for "Hungary/Romania" was extended to the Czech Republic.
  
This web site [http://www.kazer.org/] provides free and very complete xmltv listings as a downloadable XML file. You will need to run mythfilldatabase with the --file argument to use the downloaded xml.
+
Another way to get TV listings in Czech Republic is [[TVxb]].
  
=== Germany/Austria ===
+
Not public XMLTV file for czech and slovak channels (UPC channels included) can be send after request to [mailto:rfordinal@gmail.com rfordinal].
====TV Today====
 
German/Austrian XMLTV Data can be got via the tv_grab_de_tvtoday web scrapper script. It has the episodes of series all mixed up in he main title. The part Number ('Teil') of the series are not always nicely structured and spill over to the title and subtitle  field. This makes it quite impossible to  record every sequence of a series (if you do not want to make a time based recording on a single channel). I decided to write a small mySQL script to update the programm data after performing the daily load of the xmltv file. This mySQL statment is batched after the daily mythfilldatabase entry in my CRON Script. The Statment looks for sequences That have the Word 'Teil' in it. This is the German word for 'Part'.
 
  
<pre><nowiki>UPDATE IGNORE program
+
== Denmark ==
    SET    chanid=chanid
 
          ,starttime=starttime
 
          ,subtitle= concat_ws(' ', substring_index(substring(title,locate('Teil', title)-6,4+20), ' ', -2), subtitle )
 
          ,title= left(title,locate(substring_index(substring(title,locate('Teil', title)-6,4+20), ' ', -2),title) -1)
 
    WHERE title like '%Teil%';</nowiki></pre>
 
  
====Prisma====
+
In Denmark there is a group maintaining several grabbers. This group has created the following:
More information [http://www.mythwiki.de/index.php?title=HOWTO_Prisma_EPG here]. Download via [http://sfr.mythwiki.de this] or [[http://hamsta.net/myth_epg.html this]] link. The Prisma grabber is way slower than the TV Today grabber but allows to fetch 4 weeks in advance compared to 1 week for the TV Today grabber.
 
  
====TV Movie====
+
* A Wiki located at http://niels.dybdahl.dk/xmltvdk
The TV Movie paper has changed its EPG data publishment in January 2009, breaking the [http://mythtv.linux-dude.de old script].
+
* A mailing list located at http://uk.groups.yahoo.com/group/xmltvdk very 10 minutes. Only title and descr
 +
* A sourceforge repository located at http://sourceforge.net/projects/xmltvdk
 +
* Another source of info: http://xmltv.se
  
A script for the new format can be found [http://swolter.sdf1.org/software/article-tvmovie.html here].
+
Currently three grabbers are maintained:
This grabber is very fast (runs in a few seconds) and offers up to 8 days of program data for all
 
free-TV stations, including ratings, comments and actor lists.
 
  
====XMLTV.info====
+
* DR 2012 grabber written in perl (dr.dk)
You can download the TV programm as xmltv file from [http://xmltv.info xmltv.info]. There is also a fast grabber available to automatically fetch the file.
+
* An additional written in python (yousee.dk)
 +
* An universal grabber combining data from mutiple sources (tvtid.tv2.dk, yousee.dk, tv-guiden.dk, jubii.dk, ontv.dk, ahot.dk, dr.dk and swedb)
  
=== Italy ===
+
== Finland ==
The grabber in the official xmltv distribution works correctly.
+
The grabbers in the official XMLTV distribution work correctly.
 +
Alternate source: http://xmltv.se
  
=== Netherlands ===
+
== France ==
Netherlands (Holland) specific info can be found here: [[Netherlands]]
 
  
=== New Zealand ===
+
The grabber in the upcoming XMLTV release works correctly. More information is available on [http://www.kazer.org/].
A c# application to scrape sites can be found here, runs fine using mono on linux
 
# http://www.reven.co.nz/
 
  
=== Norway ===
+
== Germany/Austria ==
 +
# [http://xmltv.se/ XMLTV.se] offers many of the main channels for personal use, the grabber from the main xmltv distribution works.
 +
# Some channels are available via [http://xmltv.spaetfruehstuecken.org/xmltv/ Egon Zappt] in the upcoming release of xmltv.
 +
# TV Movie can not give out their data without additional license, see [http://www.bucksch.org/1/projects/various/xmltv/ Ben Bucksch's log].
  
To use the XMLTV Data for Norway, you will have to use:
+
== Greece ==
tv_grab_no --configure --config-file=tv_grab_no.xmltv
 
It will then ask you what channels you would like.
 
Next copy the file to your ''.mythtv'' directory.
 
  
Next, you have to get the tags for each channel.. So let's fetch the data and save it to a tmp file.
+
Try this link for Nova Greece: [http://sgcpm.com/epg/devilcosta.gz http://sgcpm.com/epg/devilcosta.gz], for example using mythfilldatabase. (Be aware that its just a dump of DVB-EIT using the service_id as channel_id which is not proper xmltv. Please consider writing a proper grabber for the file so it can be configured via the GUI or simply provide a proper guide feed with tools like [http://wiki.xmltv.org/index.php/NonameTV NonameTV].)
tv_grab_no --config-file=tv_grab_no.xmltv --list-channels --output=channels.tmp
 
  
Open that file and write down each tag for each channel. Look for lines like:
+
== India ==
<channel id="nrk1.nrk.no">
+
The grabber for What's-On-India has been added to the official XMLTV distribution.
  
As you will see in the file: '''nrk1.nrk.no''' is for '''Nrk1'''.  
+
An XMLTV grabber for all channels in India can be found at [http://ravihtpc.wordpress.com/2010/01/09/india-epg-data-for-mythtv-xmltv-listings-grabber/ http://ravihtpc.wordpress.com/2010/01/09/india-epg-data-for-mythtv-xmltv-listings-grabber/].
  
So now when you run mythtv-setup --> channel editor, in order to capture '''Nrk1''', you need to set the '''XMLTV ID''' to '''nrk1.nrk.no'''.
+
== Italy ==
 +
The grabber in the official XMLTV distribution works correctly.
  
Do this for all the channels. When you are finally finished with doing all that, just run
+
== Hong Kong ==
mythfilldatabase
+
First thanks to http://www.akbkhome.com/blog.php/View/119/Mythtv_xmltv_importer_for_HK.html.
  
When its finished, start up mythfrontend.
+
I modify his source code to get information from TVB new web site.
  
=== South Africa ===
+
    download Tv.php http://mythtv-xmltvhk.googlecode.com/files/Tv.php
 +
    download hongkong.ini http://mythtv-xmltvhk.googlecode.com/files/hongkong.ini
 +
    build xml from tvb, atv website using following command, just entry
 +
          php ./Tv.php hongkong.ini > hongkong.xml
 +
    using 'myfilldatabase' to update mythtv schedule program, just entry
 +
          mythfilldatabase --file 1 hongkong.xml
  
See the [[South Africa MUG]] WIKI page for details.
+
    (* it is my first development on php, so it may be simplified *)
 +
[Last Update by hwkit (17/04/2010)]
  
=== Spain ===
+
Since TVB web site is changed, the program need to be updated.  (My php skill is poor, so there are error when running, however it also can get schedule from TVB web site), I also change the program name called 'xmltvhk.php'. Please download it from google.
 +
     
 +
    download xmltvhk.php and copy it to /etc/xmltvhk (http://mythtv-xmltvhk.googlecode.com/files/xmltvhk.php)
 +
    download updated hongkong.ini and copy it to /etc/xmltvhk (http://mythtv-xmltvhk.googlecode.com/files/hongkong.ini)
 +
    download filltv  and copy it to /etc/cron.weekly (make it executable (chmod +x /etc/cron.weekly/filltv) (http://mythtv-xmltvhk.googlecode.com/files/filltv)
  
The included parser works (www.elpais.es/parrillatv/index.html). Although it has been found some inconsistencies with the real programation.
+
    Source code
 +
    svn checkout http://mythtv-xmltvhk.googlecode.com/svn/trunk/ mythtv-xmltvhk-read-only
  
One alternative, abeit not been as complete: http://www.miguiatv.com/todos-los-canales.html
+
[Last Update by hwkit (07/07/2010)]
  
=== Sweden ===
+
== Netherlands ==
 +
Netherlands (Holland) specific info can be found here: [[Netherlands]]
  
# http://tv.swedb.se/
+
== New Zealand ==
 +
Sky TV threatened legal action to get the [http://reven.co.nz/post/whatever-happened-to-xmltvnz xmltv grabber for NZ taken down].
 +
You can still receive the transmitted guide in MHEG5 with tools like [[XMLTV#EPG_Collector|EPG Collector]].
  
=== Switzerland ===
+
== Norway ==
 +
The grabber in the official XMLTV distribution works correctly.
 +
Another source of info: http://xmltv.se
  
# [[XMLTV_CH]]: Info on available grabbers and icon listings.
+
== Slovakia ==
# [http://xmltv.info xmltv.info] offers an xmltv file for download which includes the TV programm for a number of Swiss channels.
+
The XMLTV grabber for "Hungary/Romania" was extended to Slovakia.
  
=== United Kingdom ===
+
== South Africa ==
 +
See the [[South Africa MUG]] WIKI page for details.
  
# [[Uk_xmltv]]
+
== Spain ==
 +
Both grabbers in the official XMLTV distribution work correctly.
  
A brief guide to configuring tv_grab_uk_rt is available at the link above.
+
== Sweden ==
 +
The grabber in the official XMLTV distribution works correctly. More information is available on http://xmltv.se.
  
=== Ukraine, Russia, Belarus ===
+
== Switzerland ==
[http://www.free-x.de/xmltv/ tv_grab_ru] applicalion, converts JTV-formatted arcive with about 200 channels to xmltv. But this application does not show additional info about programmes.
+
The grabber in the official XMLTV distribution works correctly.
  
[http://tipok.org.ua/downloads/media/mythtv/tv_grab_ua/tv_grab_ua tv_grab_ua] applicalion, downloads xmltv-formatted data from sites like [http://www.teleguide.info/article1.html teleguide], [http://izbushka.kiev.ua/television/xmltv/ izbushka] or [http://www.star.poltava.ua/index.php?id=132 star.poltava.ua]. In addition it parses existing data and adds category/rating/credits into output xmltv file.
+
== United Kingdom ==
 +
The grabbers in the official XMLTV distribution work correctly.
 +
A brief [[Uk_xmltv|guide to configuring tv_grab_uk_rt]] is available, and one for [[Atlas|Metabroadcast Atlas]].
  
 +
== Ukraine, Russia, Belarus ==
 +
# [http://www.free-x.de/xmltv/ tv_grab_ru] Converts JTV-formatted archive with about 200 channels to XMLTV. But this application does not show additional info about programmes.
 +
# [http://tipok.org.ua/downloads/media/mythtv/tv_grab_ua/tv_grab_ua tv_grab_ua] Downloads XMLTV-formatted data from sites like [http://www.teleguide.info/article1.html teleguide], [http://izbushka.kiev.ua/television/xmltv/ izbushka] or [http://www.star.poltava.ua/index.php?id=132 star.poltava.ua]. In addition it parses existing data and adds category/rating/credits into output XMLTV file.
 +
# [http://www.teleguide.info/download/new3/tvguide.zip teleguide.info] XMLTV-formatted zip archive with programmes inside.
  
 +
== Latvia ==
  
 
[[Category:HOWTO]]
 
[[Category:HOWTO]]
 
[[Category:EPG Providers]]
 
[[Category:EPG Providers]]
 +
[[Category:Post Installation Tasks|1300]]

Latest revision as of 18:12, 27 March 2024


Introduction

XMLTV is the name of a system which obtains and processes program guide information. If you are in North America you can use the Schedules Direct grabber which now supports XMLTV format as well as the old Data Direct format (until v31 and above.) Using the XMLTV format with Schedules Direct gets you more days of data than the old Data Direct option. All users in the USA and other countries need to install and configure XMLTV in order to obtain program guide listings.

Installation

Install the xmltv package on your master backend server, using your distribution's package manager.

The ubuntu packages and others e.g. Debian, Raspbian are likely to have out of date version of xmltv. You can get the latest version as follows in a terminal session:

sudo apt update
sudo apt install git make xmltv perl-doc
# Note: xmltv gets the required perl modules

mkdir -p build
cd build
git clone https://github.com/XMLTV/xmltv.git
cd xmltv
perl Makefile.PL
make
make test
sudo make install

The latest version is installed in /usr/local/bin, which will be used in preference to the original versions in /usr/bin/

use tv_find_grabbers to list all grabbers, including location

Info on XMLTV is here: https://github.com/XMLTV/xmltv.

Configuration

Configuration of XMLTV must be done on the system where mythfilldatabase will be run. Normally that will be your primary backend machine, but you can run it in other places if you prefer.

MythTV Setup

If user mythtv is allowed to login and start the window manager, XMLTV can be configured through the Video Sources Setup page in mythtv-setup. Otherwise, follow the steps below and do the configuration from the command line. Depending on the grabber, it will prompt for any information it requires. If it comes down to selecting channels one at a time manually that can be a challenge if there are many channels.


Information.png Tip: Setup of XMLTV via command line is less challenging and more consistent with XMLTV instructions/usage.


Warning.png Warning: It seems users are missing the step that says run the setup as the user that will run mythfilldatabase (which is frequently user mythtv.)

Some grabbers may not be able to be fully configured through the MythTV setup interface.

The general steps are:

  1. Logon or su to the user that will run mythfilldatabase.
    • In the case of mythbuntu installation that is the user id mythtv.
  2. Run mythtv-setup using that user id.
    • If you do not want to logon to that id or you cannot, you can run the mythtv-setup from another id on the same backend.
  3. Setup Myth to use XMLTV following the instructions at Setup Video Sources.

The next steps are optional, and not recommended. By running all programs as user mythtv, as described above, duplicating the involved directories and files is not required.

  4. After completing setup, copy or link the contents of the $HOME/.xmltv directory to the mythtv home directory
  • The configuration file in $HOME/.xmltv MUST be renamed to the name supplied to mythtv-setup Video Source name. Example: Assuming you named the video source as xmltv, the configuration file name must be named $HOME/.mythtv/xmltv.xmltv.
  5. If copying or linking the files please make sure that the files and directories are read-write permitted by the mythtv user id.

Command Line Setup

Additional configuration can be done from the command line. As long as you have selected your xmltv grabber in MythTV Setup you can perform some or all configuration via the command line.

Logon to the user that will run mythfilldatabase (normally mythtv). As with mythtv setup if you cannot or will not, you can use any user id and then copy or link directories and files as described above.

List Grabbers

tv_find_grabbers

This gives a listing of grabbers, corresponding to the list you see in MythTV Setup. here is a small sample:

/usr/bin/tv_grab_zz_sdjson|Schedules Direct JSON API
/usr/bin/tv_grab_is|Iceland
/usr/bin/tv_grab_es_laguiatv|Spain (laguiatv.com)
/usr/bin/tv_grab_se_swedb|Sweden (swedb/tvsajten)
/usr/bin/tv_grab_combiner|Combine data from several other grabbers
/opt/mythtv/bin/tv_grab_zz_sdjson_sqlite|Multinational (Schedules Direct JSON web services with SQLite DB)

Find in the list the grabber you selected in MythTV Setup, and get the name of the executable from the beginning of the line. Find out information about it with the --info option.

tv_grab_zz_sdjson --info

That will give information on the usage of that grabber. Most options will be common to all grabbers. To use the --info option you need the perl-doc package installed.

Run the configuration

This is the same process that is run from MythTV Setup when you use the configure button in Video Source Setup.

tv_grab_xx_xxxxxx --configure --config-file $HOME/.mythtv/YYYY.xmltv

Use the name of the selected grabber. The value YYYY in the configuration file name corresponds to the Video Source Name. Use your video source name here. If your video source name includes spaces put quotes around it.

The grabber will prompt you required information. If your lineup includes a large number of channels and you do not subscribe to all of them, some grabbers will allow you to select the desired channels.

You should not include channels in your xmltv config which are not required. This places an unnecessarily high load on both the source websites and your own system.

For more information on editing channels see the Channel Editor [1]

Schedules Direct Setup

Schedules Direct now uses XMLTV only. In the USA the XMLTV format provides 18 days of data.

For a list of countries/regions supported see https://www.schedulesdirect.org/regions Up to 4 different lineups are supported at any one time.

Here are detailed steps for setting up to use the XMLTV format. In the following,
$HOME/.mythtv/SD.xmltv
is used as an example. That works for a user logged in (super used to) mythtv. Another example:
~mythtv/.mythtv/SD.xmltv

eliminated the confusion of the $HOME value. Users on systems that have a mythtv user that has no home directory/shell should simply use the full path to the .xmltv file. the sdjson grabbers don't understand $MYTHCONFDIR.

Be sure to read: #MythTV_Setup.

Warning.png
Beginning in v31, the Data Direct format is no longer supported. Also, the mythfilldatabase --dd-grab-all switch is no longer valid. Be sure to remove it with mythtv-setup (General -> Program Schedule Downloading Options -> Guide Data Arguements).

Types of Schedules Direct Grabbers

As XMLTV-from-Schedules Direct is supported using JSON, there are two grabbers that can be used for downloading from Schedules Direct:

  • tv_grab_zz_sdjson_sqlite - a JSON grabber that utilizes a SQL Lite database-based configuration file
  • tv_grab_zz_sdjson - a JSON grabber that utilizes a text-based configuration file

If you run tv_find_grabbers you should see these included in the list:

 tv_find_grabbers | grep Schedules
 /usr/bin/tv_grab_zz_sdjson|Schedules Direct JSON API
 /usr/bin/tv_grab_zz_sdjson_sqlite|Multinational (Schedules Direct JSON web services with SQLite DB)


Information.png Tip: You should obtain the latest version, since there have been recent changes. When you download the a more recent version, you must ensure your copies of the programs are found before the version provided for by your Linux distro - do this by placing ensuring you either replace the versions directly in /usr/bin, or adjusting PATH so that your downloaded version is found before the existing version in /usr/bin.

Use one or the other: The tv_grab_zz_sdjson_sqlite has more options and needs some command line setup that cannot be done from the mythtv-setup program. Both of them cache data allowing them to download a minimal amount of data when running. Only changes since the previous download are downloaded.

Setup with tv_grab_zz_sdjson_sqlite

If you want the very latest version of this you can download it from github.

Run MythTV Setup and select Multinational (Schedules Direct JSON web services with SQLite DB) for your listings source.

From the command line

tv_grab_zz_sdjson_sqlite --manage-lineups --config-file $HOME/.mythtv/SD.xmltv

Replace SD with the name of the source you had set up in MythTV Setup.

  • Enter your Schedules Direct user and password when prompted.
  • Select the Initialize/update the local database option.
  • Accept the default database name.
  • Accept the password hash question.
  • Add or delete lineups using the prompts.

Do not go into channel selection yet. Run this:

tv_grab_zz_sdjson_sqlite --configure --config-file $HOME/.mythtv/SD.xmltv
  • Supply the database name you selected above.
  • Accept defaults for the prompted questions.

Run these:

tv_grab_zz_sdjson_sqlite --days 0 --config-file $HOME/.mythtv/SD.xmltv
tv_grab_zz_sdjson_sqlite --manage-lineups --config-file $HOME/.mythtv/SD.xmltv
  • If you get all channels in the lineup, then select the options that will make current and future channels selected.
  • If you only get a few of the channels you will need to select the appropriate channels. Select a default for existing and future channels, then use option 5 to select which channels you need. This goes though all channels one at a time. You have to complete this for every channel in the list. You cannot stop partway through.
  • If you need to do the channel selection again because of errors or channel changes, your previous values are remembered and used as defaults.

Channel selection shortcut

The channel selection process with tv_grab_zz_sdjson_sqlite is tedious and error prone. Here is a quick and dirty python script to make it easier:

#!/usr/bin/python

# -*- coding: utf-8 -*-
#---------------------------
# Name: sdjson-sqlite-select
#---------------------------
__title__   = "sdjson-sqlite-select"
__version__ = "v0.1.0"

'''
List and update selected status of available callsigns
'''


import time
import argparse
from argparse import RawDescriptionHelpFormatter
import os
import sys
import logging
import traceback

import pymysql
import sqlite3
import json


def process_command_line():
    '''All command line processing is done here.'''

    appname = os.path.basename(sys.argv[0])

    examples = (('\nExamples:\n'
                 '  Extract all callsigns from /home/mythtv/SchedulesDirect.DB '
                 'and place them in /tmp/callsigns:\n'
                 '    {0} --list --database /home/mythtv/SchedulesDirect.DB '
                 '--file /tmp/callsigns\n'
                 '\n'
                 '  Set selected status in /home/mythtv/SchedulesDirect.DB '
                 'from callsigns in /tmp/callsigns:\n'
                 '    {0} --set --database /home/mythtv/SchedulesDirect.DB '
                 '--file /tmp/callsigns\n'
                 '\n'
                 'The callsigns file will list all the callsigns. Any callsign '
                 'with a leading explamation point (aka bang: "!") is/will be '
                 'deselected.\n\n'
                 'When callsigns are listed to the file, The callsigns will be '
                 'sorted, with selected callsigns at the bottom of the file'
                 '\n\n'
                 'Use --list to generate file, then edit the file to add "!" '
                 'in front of any callsigns you do not want downloaded. Use '
                 '--set to apply those changed to the database.')
                .format(appname)
                )

    parser = argparse.ArgumentParser(description='List and update selected '
                                     'status of available callsigns.',
                                     formatter_class=RawDescriptionHelpFormatter,
                                     epilog='{}'.format(examples))

    parser.add_argument('--debug', action='store_true',
                        help='turn on debug messages (%(default)s)')

    parser.add_argument('--version', action='version',
                        version='%(prog)s 0.1.0')

    parser.add_argument('--file', type=str, required=True,
                            metavar='<filename>',
                            help='File for callsigns')

    parser.add_argument('--database', type=str, required=True,
                            metavar='<filename>',
                            help='File for callsigns')

    parser.add_argument('--set', action='store_true',
                        help='Set selected status in SchedulesDirect.DB '
                        'from <file>')

    parser.add_argument('--list', action='store_true',
                        help='List available callsigns into <file>')

    return vars(parser.parse_args())



def list_callsigns(args):
    db = sqlite3.connect(args['database'])
    cursor = db.cursor()

    channels = {}

    for row in cursor.execute('SELECT station,selected from channels'):
        channels[row[0]] = bool(row[1])

    result = []
    for row in cursor.execute('SELECT details from stations'):
        data = json.loads(row[0])
        v = '' if channels[data['stationID']] else '!'
        result.append('{}{}'.format(v, data['callsign']))

    result.sort()
    with open(args['file'], 'w') as f:
        for callsign in result:
            f.write('{}\n'.format(callsign))
        
    return True

def set_selected(args):
    db = sqlite3.connect(args['database'])
    cursor = db.cursor()

    changed = 0

    channels = {}
    with open(args['file'], 'r') as file:
        for callsign in file:
            if callsign[0] == '!':
                selected = False
                callsign = callsign[1:]
            else:
                selected = True
            channels[callsign.strip()] = selected

    stations = {}
    for row in cursor.execute('SELECT details from stations'):
        data = json.loads(row[0])
        stations[data['callsign']] = data['stationID']
        if args['debug']:
             print('callsign {} = station {}'
                  .format(data['callsign'], data['stationID']))

    for chan in channels.keys():
        if args['debug']:
            print('Checking {}'.format(chan))
        st = (stations[chan], )
        for row in cursor.execute('SELECT selected from channels '
                                  'where station=?', st):
            if bool(row[0]) != channels[chan]:
                print('{}: selected {} -> {}'.format(chan, bool(row[0]),
                                                     channels[chan]))
                st = (channels[chan], stations[chan], )
                cursor.execute('UPDATE channels SET selected=? WHERE '
                               'station=?', st)
                db.commit()
                changed += 1

    print('Modified {} channels'.format(changed))
    return True

    
def main():
    args = process_command_line()

    opts = dict()

    if args['set']:
        return set_selected(args)
    elif args['list']:
        return list_callsigns(args)

    return False


if __name__ == '__main__':
    if main():
        sys.exit(0)
    else:
        sys.exit(1)

Use --help to get examples on how to run it.

An alternative method is as follows. First configure the grabber to select no channels. Install sqlite3. List the tables in the sqlite database. If you did not use the default database name, enter the one that you used. You may have to specify your database name with /home/mythtv instead of $HOME. In that case first make sure the file is writeable by your login user id.

sqlite3 $HOME/.xmltv/SchedulesDirect.DB 
SELECT name FROM sqlite_master WHERE type='table';

List the channels.

sqlite3 $HOME/.xmltv/SchedulesDirect.DB 
select * from channels;

You can copy the results from the screen, or pipe the results to a file. The value for selected is 0 if unselected or 1 if selected. Edit the file and create a text file like this to set the ones you want to select:

update channels set selected = 1 where channum = 2;
update channels set selected = 1 where channum = 3;
...
update channels set selected = 1 where channum = 983;
update channels set selected = 1 where channum = 986;

Update them with a command like this:

sqlite3 $HOME/.xmltv/SchedulesDirect.DB < selected.sql

Converting from DataDirect to XMLTV

There are several reasons to convert your Datadirect setup to use XMLTV.

  • You get 18 days of data instead of 14.
  • You get icons for all channels.
  • Season and episode information are included in listings.

If you decide to do this, first complete the setup of tv_grab_zz_sdjson_sqlite described above.

When you switch, the xmltvid's for the existing channels will change. Using this grabber the xmltvid has the format I11460.json.schedulesdirect.org. Using DD the xmltvid's are a numeric value.

There are two options for the conversion:

  1. Delete all your channels and let mythfilldatabase add them back. This option is not available in cases where you have to scan for channels using DVB or ATSC cards.
  2. Manually update each channel xmltvid.

I recommend that you backup your database and make sure there are no recordings scheduled for the near future before starting.

Deleting and re-adding channels

This is only possible for card types that do not need scanning for channels.

Before attempting this bear in mind the following. If any channel's chanid changes, your existing programs will show the wrong channel that the program was recorded on.

The mechanisms in MythTV for adding channels use a format of sourceid and channel number to create a chanid wherever possible. So channel 2 on source id 2 will be chanid 2002, for example. If it cannot do that then it will allocate a chanid number which is unrelated to the channel number or source id.

In order to see the chanids you will have to look at the channel table in the database. See if your existing chanid numbers match the scheme described in the previous paragraph.

If the chanids match the scheme above you should be able to safely delete them and let mythfilldatabase add them.

You can delete all of the existing channels for each video source in mythtv-setup, channel editor. Select the Video source and click Delete.

To add all of the channels you have selected as well as listings, run mythfilldatabase. Default options will add channels and listings.

Manually Updating xmltvids

If you prefer not to delete your channels or you have a DVB type tuner you will have to update the xmltvids in all of them. Using this grabber the xmltvid has the format Innnnn.json.schedulesdirect.org. You will have to change all of your existing ids to this format. The easiest place to make this change is with mythweb channel editor, although you can use the mythtv-setup channel editor as well. You can get the list of channels with their new format xmltvids by running tv_grab_zz_sdjson_sqlite --days 0 --config-file $HOME/.mythtv/SD.xmltv, which is one step of the setup process (see above). If you did not capture the output you can run this again at any time, no harm will be done.

Once you have updated all of the xmltvid's, run mythfilldatabase. Your channel names and icons will be updated to match the values supplied by schedules direct.

In a tuner type that does not use scanning, such as a cable card tuner, any channels not found with matching xmltvids are added. If you have missed updating some xmltvids that could result in duplication of the channel.

Setup with tv_grab_zz_sdjson

Run MythTV Setup and select Schedules Direct JSON API for your listings source.

You can complete the setup using the Configure button in MythTV Setup. If you prefer to use the command line, run

tv_grab_zz_sdjson --configure --config-file $HOME/.mythtv/SD.xmltv

Select the channel option: Select mode: [lineups,channels (default=lineups)] channels Replace SD with the name of the source you had set up in MythTV Setup.

  • Enter your Schedules Direct user and password when prompted.
  • For channel id format you can select any of the supplied options. Option 0 is the same format as used by tv_grab_zz_sdjson_sqlite. Option 2 is the format used by Schedules Direct Data Direct.
  • Follow the prompts to complete setup.
  • When selecting channels the selection process allows you to skip remaining channels in the list and mark them all as yes or no.
  • If you need to do the channel selection again because of errors or channel changes, your previous values are lost. You have to do the entire selection again.

Channel selection shortcut

The channel selection process with tv_grab_zz_sdjson is tedious and error prone.

An alternative method is as follows. First configure the grabber to select all channels using:
tv_grab_zz_sdjson --configure --config-file $HOME/.mythtv/SD.xmltv
.

Edit the file $HOME/.mythtv/SD.xmltv (replace SD with the name of the source you had set up in MythTV Setup). You will see a bunch of lines like this.

channel=74348
channel=11460

When a channel is selected it has =, when unselected it has !. You need to set the channels you want to use = and the others to !.

Get a list of channels by running

tv_grab_zz_sdjson --days 0 --config-file $HOME/.mythtv/SD.xmltv > list.xml

Although this requests 0 days it actually produces 1 day of programs. At the start of the file before the programs is the channel listing. Delete the program schedules and keep the channel listing at the top of the file. The channel id and display names identify each channel by its XMLTV id and the number and name. Note that the channel id in this file may have extra characters appended depending on the format you chose during configure. For purpose of the configuration file you only need the 5 numeric digit of the channel id. Using your favorite editor and your provider's channel list update the channel lines in the configuration file to have = for ones you want and ! for ones you do not.

List of Grabbers

Important.png Note: Not all grabbers on this page are XMLTV compliant. Only fully compliant grabbers are supported by MythTV and we advise against the use of the rest.

If you know a grabber listed here does not conform entirely to both the xmltv grabber specifications (XMLTV grabber spec) and XMLTV xml format (XMLTV Format) then please mark it as unsupported. If in doubt XMLTV provide a script to validate a grabber (XMLTV Validator)

There is also a list on the XMLTV wiki.

EPG Collector

EPG Collector is a .NET/Mono program for collecting the transmitted guide data from DVB or ATSC streams in various formats and creating an XMLTV file.

Multinational

Schedules Direct supports countries in North America, South America, Caribbean and Europe for $25 (US) per year. For a list of supported countries see http://schedulesdirect.org/regions. See #Schedules_Direct_Setup above

Australia

Refer to Australian TV Listings for further details.

  1. OzTivo Guide (Free, requires registration)
  2. http://www.icetv.com.au (AU$99 per year subscription)
  3. Shepherd (Free: hybrid multi source)

See also this (largely historical) list of Australian grabbers.

Belgium

The following is a list of grabbers for Belgium. It is presented only for historical purposes, because at the time of writing this (January 2011) none of them work.

  1. https://launchpad.net/belguide
  2. http://users.skynet.be/jxmltv
  3. http://pytvgrab.sourceforge.net/

Brazil

There are a working grabber, tv_grab_br_net a perl scripts (or any other Linux native solutions) for Brazilian TV programming. This grabber get data from Net [2] cable TV. No open to air TV on this source, just paid tv programme. It is detected by mythtvsetup and you are prompted (on background windows) to make your choice about your city, line-up and channels. It provide both, simple and complete description for programs.

Other functional alternative is to use TVxb, which isn't open source or even Linux, but works well with wine (at least as far as version 1.0085).

TVxb is configurable for different countries via ini-files. Check the following page for Brazilian templates. As of today, there are some for NET (PR), and others for TVA as well.

http://www.tvxb.com/country/br-ini.html

For information on how to set it up with MythTV, check out TVxb.

Croatia

The grabber in the official XMLTV distribution works correctly. More information is available on http://www.gonix.net/.

Czech Republic

The XMLTV grabber for "Hungary/Romania" was extended to the Czech Republic.

Another way to get TV listings in Czech Republic is TVxb.

Not public XMLTV file for czech and slovak channels (UPC channels included) can be send after request to rfordinal.

Denmark

In Denmark there is a group maintaining several grabbers. This group has created the following:

Currently three grabbers are maintained:

  • DR 2012 grabber written in perl (dr.dk)
  • An additional written in python (yousee.dk)
  • An universal grabber combining data from mutiple sources (tvtid.tv2.dk, yousee.dk, tv-guiden.dk, jubii.dk, ontv.dk, ahot.dk, dr.dk and swedb)

Finland

The grabbers in the official XMLTV distribution work correctly. Alternate source: http://xmltv.se

France

The grabber in the upcoming XMLTV release works correctly. More information is available on [3].

Germany/Austria

  1. XMLTV.se offers many of the main channels for personal use, the grabber from the main xmltv distribution works.
  2. Some channels are available via Egon Zappt in the upcoming release of xmltv.
  3. TV Movie can not give out their data without additional license, see Ben Bucksch's log.

Greece

Try this link for Nova Greece: http://sgcpm.com/epg/devilcosta.gz, for example using mythfilldatabase. (Be aware that its just a dump of DVB-EIT using the service_id as channel_id which is not proper xmltv. Please consider writing a proper grabber for the file so it can be configured via the GUI or simply provide a proper guide feed with tools like NonameTV.)

India

The grabber for What's-On-India has been added to the official XMLTV distribution.

An XMLTV grabber for all channels in India can be found at http://ravihtpc.wordpress.com/2010/01/09/india-epg-data-for-mythtv-xmltv-listings-grabber/.

Italy

The grabber in the official XMLTV distribution works correctly.

Hong Kong

First thanks to http://www.akbkhome.com/blog.php/View/119/Mythtv_xmltv_importer_for_HK.html.

I modify his source code to get information from TVB new web site.

   download Tv.php http://mythtv-xmltvhk.googlecode.com/files/Tv.php
   download hongkong.ini http://mythtv-xmltvhk.googlecode.com/files/hongkong.ini
   build xml from tvb, atv website using following command, just entry 
          php ./Tv.php hongkong.ini > hongkong.xml
   using 'myfilldatabase' to update mythtv schedule program, just entry
          mythfilldatabase --file 1 hongkong.xml
   (* it is my first development on php, so it may be simplified *)

[Last Update by hwkit (17/04/2010)]

Since TVB web site is changed, the program need to be updated. (My php skill is poor, so there are error when running, however it also can get schedule from TVB web site), I also change the program name called 'xmltvhk.php'. Please download it from google.

   download xmltvhk.php and copy it to /etc/xmltvhk (http://mythtv-xmltvhk.googlecode.com/files/xmltvhk.php)
   download updated hongkong.ini and copy it to /etc/xmltvhk (http://mythtv-xmltvhk.googlecode.com/files/hongkong.ini)
   download filltv  and copy it to /etc/cron.weekly (make it executable (chmod +x /etc/cron.weekly/filltv) (http://mythtv-xmltvhk.googlecode.com/files/filltv)
   Source code 
   svn checkout http://mythtv-xmltvhk.googlecode.com/svn/trunk/ mythtv-xmltvhk-read-only

[Last Update by hwkit (07/07/2010)]

Netherlands

Netherlands (Holland) specific info can be found here: Netherlands

New Zealand

Sky TV threatened legal action to get the xmltv grabber for NZ taken down. You can still receive the transmitted guide in MHEG5 with tools like EPG Collector.

Norway

The grabber in the official XMLTV distribution works correctly. Another source of info: http://xmltv.se

Slovakia

The XMLTV grabber for "Hungary/Romania" was extended to Slovakia.

South Africa

See the South Africa MUG WIKI page for details.

Spain

Both grabbers in the official XMLTV distribution work correctly.

Sweden

The grabber in the official XMLTV distribution works correctly. More information is available on http://xmltv.se.

Switzerland

The grabber in the official XMLTV distribution works correctly.

United Kingdom

The grabbers in the official XMLTV distribution work correctly. A brief guide to configuring tv_grab_uk_rt is available, and one for Metabroadcast Atlas.

Ukraine, Russia, Belarus

  1. tv_grab_ru Converts JTV-formatted archive with about 200 channels to XMLTV. But this application does not show additional info about programmes.
  2. tv_grab_ua Downloads XMLTV-formatted data from sites like teleguide, izbushka or star.poltava.ua. In addition it parses existing data and adds category/rating/credits into output XMLTV file.
  3. teleguide.info XMLTV-formatted zip archive with programmes inside.

Latvia