Python API Examples

From MythTV Official Wiki
Revision as of 17:04, 25 November 2015 by Llib (talk | contribs) (Test new template)

Jump to: navigation, search


Existing sources of information

  • Local backend: e.g. http://<backendHostNameOrIP>:6544/Channel/wsdl
  • The 'old' protocol version. Use of this mechanism is discouraged but its pages are a still a good source for (say) program data. Myth Protocol
  • The the MythTV forum has a Services API section – fast and helpful responses!
  • Source code (if desperate!): For example, the Dvr service

Getting information from the backend

The backend can be read using a standard web interface and there are a number of ways of doing this..

Web browser

In its simplest form, a browser will show the data e.g.

http:/<backendHostNameOrIP>:6544/Dvr/GetRecordedList?StartIndex=2&Count=1&Descending=true

will show a single entry from the recorded list.

This is a useful development aid but note that the browser helpfully inserts newlines or tabs for readability but these will not be present if the same page is read with perl code.

Notice that empty values like <XMLTVID></XMLTVID> will display as: <XMLTV/>.

curl

From the shell, curl can do the same, e.g.

curl <backendHostNameOrIP>:6544/Dvr/GetRecordedList?StartIndex=2\&Count=1\&Descending=true | sed "s/></>\n</g"

Note the \ to prevent the shell from spinning off the command at the &. The sed inserts newlines for readability.


Python Services API Cookbook

Some working examples using the MythTV Services API. If the user's language of choice is Perl, then see: Perl API examples

The following will be demonstrated using a Python module (included at the bottom of this page.) Users can choose to place the module in the same directory as programs that use it, or alongside the existing MythTV Python bindings.

There are excellent links that list the available endpoints (the services and actions following the port number) here: API parameters 0.27 and API parameters 0.28.

These examples were written and tested using 0.28-pre.

About the module

First, it's not necessary to use it! Users can write their own code to send information to the back/frontend. The module provides a reasonable template of what might be done. Included are checking of the parameters, some optional debugging statements, optional header choices and error handling.

Regarding the options. The module will set the value of any uninitialized option to False. But the caller should set them perhaps based on a command line argument (or just hardcode them as appropriate for their own use.) If any option isn't set, a warning messages will be printed.

Importantly, the module reformats the JSON or XML response from the server into a Python dict or an Element Tree respectively.

As with any Python module, built-in help is available. Try the following:

$ python
>>> from MythTVServicesAPI import Utilities as util # Or:
>>> import Utilities as util
>>> help(util)
>>> control-D # After the help prints

The following examples will all request the response from the back/frontend and return it in Python dictionary format.

Load the module

Choose one of the following. If the module is installed (for example) in the system's dist-packages directory, use this:

from MythTVServicesAPI import Utilities as api

That assumes there is a MythTVServicesAPI directory with the Utilities.py file underneath it (and an empty __init__.py file there too.) An example of the directory where the above would be installed is: /usr/local/lib/python2.7/dist-packages which is where users would likely find the existing MythTV Python bindings. MacPorts users might find the bindings under: /opt/dvr/bin/python2.7.

Or, if the module is installed in the same directory as the program(s) that use it, do this:

import Utilities as api

Myth/GetHostName endpoint example

Getting the backend's hostname returns the simplest response in the API. It's the easiest to parse and has the hostname of the backend (really, it's the profile name used in the DB and honors the LocalHostName line in config.xml, but that's another Wiki.))

Put the following in a file, perhaps simpleAPItest.py. Make it executable and run it.

MacPorts users replace the 1st line below with #!/opt/dvr/bin/python2.7


PythonIcon.png simpleAPItest.py
#!/usr/bin/env python
import Utilities as api

opts = {'debug':False, 'etree':False, 'nogzip':False, 'wrmi':False}

resultDict = api.Send(host='localhost', endpoint='Myth/GetHostName', opts=opts)

print 'Dictionary contents:', resultDict
print 'My backend hostname (MythTV profile name) is: ', resultDict['String']

Look at the 2 lines of output.

Dictionary contents: {u'String': u'backendHostName'}
My backend hostname (MythTV profile name) is: backendHostName

The Myth/GetTimeZone endpoint example

A slightly more complex example. Import the module and set the opts as above and then try:


PythonIcon.png getTimeZoneInfo.py
r = api.Send(host='backendHostName', endpoint='Myth/GetTimeZone')

print r
print r['TimeZoneInfo']['TimeZoneID']
print r['TimeZoneInfo']['UTCOffset']
print r['TimeZoneInfo']['CurrentDateTime']

Expect to see something like this:

{u'TimeZoneInfo': {u'TimeZoneID': u'America/Chicago', u'UTCOffset': u'-21600', u'CurrentDateTime': u'2015-11-13T19:26:18Z'}}
America/Chicago
-21600
2015-11-13T19:26:18Z

Working program using the Dvr/GetUpcomingList endpoint

Here's a simple script, with a bit of formatting, that does a practical task.

Put the following in a file, like: shortPrintUpcoming.py and type: ./shortPrintUpcoming.py backendHostNameOrIPaddress

Note the line: progs = resp_dict['ProgramList']['Programs'] below. Rather than referencing each upcoming program as: resp_dict['ProgramList']['Programs'][upcoming], they can be examined as: progs[upcoming]....

Also note that there's yet another level under progs, it's the ['Recording'] portion where just the StartTs is used in this example.

PythonIcon.png shortPrintUpcoming.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-

# shortPrintUpcoming.py: Print Start Times, Titles and SubTitles

import os.path as path, sys, Utilities as api

try:
    host = sys.argv[1]
except:
    sys.exit( '\nUsage: {} HostnameOrIP'.format(path.basename(sys.argv[0])))

resp_dict = api.Send(host=host, endpoint='Dvr/GetUpcomingList')

count = int(resp_dict['ProgramList']['Count'])
progs = resp_dict['ProgramList']['Programs']

if count < 1: sys.exit('\nNo upcoming recordings found.\n')

for upcoming in range(count):
    print u'  {}  {:45.45}  {:15.15}'.format(
        progs[upcoming]['Recording']['StartTs'],
        progs[upcoming]['Title'],
        progs[upcoming]['SubTitle']).encode('utf-8')

To see the way the response is returned, try adding lines like

print resp_dict
print resp_dict['ProgramList']['Programs'][0]
print progs[0]

The 1st displays the entire response in a dictionary, the 2nd just the 0th program returned and the last is the same as the 2nd.

Full example using Dvr/GetUpcomingList

Using the ideas above, the following is a complete program with command line arguments used to set some of the opts values, and set a variable host etc. Type: ./printUpcoming --help for details. Try the --debug argument to see the URLs generated by the program.

PythonIcon.png printUpcoming.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-

#
# printUpcoming.py: Print Start Times, Titles and SubTitles
# (optionally filtering by Titles/ChanId or limited by days.)
#
# This is an 'overly-commented', and simple example of using the Services
# API to do a task. It also forms a template for working with the API.
#
# Good Start:          printUpcoming.py --help
# Watch what it sends: printUpcoming.py --host=Name/IP --debug
#

##############################################################
# MythTVServicesAPI.Utilities handles backend connections    #
# re is used to allow things like --title to be partial and  #
#    case agnostic                                           #
# argparse is used in the following section                  #
##############################################################
from   datetime import datetime
from   MythTVServicesAPI import Utilities as API
import argparse, re, sys, time

##############################################################
# Add command arguments, as required, here. --host is likely #
# a minimum and mandatory unless the host is hardcoded into  #
# the script. Examples include string, integer and bool type #
# arguments. Printing the default values is also shown.      #
##############################################################

parser = argparse.ArgumentParser(description='Print Upcoming Programs',
    epilog='Default values are in ()s')

mandatory = parser.add_argument_group('requrired arguments')

parser.add_argument('--chanid', type=int, required=False, metavar='<chanid>',
    help='filter on MythTV chanid, e.g. 1091 (none).')

parser.add_argument('--days', type=int, required=False, metavar='<days>',
    default=7, help='number of days of programs to print (%(default)s)')

parser.add_argument('--debug', action='store_true',
    help='turn on debug messages (%(default)s)')

mandatory.add_argument('--host', type=str, required=True,
    metavar='<hostname>', help='backend hostname')

parser.add_argument('--nogzip', action='store_true',
    help='tell the backend not to return gzipped data (%(default)s)')

parser.add_argument('--port', type=int, default=6544, metavar='<port>',
    help='port number of the Services API (%(default)s)')

parser.add_argument('--title', type=str, default='', required=False,
    metavar='<prog>', help='filter by title (none)')

args = parser.parse_args()

##############################################################
# Prepare the data to send to the backend via the API and    #
# then do it.                                                #
##############################################################

endpoint = 'Dvr/GetUpcomingList'
opts = { 'debug':False, 'etree':False, 'nogzip':False, 'wrmi':False }
rest     = 'ShowAll=false'

resp_dict = API.Send(host=args.host, port=args.port, opts=opts,
                        endpoint=endpoint, rest=rest)

##############################################################
# And also get the backend's timezone offset so that the     #
# start times in the response can be converted to local time.#
##############################################################

tz_dict = API.Send(host=args.host, port=args.port,
                   endpoint='Myth/GetTimeZone', opts=opts)

tzoffset = int(tz_dict['TimeZoneInfo']['UTCOffset'])

##############################################################
# Finally, loop through the results and select the items     #
# of interest. Ignore Titles that don't match the reqular    #
# expression if the --title argument was used. Same for      #
# chanids. The results are sorted by time, so this makes an  #
# attempt to only print --days worth of data.                #
##############################################################

WHITE  = '\033[0m'
YELLOW = '\033[93m'
headerPrinted = False
one_day_in_seconds = 24 * 60 * 60
time_now = time.mktime(datetime.now().timetuple())

count = int(resp_dict['ProgramList']['Count'])
progs = resp_dict['ProgramList']['Programs']

if args.debug:
    print 'Debug: Upcoming recording count =', count

if count < 1:
    sys.exit('\nNo upcoming recordings found.\n')

print '\nUpcoming programs sorted by StartTime'
print '\n  {}{:19}  {:45.45}  {}{}' \
    .format(YELLOW, 'StartTime', 'Title', 'SubTitle', WHITE)

for upcoming in range(count):

    title    = progs[upcoming]['Title']
    subtitle = progs[upcoming]['SubTitle']
    chanid   = int(progs[upcoming]['Channel']['ChanId'])
    startts  = API.UTCToLocal(progs[upcoming]['Recording']['StartTs'],
                                 tzoffset=tzoffset)

    future_time = time.mktime(datetime.strptime(startts,
                              "%Y-%m-%d %H:%M:%S").timetuple())
    futureDays   = int((future_time - time_now ) / one_day_in_seconds)

    if futureDays >= args.days:
        break

    if (args.title  == '' or re.search(args.title, title, re.IGNORECASE)) and \
        (args.chanid == None or args.chanid == chanid):

        print u'  {}  {:45.45}  {:15.15}' \
                .format(startts, title, subtitle).encode('utf-8')

# vim: set expandtab tabstop=4 shiftwidth=4 :

By getting the time zone information from the backend, each start time in the output is converted to local time rather than the UTC returned in the Services API. Another important concept in the above is the use of print u'.... followed by .encode('utf-8). That's critical for Titles/SubTitles for example where there are characters like: ó.

The module used in the examples above

PythonIcon.png Utilities.py
# -*- coding: utf-8 -*-

""" Basic access utilities.  """

from datetime  import datetime, timedelta
from StringIO  import StringIO
from xml.etree import cElementTree as ET

import gzip, httplib, json, re, sys, urllib2

__version__ = '0.28.a1'

ServerVersion = 'Set to back/frontend version after calls to Send()'

def Send(method='GET', host='', port=6544, endpoint='', rest='', opts={}):
    """

    Form a URL and send it to the back/frontend. Error handling is done
    here too.

    Input:
    ======

    method:   Set to 'POST' for endpoints that change data. Omit method (or
              set it to 'GET' for all other endpoints. Defaults to 'GET'.

    host:     Must be set and is the hostname or IP of the back/frontend.

    port:     Used if the backend is using a different port (unlikely) or
              set to 6547 for frontend endpoints. Defaults to 6544.

    endpoint: Should be set. Example: Myth/GetHostName

    rest:     May be set if the endpoint allows it. Example for
              Myth/GetRecordedList: rest='Count=10&StorageGroup=Sports'

    opts      A dictionary of options that may be set in the calling
              program. Default values will be used if callers don't
              pass their own.

              The defaults (typically changed by command line arguments) are
              all False.

    opts['debug']:  Set to True and the developed URL will be printed (for
                    debugging.)

    opts['etree']:  If True, the response returned TO THIS MODULE will be
                    in XML. If False, JSON will be returned.

                    N.B. JSON will then be loaded into Python Dictionary
                    format. XML will be in an Element Tree.

    opts['nogzip']: Don't request the back/frontend to gzip it's response

    opts['wrmi']:   If True and method='POST', the URL is actually sent
                    to the server.

                    If opts['wrmi']=False and method='POST' ***Nothing*** is
                    sent to the server.
                    
                    This is a failsafe that allows testing. You can examine
                    what's about to be sent before doing it (wrmi = We Really
                    Mean It.)

                        The response is formatted based on the value of
                        opts['etree'] (assume the caller chooses to
                        put the response in r.)

                        True;  r.tag = 'Warning', r.text = 'wrmi=False'
                        False; r['Warning'] = 'wrmi=False'

    Output:
    ======

    An error message and program exit, or the JSON response loaded into
    a dictionary or the XML response in ElementTree format.

    Whenever Send() is used, the global 'ServerVersion' is set to the value
    returned by the back/frontend in the HTTP Server: header. It is returned
    as just the version, e.g. 0.28. Callers can check if and *may* choose to
    adjust their code work with other versions.

    IF USING method='POST', TAKE EXTREME CAUTION!!! Use opts['wrmi']=False
    1st.
    """

    optionList = [ 'debug', 'etree', 'nogzip', 'wrmi' ]

    for option in optionList:
        try:
            opts[option]
        except:
            print 'Warning: opts["{}"] is not set, using: False'.format(option)
            opts[option] = False

    #!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!#
    # The version should never be changed without testing.       #
    # If you're just getting data, no harm will be done. But if  #
    # you Add/Delete/Update anything, then all bets are off!     #
    # Anything requiring an HTTP POST is potentially dangerous.  #
    #!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!#

    global ServerVersion
    version = '0.28'

    ##############################################################
    # If there's no endpoint leave now; otherwise, form the URL. #
    ##############################################################

    if endpoint == '':
        sys.exit('\nAbort! No endpoint (e.g. Myth/GetHostName.)')

    if rest == '':  qmark = ''
    else:           qmark = '?'

    url='http://{}:{}/{}{}{}'.format(host, port, endpoint, qmark, rest)

    ##############################################################
    # Form the request to the API. Unless specified, an HTTP GET #
    # will be used, change it to a POST if asked. The debugging  #
    # provides the URL used and can be copied and tested with a  #
    # browser or curl/wget if needed. Unless specified, requests #
    # will get a JSON response from the server. The User-Agent   #
    # header will show up in server logs with the value of the   #
    # version set earlier included. And finally, responses will  #
    # be gzipped by the server unless the nogzip option is True. #
    ##############################################################

    request = urllib2.Request(url)
    request.get_method = lambda: method

    if opts['debug']:
        print '\nDebug: URL = {} [{}]'.format(url, method)

    if method == 'POST' and not opts['wrmi']:
        if opts['etree']:
            tree = ET.ElementTree(
                    ET.fromstring('<Warning>wrmi=False</Warning>'))
            return tree.getroot()
        else:
            return { "Warning": "wrmi=False" }

    request = urllib2.Request(url)

    if not opts['etree']:
        request.add_header('Accept', 'application/json')

    request.add_header('User-Agent', '{} Python Services API'.format(version))

    if not opts['nogzip']:
        request.add_header('Accept-Encoding', 'gzip,deflate')

    ##############################################################
    # Actually try to get the data and handle errors. Get the    #
    # Server: header and see if the MythTV version is recognized.#
    # Contents of server for 0.27 and 0.28 as of this writing:   #
    # MythTV/0.28-pre-3094-g349d3a4 Linux/3.13.0-66-generic UPnP/1.0
    # Linux 3.13.0-65-generic, UPnP/1.0, MythTV 0.27.20150622-1  #
    # Then decompress the gzipped response if need be. Finally,  #
    # load the data into a dictionary (if JSON) or an Element    #
    # Tree (if XML.)                                             #
    ##############################################################

    try:
        response = urllib2.urlopen(request)
    except urllib2.HTTPError, e:
        sys.exit('\nHTTP Error: {}. URL was:\n\t{}'.format(e.code, url))
    except urllib2.URLError, e:
        sys.exit('\nError: {}. URL was:\n\t{}'.format(e.args, url))
    except httplib.UnknownProtocol, e:
        sys.exit('Unknown Protocol: {}'.format(e.args))
    except:
        sys.exit('\nUndefined error. Is the backend running? URL was:\n\t{}.' \
            .format(url))

    server = response.info().get('Server')

    if server == None:
        sys.exit('\nWarning: No HTTP "Server:" header returned.')
    else:
        if re.search(version, server):
            ServerVersion = '0.28'
        elif re.search('0.27', server):
            ServerVersion = '0.27'
            print '\nWarning: {} Services API module may not work with {}.' \
                .format(version, ServerVersion)
        else:
            sys.exit('Abort: Module only tested on 0.27 and 0.28, not: {}.' \
                .format(server))

    if response.info().get('Content-Encoding') == 'gzip':
        response_string = gzip.GzipFile(fileobj = StringIO(response.read()))
        unzipped_response = response_string.read()
        if opts['debug']:
            print 'Debug: Received bytes = {}, Unzipped = {}' \
                .format(int(response.info().getheaders("Content-Length")[0]),
                    len(unzipped_response))

        if opts['etree']:
            return ET.fromstring(unzipped_response)
        else:
            return json.loads(unzipped_response)
    else:
        if opts['etree']:
            return ET.fromstring(request)
        else:
            return json.load(response)

def URLEncode(value=''):
    """
    Input:  A string. E.g a program's title or anything that has
            special characters like ?, & and special UTF characters.

    Output: The URL encoded string. E.g. ó becomes: %C3%B3 or ?
            becomes %3F.
    """

    if value == '':
        print 'Warning: MythTVServicesAPI.URLEncode called without any value'
        return value

    return '{}'.format(urllib2.quote(value).encode('utf-8'))

def CreateFindTime(host='', port='', time='', opts={}):
    """
    Normally be used to take a timestamp and convert it for use in adding
    new recordings. This always gets the UTCOffset, which the caller should
    probably do once and pass it to this function. But, since it's called
    infrequently, just do it here every time.

    Input:  Full UTC timestamp, e.g. 2014-08-12T22:00:00 (with or without
            the trailing 'Z'.)

    Output: Time portion of the above in local time.
    """

    if time == '':
        print 'Warning: MythTVServicesAPI.CreateFindTime called without any time'
        return None

    time = time.replace('Z', '')

    opts['etree'] = False

    resp_dict = Send(host=host, port=6544, endpoint='Myth/GetTimeZone', \
                            opts=opts)

    tzoffset = resp_dict['TimeZoneInfo']['UTCOffset']
    tzoffset = int(tzoffset)

    dt = datetime.strptime(time, '%Y-%m-%dT%H:%M:%S')

    return (dt + timedelta(seconds = tzoffset)).strftime('%H:%M:%S')

def UTCToLocal(utctime='', tzoffset='', omityear=False):
    """
    Does exactly that conversion. This is likely to be called frequently
    (for example if printing guide timestamps) so force the caller to get
    the TZ offset and pass it.

    Inputs:  utctime  = Full UTC timestamp, e.g. 2014-08-12T22:00:00[Z].
             tzoffset = Offset from UTC. Probably from Myth/GetTimeZone.
             omityear = If True, then drop the 4 digit year and following -.

    Output: Local time, also a string. Possibly without the year- and always
            without the T between the data/time and trailing Z.
    """

    if utctime == '' or tzoffset == '':
        sys.exit('Abort, utctime and/or tzoffset are empty!.')

    utctime = utctime.replace('Z', '').replace('T', ' ')

    dt = datetime.strptime(utctime, '%Y-%m-%d %H:%M:%S')

    if omityear: fromstring = '%m-%d %H:%M:%S'
    else:        fromstring = '%Y-%m-%d %H:%M:%S'

    return (dt + timedelta(seconds = tzoffset)).strftime(fromstring)