Perl API examples

From MythTV Official Wiki
Revision as of 16:21, 5 June 2022 by PhilB (talk | contribs) (Changes for Mythtv version 32 and API port 6744.)

Jump to: navigation, search


Author Phil Brady
Description Interfacing with the Services API interface of Mythtv from a perl script.
Supports Version27.png Version31.png Version32.png 


Introduction

This page has been created to give examples of perl code for accessing the API interface to Mythtv and includes a module to assist with this. The module was developed and tested with 0.27.4 and survived unchanged until a few tweaks were needed for the test API interface on port 5744 issued with version 32 of Mythtv.



Existing sources of information

  • Local backend: e.g. http://<backendHostNameOrIP>:6544/Channel/wsdl
  • The 'old' protocol version. Use of this mechanism is discouraged but its pages are a still a good source for (say) program data. Myth Protocol
  • The the MythTV forum has a Services API section – fast and helpful responses!
  • Source code (if desperate!): For example, the Dvr service

Getting information from the backend

The backend can be read using a standard web interface and there are a number of ways of doing this..

Web browser

In its simplest form, a browser will show the data e.g.

http:/<backendHostNameOrIP>:6544/Dvr/GetRecordedList?StartIndex=2&Count=1&Descending=true

will show a single entry from the recorded list.

This is a useful development aid but note that the browser helpfully inserts newlines or tabs for readability but these will not be present if the same page is read with perl code.

Notice that empty values like <XMLTVID></XMLTVID> will display as: <XMLTV/>.

curl

From the shell, curl can do the same, e.g.

curl <backendHostNameOrIP>:6544/Dvr/GetRecordedList?StartIndex=2\&Count=1\&Descending=true | sed "s/></>\n</g"

Note the \ to prevent the shell from spinning off the command at the &. The sed inserts newlines for readability.


Module LWP::Simple

This CPAN module will get data from the backend.

#!/usr/bin/perl -w
use strict;
use LWP::Simple;
my $url='http://192.168.1.67:6544/Dvr/GetRecordedList';
my $content=get $url;

You may need to install libwww-perl for these CPAN routines to work. With Ubuntu do this:

 sudo apt-get install libwww-perl

Module LWP::UserAgent

If getting the data fails, you may want to know why. Do this instead:

#!/usr/bin/perl -w
use strict;
use LWP::UserAgent;
my $url='http://192.168.1.67:6544/Dvr/GetRecordedList';
my $browser = LWP::UserAgent->new;
$browser->timeout(10);
my $response = $browser->get($url);
unless($response->is_success){ 
        die $response->status_line;
}
my $content= $response -> content;

The scan_database Module

If you are using other routines in the scan_database module shown below then you may find it convenient to use the ReadBackend routine. This essentially runs the UserAgent code above. It should read any simple url, not just a backend!

#!/usr/bin/perl -w
use strict;
use scan_database;
my $url='192.168.1.67:6544/Dvr/GetRecordedList';
# if you omit the http:// - it will insert it for you.
my $content;
ReadBackend($url,$content);

If you want the routine to return rather than issuing a 'die' in case of failure then call it in scalar context:

#!/usr/bin/perl -w
use strict;
use scan_database;
my $url='192.168.1.67:6544/Dvr/GetRecordedList';
# if you omit the http:// - it will insert it for you.
my $content;
if (ReadBackend($url,$content)){
    #process $content
}else{
    print "No host response\n";
}

Parsing the data

Having read the data from the backend using one of the methods above you will have a mass of XML code in a scalar variable. Before you can sensibly use this, you will need to extract from it the information you are interested in - this is called parsing.

There are many ways of parsing this - some memory intensive, some not.

Regex

A regex is an efficient way of extracting a limited amount of data. To extract protocol version and titles of all recordings for example:

#!/usr/bin/perl -w
use strict;
use scan_database;
my $url='192.168.1.67:6544/Dvr/GetRecordedList';
my $content;
ReadBackend($url,$content);
$content =~ m!<ProtoVer>(\w+)</ProtoVer>!;
print "Proto version is $1\n";
while ($content =~ m!<Title>([^<]*)<!g){
    print "$1\n";
}

Note that this will not convert the XML escape sequences such as &amp; to &.

Parsers on the CPAN database

There may be others but you may like the look of SAX or XML::Parser

The parsers in scan_database module

There are two parsing routines in the scan_database module. Both take as input the content extracted by one of the backend reading mechanisms above and both write the information you say you are interested in to a hash or a hash of hashes. Both have been written to return the minimum required information and to minimise memory usage (but if anyone can suggest improvement please do so!).

FillHashofHash

The first of these populates a hash of hashes. This would be useful if the data presented is a list of 'records' each with a set of 'fields'. eg a list of recordings with their respective fields describing filename, title, start time, filesize etc or a list of channels with their respective frequencies, call signs etc. The data would typically have been obtained with a GetSomethingList call.

eg to get a hash of hashes from GetRecordedList data with filename as a primary key and containing Title and filesize, firstly check the format of the data to determine the separator for the 'records'. You will find that the recordings are wrapped by <Program> ... </Program> or (in v32 test interface) <Program version="1.12"> ... </Program>, that the filename is in <FileName>..</FileName>, the title in <Title> ... </Title> and the size in <FileSize>..</FileSize>.

FillHashofHash takes parameters: hash, content, record separator, primary key, secondary key(s)

eg to print filename, size and title for all recordings:

#!/usr/bin/perl -w
use strict;
use scan_database;
my $url='192.168.1.67:6544/Dvr/GetRecordedList';
my $content;
ReadBackend($url,$content);
my %hash;
FillHashofHash(%hash, $content,'Program','FileName','Title','FileSize');
for (sort keys %hash){
    print "$_ $hash{$_}{FileSize} $hash{$_}{Title} \n";
}

Any of the 'key' fields can be replaced by '#' to give an incrementing (and unique) counter. The routine returns the number of records found. This might be particularly useful for guaranteeing uniqueness of the primary key. The routine returns the number of records extracted (number of filenames in the above example).

Another example would be to extract the state of a frontend where the format of the API response looks somewhat different with key and value tags but can still be handled:

#!/usr/bin/perl -w
use strict;
use scan_database;
my $frontend ='192.168.2.99';
my  $temp;my %mydata;

if (ReadBackend($frontend.':6547/Frontend/GetStatus', $temp)){
    FillHashofHash(%mydata, $temp, 'String', 'Key','Value');
    print "Frontend state is $mydata{state}{Value}\n";
}else{
    print "Frontend not running\n";
}
GetAllFields

Some of the APIs calls return a simple XML list of fields rather than a series of records. To extract ALL fields between particular start and end delimiters and put them in a hash, use

GetAllFields(%hash, $content, $start, $end)

The routine will only start looking for fields once it has encountered $start in the data and will stop if it finds $end.

A null value to start or end will indicate start and end of the whole string. eg to read the fields in the header of a GetRecordedList:

#!/usr/bin/perl -w
use strict;
use scan_database;
my $url='192.168.1.67:6544/Dvr/GetRecordedList?StartIndex=1&Count=1';
my $content;
ReadBackend($url,$content);
my %hash;
GetAllFields(%hash, $content, '', '<Programs>');
for (keys %hash){
    print "$_ $hash{$_}\n";
}

The routine returns the number of fields extracted.

Getting data from the backend and parsing in one step (deprecated)

The first cut of the module code below only included these one step routines. They have not been validated against the v32 test interface. The second cut introduced the three routines above which have a simpler interface, which are less mythtv specific and will probably need less maintenance.

The code for these one step routines has been re-written to exploit the three routines above but should maintain the same user interface. Note that hash references were needed (eg \%hash rather than %hash) and still do for compatibility. <blush> I didn't know about prototyping at the time </blush>. I suggest that you treat these routines as deprecated, but the details are here for completeness.

Readers will note that many of the API calls will return XML code which consists of

Generic 'header' information (eg Mythtv version) followed by:
Sets of data relating to an individual channel, video source, recording etc.

Code is presented here in module form which makes two routines available to calling perl scripts to reflect each type of data:

GetDBheader Deprecated

This populates a hash with the initial header information in the data returned by API calls.

Calling parameters are:

  • Reference to hash
  • IP of host
  • Name of API
  • Extra parameters to supply to the url (see below).

The hash gets populated with the header information and the return code is the number of entries found.


GetDBinfo Deprecated

This populates a hash with requested fields from the database. Calling parameters are:

  • reference to the hash
  • IP address of host
  • Name of API (eg getrecordedlist). This is case insensitive.
  • Extra information to supply to the url call to the backend (eg 'StartIndex=3&Count=4').
  • Primary key for the database. This must be unique in the database (eg 'FileName' would be unique but not 'ChanId')
  • One or more secondary keys to extract.

If extra parameters is left blank then the routines will normally return data on all recordings, video sources etc. Any initial '?' or '&' should be omitted.

If uniqueness of key is an issue then '#' can be used as a key and an incrementing counter will be returned as the value. The counter will increment at each new recording, channel etc.

The routine populates the hash and returns with the number of hash entries created.

A simple demo program should give a flavour of the routines:

Application-x-perl.png testscan.pl

#!/usr/bin/perl -w
use strict;
use scan_database;


#Example of GetDBheader

print "\nGetDBheader\n";
my %header;
my $resp= GetDBheader(\%header, '192.168.1.67','VideoSourceList');
print "$resp entries returned\n";
for (sort keys %header){
    print "$_  $header{$_}\n";
}


#Example of GetDBinfo -  Note use of # to return a counter.
# If the extra param (StartIndex=2 &Count=10 in this case) is 
# null then all will be returned

print "\nGetDBinfo\n";
my %data;
$resp= GetDBinfo(\%data, '192.168.1.67', 'GetRecordedList', 
      'Startindex=2&Count=10',  'FileName','Title','FileSize','#');

#params are:  reference to hash, host, function, extra info to url call (if any),
#             primary key for hash data, secondary keys.

print "$resp entries returned\n";
print "FileName                  #  FileSize     Title\n";
for (sort keys %data){
    print "$_   $data{$_}{'#'}  $data{$_}{FileSize}   $data{$_}{Title}\n";
}

exit 0;

The above demo needs to be able to access the module code. The module currently only supports ChannelInfoList, GetRecordedList and VideoSourceList but other routines should be straightforward to include (hint: look at the hash  %steer). Alternatively, avoid these deprecated calls.


Writing to the database

Please establish a robust database backup strategy before making any attempt to write to the database. Furthermore, if you publish any software, please include appropriate warnings for the end user.

Data is written to the database with a 'post' operation which sends a 'form' holding the new data. The 'form' is held in a hash. eg this (at least in 0.27.4) will delete a recording. Note that since original publication 'RemoveRecorded' is destined for obsolescence, but the principle remains.


Application-x-perl.png expire.pl

#!/usr/bin/perl -w 
use strict; 
use warnings FATAL => 'uninitialized'; 
use LWP::UserAgent; 

my $StartTS = '2012-01-18T21:26:00Z';	#extracted from GetRecordedList parameter <StartTS> 
my $ChanId = '1002';				# ditto but <ChanId> 

#open the browser 
my $browser = LWP::UserAgent->new; 
$browser->timeout(10); 

#issue post 
my $response = $browser -> post( 
	'http://192.168.1.67:6544/Dvr/RemoveRecorded', 
	[ 
	'StartTime' => $StartTS,          #NB StartTS not StartTime 
	'ChanId'	=> $ChanId, 
	], 
     ); 

unless($response->is_success){die "Error expiring recording"}; 
my $url_buffer = $response -> content; 
unless ($url_buffer =~ /<bool>true/) {print "Bad response expiring\n"}; 
exit 0;

For most of the POST operations there is a closely similar GET operation. eg GetChannelInfo and UpdateDBChannel

This similarity is useful if you only wish to change a very limited number of fields (eg to hide a channel). The intention is clearly that one should issue a 'get', extract the information into a hash, change the required value and then post the hash back. Unfortunately, it is not quite that simple because:

1. The API system is still new and will take a while to bed down.

2. There are inconsistencies in parameter naming (eg GetChannelInfo has 'Visible', 'ChanId' and IconURL' whilst UpdateDBChannel expects 'visible', 'ChannelID' and 'Icon').

3. Some API calls are destined for replacement (eg RemoveRecorded will become DeleteRecorded). Additionally, allowable parameters will change.

4. Documentation is lagging behind the code.

5. Some of the calls have in excess of 40 parameters. The opportunity for human error is high and so the potential for database corruption or 'empty fields' is high.

You did have a robust database backup strategy?

The wsdl interface is the key to resolving this as it shows you which API routines are available and which parameters are needed. Since the wsdl pages are machine readable, I offer you three routines which may assist you in automating checking and writing code for this error prone task. They are offered without warranty and have only been tried against Mythtv 0.27.4.

All are in the module at the foot of this wiki article.

ValidatePost

This routine performs checking on a hash (form) before posting it.

The routine ValidatePost takes 4 parameters as input:

  • The proposed hash for posting to the backend.
  • The url of the post call.
  • An 'action' parameter'.
  • Sanity check count

The routine will

  • work out the appropriate wsdl call from the url and read the wsdl
  • Extract the names of the parameters needed by the API (and cache these for future calls).
  • Fuzzy match those from the proposed hash.
  • Expand ampersands, quotes etc in parameter values to allow transmission over the web interface
  • Post the resulting hash.
  • Check the response.

The fuzzy matching is deliberately very conservative; it tries

  • case insensitivity,
  • 'channel' instead of 'chan',
  • 'frequency' instead of 'freq',
  • 'number' in place of 'num' and
  • 'authority' in place of 'auth'.

I draw the line at translating StartTime ->StartTS as I think that too dangerous for the general case. If you encounter these or other discrepancies you can always add these to the proposed hash before calling ValidatePost.


If there is a parameter which is optional (this is not obvious from the wsdl page) then this can be indicated by inserting # in front of the parameter name as a key in the hash. In this case, the parameter will not be posted but neither will it raise an error.


In case of error such as the host is not reachable, an API call cannot be identified, the API call is a GET, a parameter cannot be matched (or found as #name), the number of parameters matched is below sanity level or if the post fails then the routine dies in the case of 'raw' and 'post' actions. For the two test options, it will return 0.

No type checking (eg numeric, text, date) of parameters is performed and the hash is not checked for duplicates.


The action parameter must be one of:

  • 'post' which issues the post if all is well, and
  • 'raw' which omits the ampersand checking. Any inconsistencies in these two routines will result in a 'die' to prevent any database corruption.

Both these routines return the text returned by the server or they die trying. The returned text often has <bool>true</bool> to describe success, but other API calls return an integer.

  • 'test' which validates the parameters, complains if necessary, then returns without issuing the post. It returns 0 for failure, 1 for success.
  • 'quiettest' similar but does not print error messages.

The sanity check parameter is mandatory. The 'post' will only be issued if the number of matched parameters is at least this threshold. eg for UpdateDBChannel I use a figure of 12. This check has been included to guard against the situation where a change in format to the wsdl pages causes NO parameters to be returned resulting in a 'post' which submits an empty hash and causes chaos. This will also allow me to sleep soundly at night!


Thus to hide channel 1003:

my %hash;  my $content;
ReadBackend('192.168.1.67:6544/Channel/GetChannelInfo?ChanID=1003', $content);
GetAllFields(%hash, $content, '<ChannelInfo', '</ChannelInfo>');
$hash{Visible}='false';  #the value we are changing
$hash{'#Icon'}='';        #allow myth to assume default since we know no better
ValidatePost(%hash, '192.168.1.67:6544/Channel/UpdateDBChannel', 'raw', 12);

Note that from v31 you also need to set $hash{ExtendedVisible}='Not Visible'. Did I test this? Yes, I hid then un-hid all BBC channels. Worked great! Didn't work so well with the channel 'Create & Craft' until I switched 'post' to 'raw' though! See traps for the unwary below. It's also the basis of the perl channel editor.

APISupported

This routine is a helper routine for 'ValidatePost' but is presented as it may prove useful.

It will read the appropriate wsdl page and determine whether the url is a valid API name. It return 2 if the function is a POST, 1 for a GET function and 0 if unsupported. eg

unless (APISupported('192.168.1.67:6544/Dvr/RemoveRecorded') {
        #try DeleteRecording instead
}

GetAPIParameters

This routine is also a helper routine for 'ValidatePost' and is also presented as it may prove useful.

It reads the wsdl and returns an array holding the names of the calling parameters. eg

my @array=GetAPIParameters('192.168.1.67:6544/Channel/UpdateDBChannel');

It will use cached values if available but otherwise issue a wsdl call.

Two hashes used in the module are also made available externally and are mentioned here for completeness. These are the caches of information gathered from wsdl calls:

%scan_database::APICallingParameters which is a hash of arrays holding calling parameters and %scan_database::APItype which holds API type. eg

my @parameters=@{${scan_database::APICallingParameters{'Channel/GetChannelInfo'}}};
my $state=$scan_database::APItype{'Channel/GetChannelInfo'};   #returns 1 as its a get


The scan_database Module

The module needs to be put in a file called scan_database.pm with read permissions. Note the .pm used for modules rather than the .pl of perl scripts. It will work if placed in one of the directories in the perl path. The perl path for your system is different to that used by the command interpreter and can be determined by printing the perl variable @INC. eg

perl -e 'print join ("\n", @INC), "\n"'

If you want to place the module in the current folder you will need to include this directive before your 'use' :

use lib '.';

The libwww-perl module needs loading - if using ubuntu try

sudo apt-get install libwww-perl


Here is the module:

#!/usr/bin/perl -w
package scan_database;
use strict;
use Exporter;
use LWP::UserAgent;  # needs 'sudo apt-get install libwww-perl'

#Version 1.01 25 Aug 2015.     APISupported ValidatePost and GetAPIParameters added.
#Version 1.02 26 Aug 2015.     Changes to ValidatePost:
#Version 1.03 15 Sep 2015.     Fixed bug:  Misuse of $_ in KillEscapes causes various problems with escape sequences.
#Version 1.04. 23 Sept 2015.   ValidatePost 'test' shows values extracted. 
#Version 1.05  24 Sept 2015.   Allow ReadBackend in scalar context - return 1 if ok, 0 if host not responding.
#                              Still die in void context.
#Version 1.06  29 Sept 2015.   Allow external access via 'our' %APICallingParameters.
#                              Fix bug if wsdl shows API call has no calling parameters.  Affects APISupported and 
#                              GetAPIParameters.
#Version 1.07   1 Oct 2015.    Rework of wsdl with no parameters 29/9/15.  (oops!).
#version 1.08  17 Oct 2015     Extend APISupported to return 1 if a get, 2 if a post. 
#                              Tweak to accept 0.28 wsdl format
#                              ValidatePost to return the POST response eg <bool>true</bool>.
#                              ValidatePost to check that it is a post function
#Version 1.09  10 Nov 2015     Request compressed content in ReadBackend.
#Version 1.10  10 May 2022     Allow new API inteface on :6744 - tags eg <VideoSource version ...> as well as <VideoSource>
#              12 May 2022     fix to regex in APISupported  changed \s? to \s* 
#Version 1.11  19 May 2022     FillHashofHash improved - use regex throughout instead of index.
#Version 1.12   1 June 2022    Ditto for GetAllFields, Corrected count entry in FillHashofHash 


# DOCUMENTATION:  See https://www.mythtv.org/wiki/Perl_API_examples


our $VERSION     = 1.12;
our @ISA         = qw(Exporter);
our @EXPORT      = qw(ReadBackend FillHashofHash GetAllFields APISupported ValidatePost GetDBinfo GetDBheader GetAPIParameters);

my $browseropen=0;
my $browser;
my $counter;
our %APICallingParameters;
our %APItype;


#steering file for GetDBHeader and GetDBinfo
my %steer = (
    getchannelinfolist  => ['Channel/GetChannelInfoList', 'ChannelInfo','ChannelInfos'],
    getrecordedlist     => ['Dvr/GetRecordedList?Descending=true', 'Program','Programs'],
    videosourcelist     => ['Channel/VideoSourceList', 'VideoSource','VideoSources'],
    
    #add more like this => [url, separator for getDBinfo, separator for getDBheader],  
);

#--------------------------
# published routines follow
#--------------------------

sub ReadBackend($\$) {
 
    my ($url, $buff)=@_;
    unless ($browseropen){
        $browser = LWP::UserAgent->new;
        $browser->timeout(10);
        $browser->default_header('accept-Encoding' => 'gzip,deflate'); 
        $browseropen = 1;
    }
    my $response = $browser->get(tidyurl($url)); 
    unless($response->is_success){
        (defined wantarray()) || die $response->status_line;
        return 0;
    }
    $$buff = $response -> decoded_content;
    return 1;
}
#------------------------------
sub FillHashofHash(\%\$@){
    my ($hashref, $buff, $separator, $key_name, @params)=@_;
    %$hashref=();
    my $count=0;
    my $key;
    my $seek="(<$separator>|<$separator" . '\s[^>]*?>)(.*?)</' . $separator . '>';
    while ($$buff =~ m!$seek!g){
		$key=$count;
		if ($key_name ne '#'){$key=myget($2, $key_name)};
		for my $item (@params){
			if ($item eq '#'){
				$$hashref{$key}{$item}=$count
			}else{
				$$hashref{$key}{$item}=myget($2,$item)
			}
		};
		$count++;	
	}
	return $count;
};

#-------------------------
sub GetAllFields(\%\$@){

    #   Read the buffer and populate a hash with all tags found between delimiting text.
    #   GetAllFields(%hash, $buffer, $start_text, $end_text)
    #       if $start_text or $end_text are empty strings, then start or end of buffer are assumed.
    #       eg
    #       ReadBackend('192.168.1.67:6544/Channel/GetChannelInfoList', $buffer)
    #       GetAllfields(%hash, $buffer, '', '<ChannelInfos>')
    #       print "$hash{'ProtoVer'}\n";
    #   rewritten 1 June 2022
    
    my ($hashref, $buff, $start_text,$end_text)=@_;
    $end_text |='$';    #defaults
    $start_text='^';
    %$hashref=();
    my $counter=0;
    my $buff2;
    my $seek="$start_text(.*?)$end_text";
    $$buff =~ m!$seek!;
    $buff2=$1;
    unless (defined $buff2){die "Faulty start or end"};
    
    #Extract instances of  <key>something</key>
    while ($buff2 =~ m!<(\w*)>(.*?)</\1!g){
		$$hashref{$1}=&KillEscapes($2);
		$counter++;
	}
	
	#and of <key/> 
	while ($buff2 =~ m!<(\w*)/>!g){
#-----------------------------
		$$hashref{$1}='';
		$counter++;
	};
	$counter;
}
#--------------------------

sub APISupported($){
    #Check API against wsdl.
    #return 0=not known, 1=a GET, 2=a POST
    my ($APIurl)=@_;
    if ($APIurl =~ /\?/) {die "APISupported does not expect '?' in $APIurl"};
    my @URLbits = split '/', $APIurl;
    my $apicall=pop(@URLbits);
    my $service=$URLbits[-1];
    $apicall = $service . '/' . $apicall;
  
    #Do we know about this api service?  
    unless (defined $APICallingParameters{$service}){
        # Nope!  do a wsdl call        
        push  @URLbits, 'wsdl';
        my $url=join '/', @URLbits;
        my $content;
        ReadBackend($url,$content);
        #
        while ($content =~ m!<xs:element name="(\w*)">(.*?)</xs:complexType>!gs){
            (my $func, my $list)=($1, $2);
              
            unless ($func =~ /Response$/){
                @{$APICallingParameters{$service .'/' . $func}}=();  #In case no params found
                while ($list =~ m!name="(\w*)"!g){
                    push ( @{$APICallingParameters{$service .'/' . $func}}, $1);
                }
            }
        } 
        $APICallingParameters{$service}=1;  #to allow caching
        
        #now see if post or get: sample text is:
        #     <operation name="AddVideoSource"><documentation>POST </documentation>
        while ($content =~ /<operation name="([\w]*)">\s*<documentation>([\w]*)/g) {
			#  regex modified 12 May 2022:  was ...\s?  change needed for API backend on post 2744
            if ($2 eq 'POST') {
                $APItype{$service .'/' .$1}=2;
            }elsif ($2 eq 'GET'){
                $APItype{$service .'/' .$1}=1;
            }else{
                die "Not recognised operation type $2 for API $1 in APISupported"
            }
        }    
    }
    return $APItype{$apicall} || 0;
}
#----------------------
sub GetAPIParameters($){
    my ($p)=@_;
    unless (APISupported($p)){die "Unsupported API $p"};
    my @URLbits = split '/', $p;
    $p=$URLbits[-2] . '/' .$URLbits[-1];
    @{${APICallingParameters{$p}}};
}


#------------------------
sub ValidatePost(\%$$$){

    #Validate data for a POST before issuing it
    my ($hashref, $APIurl,$action, $SanityCount)=@_;

    unless ($APIurl =~ /^http:/) {$APIurl= 'http://' . $APIurl};
    unless ($action =~ /^post$|^test$|^raw$|^quiettest$/){
        die "ValidatePost actions are post, test, quiettest or raw.  $action not supported"
    };

    unless (&APISupported($APIurl)==2){
        if ($action !~ /quiettest/){print "$APIurl not supported or not a post on this backend"};
        if ($action =~ /test/){return 1};
        die '';
    }
     
    #split the url
    
    my @URLbits = split '/', $APIurl;
    my $apicall=pop(@URLbits);
    my $service=$URLbits[-1];
    $apicall = $service . '/' . $apicall;

    #ok, start by generating equivalent parameter names
    my %alias;my $k, my $v;
    while (($k, $v) = each %$hashref){
            $_=$k; 
            unless ($action =~ /raw/){$v=&InsertEscapes($v)};
            $alias{$_}=$v;
            $_=lc $_; $alias{$_}=$v;
            s/chan/channel/ unless (/channel/);
            s/freq/frequency/ unless (/frequency/);
            s/num$/number/;
            s/auth$/authority/;
            $alias{$_}=$v;
    }

    
   #now let us check against the wsdl output - build up a valid hash in %final
    my $missing=0;my %final;my $found=0;
    for (GetAPIParameters($APIurl)){    
        if (defined $alias{lc $_}){
            if ($action eq 'test'){print "   $apicall found $_: Setting to: $alias{lc $_}\n"};
            $final{$_}=$alias{lc $_};
            $found++;
        }elsif (defined $alias{"#$_"}){
            #ok to omit this one - user declares it as optional.
            $found++;
        }else{
            print "** $apicall needs $_ **\n" unless  ($action eq 'quiettest');
            $missing++;
        }
    }
    if ($found<$SanityCount){
        #something has gone horribly wrong!
        $missing++; #to force exit
        unless ($action =~ /quiet/){
            print "Failed sanity check:  found $found params but expected at least $SanityCount\n";
        }
    }
    
    if ($action =~ /test/){return ($missing==0)?1:0};
    if ($missing){die "Unsafe to proceed with ValidatePost "};
    
    #Now post it!
    my $response = $browser -> post($APIurl,\%final);
    unless($response->is_success){die "Error updating recording"}; 
    my $url_buffer = $response -> content; 
    return $url_buffer;
}


#--------------
sub GetDBheader{
    #deprecated.
    #expects hash reference as first item; returns count of header items. 

    my ($hashref, $ip_addr, $class,$extra)=@_;
    $class=lc $class; 
    unless (defined ($steer{$class})) {die "Sorry - $class not supported"};
    $counter=0;
    #read from backend 
    my $url= &FormURL($ip_addr, $steer{$class}[0], $extra);
    my $buffer;
    ReadBackend($url, $buffer);
    return GetAllFields(%$hashref, $buffer,'',$steer{$class}[2]);
}
#---------------------

sub GetDBinfo{

my ($hashref, $ip_addr, $class, $extra, $key_name, @params)=@_;

    $class=lc $class; 
    unless (defined ($steer{$class})) {die "Sorry - $class not supported"};
 
    #read from backend 
    my $url= &FormURL($ip_addr, $steer{$class}[0], $extra);
    my $buffer;
    ReadBackend($url, $buffer);
    #process buffer
    FillHashofHash(%$hashref, $buffer, $steer{$class}[1], $key_name, @params);
}


#-------------------------
#internal routines follow
#-------------------------

sub myget{
	my ($content, $tag)= @_;
	my $seek="(<$tag>|<$tag" . '\s[^>]*?>)(.*?)</' . $tag . '>';
	if ($content =~ m!$seek!){
		return $2;
	}else{
		$seek="<$tag/>";
		if ($content =~ m!$seek!){return ''};
		return '';  #or your preferred action on missing tag.  
		            #Beware: Port 6744 omits many null values and v33 will probably do so too. 
	}
}

sub tidyurl{
    my ($url)=@_;
    $url='http://'. $url unless ($url =~ m!^http://!);
    $url;
}


sub FormURL{
    #used in 
    my ($ip, $class, $extra)=@_;
    my $addr="http://$ip:6544/$class";
    if ($extra){
        if ($addr =~ /\?/){
            $addr .= '&' . $extra;
        }else{
            $addr .= '?' . $extra;
        }
    }
    return $addr;
}


sub KillEscapes{
    my ($data)=@_;
    return $data unless ($data =~ /&\w+;/);
    my $a=';';    #trick to prevent wiki from corrupting this code 
    $data =~ s/&quot$a/"/g; 
    $data =~ s/&lt$a/</g; 
    $data =~ s/&gt$a/>/g; 
    $data =~ s/&apos$a/'/g; 
    $data =~ s/&amp$a/&/g; 
    $data;
}

sub InsertEscapes{
    my ($data)= @_;
    my $a=';';
    $data =~ s/&/&amp$a/g; 
    $data =~ s/"/&quot$a/g; 
    $data =~ s/</&lt$a/g; 
    $data =~ s/>/&gt$a/g; 
    $data =~ s/'/&apos$a/g; 
    $data;
}
1;

Traps for the unwary

Apart from the issues above, there are particular traps:


1. Many of the routines have parameters StartIndex and Count. If neither is supplied, then all entries are assumed. Note that there is an inconsistency with StartIndex.

With GetChannelList, StartIndex=0 must be supplied in order to get the first entry. That's logical.

With GetRecordedList, StartIndex=0 and StartIndex=1 return the same data. Thus code which grabs the data in chunks eg

StartIndex=0&Count=100
StartIndex=100&Count=100

will have a duplicate returned because he first call will return 1 to 100 inclusive, the second 100 to 199.

2. GetRecorded and RemoveRecorded both have a 'StartTime' parameter.

Note that the value supplied should be the time a recording started and not the time in the schedules. These will differ in cases where a recording is requested after the start of the recording or if all recordings are started early. If you extract the time from a GetSomethingList call then you need to extract 'StartTS' not 'StartTime'.

3. Note that after a 'post', a subsequent 'get' may not reflect the changes just made. (Cache action? What is the solution?).

4. 'get' calls appear to require translation of escape sequences (eg &amp; to &) but the UpdateDBChannel post does not need the converse so a ValidatePost call needs a 'raw' action. You may need to check the particular 'post' API you are using.

5. Note that there were some changes to the API output with the new interface in v32 via port 6744 which may require changes to parsers and which prompted the release of version 1.12 of scan_database.pm above. See https://forum.mythtv.org/viewtopic.php?f=33&t=4848 If you need to check the version of scan_database.pm you will find it in the variable $$scan_database::VERSION.

Most calls have tags around items eg for GetChannelInfoList the information for each channel was wrapped between <ChannelInfo> and </ChannelInfo>. With port 6744 the tags are: <ChannelInfo version="2.2"> and </ChannelInfo>

Many 'null' entries which may previously have been returned by <tag></tag> or <tag/> are missing. Whereas scan_database would previously have claimed user error ('that tag does not exist') version 1.12 now needs to return a null value.

Some calls are now 'POST' rather than 'GET'.


More information please!

Have fun and do help to expand this page or add comments in the discussion!