ExternalRecorder

From MythTV Official Wiki
Revision as of 20:04, 4 April 2020 by Jpoet (talk | contribs) (Writing an External Recorder)

Jump to: navigation, search

External Recorder

As of v0.28-pre-1073-ga9ce4c09b mythbackend has the ability to communicate with an external "Black Box" application for recording. This external application module does whatever it needs to do to generate a transport stream which mythbackend then reads and records.

Using an External Recorder

Run mythtv-setup and choose "External Recorder" for the type of capture card. Fill in the path to the application and include any command line arguments needed. The standard myth arguments will also be passed (e.g. -q --syslog none --logpath -v).

Myth will expect a MPTS from the application, but that does not mean there must be more than one program in the transport stream. If the application reports that it has a 'tuner', then mythbackend will try to use it to tune to an appropriate channel. Depending on the source, you may need to manually enter channels via "6. Channel Editor" in mythtv-setup. If you do add channels, make sure the ServiceID is set to a non-Null value.

As of 16-October-2018, An External Recorder can support the ability to "scan" channels. After configuring the recorder, use mythtv-setup to scan for channels, and those channels will automatically be added to MythTV.

A couple of examples of External Recorders are:

Hauppauge HD-PVR2 and Colossus2

https://github.com/jpoet/HauppaugeUSB

Silicon Dust HDHomeRun Premium

https://github.com/garybuhrmaster/mythhdhrrecorder.git

Using the "Generic" External Recorder

As of 16-October-2018 (master) and 01-November-2018 (fixes/29), MythTV includes a "Generic" External Recorder application which is installed as mythexternrecorder. It uses configuration files to control its function. The primary configuration file tells it what "system" application to run. It can also reference a secondary channel configuration file which is used to populate MythTV's guide and to tune different channels.

Configure the External Record in mythtv-setup using the full path to mythexternrecorder and include "--conf <config.conf>", for example '/usr/bin/mythexternrecorder --conf /usr/share/mythtv/vlc.conf'.

Adjust your configuration files as appropriate. The result of the command needs to be a transport stream. The command listed in the configuration file should provide all the arguments necessary.

Example Configuration files

Log files will be created in the same directory as your other mythtv logs. Important logging will also show up in your mythbackend log file. Logging levels will be inherited from mythbackend. mythexternrecorder logs most messages under the channel or record categories, so use "-v channel,record" with mythbackend to get useful messages.

Writing an External Recorder

When mythbackend wants to start a recording using the 'External Recorder', it executes the application, and then communicates with it via STDIN, STDOUT and STDERR. Commands will show up on the applications STDIN. The application will respond to the commands via its STDERR. When told to stream a MPTS (or SPTS) the application will send the data out STDOUT.

There are two API levels supported The difference is that version 2 includes a <serial#> as part of each message. The External Recorder is expected to mimic that <serial#> in its response. This helps mythbackend ensure that it has not missed the response to the request. If a message from the External Recorder starts with "0:STATUS:" (serial # is 0 (zero)), the message will be considered to be "out of band"; the STATUS word will be stripped off and the rest of the message will just be logged.

These are the commands that the external recorder application (module) needs to understand:

  • "APIVersion?" (Valid response is "OK:" followed by an API version number) Version 1 or 2 is currently supported
  • "<serial#>:APIVersion:<value>": (Valid response is "OK". Will only be sent if Application responded to "APIVersion?". Lets the applicaton know what API version the backend will be using.
  • "<serial#>:Version?" (Valid response is "<serial#>:OK:" followed by any version information the application wishes to provide.
  • "<serial#>:Description?" (Valid response is "<serial#>:OK:" followed by a description of the recording. This is used in mythbackend logging messages.
  • "<serial#>:IsOpen?" (Valid response is "<serial#>:OK:Open" or "OK:No") Is the recorder application in a good state?
  • "<serial#>:CloseRecorder" (Valid response is "<serial#>:OK:Terminating", and then the application will shut down)
  • "<serial#>:HasTuner?" (Valid response is "<serial#>:OK:Yes" or "OK:No") Does the recorder have a tuner?
  • "<serial#>:LoadChannels" (Valid response is "<serial#>:OK:" followed by the number of channels for this source)
  • "<serial#>:FirstChannel" (Valid response is "<serial#>:OK:" followed by "<channum>,<channame>,<callsign>,<xmltvid>")
  • "<serial#>:NextChannel" (Valid response is "<serial#>:OK:" followed by "<channum>,<channame>,<callsign>,<xmltvid>")
  • "<serial#>:TuneChannel:<value>" (Valid response is "<serial#>:OK" or "<serial#>:InProgress") If "InProgress" is return, then the "TuneStatus?" message must be supported
  • "<serial#>:TuneStatus?" (Valid response is "<serial#>:OK" or "<serial#>:InProgress")
  • "<serial#>:HasPictureAttributes?" (Valid response is "<serial#>:OK:Yes" or "<serial#>:OK:No") Does the recorder support picture adjustments (hue, brightness, etc.)?
  • "<serial#>:LockTimeout?" (Valid response is "<serial#>:OK:<value>", where value indicates a value in ms indicating when to give up on acquiring a signal lock)
  • "<serial#>:SignalStrenghtPercent?" (Valid response is "<serial#>:OK:<value>", where value indicates the strength of the signal in a range of 0-100). App should return "OK:100" if the concept of a signal strength does not apply.
  • "<serial#>:HasLock?" (Valid response is "<serial#>:OK:Yes" or "<serial#>:OK:No") If not applicable, app should return "OK:Yes"
  • "<serial#>:FlowControl?" (Valid response is "<serial#>:OK:Polling" or "<serial#>:OK:XON/XOFF").
  • "<serial#>:BlockSize:<value>" (Valid response is <serial#>:OK") This tells the app the maximum number of bytes that can be sent without mythbackend blocking.
  • "<serial#>:StartStreaming" (Valid response is "<serial#>:OK:Started", and the Transport Stream should be sent to STDOUT)
  • "<serial#>:StopStreaming" (Valid response is "<serial#>:OK:Stopped", and data should no longer be sent to STDOUT)
  • "<serial#>:XON" (Valid response is "<serial#>:OK", and the app is allowed to write to STDOUT)
  • "<serial#>:XOFF" (Valid response is "<serial#>:OK", and the app should stop writing to STDOUT)
  • "<serial#>:SendBytes" (Valid response is "<serial#>:OK")

For any response that can just be "<serial#>:OK", the application may send other information after the "<serial#>:OK". For example, a response of "<serial#>:OK:5632423 bytes sent" is a valid response for the "SendBytes" command.

If the application returns "<serial#>:OK:Yes" to "HasTuner?", then it needs to handle:

  • "<serial#>:TuneChannel:<value>"

<value> will be a channel number, possibly looking like 1234-23 to indicate major-minor elements.

As of MythTV v32-Pre 1dd0408e, the external application may return "InProgress" in response to a "TuneChannel" request.

If the application returns "<serial#>:OK:Yes" to "<serial#>:HasPictureAttributes?", then it also needs to deal with these commands:

  • "<serial#>:SetBrightness:<value>"
  • "<serial#>:SetContrast:<value>"
  • "<serial#>:SetColour":<value>"
  • "<serial#>:SetHue":<value>”

Note: The PictureAttribute commands are currently unimplemented.

If FlowControl is set to Polling, then the application needs to processes the SendBytes command. If FlowControl is set to XON/XOFF, then the application needs to process XON and XOFF commands. Polling mode works better for applications which generally end up waiting on mythbackend to send data. XON/XOFF works better for applications which are event driving, and spend more time waiting on the recording source.

When the external module receives the "XON", "XOFF" or "SendBytes" command, it will respond in one of four ways via its STDERR:

  • "<serial#>:OK"
  • "<serial#>:OK: some descriptive text" -- same as just "OK", but allows the external module to tell mythbackend to log the external module's 'state'.
  • "<serial#>:WARN: some descriptive text" -- which may indicates that the external module is not able to send any bytes right now, but to try again.
  • "<serial#>:ERR: some descriptive text" -- which indicates that the external module is in an error state, and the recording needs to abort and possibly restart.

This enables a "flow control" between the external module and mythbackend and allows the external module to report problems.

The external module has the burden of buffering the transport stream. If mythbackend does not read the transport stream fast enough from the external module, the external module will have to decide if it needs to increase its buffer, or if it should drop packets. If the external module needs to drop packets, that might be a case where it would respond with something like "OK: # packets have been dropped".

When mythbackend first starts up it will invoke the external application and send it various commands like "Version?". If no errors are detected then that "input" will be put into a "good" stage, and the "CloseRecorder" command will be sent to the app. When LiveTV or a recording is ready to start, the application will be launched again and will be kept active until it is no longer needed.

When a recording or LiveTV is starting, the "LockTimeout?" message will be sent. When mythbackend is first starting up and it is just checking to see if the recorder is valid, the "LockTimeout?" message is not sent. This can be useful for the External Recorder to know. For example, there is no need to actually initialize the transport stream until the "LockTimeout?" message is received.

The external application will be passed several arguments in addition to whatever is configured when setting up the recorder. These typically are: "--verbose <list> --logpath <path> --loglevel <level> --quiet --inputid <id>"

More Details

APIVersion?

Currently two API versions are supported. The main difference is the inclusion of the <serial#> as part of each message. If an External Recorder does not respond to this message, then API version 1 is assumed.

<serial#>:APIVersion:<value>

This indicates which API version mythbackend has selected to use, based on the response from "APIVersion?". For example, if the External Recorder responded with '3', but this version of mythbackend only understands version 2, then it uses this message to indicate that version '2' will be used.

<serial#>:Version?

This lets mythbackend query and log the External Recorder version.

It is also sent occasionally as a "Are you still there?" message. If the External Recorder does not respond, then mythbackend will assume it is wedged and it will try to kill and restart the External Recorder.

<serial#>:Description?

This is used in the mythbackend logs to make it easier to track which External Recorder is be referenced.

<serial#>:IsOpen?

mythbackend uses this query to determine if the External Recorder is ready to produce a transport stream

<serial#>:CloseRecorder

This tells the External Recorder that it services are not longer needed, and it should clean up and terminate.

<serial#>:HasTuner?

If this returns yes, then LoadChannels, FirstChannel, NextChannel and TuneChannel may be used.

<serial#>:LoadChannels

This indicates to the ExternalRecorder than FirstChannel and NextChannel are about to be called. If the External Recorder needs to do anything to prepare for that, then it should do it now.

For example, the "Generic" External Recorder can be configured to run a script which generates a "channel.conf" file.

<serial#>:FirstChannel

Returns the first channel "tune-able" by this External Recorder. For example:

2012,"Channel Name","SomeChannel","xmltvid-string"

The information returned by this and NextChannel will be used to populate the MythTV channel table.

Note: Each time a channel scan is performed, the data for this "source" is replaced, not added.

<serial#>:NextChannel

Returns the next channel "tune-able" by this External Recorder.

<serial#>:TuneChannel:<value>

Tells the External Recorder to tune to this channel.

Note: LoadChannels, FirstChannel and NextChannel are not necessary for this to work. Those are only used if the External Recorder supports "scanning" for the list of channels, as apposed to using some other channel data source.

<serial#>:HasPictureAttributes?

Not currently supported, but it may be in the future. The External Recorder should just respond with "<serial#>:OK:No"

<serial#>:LockTimeout?

mythbackend will use this value to know when to give up on getting a lock on the transport stream.

This query is sent when mythbackend is ready to test the "signal quality", in preparation for recording. The External Recorder can use this message as an indicator that it is about to "get real", and any "lazy" initialization should now be performed.

Note: When mythbackend first starts up, it tests each recorder input to validate its availability. mythbackend will not be ready until each recorder has been verified. If the External Recorder performs any time-intensive operations when it first starts up, it can significantly delay the availability of mythbackend.

<serial#>:SignalStrenghtPercent?

This query is used for informational purposes only and can be used by the External Recorder to indicate the "progress" in getting the transport stream ready.

<serial#>:HasLock?

mythbackend will wait until the External Recorder returns "<serial#>:OK:Yes" to start reading the data.

<serial#>:FlowControl?

The External Recorder should respond with "<serial#>:OK:XON/XOFF". Polling is also supported, but is far less efficient.

<serial#>:BlockSize:<value>

This suggests a good chunk size for the data. The External Recorder is free to send as much data as the underlying operating system can support.

<serial#>:StartStreaming

The External Recorder should start buffering transport stream data, so it is available to send to mythbackend.

<serial#>:StopStreaming

The External Recorders can stop buffering transport stream data.

<serial#>:XON

The External Recorder can send transport stream data to mythbackend (via STDOUT).

<serial#>:XOFF

The External Recorder must stop sending transport stream data to mythbackend. It should continue to buffer the data so there is not break in the data when mythbackend is ready to resume processing.

<serial#>:SendBytes

This tells the External Recorder to send a block of transport stream data. This query is only used when polling flow control is used.

Out of Band

The External Recorder can send an "out of band" status message at any time. For example:

0:STATUS:Some status message
0:STATUS:WARN:Something unexpected but not critical has happened.
0:STATUS:ERR:An error has occurred

These message will show up in the mythbackend log. The 'STATUS:ERR' message will be treated as fatal and mythbackend will probably close down the external recorder, and restart it.

It is a good practice to always use 0 (zero) as the serial# for such messages. At the very least, the serial# needs to be less than any serial# previously seen.

mythfilerecorder

Bundled with myth is the mythfilerecorder application. It is an "External Recorder" which operates using polling flow control. Configure it in mythtv-setup with a command line like "/usr/local/bin/mythfilerecorder --infile <somefile>". By default it will loop back to the beginning of the file when it reaches the end of the file. It can be given the --noloop argument to disable this behavior.

mythfilerecorder uses polling and is not necessarily a good example of an External Recorder.