IPTVRecorder

From MythTV Official Wiki
Jump to: navigation, search

General MythTV recorder architecture

The five most important classes in the recorder architecture are:

 * TVRec -- same for all recorders
 * RingBuffer -- same for all recorders
 * DeviceReadBuffer
 * RecorderBase
 * SignalMonitor
 * ChannelBase

TVRec

When either Scheduler or a TVPlay instance wants to start recording they call TVRec::StartRecording() with the ProgramInfo of the program they wish to record. Once recording the Scheduler or TVPlay do not need to end the recording, TVRec takes care of ending the recording.

TVRec runs its own event loop in TVRec::run(). This event loop predates the ability to run Qt event loops in any thread and is custom to TVRec. It handles three things: state change commands initiated from TVRec::ChangeState(), tuning requests from the tuningRequests deque, and it handles ending in progress recordings.

RingBuffer

This class provides the interface that RecorderBase uses to write data to disk. Internally it uses a ThreadedFileBuffer, but the most important thing to RecorderBase is that the RingBuffer::write() does not block for more than a few milliseconds at a time, unlike disk writes themselves which can block for a number of seconds on a system that is operating normally.

This class' public methods are thread-safe.

DeviceReadBuffer

Like the RingBuffer buffers reads for file the DeviceReadBuffer buffers reads from character devices. This is used by both some DTVRecorder based recorders (via their StreamHandler), and MPEGRecorder. If you are reading from a character device in your recorder it is highly recommended you use this class.

RecorderBase

This class does two main things:

    * Coordinates moving data from a device to a RingBuffer
    * Writes any metadata pulled from the streams to disk or database.

This class is a QRunnable and is given its own thread by TVRec.

This class' public methods are thread-safe.

Instances of this class are created when recording starts and destroyed when recordings end. With back-to-back recordings this class will live through the program transitions.

ChannelBase

This class handles changing channels.

In some cases this responsibility is delegated to other classes, but this class provides the interface that TVRec and SignalMonitor interact with.

This class' public methods are thread-safe.

Instances of this class are created when TVRec starts up, deleted when TVRec is torn down. Currently these are only destroyed on mythbackend shutdown, but the plan is for these to sometimes be torn down while the backend is running as part of the HTML configuration of the backend work.

SignalMonitor

This class tells us if there is data coming in on our device that we can record. These are started up between the time we tune to a channel using ChannelBase and start recording with RecorderBase. Using a signal monitor is only required if it is possible for a recorder to fail to tune to a channel. Currently all recorders employ a SignalMonitor.

Digital TV Recorders

Digital Television in the world today has some attributes that both make it more difficult to record than analog television but also has many commonalities that allow a great deal of code reuse. Most notably every DTV in operation today, from ATSC to DVB to OpenCable to ISDB are built on top of the MPEG ISO 11172 standard adopted in 1993 (except possibly China). And even more fortuituously they all use MPEG transport streams. There are two sets of data we care about in any MPEG transport stream. The first is Program System Information Tables or PSIP, and the other is the location of keyframes in the stream.

An MPEG Transport stream (MPEG-TS) contains 188 byte packets each containing a 13 bit PID and 184 bytes of payload. PID 0 always contains the Program Association Sections that make up the Program Association Table or PAT. The PAT in turn contains a list of PIDs and information on those PIDs. Most notably, it lists all the active Program Map Table PIDs, or PMT PIDs. Each PMT describes a 'channel' or 'program stream'. That is it lists all the video and audio stream PIDs associated with the program and optionally provides metadata about the program stream. These two tables are all we need to tune to an MPEG program stream. If a channel has it's sistandard field in the database set to 'mpeg' or 'opencable' then what we expect for a DTVSignalMonitor to be happy is a PAT containing the program number or 'serviceid' of the program in it's list, and then on the PID pointed to by the PAT a complete PMT. For ATSC and DVB streams some additional tables are required, but the concept is the same. For instance for ATSC we have 'major' and 'minor' channel numbers we find listed in the Virtual Channel Table or VCT and this VCT lists the program number, which we look up in the PAT to so we can find the complete PMT.

TBD

  *Explain keyframe finding done by DTVRecorder::FindXXXKeyframes()

We don't want everyone writing a recorder to need to be an expert in digital television to write a recorder. If you look at HDHRRecorder::run() you will see these three lines:

   m_stream_data->AddAVListener(this);
   m_stream_data->AddWritingListener(this);
   m_stream_handler->AddListener(m_stream_data);

This line:

   m_stream_data->AddAVListener(this);

tells our MPEGStreamData instance to call this classes Audio/Video handling methods whenever it sees audio and video data. The MPEGStreamData instance is given to HDHRRecorder by TVRec and may very well be a DVBStreamData or a ATSCStreamData, but most recorders don't need to care about those details. Note: HDHRRecorder inherits from DTVRecorder so it doesn't actually need to implement any of these Audio/Video handling methods. DTVRecorder finds keyframes in the data and schedules the packets for writing to disk. The MPEGStreamData instance also performs all the necessary PID filtering. PID filtering is sometimes handled in elsewhere by the recorder, but this is only for efficiency reasons. For instance, the HDHomeRun can in most cases filter out PIDs we don't care about on the device itself and this avoids sending this data we will throw away over the network.

This line:

   m_stream_data->AddWritingListener(this);

tells our MPEGStreamData instance to send other data marked for writing to us as well. This might include something like the PCR timing stream when it is sent separately from the audio and video streams.

This line:

   m_stream_handler->AddListener(m_stream_data);

tells the StreamHandler instance that HDHRRecorder manages to write all its data to our MPEGStreamData instance. StreamHandlers are not used by all recorders, in fact we could just read data from a some location in the HDHRRecoder::run() loop and write it to the MPEGStreamData by calling it's ProcessData() method. This is what the RecorderBase interface was originally written for, but StreamHandlers were added to support recording multiple programs received on the same device by sharing a device with several recorders each of which writes one file to disk.

As an important aside for someone writing a recorder, there are seven main responsibilities Recorder::run() fulfills:

 1/ Calls Open() and exits if the open fails.
 2/ Sets recording to true under the pauseLock and wakes up any thread waiting for the recorder to start or stop
 3/ Calls StartNewFile()
 4/ Exits promptly if IsErrored() returns true
 5/ Exits promptly if IsRecordingRequested() returns false
 6/ Temporarily pauses recording if request_pause is true, resumes when request_pause is false.
 7/ After IsErrored() is true or IsRecordingRequested() is false:
    7.a/ Calls Close()
    7.b/ Calls FinishRecording();
    7.c/ Sets recording to false under the pauseLock and waits for any thread waiting for the recorder to start or stop

Number 6 is interesting and a possible source of error. There tends to be a fair bit of set up and tear down involved with the use of a recorder; this is especially true with the analog framegrabber recorders MythTV originally supported. To avoid a teardown and rebuild when switching channels in LiveTV mode this pausing mechanism was devised. This probably wouldn't be designed this way today but it isn't that hard to get right either. All you need to do is ensure no data is written while the recorder is paused. In the case of the HDHRRecorder there is a custom HDHRRecorder::PauseAndWait() that just calls m_stream_handler->RemoveListener(_stream_data); when the recorder is paused, and m_stream_handler->AddListener(_stream_data); when it is unpaused.

In the HDHRRecorder::run() loop we spend most of our time sleeping in either PauseAndWait() or unpauseWait.wait(&pauseLock, 100); This is pretty typical of most recorders. You want this loop to be able to stop or pause recordings quickly because the quicker you respond the more quickly channel changes can happen.

MPEGStreamData

This class provides several services for digital tv recorders:

* Determining which Transport Stream to record
* Transport Stream Packetization
* PID Filtering
* PSIP table caching

This class, and the children ATSCStreamData and DVBStreamData, are what allow us to demultiplex broadcast streams containing multiple programs into single program streams. They are essential there, but irrelevant to recorders that only see a single program stream, such as the MPEGRecorder which handles HD-PVR recordings.

TBD

StreamHandler

When the V4L and MPEG recorders were originally written the recorder's run method called read on the /dev/video device and piped that directly into the key-frame detector and on to RingBuffer::write(). A similar mechanism was later applied to Digital TV recorders, but many people wanted to be able to record multiple program streams from the same multiplex. This was especially true in Northern Europe where whole countries carried all the TV channels on a single transport. From this need the StreamHandler was born. It is an abstraction that lives on between a DTVRecorder and the actual device and allows multiple recorders to share that device. There are several specializations of this, the DVBStreamHandler, HDHRStreamHandler, ASIStreamHandler, but they do basically the same thing. Currently the StreamHandler is only used for MPEG-TS streams. In theory a StreamHandler class could be used by recorders like MPEGRecorder to allow overlapping recordings, but this hasn't been attempted yet and there may be issues doing this.

IPTV Recorder Architecture

The Digital TV recorders are designed to demultiplex a single multi-program MPEG-TS multiplex and write it to disk. Multiple DTVRecorder classes sharing a single StreamHandler class can record different streams from any single multiplex, but the multiplex is tied to a single physical 'channel' at a time. Because of this tie a DVBRecorder or a HDHRRecorder always uses the same StreamHandler tied to the same physical device, for example "/dev/dvb/adapter0/frontend0".

IPTV streams are different. While there may be a limitation on the number of simultaneous recordings from a content provider you are never tied to a 'channel' and generally each MPEG-TS stream only carries a single program. Because of this difference an IPTVRecorder instance doesn't use a single IPTVStreamHandler. Instead, it gets the StreamHandler from the IPTVStreamHandler::Get() factory by giving the factory method IPTVTuningData. And the IPTVStreamHandler takes care of getting the remote source to send the data and reading from the actual stream.

IPTVTuningData consists of one to three URLs. At minimum a data URL, and optionally up to to FEC URLs and/or bitrate info. Currently we don't use FEC data, but a more robust IPTV recorder would use this to deal with dropped packets. The tuning data is stored in the iptv_channel data and is associated with the rest of the tuning info by the chanid; see IPTVTuningData ChannelUtil::GetIPTVTuningData(uint chanid) for details.

The IPTVRecorder currently handles two basic types of streams, MPEG-TS streams over UDP or RTP. And these may contain any of the various audio and video codecs other MythTV recorders support.

UDP isn't much of a protocol, it just means the MPEG-TS stream is sent as a series of UDP packets, each containing 7 or so 188 byte MPEG-TS packets. If a packet is lost it is lost forever; if packets are reordered the stream is scrambled. RTP on the other hand is a huge protocol and we support only the common subset used by IPTV broadcasters. What we do support in RTP is packet reordering and we have some of the parts needed to support Forward Error Correction. The way packet reordering is handled is via the PacketBuffer abstraction, with a specialization RTPPacketBuffer for RTP streams. Notably, RTP streams are also used by the CetonRecorder and the CetonRecorder's CetonStreamHandler is just a specialization of IPTVStreamHandler.