Release Notes - 31 Video decoding and playback

Introduction

There have been significant changes in the decoding and playback of video for the 0.31 release.

Please read these notes carefully to ensure you get the best performance and quality from your system.

OpenGL is now a requirement

As of version 0.31, all video playback requires a working OpenGL installation.

This maximises performance, quality and consistency of display across a wide range of devices and platforms. It also allows MythTV to efficiently display video from the variety of hardware accelerated video decoders that are supported.

There are no strict requirements on the version of OpenGL required - any version from OpenGL 2.0 and OpenGL ES 2.0 and above should work. In the future, more modern versions of OpenGL may be required for some advanced features but the intention is that these will be remain optional.

If OpenGL is not available when starting Mythfrontend, MythTV will fallback to Qt painting to display the GUI and a notification will be displayed that OpenGL (and hence video playback) is not available.

Video decoders

Full accelerated hardware decoding is now available with the following interfaces:-

- VAAPI
  For Linux systems using Radeon and Intel (integrated) GPUs.

- VDPAU
  For Linux systems using NVidia GPUs and their proprietary video drivers.

- NVDEC
  For Linux systems using more modern NVidia GPUs and their proprietary video drivers (including CUDA).

- VideoToolBox
  MacOSX.

- Video4Linux2 Codecs(V4L2-codecs)
  For certain 'embedded' Linux devices including the Raspberry Pi and other supported 'SoCs'.

- MMAL.
  For the Raspberry Pi.

- MediaCodec.
  Android.

OpenMax support on the Raspberry Pi has been removed, as has support for the long deceased CrystalHD discrete video decoder. VideoToolbox decoding for OSX is a replacement for VDA decoding.

Video display profiles

With the move to OpenGL, decoder changes and significant changes to deinterlacing preferences, the Video Display Profile settings pages have changed significantly.

All new and existing users are strongly advised to review their video display profiles.

Users should take particular note of their decoder preferences and deinterlacing settings (see below).

As part of the update to version 0.31, all video display profiles stored in the database have been modified/upgraded to ensure they still function with the new setup. The update attempts to make sensible decisions about the appropriate new settings and in the vast majority of cases, this should be seamless.

There will however be cases where an updated profile needs editing. This is most likely to impact users who previously used VDPAU and/or VAAPI for rendering of software decoded video; this use case is no longer supported.

Video frame formats

Previous versions of MythTV only supported displaying frames in the YUV420P frame format - which has been the de facto standard for many years for broadcast and media based (i.e. DVD, BluRay) video.

All other formats were converted to YUV420P prior to display. This led to a potential loss of quality and added an additional processing stage which reduced performance.

The OpenGL renderer can now display all of the common YUV frame formats - YUV420P, YV422P, YUV444P and NV12 - in all bit depths (e.g. YUV420P10, P010 etc).

This improves performance, allows for lossless display of higher bit depth material (see below) and integrates well with hardware decoders that typically return NV12/P010 frame formats when decoding.

Note: OpenGL ES2 only supports display of YUV420P, YUV422P and YV444P formats. All other formats will be converted in software to one of these types.

Note: When decoding for preview generation, commercial flagging etc, the code will still always convert to YUV420P.

Video renderers

When software decoding, two OpenGL Video renderer options are available; 'OpenGL' and 'OpenGL YV12'.

'OpenGL' will convert YUV420P frame formats to an intermediate format before uploading to an OpenGL texture for display. This adds a little software overhead but its main advantage is that it speeds up OpenGL deinterlacing. Hence it may be useful when the majority of content is interlaced and the OpenGL hardware is old - and slow. If the frame format is not YUV420P, the renderer will always fallback to 'OpenGL YV12'. Note: this approach is likely to be removed for MythTV version 0.32.

'OpenGL YV12' should be considered the default OpenGL renderer. It takes video frames and uploads them to OpenGL textures for display - unaltered.

When hardware decoding is selected, the only available Video renderer is 'OpenGL Hardware'. In the event that hardware decoding is not available or fails, this will always fallback to 'OpenGL YV12'.

Video colourspaces

Colourspace conversion has been re-written for version 0.31.

A single conversion matrix is used in the OpenGL shaders to handle the conversion to RGB (for display) as well as processing all picture adjustments (brightness, contrast, colour and hue). This also applies to most hardware decoders although some hardware decoders may have limited support.

An additional conversion is also available when the video colourspace differs from the display colourspace.

The behaviour of this additional stage is controlled by a new setting 'Primary colourspace conversion' (Setup->Video->Playback->Advanced Playback Settings).

This defaults to 'Auto' and most users will have no need to change from this default.

When 'Auto' is selected, the primary colourspace conversion will be enabled when the display colourspace is significantly different from the video. This typically means it will be enabled for High Dynamic Range (HDR) material. For most 'traditional' colourspaces, the differences between the video and display colourspaces are small enough that most people cannot tell the difference - and the extra processing stage is ignored. (This generally means displaying any Standard Range video (SDR) on traditional SDR displays - most of which use the Rec. 709 colourspace).

When 'Exact' is selected, the conversion is enabled whenever there is a difference between the video and the display, regardless of how small that difference may be.

Use 'Disable' to disable entirely.

Note: The previous setting for 'Studio levels' that was available from the menu during playback has been replaced with a global setting 'Use full range RGB output' (Setup->Appearance->Theme/Screen settings). This ensures consistent range adjustment for the UI as well as video. The associated keybinding has also been removed.

10/12bit Video

Decoding of higher depth video is supported by FFmpeg (software decoding) and some hardware decoders.

With the updates to the OpenGL video renderer, there should be no loss in precision when processing and displaying these video streams.

MythTV will always try and retain precision, including the use of 16bit OpenGL framebuffers and textures while rendering. The final rendering stage may (will) however lose precision when using OpenGL ES 2.0 and when using OpenGL ES3 on Qt versions before Qt5.12 (support was not complete before this version of Qt).

Lossless display of 10bit video does however require a 10bit display framebuffer.

This is not currently widely supported in Linux; though improvements are being made. Setting up a 10bit display is beyond the scope of these notes (and may well break a number of non-MythTV desktop applications).

Retaining 10bit precision throughout the MythTV code will improve display quality regardless.

High Dynamic Range (HDR) Video

While HDR video will be decoded and displayed, there is no support in version 0.31 for signalling to the display that the current content is High Dynamic Range. This will hopefully be at least partially included in version 0.32.

Support for signalling the correct data is only just becoming available in the latest Linux kernels and on other devices and platforms requires a significant amount of new code.

The visual impact of displaying HDR material without this support depends on the type of HDR material:-

Hybrid Log Gamma (HLG)

- HLG is designed to display on both HDR and standard range (SDR) displays. As such, it should display in good quality even without explicit HDR support.

HDR10

- HDR10 material tends to look 'washed out' when display without HDR support. There is currently no workaround for this problem (see Tonemapping below).

Tonemapping of HDR material

Tonemapping is the process of adjusting HDR material so that it displays 'correctly' on displays with a more limited range.

This will be available in version 0.32.