[mythtv-users] Flash, VDPAU, MythNetVision and You

[Michael T. Dean]

  All the below is TTBOMK--I may well be wrong in my interpretation of 
the information I have.


On 12/03/2010 10:27 PM, Eric Sharkey wrote:
> On Fri, Dec 3, 2010 at 10:08 PM, Jarod Wilson wrote:
>> Xv support gets you what? Colorspace conversion?
> I'm certainly no expert, but I believe scaling is done in hardware with xv.

Yes. Really the biggest problem with Flash is software scaling of video.

Decoding low-bitrate video (even H.264) isn't that tough (at least not 
for a real CPU :). And, yes, even Hulu "high-definition" video is 
low-bitrate (480kbps, 700kbps, and 2.5Mbps On2 Flash VP6 or 1000kbps 
H.264 Flash--see 
http://en.wikipedia.org/wiki/High-definition_video#HD_on_the_World_Wide_Web.2FHD_Streaming 
).

Scaling low-resolution H.264 to full 1920x1080 in software is /very/ tough.

I'm pretty sure this is why those who tried the CrystalHD decoding in 
Flash were somewhat disappointed by the results. (I.e. it solved the 20% 
problem (decoding), leaving the 80% problem (software scaling) still 
there--or should that be the 5%/95% problems.)

Note, also, that Flash has had hardware acceleration support on 
GNU/Linux since 9.0.115.0 (in May 2008) using OpenGL (and I'm pretty 
sure that includes the 64-bit Flash versions). This allows the video to 
be dumped to OpenGL in RGB (the colorspace that Flash uses***). Then 
OpenGL can do the video scaling, and full-size graphic elements can be 
composited on top of the video by OpenGL. (Then again, with OpenGL 
support, I /think/ they could dump the unscaled video in YUV to the 
graphics card, let OpenGL scale it, then dump a full-size graphic 
element in RGB, and let the graphic card composite it on top of the 
video--doing all required colorspace conversions--but I'm not positive. 
And even if they could, I don't know if they actually do it that way 
versus just doing the YUV->RGB conversion in software.)

However, hardware acceleration in Flash only works with content authored 
to take advantage of this. See:
http://blogs.adobe.com/penguinswf/2008/05/flash_uses_the_gpu.html
and
http://www.kaourantin.net/2008/05/what-does-gpu-acceleration-mean.html
for details.

TTBOMK, this just means that for 2 1/2 years, it's been possible for 
sites like Hulu/NBC and Fox and CWTV (which use wmode transparent) and 
CBS (which uses wmode opaque) to have provided content that allows 
hardware-accelerated playback in Flash, but they chose not to.

Of course, now that Adobe is making a big deal out of it, they're likely 
to do things right. However, AFAIU, that means that anyone with OpenGL 
support will also benefit.

And, Eric, FWIW, the Adobe blog post has the reason why Adobe doesn't 
use Xv:
---

    * FAQ regarding hardware acceleration: Why doesn’t the Flash Player
      on Linux user[sic] the X video extension (Xv)?
    * Answer: Because Xv scales YUV data. Flash Player operates on RGB data.

For the uninitiated, many video codecs operate in a YUV colorspace. 
Unix/X11 has an extension called X video that allows hardware scaling of 
YUV images. This is a very mature system on Linux which has allowed 
seamless, low CPU usage, fullscreen video playback on Linux for many 
years. Unfortunately, the Flash Player can not easily make use of this 
since Sorenson, On2, or H.264 video data — even though it is decoded as 
YUV — has to be converted to RGB and possibly combined with other 
graphical elements. This is why RGB scaling via OpenGL is the future of 
Flash.
---

Well, now it seems they're saying that VDPAU is the future of Flash... 
Either way, TTBOMK, fixing the Flash to work with VDPAU will also fix it 
to work with OpenGL.

See, also, 
http://blogs.adobe.com/penguinswf/2010/01/solving_different_problems.html

Also, take note of who wrote those 2 Adobe blog posts. He should know 
what he's talking about when it comes to FOSS video playback... (Search 
on his name and look at Google's, "Searches related to..." list.)

Mike

*** Whereas Xv only works with YUV. Flash actually decodes H.264, which 
results in YUV. Then it does a software colorspace conversion to RGB so 
that it can do compositing of graphic elements on top of the video. They 
also do software scaling of the video. And, they are likely doing 
compositing after scaling so that they can render the graphic elements 
at full display size rather than have them tied to the video size--play 
back a 160x120 video in MythTV on a 1920x1080 screen using the Xv 
renderer and display the OSD to see why. If you instead use the OpenGL 
renderer on that same 160x120 video on a 1920x1080 screen, you get a 
nice OSD because it's not compositing before scaling the video.