I love catchy titles. I know nothing about the next major version of Flash and all of this is just speculations.
Anyway, as the whole HTML5 versus Flash battle is raging, it appears Flash relatively poor performances are indeed criticizable at best. Beside poor developers, the Flash Platform suffers from a very very slow software renderer. Or at least much slower than the rest of the platform. It’s no secret: hardware acceleration is a key feature for the future of the Flash Player. It’s even hard to believe it is not available yet!
As you must already know, Flash 10.1 will support OpenGL ES 2 on mobile devices to leverage the lack of CPU horsepower. While hardware HD video decoding will be available on the desktop too, the drawing API will only be accelerated on mobile devices.
Anyone knowing a bit about OpenGL ES knows it is a subset of OpenGL. Thus if it works with OpenGL ES, it should work with OpenGL. With this in mind, a few questions:
- Why isn’t Flash 10.1 drawing API hardware accelerated on any OpenGL capable platform, including the desktop?
- How will it work?
- Will Pixel Bender be hardware accelerated?
1. Hardware Accelerated Desktop Flash Player
Let’s face it: there must be an hardware accelerated desktop Flash Player in the works. As I said previously, OpenGL supports all features of OpenGL ES so this is not far fetched at all. Yet, it is neither released nor announced. Why?
My first guess is OpenGL provides with a lot of features that would make the desktop experience a lot smoother than just using what offers its little brother OpenGL ES. So at some point Adobe had to make a choice :
- release a fully hardware accelerated Flash 10.1 on both mobile and desktop platforms, with the last one being very far from what desktop hardware is actually capable to handle
- or release Flash 10.1 focusing on mobile devices and announce hardware acceleration for the desktop just after its the final release… and I’m guessing it might be a key feature of Flash 11
When I was at the French Flash User Group (TTFX – les TonTons FleXeurs) a few months ago, I spoke with Lee Brimelow and Mike Chambers about the just announced microphone raw-data access. I asked them why it was announced only for AIR 2.0 and not Flash 10.1. The answer was something about the “quality guys” making sure the feature was well suited on both the roadmap and the logic of the incoming updates. And this feature eventually made its way into the Flash Player! I think that’s what is happening with hardware acceleration on the desktop.
But if you don’t believe me, you don’t have to take my word for it! What about Adobe’s word? Cnuuja, one of the engineer working on the Flash Player, posted this very message on the Flash 10.1 Forum:
Can OpenGL 3.3/4.0 improve Flash 10.x ?
“Yes…. with a lot of work. We have spent the last year writing new code which allows OpenGLES2 to render flash content on mobile devices. Performance varies significantly from one gpu to the next, with some gpus being slower than the software renderer. What Flash does is significantly different from the 3d triangles+shaders GPUs were designed to support. Its a lot of work to make OGL/D3D usable as our renderer, but we’re working on it
2. How will it work?
Just like in Flash 10.1 for mobile devices. But much faster thanks to OpenGL and Direct3D.
The internals of such feature is very important: developers must know and understand how it works to make the best out of it. Adobe already talked about how the drawing API is accelerated in Flash 10.1 on mobile devices:
“When a GPU renders vector graphics, it breaks them up into meshes made of small triangles before drawing them, a process called tesselating. There is a small cost to doing this, which increases as the complexity of the shape increases. To minimize performance impact, avoid morph shapes, which must be retesselated on every frame.”
It’s straight forward and I think it’s actually the best (and only…) way to do it. Tesselation will create triangles by computing sets of vertices/indices (also called Vertex and Index Buffers) and push them to the graphics hardware. The end of the quote suggests such data is cached and should not be recomputed if no redraw occurs.
Something very important though: z-sorting. People might think hardware acceleration implies z-sorting. But it doesn’t. When you know how 3D hardware and APIs work, you know it will be very tricky to make it work with something as general purpose as Flash. If Adobe wants to use the z-buffer, they will have to cut the compatibility with the software renderer. And I don’t think this will happen anytime soon.
3. Hardware Accelerated Pixel Bender
Pixel Bender is already hardware accelerated pretty much everywhere except the Flash Platform. I’m not sure why. Still, it’s hardware accelerated in other products of the Creative Suite so I guess that an OpenGL/Direct3D shader languages compliant intermediate represenation of Pixel Bender kernels does exist.
This said, it’s just a matter of how to make it work with the very general purpose Flash Player. Considering Flash 10.1 is using tesselation, my guess is pixel shader should follow quite easily.