You can watch the entire conference here.

As I promised, here are the slides to this presentation. They are pretty heavy because they embed some videos. Here is the outline of the content of the presentation:

• Community SDK
• Scripting
• Shaders and GPU programming
• Scene editor
• Professional SDK
• Physics
• Optimizations for the web and mobile devices

At the end of the presentation, I also demonstrated how Minko can load and display Crytek's Sponza smoothly on the iPad and the Nexus 7 in just a few minutes of work thanks to the editor and the optimizations granted by the MK format. You will soon here more about this very demonstration wiht a clean video demonstrating the app. but also the publishing process. This is incredibly cool since Sponza is quite a big scene with more than 50 textures including normal maps, alpha maps and specular maps for a total of 200+MB (only 70MB when published to MK).

Don't forget to have a look at all the online resources for Minko:

• Minko's downloads page
• Minko's GitHub repository
• Minko's examples GitHub repository
• The tutorials and the online developers reference
• Answers, the official support forum

As stated in the presentation, Minko's editor public beta should start next week. So stay tuned!

That's why I've compiled a new documentation article that explains how to properly export Collada files from 3D Studio Max:

Export Collada files from 3D Studio Max on Aerys Hub

The article details the export procedure itself but also provides guidelines to make sure the exported file will display properly with Minko. It also give details about the supported features, including material, material properties and how the engine will use them. Similar articles will be released for different editors and formats according to the community's needs.

Of course, as soon as the Minko editor and the MK format will be available we will strongly discourage the use of Collada files in production for many reasons (mainly performances). But you will still need to import those Collada files into the editor first... So here is a little video to show how simple importing assets is with the editor:

Yes: it's as simple as a drag'n'drop! You will also notice that the textures load automatically and that the editor will let you play the animations and make sure everything works out of the box. The editor will be available in open beta in March...

Tonight's experiment was focused on post-processing. My goal was to implement a simple anamorphic lens flare post-processing effect using Minko. It was actually quite simple to do. Here is the result:

The 1st pass applies a luminance threshold filter:

public class LuminanceThresholdPass extends Shader
{
    private var _postProcessing : PostProcessingShaderPart;
 
    public function LuminanceThresholdPass(renderTarget  : RenderTarget  = null,
                                           priority      : Number        = 0.0)
    {
        super(renderTarget, priority);
 
        _postProcessing = new PostProcessingShaderPart(this);
    }
 
    override protected function getVertexPosition():SFloat
    {
        return _postProcessing.vertexPosition;
    }
 
    override protected function getPixelColor():SFloat
    {
        var rgb			: SFloat	= _postProcessing.backBufferPixel;
        var rgbToLuminance 	: SFloat 	= float3(0.2126, 0.7152, 0.0722);
        var luminance		: SFloat	= dotProduct3(rgb, rgbToLuminance);
 
        return multiply(
            _postProcessing.backBufferPixel,
            greaterEqual(luminance, sceneBindings.getParameter('luminanceThreshold', 1))
        );
    }
}

Then I use a multipass Gaussian blur with 4 passes: 3 horizontal passes and 1 vertical passes. The trick is to apply those 5 passes (1 luminance threshold pass + 4 blur passes) on a texture which is a lot taller than wide (32x1024 in this case). This way, everything gets streched when the flare are composited with the rest of the backbuffer.

The Problem

That might look like a complicated post title... but this is rather complex than really complicated. Here is how it starts: rendering a 3D object require to execute a graphics rendering program - or "shader" - on the GPU. To make it simple, let's just say this program will compute the final color of each pixel on the screen. Thus, the operations performed by this shader will vary according to how you want your object to look like. For example rendering with a solid flat color requires different operations than rendering with per-pixel lighting.

Any programming beginner will understand that such program will test conditions - for example whether to use lighting or not - and perform some operations according to the result of this test. Yes: that's pretty much exactly what an "if" statement is. It might look like programming trivia to you. And it would be if this program was not meant to be executed on the GPU...

You see, the GPU does not like branching. Not one bit (literally)! For the sake of parallelization, the GPU expects the program to have a fixed number of operations. This is the only efficient way to ensure computations can be distributed over a large number of pipelines without having to care too much about their synchronization. Thus, the GPU does not know branching and each program has a fixed number of instructions that will always be executed in the same order.

Conclusion: shader programs cannot use "if" statements. And of course, loops are out of the game too since they are pretty much pimped out "if" statements. Can you imagine what such logic would imply on your daily programming tasks? If you simply try to, you will quickly understand that instead of writing one program that can handle many different situations you will have to write many different programs that will handle a single situation. And then manually choose which one should be launched according to your initial setup...

Workarounds...

Mutables

The simplest workaround is to find "some way" to make sure useless computations do not affect the actual rendering operations. For example, you can "virtually disable" lighting by setting all lights diffuse/specular components to 0.

As you can imagine, this is really a suboptimal option. Performance wise, it's actually the worst possible idea: a lot of computations happen and most of them are likely to be useless in most cases.

If/else shader intrinsic instructions

After a few years, shaders evolved and featured more and more instructions. Those instructions are now usable through higher level languages such as CG or GLSL. Those languages feature "if" statements (and even loops too). How are they compiled into shader code that can run on a GPU? Do they overcome the challenges implied by parallelization?

No. They actually fit in in a very straight forward and simple way. As a shader program must feature a single fixed list of instructions, the two parts of a if/else statement will both be executed. The hardware will then decide which one should be muted according to the actual result of the test performed by the conditional instructions.

The bright side is that you can use this technique to have a single shader program that handles multiple scenarios. The dark side is that this shader is still very inefficient and might eventually break the limit number of instructions for a single program. On some older hardware, the corresponding hardware instructions simply do not exist...

So even this "brand new" feature that will be introduced in Flash 11.7 and its "extended" profile is far from sufficient.

Pre-compilation

Some engines will use high level shader programming languages (like CG or GLSL) and a pre-compilation workflow to generate all the possible outcomes. Then, the right shader is loaded at runtime according to the rendering setup. This is the case of the Source Engine, created by Valve and used in famous games like Half Life 2, Team Fortress 2 or Portal.

This solution is efficient performance wise: there is always a shader that will do exactly and strictly the required operations according to the rendering setup. Plus it does not have to rely on some hardware features availability. But pre-compilation implies a very heavy and inefficient assets workflow.

Minko's Solution

We've seen the common workarounds and each of them has very strong cons. The most robust implementation seems to be the pre-compilation option despite the obvious workflow issues. Especially when we're talking web/mobile applications! But the past 10 years have seen the rise of a technique that could solve this problem: Just In Time (JIT) compilation. This technique is mostly used by Virtual Machines - such as the JVM (Java Virtual Machine), the AVM2 (Actionscript Virual Machine) or V8 (Chrome's JavaScript virtual machine). It's purpose is to compile the virtual machine bytecode into actual machine opcodes at runtime in order to get better performances.

How would the same principle apply to shaders? If you consider your application as the VM and your shader code as this VM execution language, then it all falls into place! Indeed, your 3D application could simply compile some higher level language shader code into actual machine shader code according to the available data. For example, some shader might compile differently according to whether lighting is enabled or not or even according to the number of lights.

With Minko, we tried to keep it as simple as possible. Therefore, we worked very hard to find a way to be able to write shaders using AS3. As the figure above explains, the AS3 shader code you write is not executed on the GPU (because that's simply not possible). Instead, the application acts as a Virtual Machine and as it gets executed at runtime, this AS3 shader code transparently generates what we call an Abstract Shader Graph (ASGs). You can see it as an Abstract Syntax Tree for shaders (you can even ask Minko to output ASGs in the consoleas they get generated using a debug flag). This ASG in then optimized and compiled into actual shader code for the GPU.

For example: everytime you call the add() method in your AS3 shader code, it will create a corresponding ASG node. This very node will be linked with the rest of the ASG as you use it in other operations until it is finally used as the result of the shader. This result node becomes the "entry point" of the ASG.

Here is what a very simple ASG that just handles a solid flat color rendering looks like:

Here is what a (complicated) ASG that handle multiple lights looks like:

Your AS3 shader code is executed at runtime on the CPU to generate this ASG that will be compiled into actual shader code that will run on the GPU (in the case of Flash it will actually output AGAL bytecode that will be translated into shader machine code by the Flash Player). As such, you can easily perform "if" statements that will shape the ASG. You can even use loops, functions and OOP! You just have to make sure the shader is re-evaluated anytime the output might be different (for example when the condition tested in a "if" changes). But that's for another time...

Using JIT shaders, Minko can efficiently dynamically compile shaders shaped by the actual rendering settings occuring at runtime. Thus, it combines the high performance of a pre-compilation solution while leveraging all the flexibility of JIT compilation. In my next articles, I will explain how JIT shaders compilation can be efficiently automated and how multi-pass rendering can also be made more efficient thanks to this approach.

If you have questions, hit the comments or post in the Minko category on Aerys Answers!

I will share extensive details about this new physics engine during the Stage3D online conference in February. Make sure you attend: it's online and it's free!

The Demo

Anyway, I just wanted to tease the community with one of the stress tests we're using here to benchmark the engine. It's a really simple test and it uses only a very very small subset of the available features.

Here you go (just press "space" to throw some balls):

About Scalability

There are actually quite a few interactive objects active at once in this scene: more than 350! And if you throw some balls (using "space"), it should go even higher. Yet the application should run very smoothly on a decent computer: the framerate doesn't get below 24 fps on my 2011 laptop using a core i7@1.5Ghz/Intel HD 3000.

The number of contact points/collisions and therefore the stacking appears to be the main bottleneck here. And that's a very good thing since it's precisely the point of this stress test.

When the application starts, you might also notice that the framerate doesn't reach 60 fps until a few seconds. This is because colliders need this time to stabilize and "fall asleep" when they finally stop moving for long enough. After a few seconds, you should get 60 fps. If you don't, it means your GPU is the bottleneck since sleeping physics colliders are actually virtually free. This also means the engine can easily handle very large physical worlds and detect/compute collisions in a very optimized fashion.

Future Improvements

You might think that the fact we're using boxes might makes things a lot faster for the physics engine. Yet we've implemented nothing special for boxes collisions tests so this is a real glimpse at the actual general performances of the engine. Other stress tests will enlight that of course, and we might implement box/box optimization in the future to get even more performances.

When you press space to throw a ball, you might also notice a performance drop. This is unlikely to have anything to do with physics. I think it's rather related to the fact that adding new scene nodes will require to rebuild the local to world transforms tree into a linearized list. This might be avoided by re-using the "fallen" balls instead of building new ones everytime "space" is pressed.

Of course, feel free to discuss this demo and share your results in the comments or on Answers.

TL;DR

There is a new 3D transforms API available in the dev branch that provides a 50000% 25000% boost on scene nodes' matrices update in the best cases, making it possible to display 50x 25x more animated objects. You can read more about the changes on Answers.

The Problem

Here is how it worked until very recently: each scene node was the target of a TransformController. The job of this controller was to listen to the changes of the ISceneNode.transform property in order to update the localToWorld and worldToLocal properties accordingly. Of course, as those properties rely on the parent's transform, the controller also listen to it and that's where trouble starts: modifying the transform of a node will trigger a lot of signals executions, implying a lot of overhead.

The worse case scenario is vertex skinning: the skeleton has to be updated top to bottom (because there is no other way to do it really...). And each update on a node will trigger updates on all of its descendants, leading to a factorial complexity and a huge overhead.

The good point with this method is that all the scene nodes transforms are always available and "synchronized" with each other. The very bad point is that it does a lot of unnecessary computations using recursive signals that imply a huge function calls overhead.

The Solution

We had three priorities:

1. Provide a new, robust and fast 3D transformation matrices computation strategy.
2. Keep API changes to a minimum.
3. Make sure we don't remove any feature.

To fasten the local to world matrices computation, we decided to update them in a batched fashion: all the local to world matrices of a (sub)scene had to be updated in a single method call using a single loop. It forbids updating the (sub)scene as a tree data structure, since it would mean recursion - and a big function call overhead - or using stacks - which is not the strong point of AS3. Therefore, we had to linearize the (sub)scene in order to get a list of transforms. To do this, we simply do a breadth first traversal of the (sub)scene tree when it has changed in order to get a flat list of matrices. We also store a few other things in other lists - such as the number of children or the id of parent of each node - in order to preserve the data we need to traverse that very list as a linearized tree structure.

This job is done by the TransformController.updateTransformsList() method:

private function updateTransformsList() : void
{
    var root 	: ISceneNode 	= _target.root;
    var nodes 	: Vector. 	= new [root];
    var nodeId 	: uint 		= 0;
 
    _nodeToId = new Dictionary(true);
    _transforms = new [];
    _localToWorldTransformsInitialized = new [];
    _localToWorldTransforms = new [];
    _worldToLocalTransforms = new [];
    _numChildren = new [];
    _firstChildId = new [];
    _idToNode = new [];
    _parentId = new [-1];
 
    while (nodes.length)
    {
        var node 	: ISceneNode 	= nodes.shift();
        var group 	: Group 	= node as Group;
 
        _nodeToId[node] = nodeId;
        _idToNode[nodeId] = node;
        _transforms[nodeId] = node.transform;
        _localToWorldTransforms[nodeId] = new Matrix4x4().lock();
        _localToWorldTransformsInitialized[nodeId] = false;
 
        if (group)
        {
            var numChildren 	: uint = group.numChildren;
            var firstChildId 	: uint = nodeId + nodes.length + 1;
 
            _numChildren[nodeId] = numChildren;
            _firstChildId[nodeId] = firstChildId;
            for (var childId : uint = 0; childId < numChildren; ++childId)
            {
                _parentId[uint(firstChildId + childId)] = nodeId;
                nodes.push(group.getChildAt(childId));
            }
        }
        else
        {
            _numChildren[nodeId] = 0;
            _firstChildId[nodeId] = 0;
        }
 
        ++nodeId;
    }
 
    _worldToLocalTransforms.length = _localToWorldTransforms.length;
    _invalidList = false;
}

In order to avoid unnecessary computations, we decided to update them on a frame-to-frame basis. To make sure this update happens just before rendering and that all matrices are actually up to date, we've added the Scene.renderingBegin signal. As you might have guesses, this signals is simply executed right before the scene starts the actual rendering operations when Scene.render() is called (so it's called after Scene.enterFrame, which is the signal which should be used to update the scene).

This is the job of the TransformController.updateLocalToWorld() method:

private function updateLocalToWorld(nodeId : uint = 0) : void
{
    var numNodes 		: uint 		= _transforms.length;
    var childrenOffset		: uint		= 1;
    var rootLocalToWorld	: Matrix4x4	= _localToWorldTransforms[nodeId];
    var rootTransform		: Matrix4x4	= _transforms[nodeId];
    var root			: ISceneNode	= _idToNode[childId];
 
    if (rootTransform._hasChanged || !_localToWorldTransformsInitialized[nodeId])
    {
        rootLocalToWorld.copyFrom(rootTransform);
 
        if (nodeId != 0)
            rootLocalToWorld.append(_localToWorldTransforms[_parentId[nodeId]]);
 
        rootTransform._hasChanged = false;
        _localToWorldTransformsInitialized[nodeId] = true;
        root.localToWorldTransformChanged.execute(root, rootLocalToWorld);
    }
 
    for (; nodeId < numNodes; ++nodeId)
    {
        var localToWorld 	: Matrix4x4	= _localToWorldTransforms[nodeId];
        var numChildren		: uint		= _numChildren[nodeId];
        var firstChildId	: uint		= _firstChildId[nodeId];
        var lastChildId		: uint		= firstChildId + numChildren;
        var isDirty		: Boolean	= localToWorld._hasChanged;
 
        localToWorld._hasChanged = false;
 
        for (var childId : uint = firstChildId; childId < lastChildId; ++childId)
        {
            var childTransform		: Matrix4x4		= _transforms[childId];
            var childLocalToWorld	: Matrix4x4		= _localToWorldTransforms[childId];
            var childIsDirty		: Boolean		= isDirty || childTransform._hasChanged
                || !_localToWorldTransformsInitialized[childId];
 
            if (childIsDirty)
            {
                var child	: ISceneNode	= _idToNode[childId];
 
                childLocalToWorld
                .copyFrom(childTransform)
                    .append(localToWorld);
 
                childTransform._hasChanged = false;
                _localToWorldTransformsInitialized[childId] = true;
                child.localToWorldTransformChanged.execute(child, childLocalToWorld);
            }
        }
    }
}

You can read the changelog and more details about the list of API changes on Aerys Answers.

Future Improvements

We could make the TransformController.updateLocalToWorld() method a bit faster by avoiding taking identity matrices into account or do a special case for matrices that have nothing but a translation.

Memory wise, we could also avoid using the _localToWorldTransformsInitialized vector all together and check whether _localToWorld[nodeId] is null or not.

This Flex SDK can be downloaded directly on their site:

Combined version of the Flex 4.6 and AIR 3.5 SDK for the lazy ones on swfgeek.

• The JointsDebugController makes it possible to display the joints and bones of a skinned mesh in order to check if everything works as expected.



• The VertexPositionDebugController will display the position of each vertex of a mesh.
• The VertexNormalDebugController will display the normal of each vertex. You can use it with the VertexNormalShader to display and debug the normals of a mesh.

Examples

• VertexIterator Example
• JointsDebugController Example

Answers

• Mapping Flex UI components to textures in Minko: an interesting thread about how to emulate mouse-enabled 2D components - such as Flex components - mapped as 3D textures

Tutorials

• Display the position and the normal of each vertex
• Display the bones and joints of a skeleton
• Select scene nodes with XPath

Fixes

• minko-collada will now load the vertex RGBA data when it's available
• min/max computation is always possible upon creation of a VertexStream regardless of its StreamUsage
• frustum culling will now also test the center of the bounding box and not only the 8 corners

Wieliczka Salt Mine website and apps

Created by the Polish agency GoldenSubmarine, the Wieliczka Salt Mine web site offers an interative experience with HD 360° panoramas built with Minko. Thanks to Adobe AIR, the application is also available on mobile platforms such as the iPad, the iPhone and Android.

This awesome project was just rewarded with the "FWA Mobile of the Day" award! This is the very first FWA for a project built with Minko so we are very proud of course!

Videos

This video presents yet another example of a very cool shader built by developer Jeremy Sellam from the Les Chinois, a French digital agency based in Paris, France. It was built with the public beta of the ShaderLab of course!

Features

ByteArray streams

Geometry is now stored in ByteArray objects. It reduces the memory consumption and should provide a significant performances boost. Especially on mobile devices. You can read more about this feature on my previous blog post.

Parametric frustum culling

Minko now implements frustum culling in a very flexible fashion. You can chose on what planes and using what volume (sphere or box) frustum culling is computed. For example, to use the bounding sphere on all planes but the box only on the near/far planes, you can write:

mesh.frustumCulling = FrustumCulling.SPHERE | FrustumCulling.BOX_NEAR | FrustumCulling.BOX_FAR;

Such flexibility should make it possible to optimize performances to every use case. It's also nice to consider computations are kept to a minimum by storing world space versions of the bounding sphere/box. You can view the entire code for this feature in the VisibilityController.

Per-triangle ray casting

Geometry.cast() now makes it possible to get the triangle ID (the first indice of the triangle) that crosses the specified ray. Combined with Mesh.cast() and Group.cast(), it makes it very easy to perform ray-casting at any level, from the root of the scene to the geometry level.

Local/world gizmos in Minko Studio

Gizmos are the key to any WYSIWYG editor. We've enhanced Minko Studio's gizmos not only by adding scale/rotation gizmos but also the possiblity to chose whether those gizmos should be used in local or world space.

Smooth shadows in Minko Studio

Shadows are a very important part of any real-time 3D immersive application. Thus, being able to control the quality of the shadows while keeping interactive framerates is very important. The minko-lighting plugin already introduced soft shadows a few weeks ago, and now the corresponding options are available right inside Minko Studio! It's now easier than ever to fine tune the lighting engine in order to get the best performance/quality ratio.

Documentation

Minko now has a new, clean and fast growing wiki: the developers Hub. You will find a lots a old and new tutorials. But also lots a new examples and projects done with Minko!

• Add pixel-perfect 3D mouse interactivity
• The KeyboardScriptExample demonstrates how to build a very simple script to handle focus 3D objects and move them with the keyboard. This example really highlights how easy it is to create controllers and have them working on any scene, thus making it possible to code things once and re-use them at any time!

Fixes

• Geometry.fillNormalsData() and Geometry.fillTangentsData will reset the corresponding buffers to make sure normals/tangents are not incremented everytime they are supposed to be updated.
• Picking has been completely refactored: all the known bugs have been fixed and we are now using a 1*1 BitmapData/scissor rectangle to get better performances.
• The Collada loader will now inspect 3D transforms in order to handle negative scales properly. It will also invert the z axis at the vertex level in order to perform right to left handed coordinates conversion. It fixes problems occuring when loading animations build using symmetry.

Don't forget to "watch" Minko's github repository to get your daily dose of new features and fixes!