ActionScript 3.0 | Jean-Marc Le Roux

11Apr/121

Minko 2 New Feature: XPath Selectors

Manipulating a scene graph is sometime quite a challenge. If you take the classic Flash display list for example: fetching tree nodes and modifying them as batches is not possible. You have to write a lot of code to simply fetch all the nodes you need reqgarding a set of creteria. Then you have to loop on this set to call the proper functions and set the right properties.

Working with trees as a representation model is something XML is very good at. XPath is a part of the XML specification and it defines a specific syntax to select a subset of XML nodes in an XML document. I think the syntax is quite obvious and very intuitive as it inherits most of its design from the UNIX/URI paths.

That's why we chose to add a simple XPath implementation in Minko 2. You can now write this kind of things:

var visibleMeshes : SceneIterator = group.get("/group/mesh[visible=true]");

The Group.get() method takes the XPath you wan to match and returns a SceneIterator object. This object acts as a wrapper for the fetched collection of nodes that matched your XPath request:

var visibleMeshes : SceneIterator = group.get("/group/mesh[visible=true]");
var numMeshes : uint = visibleMeshes.length;
 
for (var i : uint = 0; i < numMeshes; ++i)
  visibleMeshes[0].visible = false;

The "[]" operator can be used to get the nth item of the fetched collection. To override this operator, we simply extend the Proxy class and override the flash_proxy getProperty() method. Extending the Proxy makes it possible to add two new features:
- iterator: our SceneIterator object can be used in "for each" loops if we override the proper Proxy methods
- property setters/getters overriding: set/get properties on all the selected nodes at once

// fetch all the visible meshes and hide them
group.get("//mesh[visible=true]").visible = false;
 
// fetch all the white meshes and make them white
group.get("//mesh[@diffuseColor=0xffffffff]").bindings.setProperty(
  "diffuseColor",
  0xff0000ff
);
 
// fetch all the visible white meshes and remove them from their parent
group.get("/group/group/mesh[@diffuseColor=0xffffffff][visible=true]").parent = null;

Tagged as: 3D, ActionScript 3.0, Minko, Minko 2 1 Comment

5Mar/127

Minko on the iPad/iPhone

Tom Krcha - Adobe gaming evangelist - was kind enough to recompile my previous benchmark to see how it performs on the iPad:

Half a million polygons is not too bad! It's actually far enough to build amazing 3D games. If you want to see a live application, here you have the 3D data visualization globe demo compiled for the iPad/iPhone:

The application loads 35 000 geo-localized hot spots and uses Minko's streams API to create the corresponding 3D geometry dynamically. You can find the code on Aerys' GitHub. A beta for Minko 2 should be available this week on GitHub. Stay tuned!

Tagged as: ActionScript 3.0, AIR 3.2, data visualization, Flash 11, Flash-to-iPhone packager, Minko, Minko 2, Stage3D 7 Comments

2Feb/1217

AIR 3.2, Stage3D and Minko: 100k polygons at 26fps!

AIR 3.2 beta 5 is available on Adobe Labs, but one of the most ancitipated new features is the addition of Stage3D for Android and iOS devices (iPhone and iPad). Thanks to AIR 3.2 and Stage3D, Flash developers will now be able to create 3D applications targeting mobile devices. According to Adobe's engineer, GPU-wise performances are very close to what you would have with a native apps. And that's really incredible (I guess we now just have to wait for the VM to catch up but thing are getting better and better...).

As a member of the private prerelease program, I have access to AIR 3.2 for Android/iOS with Stage3D enabled. So I decided to give it a try with the latest prerelease drop. And boy was I impressed! My test device is an HTC Desire HD with Android 2.2 (official HTC rom). I build a very simple mobile application with Flash Builder 4.5 and Minko (thank you Thibault for letting me post that photo!):

The framerate, memory usage and other values are tracked with monitor. And that's right, it's a bit more than 100 000 polygons at 26 frames per second! Here is the code:

public class Teapot extends Sprite
{
  private var _viewport		: Viewport	= new Viewport();
  private var _scene		: StyleGroup	= null;
 
  private var _shader		: IShader	= new CelShadingShader();
  private var _lightMatrix	: Matrix4x4	= new Matrix4x4();
 
  public function Teapot()
  {
    super();
 
    // simple scene: camera and a teapot with normals
    _scene = new Group(
      new Camera(new Vector4(0, 0, -12), new Vector4(0, 0, 0)),
      new NormalMeshModifier(new TeapotMesh(40))
    );
 
    // use the cel shading effect by default
    _viewport.defaultEffect = new SinglePassRenderingEffect(_shader);
    // grey background
    _viewport.backgroundColor = 0x666666;
 
    // setup the stage
    stage.frameRate = 30.;
    stage.addChild(_viewport);
    stage.addChild(Monitor.monitor.watch(_viewport, ["numTriangles"]));
    stage.addEventListener(Event.ENTER_FRAME, enterFrameHandler);
  }
 
  private function enterFrameHandler(event : Event) : void
  {
    // rotate the light
    _lightMatrix.appendRotation(0.01, ConstVector4.Y_AXIS);
    _shader.setNamedParameter(
      "light direction",
      _lightMatrix.transformVector(ConstVector4.Z_AXIS)
    );
 
    // render
    _viewport.render(_scene);
}

You can find the full source code on my GitHub.

That's only a few lines of code, and yet it displays a Utah teapot with cel shading and a rotating light. An important thing to remember is that this cel shading effect is coded using ActionScript shaders. AS shaders is a feature of Minko: you program the GPU with ActionScript code and Minko's JIT compiler turns it into AGAL bytecode at runtime. And the ActionScript to AGAL shader compiler runs just fine on mobile devices. Even better: Minko 2 will feature a new and even more optimized compiler so it will be very easy and very efficient to program mobile GPUs with ActionScript code!

Considering my phone is not supposed to perform as well as the Samsung Galaxy SII or the iPad 2, I can just imagine what you can do on those! But I'll have an iPad 2 tomorrow and I'll try to see how BlackSun performs on it, so just be patient

AIR 3.2 will be available "very soon" and so will be Minko 2. In the meantime, you can start playing with Minko 1 and get ready to rock on mobile devices! As usual, if you have questions/suggestions, please post them in the comments or on Aerys Answers.

Tagged as: 3D, ActionScript 3.0, AIR, AIR 3.2, Flash 11, Minko, Stage3D 17 Comments

23Jan/124

Single Pass Cel Shading

Cel shading - aka "toon shading" - is an effect used to make 3D rendering look like cartoon. It was used in many flavours in a few games such as XIII or Zelda Wind Wakers.

TL;DR

Click on the picture above to launch the demonstration.

The final rendering owns a lot to the orginal diffuse textures used for rendering. But this cartoon style is also achieved using two distinct features on the GPU:

The light is discretized using a level function.
A black outline on the edges.

This is usually done using two passes (or even more). One pass renders the scene in the backbuffer with the lighting modified by the level function. Another pass renders the normals only and then pass is done as a post-process to perform an edge detection algorithm (a Sobel filter) to add the black outline.

Another technique uses two passes: one to render the object with the lighting, and a second one with front-face culling and a scale offset to render the outline with a solid black color.

But I thought it might be done more efficiently in one pass using a few tricks. The most difficult part here is how to get the black outline. Indeed, we are working on a per-vertex (vertex shader) or per-pixel (fragment shader) basis. Thus, it's pretty hard to work on edges. It's pretty much the same problem we encountered when working on wireframe, except here we want to outline the object and not the triangles. But this little difference actually makes it a lot easier for us.

Why? Because detecting the edges of the 3D shape is much easier.

The Outline

The trick is to get the angle between the eye-to-vertex vector and the vertex normal. If that very angle is close to PI/2 (=90°), then the vertex is on an edge. If the vertex is on an edge, then we will displace it a bit toward its normal. The vertices deplaced this way will form an outline around around shape:

public class CelShadingShader
{
  private var _isEdge	: SValue	= null;
 
  override protected function getOutputPosition() : SValue
  {
	var eyeToVertex : SValue = normalize(subtract(vertexPosition, cameraLocalPosition));
 
	_isEdge = lessThan(
		-0.05,
		dotProduct3(vertexNormal, eyeToVertex)
	);
 
	var thickness : SValue	= getNamedParameter("thickness", .04);
	var delta : SValue = multiply(_isEdge, vertexNormal.xyz, thickness);
 
	return multiply4x4(
		add(vertexPosition, float4(delta, 0)),
		localToScreenMatrix
	);
  }
 
  override protected function getOutputColor() : SValue
  {
	var outline : SValue = lessThan(interpolate(_isEdge), 0.1);
 
	return float4(float3(outline), 1);
  }
}

Our fragment shader is pretty simple here: _isEdge is supposed to contain 1 if the vertex is on an edge, 0 otherwise. Therefore, as we want our outline to be black, we simply use the "lessThan" operation. If the vertex is on an edge, the outline value will be 0 and 1 otherwise. We just have to multiply "outline" with whatever color we want to use.

You'll get the following result:

The Light Level Function

There are many ways to transform a continuous per-pixel lighting equation into another one that will use levels. The best way to make it possible for the artists to customize it is to use a light texture. The Lambert factor is then used as the UVs to sample that texture.

But here I wanted this effect to rely on no textures (except a diffuse one eventually). So I implemented a very simple level function using an euclidian division:

private static const NUM_LEVELS			: uint		= 6;
 
private static const DEFAULT_THICKNESS		: Number	= 0.04;
private static const DEFAULT_AMBIENT		: Number	= .2;
private static const DEFAULT_LIGHT_DIRECTION	: ConstVector4	= new ConstVector4(1., -1., 0.);
private static const DEFAULT_DIFFUSE_COLOR	: ConstVector4	= new ConstVector4(1., 1., 1., 1.);
 
override protected function getOutputColor() : SValue
{
  var vertexNormal : SValue = normalize(interpolate(vertexNormal));
  var lightDirection : SValue = getNamedParameter("light direction", DEFAULT_LIGHT_DIRECTION);
 
  lightDirection.normalize();
 
  // diffuse lighting
  var lambertFactor : SValue = saturate(
    negate(dotProduct3(lightDirection, vertexNormal))
  );
 
  // cel shading
  lambertFactor = multiply(lambertFactor, NUM_LEVELS);
  lambertFactor = subtract(lambertFactor, fractional(lambertFactor));
  lambertFactor = divide(lambertFactor, NUM_LEVELS);
 
  // ambient lighting
  var ambient	: SValue = getNamedParameter("ambient", DEFAULT_AMBIENT);
 
  lambertFactor.incrementBy(ambient);
 
  var diffuseColor : SValue = getNamedParameter("diffuse color", DEFAULT_DIFFUSE_COLOR);
 
  return float4(
    multiply(diffuseColor.rgb, lambertFactor),
    diffuseColor.a
  );
}

Here is what you get:

The Final Shader

You can combine both effects by simply multiply the Lambert factor by the outline value we used to output.

And voilà!

public class CelShadingShader
{
  private static const NUM_LEVELS		: uint		= 6;
 
  private static const DEFAULT_THICKNESS	: Number	= 0.04;
  private static const DEFAULT_AMBIENT		: Number	= .2;
  private static const DEFAULT_LIGHT_DIRECTION	: ConstVector4	= new ConstVector4(1., -1., 0.);
  private static const DEFAULT_DIFFUSE_COLOR	: ConstVector4	= new ConstVector4(1., 1., 1., 1.);
 
  private var _isEdge	: SValue	= null;
 
  override protected function getOutputPosition() : SValue
  {
	var eyeToVertex : SValue = normalize(subtract(vertexPosition, cameraLocalPosition));
 
	_isEdge = lessThan(
		-0.05,
		dotProduct3(vertexNormal, eyeToVertex)
	);
 
	var thickness : SValue	= getNamedParameter("thickness", .04);
	var delta : SValue = multiply(_isEdge, vertexNormal.xyz, thickness);
 
	return multiply4x4(
		add(vertexPosition, float4(delta, 0)),
		localToScreenMatrix
	);
  }
 
  override protected function getOutputColor() : SValue
  {
    var vertexNormal : SValue = normalize(interpolate(vertexNormal));
    var lightDirection : SValue = getNamedParameter(
      "light direction",
      DEFAULT_LIGHT_DIRECTION
    );
 
    lightDirection.normalize();
 
    // diffuse lighting
    var lambertFactor : SValue = saturate(
      negate(dotProduct3(lightDirection, vertexNormal))
    );
 
    // cel shading
    lambertFactor = multiply(lambertFactor, NUM_LEVELS);
    lambertFactor = subtract(lambertFactor, fractional(lambertFactor));
    lambertFactor = divide(lambertFactor, NUM_LEVELS);
 
    // ambient lighting
    var ambient	: SValue = getNamedParameter("ambient", DEFAULT_AMBIENT);
 
    lambertFactor.incrementBy(ambient);
 
    var diffuseColor : SValue = getNamedParameter(
      "diffuse color",
      DEFAULT_DIFFUSE_COLOR
    );
 
    // outline
    var outline : SValue = lessThan(interpolate(_isEdge), 0.1);
 
    lambertFactor.scaleBy(outline);
 
    return float4(
      multiply(diffuseColor.rgb, lambertFactor),
      diffuseColor.a
    );
  }
}

Tagged as: 3D, ActionScript 3.0, Flash 11, hardware acceleration, Minko, shader, Stage3D 4 Comments

7Dec/110

NAO: Orbit VS XMPP

A few weeks ago I had the opportunity to work on a robotics project involving Nao, the 60cm humanoïd robot created by Aldebaran Robotics. After this first successful experience with the robot, we decided to take it a step further: we installed an Orbit application right on the robot. As we pretend Orbit can work on any device, a robot might actually be the ultimate test!

Nao uses XML-RPC embed in the XMPP protocol in order to provide network APIs. Those APIs are the very one mapped in the nao-as3-api project than can be used to program the robot using ActionScript 3.0. This is the protocol we've used to build a remote controller application for Dell. It was built on top of AIR 3.1, Stage3D and Minko. You can see it in action in the following video demonstration the application on a Dell XT3 tablet PC:

But XML-RPC/XMPP is a very slow and expensive option: the XML parsing alone is quite the CPU hog, and it sure is even harder on the bandwidth. We thought Orbit might be a good replacement so we did a little test application. It was built using Minko and the nao-as3-api. You can see it in action in the following video:

As you can see in the video, XML-RPC/XMPP (right) doesn't perform quite as well as Orbit (left). The synchronization is stuttering and far from smooth when using XML-RPC/XMPP, but it is just perfect when using Orbit. Event better: the Orbit version takes 30 times less CPU, and 100 times less bandwidth!

This gives me real hopes for intuitive, real-time and smooth multi-devices experiences working across tablets, phones, servers... and now robots! If you want to know more about what you can do with Orbit, you can take a look at the 2010/2011 Orbit showreel.

Tagged as: ActionScript 3.0, Minko, Orbit No Comments

29Oct/111

Minko “Shader Lab” Alpha Preview 1

Working on shaders might be one the most complicated - and yet most essential - tasks when it comes to the new Stage3D API features in Adobe Flash 11. This is why Minko features ActionScript 3.0 shaders. This feature makes it possible to do GPU programming using nothing more than good old ActionScript. You can already try this feature and start working with Minko today.

But today I want to present an early alpha preview of one of the tools we will release with the next version of Minko: the Shader Lab. The idea is simple: provide a visual environment to create, test and debug shaders without doing any programming. The tool will feature a lot of higher level operations to make it possible for artists to work on complex shaders and materials.

You can see the Shader Lab in action in the following video:

In the last two examples, we show how sound can be sampled and streamed to the GPU to tune the geometry without any actual programming skills.

Tagged as: 3D, ActionScript 3.0, Flash 11, Flex, Minko, RIA, Stage3D 1 Comment

30Sep/113

NAO AS3 API released as open-source

Long time no see! As you can imagine I'm quite busy with Minko and Flash 11 stuff right now. Still, I would like to introduce some related work. Do you know Nao?

Nao is a quite impressive humanoïd robot. It has a lot of features, from text-to-speech to face recognition. And of course, it can walk and look around... seeing it in action is actually completely amazing! As I introduced Aerys' work to Aldebaran, they asked me to work on an ActionScript 3.0 wrapper for Nao's remote APIs as a part of a bigger project I will speak about soon.
Technical details and code samples after the jump...

Tagged as: 3D, ActionScript 3.0, Aerys, AIR 3.0, Flash, Flash 11, Minko, NAO, NAO AS3 API, RIA, robotics, Stage3D 3 Comments

6May/114

About Minko’s plugin system

Building a flexible and extensible 3D framework has always been our priority. This goal has a huge influence on the way we work and what we provide as a final result. This is also a main difference with other 3D engine/frameworks that tends to provide everything in a single monolithic library.

One of the side effects of such politics is Minko's plugin system. Here is how it works:

Every core APIs are in the actual Minko project
Any additional feature will be implemented as a "plugin"

About the core APIs

Those APIs are the foundations of the framework. It will be released as an open-source project. They are very loose coupled so that even Minko's core can be splitted into distinct entities. There are two main APIs:

The Scene Graph API: it provides interfaces and base classes to build a scene using a data structure called a "graph". In the end, building a Minko 3D scene is just as simple as building a 2D scene with the good old Flash display list API.
The Rendering API: it provides everything you need to perform rendering operations using the Molehill API and shaders.

We define a simple mechanism to pass data from the Scene Graph API to the Rendering API. We also provide basic scene and rendering algorithm to make it possible to start building a simple 3D application from scratch.

About the additional features

So what are additional features? They are features that will rely on the core APIs but that are not essentials or required in every project. Those features are (the following list is not exhaustive):

Dynamic lights
Dynamic shadows
Animations
Skinning
Dynamic reflections
File formats parsers (3DS, Collada ...)
Physics
Particles
Mouse interactivity

Every new feature will be implemented as a "plugin" in a separate project. If you don't want it, don't use it! If you want to provide a different/better way to do it, feel free to do so! Does Minko lack a feature you need? You can create your own plugin and share it as you like.

Those plugins also make it possible to create bindings between Minko and thrid party projects such as jiglibflash or Stardust.

Why ?

The main goal is to ensure the core project exposes the right APIs. This way, it will be extensible enough and anyone should be able to work and build against it to provide more and more plugins... We also want people to be able to pick what they need and build a community around the project they like the most. And clearly, some plugins will be a lot more technical than others.

Finally, every plugin can be distributed under a different license...

Tagged as: 3D, ActionScript 3.0, Minko, Molehill 4 Comments

23Mar/112

New Minko demonstration: Citroen DS3

Update: The actual application is available here.

The above video demonstrates what Minko is capable of:

This application simple car customizer was created using Minko, a new 3D engine targeting the Adobe Flash Platform. The exhibited car is a Citroën DS3 with more than 150 000 polygons. It is rendered with dynamic lighting and reflection effects (use arrow keys to move the light, see the "Controls" section for more details). The car is made of close to 400 different objects (wheels, buttons, lights, ...).

This application uses:

a 3DS file parser
a high polygon car model, with close to 400 different objects
dynamic Phong lighting with specular effect
reflection effect with dynamic spherical environment mapping
lighting/reflection blending using multipass
multiple cameras in a single scene (inside/outside the car)
up to 300 000 polygons rendered per frame

The actual application will be released tomorrow during Adobe Lightning Talks.

Tagged as: 3D, 3DS, ActionScript 3.0, Adobe, Aerys, demo, hardware acceleration, Minko 2 Comments

21Feb/110

Quake 3 HD with Flash “Molehill” and Minko

This video demonstrates what can be achieved using Minko, the next-gen 3D engine for Flash "Molehill" I'm working on. The environment is a Quake 3 map loaded at runtime and displayed using high definition textures. The demo shows some technical features such as:

Quake 3 BSP file format parser
Potentially Visible Set (PVS)
complex physics for spheres, boxes and convex volumes using BSP trees
light mapping using fragment shaders
materials substitution at runtime
dynamic materials using SWF and the Flash workflow
first person camera system

Quake 3 BSP file format

Quake 3 uses the version 46 of the BSP file format. It's farily easy to find unofficial specifications for the file format itself. Yet, parsing a file and rendering the loaded geometry are two very different things. With Minko, it is now possible to do both without any knowledge of the specific details about the BSP format and algorithms. Still, those are interesting algorithms so I wanted to explain how it works.

Binary Space Partioning (BSP) is a very effective algorithm to subdivise space in two according to a "partition". To store the subdivided spaces, we use a data structure called a BSP tree. A BSP tree is just a binary tree plus the partition that separates both children. This picture shows the BSP algorithm in 2 dimensions:

In that case, each node of the BSP tree will store the partition line. When working with 3D the partition will be a plane. We can then walk through the tree by simply comparing the location of a particular point to the partition. A good use case is 3D z-sorting: we simply have to compare the location of the camera to the planes stored in the BSP nodes to know which child should be walked through (or renderered) first. But in Quake 3, the BSP tree is used for:

physics: the tree makes it possible to optimize collisions detection by performing only the required computations and tests
rendering optimization: by locating the node where the camera is, we can know what is the Potentially Visible Set (PVS) of polygons that should be renderered and ignore the rest

In each "*.bsp" file are stored the BSP tree itself, the partition planes, the geometry, the PVS data and even bounding boxes/spheres for each node to perform frustum culling. This data is very precious and is compiled directly in the file by tools like GTKRadiant.

Credits

HD textures were provided by the ioquake3 project.

Tagged as: 3D, ActionScript 3.0, demo, experiment, hardware acceleration, Minko, Molehill, physics, Quake 3 No Comments

Older Entries »

Aerys

Twitter: promethe42

@makc3d @inspirit @joa @pleclech we created AS3 shaders, much more powerful than GLSL/FLSL, why don't you use it? 11:37:54 PM May 14, 2012 from TweetDeck in reply to makc3d Reply Retweet Favorite
@makc3d yesterday I implemetend a radial blur in 10 minutes. Too bad you still refuse to work with tools for grown ups :) 11:34:00 PM May 14, 2012 from TweetDeck in reply to makc3d Reply Retweet Favorite
just implemented BF3-like screen space lens flare with #minko #as3 #shader http://t.co/kXUZtC0a 11:22:20 PM May 14, 2012 from TweetDeck Reply Retweet Favorite
Someone implemented AS3Mod for Away3D 4 on the CPU... SERIOUSLY?! You REALLY don't want to learn anything new Flashers?! man shaders! 11:03:19 PM May 11, 2012 from TweetDeck Reply Retweet Favorite
RT @DisneyPixar: One thing you’ll never regret is giving your best effort. 02:07:10 PM May 06, 2012 from Twitter for Android Reply Retweet Favorite

@promethe42

Jean-Marc Le Roux Web, RIAs and chocolate spaghettis…