Jean-Marc Le Roux Web, RIAs and chocolate spaghettis…

Render To Texture With Minko 2

Update: If you want to give it a try, you can start working with Minko 2 beta today!

During last week "Starting With Shaders" workshop, nicoptere asked me about Render To Texture (RTT). RTT is a very common technique used for multi-pass rendering. The principle is quite simple: the pre-passes render the scene in one or more textures. This texture can then be sampled to provide per-pixel data during the rendering pass.

A very common use for RTT is shadow mapping. Shadow mapping is a technique to create projected shadows. The shadow mapping rendering pass will require access to the depth of each pixel as seen from the light source which is "emitting" the shadow. Therefore, each mesh will have more than just a rendering pass. Indeed, we will have to add a pre-pass for each light in the scene to generate it's "depth map".

RTT can also be used to create portals! So it's a very cool feature.

RTT might look quite simple. It reads like it's just like rendering in a texture. But it's not that simple: the goal of rendering in a texture is most likely to use that texture afterward during another pass. Therefore, some rendering passes will depend on other pre-passes. So you have to be able to sort all the passes. It's yet another symptom of the scene tree VS draw calls list paradox: the 3D scene is a tree but the rendering is performed as a list of passes/draw calls. As a developer, you want to affect the rendering effects to the meshes. It's a scene node/geometry centric paradigm. But the rendering API expects a list of passes and each pass will perform a draw call for each mesh it affects.

With Minko 2, it's very easy to perform RTT or multi-pass rendering. Each Mesh as an Effect object. And this very Effect object is a collection of passes. Each pass is a shader, and each shader can render in a texture or directly in the back-buffer. The RenderingController does all the job and creates, organizes and performs all the mesh-to-passes operations. Here is a simple demo:

// setup lighting
scene.bindings.setProperties({
	lightDirection		: new Vector4(.1, .1, 1.),
	lightDiffuseColor	: 0xffffff,
	lightDiffuse		: 1.,
	lightEnabled		: true
});
 
// create render targets
var normalsTexture : RenderTarget = new RenderTarget(
	128, 128,
	new TextureResource(128, 128),
	0,
	0x000000ff
);
var positionsTexture : RenderTarget = new RenderTarget(
	128, 128,
	new TextureResource(128, 128),
	0,
	0x000000ff
);
 
// create the teapot
var teapot : Mesh = new Mesh(
	new TeapotGeometry(),
	{
		diffuseColor	: 0xffffff00,
		lightEnabled	: true
	},
	new Effect(
		// RTT passes
		new VertexNormalShader(normalsTexture),
		new VertexPositionShader(positionsTexture),
		// rendering pass
		new BasicShader()
	)
);
 
teapot.transform
	.appendUniformScale(.5)
	.appendTranslation(0, -0.6);
 
scene.addChild(teapot);
 
// create the sprites to see the result of the RTT
scene.addChild(new Sprite(
	10, 10,
	128, 128,
	{ diffuseMap : normalsTexture.resource }
));
 
scene.addChild(new Sprite(
	10, 148,
	128, 128,
	{ diffuseMap : positionsTexture.resource }
));

The shaders I use for the RTT are debug shaders provided with Minko 2. Here is the code for the VertexNormalShader for example:

public final class VertexNormalShader extends BasicShader
{
	public function VertexNormalShader(target	: RenderTarget	= null,
					  priority	: Number	= 0.)
	{
		super(target, priority);
	}
 
	override protected function getPixelColor() : SFloat
	{
		var normal : SFloat = interpolate(
			vertexAnimation.getAnimatedVertexNormal()
		);
 
		normal = normalize(normal);
		normal = divide(add(normal, 1), 2);
 
		return float4(normal.xyz, 1);
	}
}

A very interesting thing here is that this shader extends the BasicShader. BasicShader handles all the "usual stuff" and its vertex shader does all the animations. So being able to extend it only to override the behavior of the pixel shader is a very cool feature.

RTT combined with ActionScript shaders is incredible. You have to remember that every AS shader you write can fork into many actual shaders on the GPU to handle all the rendering situations. It's yet another problem: it means a "pass" is not just a "shader", but multiple shaders forked from the original AS shader you wrote. For example, a normal pre-pass (rendering the normals in a texture) will have to work for both animated and static objects. So it's actually two passes: the normal pre-pass for the animated objects and the normal pre-pass for the static ones.

Thanks to the AS shaders and the rendering API, you write your shader once and Minko will figure it out all by itself: when to fork your shaders, how many passes are actually needed, what objects should be rendered during each pass, what pass has to be rendered first, etc...

If you have questions or comments, you can post here or on Aerys Answers.