Stepping through the process of creating a realtime median filter shader for Stage3d that pushes affectionately against the platform’s limits.

Goal

Before setting out to make any shader, it’s a good idea to have an idea of what you’re aiming at. In this case we’re in luck as a median filter is fairly well defined. It’s a convolution filter (so each pixel has its state determined by several neighbouring pixels) that calculates the median value (place all the values in order, and take the middle one) of the surrounding pixels. In our case we’ll be making a median filter with a 3×3 area as we expect our noise to be mostly single pixels that have been altered. Also in our case we’ll be filtering on a per-channel basis, so finding the median red value, median green value, and median blue value rather than finding the colour value with the median brightness – or whatever other measurement you might like to use.

Median filters are useful in image processing for removal of small scale noise while maintaining the edges of the original image. They’re used very often in digital cameras to remove the dusting of noise that appears when shooting in low light, using a low cost sensor, or just when hit by interference.

A composite of an image afflicted by noise before and after being put through a median filter. Click for full size.

 A First Draft

Let’s get straight in to designing our shader. I’ll mostly be writing code in Haxe’s HxSL, but for the very start let’s have some pseudo-code for a simple median filter.

declare an array of float3, length 9, named v

for x in -1 to +1 { 
  for y in -1 to +1 {
    v[x + y*3] = sample texture at (current location + (x,y))
  }
}

sort each channel of v by value
return v[4]

Two implementation issues appear when it comes to writing that as an actual shader.

Firstly how do we sort values in a shader? Shaders have no support for looping or conditional execution, so there can be no straight forward “if a is larger than b then …” or “repeat until in order”. It actually ends up being quite easy to do without such constructs; it just takes a little change of thinking.

Secondly a lack of register space. Stage3d’s shaders only have write-access to 8 temporary registers, which is notable for being one less than the 9 required to store all the values from the 3×3 area sample area.

Sorting in a Shader

As every young computer scientist knows, bubble sort is the poor relation of the sorting family. But let’s use it in our shader anyway.

The advantage for us is that bubble sort is delightfully simple. Just advance through the array, compare each value to its neighbour and swap them if they’re in the wrong order. Do that enough times and you end up with the array in order. The question then is how do we perform the swapping of unordered neighbours?

Instead of thinking of things swapping positions, think of how you want the data to be after the operation.

// after sorting, v0 should have the smaller value, v1 the larger
v0 = min(v0, v1);
v1 = max(v0, v1); // problem!
// because we just wrote to v0 it might have changed value,
// so the second line doesn't do what we expect.
// let's keep a copy of v0 safe in a temporary variable

temp = v0;
v0 = min(temp, v1);
v1 = max(temp, v1);

That has placed two values in the correct order. Let’s now extend that to sort three values.

var v0 = 6;
var v1 = 3;
var v2 = 1;
var temp;

temp = v0;
v0 = min(temp, v1);
v1 = max(temp, v1);

temp = v1;
v1 = min(temp, v2);
v2 = max(temp, v2);

// 1st iteration of bubble sort is complete
// we now know that v2 holds the maximum value
// but v0 and v1 might still be in the wrong order

temp = v0;
v0 = min(temp, v1);
v1 = max(temp, v1);

// 2nd iteration of bubble sort is complete
// we now know that v0, v1 and v2 hold values
// in ascending order

It’s old news to our young computer scientist, but each iteration of bubble sort guarantees that the final value is correctly placed. That means we don’t need to touch it on future iterations (so future iterations are shorter), and it means that for an array of n elements every element is certain to be in its correct place after n-1 iterations. As we’re doing all this to find the median value, we actually don’t have to do the full n-1 iterations as we only need to be sure that the middle value is correctly placed in order to read off the median.

Taking into Account the Three Channels

At this point it’s important to remember that our shader is dealing with Float3 variables (that is, each variable has an x, y, and z component representing the three colour channels) rather than the Float variables we’ve used in the examples so far. AGAL’s min and max operations work on each channel separately, so we are performing our sort on a per channel basis. This has the result that we might end up with our “median colour” being a colour that isn’t actually present in the source image – it may be built up from the red of one colour, the green of another and blue of a third. In my brief experiments it seems that this per-channel sorting actually produces better results, but it’s worth showing how to adapt the sort so that it maintains colours.

If we’re not going to sort each channel separately on its value, we’ll need some other metric by which to sort. For simplicity we’ll sort on the sum of the three channels, but you could calculate the colour’s hue, saturation, value, or whatever you like that can be expressed as a single number. Stage3d’s registers all consist of four components, and our colour values are only using three, so let’s use the fourth to store the value we’ll sort them by.

// put value to sort by in .w component
v0.w = v0.x + v0.y + v0.z;
v1.w = v1.x + v1.y + v1.z;
v2.w = v2.x + v2.y + v2.z;

Rather than using min and max to perform the sorting, we’ll use lt (less than) and gte (greater than or equal). These operations return 1 if they are true, and 0 otherwise. Note that like most operations these are performed on all components of a variable, and as we want just a single value we give them just a single component – the one by which we want our values sorted.

firstIsSmaller = lt(v0.w, v1.w);
secondIsSmaller = gte(v0.w, v1.w);

As we know that these are always either 0 or 1 and that when one is 1 the other is 0, we can construct the desired new value of v0 and v1.

temp = v0;
v0 = firstIsSmaller * v0 + secondIsSmaller * v1;
v1 = secondIsSmaller * v0 + firstIsSmaller * v1;

Using this technique you can do all kinds of conditional-based stuff in shaders, despite the lack of an if statement.

Squeezing into a Size Eight

Being able to sort a set of variables isn’t much use if we can’t actually store them in the first place, and with just 8 temporary registers available to us things are looking pretty tight when we’re sampling 9 locations, and need a temporary variable to perform the sort.

As I wrote this section I realised it would be possible to fit all 9 Float3s and the one temporary Float3 used during sorting in the 8 available Float4 registers. It would require several of the variables to be split between multiple registers, which would make your code joyfully unreadable but it would work. But if you’re using Float4s for each sampled location then that of course could not fit in 8 registers.

Thankfully we can (I think!) arrive at the median by analysing subsets of the data. We need to find the median of 9 values, but are unable to store all 9 in memory at any one time. Instead we shall store the first 3 values, find and store their median. We do the same for the next subset of 3 values, and then the final 3. By finding the median of those three medians we should arrive at the median for the whole set.

I’m fairly sure that this is mathematically true so long as the subsets are all of equal size, all have an odd number of members, and there’s an odd number of subsets. It does seem to give the expected results, and should give something close to the true median if those requirements are not met.

var median0:Float3;
var median1:Float3;
var median2:Float3;

var textureUV:Float2;

var v0:Float3;
var v1:Float3;
var v2:Float3;
var temp:Float3;

// sample locations (-1,-1) (0,-1) (+1,-1) into v0 v1 v2
// bubble sort on v0 v1 v2
median0 = v1;

// sample locations (-1,0) (0,0) (+1,0) into v0 v1 v2
// bubble sort on v0 v1 v2
median1 = v1;

// sample locations (-1,+1) (0,+1), (+1,+1) into v0 v1 v2
// bubble sort on v0 v1 v2
median2 = v1;

// bubble sort on median0 median1 median2
return median1;

Even with that we’re flying pretty close to the register limit. Further savings can be made by reusing registers for multiple purposes. For instance median2 is only used after v0, v1 and v2 have stopped being used, so it can share a register with one of those. Similarly the variable used to locate texture sampling is never in use at the same time as the temp variable needed for bubble sorting, so they can share a register too.

Implementation Details

I had planned to write a section on manually converting this code into AGAL, but I think this article is long enough already. It’s a fairly robotic process anyway, just make sure you keep good notes, and keep reminding yourself to switch to Haxe as soon as you can.

Speaking of Haxe and HxSL, something I learned writing this shader is that …

median0 = v1;

… is not the same as …

median0 = v1 + [0,0,0];

If you use the first then median0 seems to change when you alter v1. Presumably this is part of an effort to save register space (after all why keep two registers that have identical contents?) but the behaviour is a little unexpected.

Results

My motivation for writing a median filter shader was to produce some simple realtime painterly effects. It turns out that my 3×3 sample area is not enough to produce anything particularly noticeable. Which shouldn’t be surprising as the strength of the median filter in image processing is that it preserves the image while removing noise.

However after some simple modifications I’ve made two shaders that sample at 1×9 and  9×1. By applying both in turn it effectively finds the median from a 9×9 cross shaped sample area. This new shader is of limited use in noise removal, but it does produce some quite pleasing results, all running very comfortably at 60fps.

Photo after passing through a larger scale cross-sample median filter. Click to enlarge.

Comparison showing the effect of the enlarged median filter on the flat-shaded landscape of a game. Click to enlarge.

 The full fragment shader for the 3×3 median filter, in HxSL:

function fragment(texture:Texture, sizeOfTexel:Float2)
{
 // due to limited availability of temporary variables
 // we cannot hold samples of all pixels in a 3x3 area.
 // instead we'll sample the first row, find its median,
 // sample the second row, find its median,
 // sample the third row, find its median,
 // then find the median of those three medians

 var median0:Float3;
 var median1:Float3;

 // fill v[n] with sampled values
 var localUV:Float2;
 localUV.y = tuv.y - sizeOfTexel.y;
 localUV.x = tuv.x - sizeOfTexel.x;
 var v0:Float4 = get(texture, localUV, clamp, nearest);
 localUV.x = tuv.x + 0;
 var v1:Float4 = get(texture, localUV, clamp, nearest);
 localUV.x = tuv.x + sizeOfTexel.x;
 var v2:Float4 = get(texture, localUV, clamp, nearest);

 var t:Float3;

 t = v0.xyz + [0, 0, 0];
 v0.xyz = min(t, v1.xyz);
 v1.xyz = max(t, v1.xyz);

 t = v1.xyz + [0, 0, 0];
 v1.xyz = min(t, v2.xyz);
 v2.xyz = max(t, v2.xyz);

 // after first iteration of bubble sort v2 is certain to be the maximum value
 // but median could be either v0 or v1
 t = v0.xyz + [0, 0, 0];
 v0.xyz = min(t, v1.xyz);
 v1.xyz = max(t, v1.xyz);

 // after a second iteration of bubble sort
 // v1 is also certain to have correct value
 // so v1 contains the median for this set of three values.
 // incidentally, v0 also has the correct minimum value.
 median0 = v1.xyz + [0, 0, 0];

 // -- now do all of that again to find median of second row --
 localUV.x = tuv.x - sizeOfTexel.x;
 v0 = get(texture, localUV, clamp, nearest);
 localUV.x = tuv.x + 0;
 v1 = get(texture, localUV, clamp, nearest);
 localUV.x = tuv.x + sizeOfTexel.x;
 v2 = get(texture, localUV, clamp, nearest);

 t = v0.xyz + [0, 0, 0];
 v0.xyz = min(t, v1.xyz);
 v1.xyz = max(t, v1.xyz);

 t = v1.xyz + [0, 0, 0];
 v1.xyz = min(t, v2.xyz);
 v2.xyz = max(t, v2.xyz);

 t = v0.xyz + [0, 0, 0];
 v0.xyz = min(t, v1.xyz);
 v1.xyz = max(t, v1.xyz);

 median1 = v1.xyz + [0, 0, 0];

 // -- and again for third row --
 localUV.x = tuv.x - sizeOfTexel.x;
 v0 = get(texture, localUV, clamp, nearest);
 localUV.x = tuv.x + 0;
 v1 = get(texture, localUV, clamp, nearest);
 localUV.x = tuv.x + sizeOfTexel.x;
 v2 = get(texture, localUV, clamp, nearest);

 t = v0.xyz + [0, 0, 0];
 v0.xyz = min(t, v1.xyz);
 v1.xyz = max(t, v1.xyz);

 t = v1.xyz + [0, 0, 0];
 v1.xyz = min(t, v2.xyz);
 v2.xyz = max(t, v2.xyz);

 t = v0.xyz + [0, 0, 0];
 v0.xyz = min(t, v1.xyz);
 v1.xyz = max(t, v1.xyz);

 //median2 = v1.xyz + [0, 0, 0];
 // just directly use v1 instead of median2

 // we now have a median for each row
 // bubble sort on them to find the median of medians
 t = median0.xyz + [0, 0, 0];
 median0 = min(t, median1.xyz);
 median1 = max(t, median1.xyz);

 t = median1.xyz + [0, 0, 0];
 median1 = min(t, v1.xyz);
 v2.xyz = max(t, v1.xyz);

 t = median0.xyz + [0, 0, 0];
 median0 = min(t, median1);
 median1 = max(t, median1);

 out = [median1.x, median1.y, median1.z, 1];
}

A collection of the most useful Stage3D links I’ve come across.

This link will give you all the posts on my site about Stage3D.
Getting the inevitable self-advertising out the way.

How Stage3D works
Working with 3D cameras
What is AGAL
Vertex and Fragment Shaders
Hello Triangle
Perspective Projection
A series of quality and thorough tutorials on the fundamentals of working with Stage3D from Macro Scabia. I dropped my plans to write some very similar tutorials when I found these as they cover everything I had planned to.

Byte Array
The blog of Thibault Imbert, a product manager for the Adobe Flash Runtime. A good source of information on the most recent developments, which has mostly been Stage3D related of late. Browse through and you’ll quickly find important and practical details about the Stage3D technology.

Flashing in Public
Ben Whiting’s blog on Flash development, with a healthy emphasis on Stage3D. Really rather nice examples of the technology.

McFunkyPants
The blog of Chris Kaitila, who has written a [the?] book on Stage3D, which follows through a practical example of creating a 3D game from scratch.

Official Documentation
Don’t forget there actually is documentation for Stage3D! For some reason most the contents of the standard documentation for Stage3D features is currently blanked out, but this link will take you to the beta version which is intact.

Finally a note that back when Stage3D was younger it was known as Molehill, so you may see references to that in some resources. Very little has changed with regards how everything works since then so you should be safe following older tutorials. The only difference in the API I know of is that Molehill used slightly different syntax for setting up the size of a Context3d’s back buffer. Switching to the new version is no work at all, but it does mean older examples you find might simply fail to work.

A thorough tutorial on the use of registers for AGAL shaders. How to go about setting their values and accessing them in the shader.

This tutorial is weighted towards theory rather than the practical writing of code. You might find it beneficial to refer to my simple rendering example to see how this all gets applied.

Registers:

Each register consists of four components, which are floating point values. A particular component is accessed by the notation [registerName].[componentLetter], where componentLetter is x, y, z, or w. For example va0.x is the first component of the first vertex attribute register. For a listing of the available registers, please see my AGAL reference page.

Vertex Buffer:

The vertex buffer is how you pass per-vertex data into your vertex shader, where it will appear in the va (vertex attribute) registers. Usually this will include the position of that vertex in the world and some data used to colour it – either by passing in a colour for the vertex, uv coordinates for texture mapping, or something more complex. Keep in mind that the vertex buffer is just a series of floating point numbers which you can use for anything you like in your shader.

To deal with the vertex buffer you first create a VertexBuffer3D object by using the function context3D.createVertexBuffer (you cannot just instantiate it yourself with new VertexBuffer3D().)  This object will store the values until you’re ready to upload them into registers and have a shader use them.

Imagine the VertexBuffer3D object as a table. When you create it with context3D.createVertexBuffer you specify the number of vertices (the number of rows in the table) and the number of data32 per vertex (the number of columns in the table). In this illustration I’ve labelled each column with what I plan to use the values for, but remember they’re just a series of meaningless numbers as far as Flash is concerned.

You upload data into the VertexBuffer3D table using vertex3D.uploadFromVector, where you supply:

• data - A Vector.<Number> of the values you’re going to upload. Must be a whole number of rows’ worth of data.
• startVertex - The row in the vertex3D table at which to start writing.
• numVertices - The number of rows of data to copy.

In this illustration see how the startVertex determines where in the VertexBuffer3D object our Vector of numbers will be placed. If numVertices was set to 1 then only one row of the table would have been filled.

You set the va registers using context3D.setVertexBufferAt. When doing so you specify:

• index - Which va register you want to upload into, e.g. 0 for va0.
• buffer - The vertexBuffer3D object to use.
• bufferOffset - Which column in the vertexBuffer3D table to start from when reading in data
• format - How many columns of the table to read in, from FLOAT_1 for one to FLOAT_4 for four. Alternatively read in a single column but interpret it as four individual bytes instead of a single float.

As seen here, bufferOffset (0 in this case) and format (FLOAT_2) determine which columns of the VertexBuffer3D object to copy. Index (0 for the va0 register) determine which register those values should be placed in.

In this second example bufferOffset is 2, format is FLOAT_4, and index is 1. This results in copying the four columns shown from the VertexBuffer3D and using them to fill the va1 register.

Keep in mind that when performing the setVertexBufferAt operation you are setting the registers for as many vertices as you have rows in the vertexBuffer3D object you’re using.

Note! Rendering will fail if you have set values for registers that are then not used by your shader. Similarly it will fail if your shader tries to access a va register that hasn’t had data uploaded to it. Make sure you have context3D.enableErrorChecking set to true so you receive these error messages.

Vertex and Fragment Constants:

The other way to get numbers into a shader is through the vertex constant and fragment constant registers. These are accessed in the shader at vc0 to vc127 and fc0 to fc27. Keep in mind that only the vertex shader can access vc registers and only the fragment shader can access fc registers. Unlike the va and v (see below for details on these Varying registers) registers, the contents of vc and fc are the same for all vertices and all fragments.

To set values in the constant registers you use the context.setProgramConstantsFromVector or context.setProgramConstantsFromMatrix functions.

When using .setProgramConstantsFromVector you supply:

• programType – Must be either Context3DProgramType.FRAGMENT or Context3DProgramType.VERTEX, use this to indicate if you want to set a fc or a vc register.
• firstRegister - Which register to set, or if you supply enough data for multiple registers which is the first you’d like to set. For example use 0 to set fc0 or vc0.
• data - The Vector.<Number> of values you want to upload. This must contain a multiple of 4 values so as to fill the x, y, z and w components of however many registers you’re writing to. Even if you only actually need to upload a single number, you are not allowed to have only some components of a constant register set.
• numRegisters - Optional. If your data has enough for 3 registers but you only want to upload into 1 then you can do that here.

When using .setProgramConstantsFromMatrix you supply:

• programType - Must be either Context3DProgramType.FRAGMENT or Context3DProgramType.VERTEX, use this to indicate if you want to set  fc or  vc registers.
• firstRegister - The first register to set with the data from your matrix.
• matrix - The Matrix3D object you want to upload into constant registers. Matrix3Ds are always 4×4 matrices so four registers are needed to store their contents.
  If you specify the firstRegister as 2 and programType as Context3DProgramType.FRAGMENT, then fc2 fc3 fc4 and fc5 will be written to.
• transposedMatrix - If set to false the matrix is written into the registers with the first register being the first row of the matrix. If set to true the matrix is written in transposed order, meaning the first register stores the first column of the matrix.

Unlike vertex attributes, setting a constant and then not using it seems to pass without error. Obviously a shader trying to access a constant that you haven’t set will fail.

Varying Registers

Varying registers (v0 to v7) allow per-vertex data to be indirectly used in your fragment shader. To access a varying register within a fragment shader your vertex shader must be feeding it values by including an operation that writes data to one of the v registers.

Each triangle that is drawn will of course have three vertices, each of which provide a value to a particular v register. Every fragment contained within that triangle will have its corresponding v register contain a value that has been interpolated between that of the three vertices in that triangle.

In this illustration we look at a single register component as it is interpolated between the three vertices of a triangle. The image is shaded to indicate what value v0.x has at each point on the triangle. The value increases at positions closer to the vertex which has it set to 1.

Using Registers Within a Shader

Now that we have assigned values to the various registers we can deal with actually using them in a shader. To do anything in a shader you must use operations, which you can find outlined on my quick reference page.

Here we will deal with how to use registers in general rather than how to use specific operations. This mainly comes down to understanding how the components which make up each register work.

Recall that every register consists of four values, which are accessed through the a dot-suffix notation. For instance fc3.x is the first component of the register “fragment constant 3″. The components are referred to as .x .y .z and .w, you can also refer to them as .r .g .b and .a. Keep in mind that fc3.x is exactly the same thing as fc3.r; for this reason I prefer to stick with using the xyzw notation throughout my shaders even when I’m dealing with colours.

AGAL operations almost all act in a “component-wise” fashion, meaning we can supply them with more than a single component and they will apply the operation to all components at once. Some operations require multiple components be given, namely those meant to operate on vectors. To specify multiple components from a register simply use multiple component letters after the register name, for instance fc3.xyz gives the first three components of the “fragment constant 3″ register.

Note! If you specify just a register name then AGAL assumes you want to use all four components. So fc3.xyzw is exactly the same as fc3.

How Component-wise Operations Work

add vt0 va0 va1
or
add vt0.xyzw va0.xyzw va1.xyzw

before:
 vt0: ? ? ? ?
va0: 2 7 4 2
va1: 1 4 9 0

after:
 vt0: 3 11 13 2
va0 and va1 unchanged

Each component of va0 is put through the addition operation with the matching component of va1 and the result is put in the matching component of vt0.

In the above example:
vt0.x = va0.x + va0.x
vt0.y = va0.y + va0.y
vt0.z = va0.z + va0.z
vt0.w = va0.w + va0.w

Warning! I’ve encountered some situations where component-wise multiplication doesn’t work as I would expect. Specifically the operation:
mul vt0.xy va0.xy vc0.xy

Gives a result which is different from that of performing these two operations:
mul vt0.x va0.x vc0.x
mul vt0.y va0.y vc0.y

I’ve yet to work out exactly what’s going on. Expect this section to be edited.

Limited Component Operations

add vt0.xy va0.xy va1.xy

before:
 vt0: ? ? ? ?
va0: 2 7 4 2
va1: 1 4 9 0

after:
 vt0: 3 11 ? ?
va0 and va1 unchanged

Does as you’d expect by adding the values on a component-by component basis. As the .zw components of vt0 are not included in the target they are left untouched by the operation.

Mismatched Component Counts

add vt0 va0.xy va1.xy

Here we’re assigning the result of a two-component operation to a four-component target (remember that vt0 is the same as saying vt0.xyzw). In this case I believe it will put the result of the addition in vt0.xy, but your code isn’t clear so it is much better to specify where you want the result to go.

Swizzled Components

mov vt0.xy va0.zw

before:
vt0: ? ? ? ?
va1: 1 2 3 4

after:
 vt0: 3 4 ? ?
va1: 1 2 3 4

The fact that the source and destination components are not the same doesn’t matter. So far as AGAL is concerned it’s just a group of numbers. You can also use the same components but in a different order such as:

mov vt0.xyz va0.zyx

before:
vt0: ? ? ? ?
va1: 1 2 3 4

after:
 vt0: 3 2 1 ?
va1: 1 2 3 4

Finally, you can even use the same component multiple times.

mov vt0.xyz va0.xxx

before:
vt0: ? ? ? ?
va1: 1 2 3 4

after:
 vt0: 1 1 1 ?
va1: 1 2 3 4

Next Time

That’s quite enough theory. Next tutorial will definitely be something more direct and practical. Probably stepping through the process of creating a shader, from initial idea to final code.

The power of being given access to GPU acceleration for graphics isn’t just in getting images to be drawn really fast. If we use vertex shaders to make things move around the world then we can offload a significant amount of per-frame calculations over to the graphics card, which frees up the CPU for more interesting tasks like AI and physics.

As an example of the CPU time saved by using graphics hardware, here is virtually the same program embedded twice. The first version attempts to use your graphics hardware while the second is forced to use the fall-back software implementation. If you’re viewing this on a device that isn’t currently supported by Flash’s hardware acceleration then both versions will be using software rendering. You will of course need Flash Player 11 to see the demo.

If your results are anything like mine then the difference in performance even with this relatively small scale example will be dramatic. That said I’ve got to take a moment to say how impressed I am by the performance of the software implementation. It multi-threads and spreads itself across my CPU cores beautifully, and most importantly produces output which is so far as I can tell completely identical to that coming out of the graphics hardware version.

Perhaps more interesting than the raw increase in speed is the difference in CPU use. For me the hardware accelerated version’s main loop is measured at taking 0ms, and as you would expect the software version’s is much higher at around 20ms. Not only is the hardware accelerated version pumping out frames faster but it’s barely touching the CPU, which is left free for running the rest of the game.

Implementation

Writing the shader for these particles was good fun. If stripped of comments the final code is 54 lines (of a maximum 200 allowed) of virtually indecipherable opcodes and registers, but the process it represents is simple enough.

The classic way to handle moving objects in game programming is to adjust their position each update cycle by adding on a value representing their velocity. This technique doesn’t work for objects that are to be handled entirely through AGAL shaders as there is simply no place where their position could be updated by the shader and stored between frames.

Instead the shader needs to be able to produce the particle’s position when provided with the current time. So the process of writing the shader starts with finding an equation in terms of time that gives position (or rotation, colour, scale, or any other property of your particles that you want to have change). If the particles are moving at a constant velocity this is simple enough:

position = startPosition + startVelocity * time

The variables like startPosition and startVelocity will be passed into the shader through the vertex buffer (as they will be different for each particle), while time will be passed in through the shader constants as it is the same for all particles being drawn that frame. If you want the particles to undergo acceleration (such as due to gravity) then you can just add on a new term to account for that.

position = startPosition + startVelocity * time
           + 0.5 * constantAcceleration * time^2

Effects like the particles bouncing once when reaching a certain y-value are a little more complex but it still all comes down to making an equation that gives the position of the particle in terms of time.

Bonus Content!

Not particularly related to particle systems, but I decided to reproduce an old example of normal mapping I made back when Pixel Bender was the closest to hardware acceleration Flash had available. This version is of course running entirely on the GPU.

]]> http://www.saltgames.com/2011/accelerated-particle-effects/feed/ 4 http://www.saltgames.com/2011/simple-annotated-stage3d/ http://www.saltgames.com/2011/simple-annotated-stage3d/#comments Wed, 05 Oct 2011 17:08:03 +0000 Sam Driver http://www.saltgames.com/?p=539 Continue reading →]]> The first step is often the hardest. Here I provide a very simple example of rendering using Stage3D, complete with extensive comments.

All this does is draw a single multicoloured triangle on the screen. Not much, but it’s a good simple place to start.

This example makes use of the MiniAGALAssembler, which is a neat little class allowing the creation of AGAL shader code from within our Actionscript. It is available from the original source (bundled with a couple of other things), or mirrored on this site.

My apologises for the slightly ugly way the code gets displayed on this page. If you prefer it is also available as a FlashDevelop project, with the included Actionscript files of course readable in any text editor.

Main.as

package
{
import flash.display.Sprite;
import flash.display.Stage3D;
import flash.display3D.Context3D;
import flash.display3D.Context3DProgramType;
import flash.display3D.Context3DVertexBufferFormat;
import flash.display3D.Program3D;
import flash.events.Event;
import flash.geom.Matrix3D;

/**
* @author Salt
* www.saltgames.com
*/
public class Main extends Sprite
{
  private const renderAreaWidth:int = 600;
  private const renderAreaHeight:int = 400;

  private var myContext3D:Context3D;
  private var myStage3D:Stage3D;

  private var aListOfBundlesToRender:Vector.<BundleToRender>;

  public function Main():void
  {
   if (stage) init();
   else addEventListener(Event.ADDED_TO_STAGE, init);
  }

  private function init(e:Event = null):void
  {
   removeEventListener(Event.ADDED_TO_STAGE, init);

   // Start here <<

   // to do any rendering a Context3D is needed, and they can only be made by a Stage3D object
   // fortunately the Stage already has a list of Stage3D objects ready for us to use

   myStage3D = stage.stage3Ds[0];
   // creating a Context3D is not instant so we have to wait for an event telling us when it is ready
   myStage3D.addEventListener(Event.CONTEXT3D_CREATE, context3DCreated);
   myStage3D.requestContext3D();
  }

  private function context3DCreated(e:Event):void
  {
   // as the CONTEXT3D_CREATE event has triggered, we know that myStage3D has finished creating its Context3D
   myContext3D = myStage3D.context3D;

   // set up the back buffer of this Context3D, this is the 'canvas' which will be displayed.
   var antiAliasLevel:int = 0;	// valid values are 0, 2, 4, 8, 16
   var enableDepthAndStencilBuffers:Boolean = true;
   myContext3D.configureBackBuffer(renderAreaWidth, renderAreaHeight, antiAliasLevel, enableDepthAndStencilBuffers);

   // if you want the rendered area to be offset from the top-left corner, the position of myStage3D can be changed
   myStage3D.x = 0;
   myStage3D.y = 0;

   // myContext3D is now ready for rendering to take place on it
   addEventListener(Event.ENTER_FRAME, perFrame);

   // create a test item to render
   aListOfBundlesToRender = new Vector.<BundleToRender>;
   aListOfBundlesToRender.push(createTestBundleOfTrianglesToRender());
  }

  private function perFrame(e:Event):void
  {
   performRender();
  }

  private function performRender():void
  {
   // the back buffer must be cleared before we start trying to draw new stuff on it
   // here you can specify what it should be cleared to - including the depth and stencil buffers
   myContext3D.clear();

   // if you want your view of the world to be changed, simply translate the cameraMatrix
   var cameraMatrix:Matrix3D = new Matrix3D();

   // the cameraMatrix and orthographicProjectionMatrix are combined to give a complete transformation matrix
   // this matrix describes how each vertex in the world should be transformed (moved) to appear in the correct place on the screen
   var completeProjectionMatrix:Matrix3D = new Matrix3D();
   completeProjectionMatrix.append(cameraMatrix);
   completeProjectionMatrix.append(createOrthographicProjectionMatrix(renderAreaWidth, renderAreaHeight, 0, 1));

   // now that the transformation matrix has been created, it is loaded into the vertex constant register vc0,
   // ready to be accessed by our vertex shader program.
   myContext3D.setProgramConstantsFromMatrix(Context3DProgramType.VERTEX, 0, completeProjectionMatrix, true);

   for each (var bundleToRender:BundleToRender in aListOfBundlesToRender)
   {
    // set the shader program
    var shaderProgram:Program3D = myContext3D.createProgram();
    shaderProgram.upload(bundleToRender.vertexShaderAssembler.agalcode, bundleToRender.fragmentShaderAssembler.agalcode);
    myContext3D.setProgram(shaderProgram);

    // load vertex position into the vertex registers
    myContext3D.setVertexBufferAt(0, bundleToRender.vertexBuffer, 0, Context3DVertexBufferFormat.FLOAT_2);
    //                                                                                            ^ load a pair of values for each vertex - x and y position
    //                            ^ load into the va0 (vertex attribute zero) register

    // load vertex colour into the vertex registers
    myContext3D.setVertexBufferAt(1, bundleToRender.vertexBuffer, 2, Context3DVertexBufferFormat.FLOAT_3);
    //                                                                                            ^ load three values for each vertex - r,g,b to define a colour
    //                                                            ^ skip over the first two values for each vertex (those two values are the x and y coordinates)
    //                            ^ load into the va1 (vertex attribute one) register

    // actually perform the rendering operation
    myContext3D.drawTriangles(bundleToRender.indexBuffer, 0, bundleToRender.numberOfTriangles);
   }

   // once all the rendering is complete, present the back buffer so that the user actually sees it
   myContext3D.present();
  }

  private function createOrthographicProjectionMatrix(viewWidth:Number, viewHeight:Number, near:Number, far:Number):Matrix3D
  {
   // this is a projection matrix that gives us an orthographic view of the world (meaning there's no perspective effect)
   // the view is defined with (0,0) being in the middle,
   //	(-viewWidth / 2, -viewHeight / 2) at the top left,
   // 	(viewWidth / 2, viewHeight / 2) at the bottom right,
   //	and 'near' and 'far' giving limits to the range of z values for objects to appear.
   return new Matrix3D(Vector.<Number>
   ([
    2/viewWidth, 0, 0, 0,
    0, -2/viewHeight, 0, 0,
    0, 0, 1/(far-near), -near/(far-near),
    0, 0, 0, 1
   ]));
  }

  private function createTestBundleOfTrianglesToRender():BundleToRender
  {
   var testBundle:BundleToRender = new BundleToRender();

   // generate the vertex and index buffers for a single triangle
   // in this case the vertex buffer will include both the position and colour data
   // if we were to be using texturing, uv coordinates would be loaded in the vertex buffer too

   // first prepare the vertex buffer, specifying its dimensions
   var verticesForATriangle:int = 3;
   var numberOfTriangles:int = 1;
   var dataCountForEachVertex:int = 5; // for x, y, red, green, blue
   testBundle.vertexBuffer = myContext3D.createVertexBuffer(verticesForATriangle * numberOfTriangles, dataCountForEachVertex);

   // similarly prepare the index buffer
   var numberOfIndicesForATriangle:int = 3;
   testBundle.indexBuffer = myContext3D.createIndexBuffer(numberOfIndicesForATriangle * numberOfTriangles);

   // create the vertex data that will be loaded into the buffer
   var vertexData:Vector.<Number> = new Vector.<Number>();
   //                x     y   r  g  b
   vertexData.push(   0, -100, 1, 0, 0);
   vertexData.push(-150,  150, 0, 1, 0);
   vertexData.push( 200,   50, 0, 0, 1);

   // load the vertex data to the buffer
   testBundle.vertexBuffer.uploadFromVector(vertexData, 0, verticesForATriangle * numberOfTriangles);

   // create the index data to be loaded in
   var indexData:Vector.<uint> = new Vector.<uint>();
   indexData.push(0, 1, 2);

   testBundle.numberOfTriangles = numberOfTriangles;

   // load the index data to the buffer
   testBundle.indexBuffer.uploadFromVector(indexData, 0, numberOfIndicesForATriangle * numberOfTriangles);

   // create both the vertex and fragment shader programs to be used when rendering this bundle of triangles
   // the vertex data created above will be available to these shaders through registers:
   // the first 2 values for each vertex (the x and y position) will appear in va0
   //	the remaining 3 values for each vertex (r, g, b for colour) will appear in va1
   testBundle.setVertexShaderCode(
    "m44 op, va0, vc0 \n" + // apply the matrix transform to va0 (vertex position), putting result in op (the position output)
    "mov v0, va1"           // move va1 (vertex colour) to v0 where the fragment shader will be able to access it
    );

   testBundle.setFragmentShaderCode(
    "mov oc, v0"				// simply put the vertex colour directly into oc (the colour output)
    );

   return testBundle;
  }
}
}

BundleToRender.as

package
{
import flash.display3D.Context3DProgramType;
import flash.display3D.IndexBuffer3D;
import flash.display3D.VertexBuffer3D;
/**
* @author Salt
* www.saltgames.com
*/
public class BundleToRender
{
  // I use the not entirely technical term of "bundle" for a group of triangles being rendered all in one batch.

  public var vertexShaderAssembler:AGALMiniAssembler;
  public var fragmentShaderAssembler:AGALMiniAssembler;

  // the vertexBuffer stores all the per-vertex data
  public var vertexBuffer:VertexBuffer3D;

  // the indexBuffer is used to determine how the vertices should be connected to form triangles
  public var indexBuffer:IndexBuffer3D;

  // the number of triangles for which we have data, and we want to draw.
  public var numberOfTriangles:int;

  public function BundleToRender()
  {

  }

  public function setVertexShaderCode(vertexCode:String):void
  {
   vertexShaderAssembler = new AGALMiniAssembler();
   vertexShaderAssembler.assemble(Context3DProgramType.VERTEX, vertexCode);
  }

  public function setFragmentShaderCode(fragmentCode:String):void
  {
   fragmentShaderAssembler = new AGALMiniAssembler();
   fragmentShaderAssembler.assemble(Context3DProgramType.FRAGMENT, fragmentCode);
  }
}
}

]]> http://www.saltgames.com/2011/simple-annotated-stage3d/feed/ 4 http://www.saltgames.com/2011/stage-3d-shader-cheatsheet/ http://www.saltgames.com/2011/stage-3d-shader-cheatsheet/#comments Wed, 05 Oct 2011 15:08:44 +0000 Sam Driver http://www.saltgames.com/?p=523 Continue reading →]]>

A quick reference for working with AGAL, the new shader language for use with Stage3D introduced with Flash Player 11.

I’ve been working on a series of tutorials for Stage3D which aren’t yet ready, but with the recent release of Flash Player 11 now seems a good time to get at least some information out. If you’re new to Stage3D then this may not make much sense until you’ve read some other resources, but I hope it serves as a useful reference as you experiment and learn.

Registers:
Each register consists of four components, which are floating point values. These components are accessed by registerName.x, registerName.y, registerName.z, registerName.w. They’re named for dealing with 3D positions – with the ‘w’ for rotation in the style of quaternions – but they can just as well be used to hold a colour (in fact you can also access the components with .r .g. b. a) or any other values you want to use.

There are several registers of each type available. For instance you might have va0 giving the 3D position of a vertex in space, and va1 giving the uv mapping coordinate for that vertex. The nice thing about having registers made up of components is you can do things like perform a basic addition operation va0 and vc0, and the addition will be performed correctly on each component.

Registers for Vertex Shaders

• va[0 to 7] Vertex Attribute. The contents of the vertex buffer, as set with context3D.setVertexBufferAt. Each vertex has its own space in the vertex buffer which only it can access.
• vc[0 to 127] Vertex Constant. Passed into the shader with context3D.setProgramConstantsFromVector or context3D.setProgramConstantsFromMatrix. These registers can be read by all vertices, but cannot be written to by the shader.
• vt[0 to 7] Vertex Temporary. A handy temporary register where you can put values during a calculation.
• op Vertex Output or “Output Position”. The output: op.x and op.y is where in the 2D space of the screen this vertex will be drawn. The op.z value is used for depth checking and writing to the depth buffer if you have those enabled. So far as I know the op.w value is not actually used.

• v[0 to 7] Varying. A magical (not actually magical) register that allows you to pass values from the vertex shader to the fragment shader. The value that arrives in the fragment shader will be interpolated between the value of the three v registers of the vertices which make up the triangle in which the fragment falls.
  Note!  The fragment shader cannot directly access the vertex buffer, so anything it needs from there has to be passed through the v registers. For instance if you’re using texture mapping this register will need to pass the uv coordinates to the fragment shader.

Registers for Fragment Shaders

• fc[0 to 27] Fragment Constant. Much like vc for the vertex shader, this register is set by context.setProgramConstantsFromVector or context.setProgramConstantsFromMatrix, can be read by each fragment, and not written to by the shader.
• ft[0 to 7] Fragment Temporary. Again just like vt for the vertex shader, this is a temporary store useful for performing calculations.
• fs[0 to 7] Texture Sampler. This is where the fragment shader is able to access whatever texture(s) were bound using context3D.setTextureAt.
• oc Fragment Output or “Output Colour”. The output: oc.x oc.y oc.z oc.w are the red, green, blue and alpha values respectively for the fragment to be drawn.

Operations:
Shaders are made up of a series of operations, with one operation on each line. First the operation to perform is identified by a three letter opcode such as “add”, then the parameters for that operation are given. The parameters are (almost) always specified as registers. If you want to use a number in your shader, it should be supplied through the vc or fc registers.

In the parameters the target register is always specified first. The target register is where the result of the operation is placed. No change is made to a register other than the target register. Some operations require two further parameters, others just one. The tex operation used for texture sampling is a special case that has six parameters, three of which are given as strings rather than registers. tex is a pretty wild guy.

At first look the mess of opcodes and registers names that make up the AGAL code for a shader can look intimidating but they’re actually quite simple. Just remember that each operation does exactly one thing, and writes to exactly one register. AGAL doesn’t allow for conditional statements like if then or any form of looping, so following along with what a shader is doing is extremely easy: it always just proceeds to the next operation.

With that said, AGAL code is not nearly as intuitive to glance at and understand what it does as (well written) AS3 code is. Taking a minute to type out some comments for the AGAL code you write is a very good idea.

You are limited to 200 operations in a single AGAL shader.

m44 op, va0, vc0
mov v0, va1

tex oc, v0, fs0 <2d,clamp,linear>

A practical test of Pixel Bender powered normal mapped lighting in a game setting, in contrast to a less practical but more pretty example.

W, A, S and D to move the player ship, mouse to shoot. Press Q to spawn a group of enemies.  Be aware – there is sound in this example!

A single light source follow’s the player, and additional light sources are created for each enemy destroyed. All light sources illuminate a generated layer of textured and bump mapped rock.

To attempt to match the various performance requirements of a finished game, there is a simple particle animation system, enemy AI and parallax background. A circular distortion effect also powered by Pixel Bender can be seen if an enemy collides with the player, or the player detonates a bomb using Spacebar.

The lighting effect can be disabled from the menu accessible by right-clicking while the game is running.

The source code for this simple example is available here. Although be aware it uses the Box2D physics library for handling the shapes, so some of the source may be confusing if you’re not familiar with that library.

A demonstration of the 2D shadow effects that I discuss in a tutorial on the TIGSource forums that I wrote a while ago.  Just a series of rooms that you can walk through. No source code for this one because frankly it’s a bit of a mess. It’s the same technique as used in the first example, just applied to a more overtly game-like setting.

W, A, S and D to move, space bar to toggle the light on your faerie companion.

A quite detailed tutorial on creating custom shaders in Pixel Bender and using them in your Flash projects.

This is quite a lengthy tutorial.  It includes a brief introduction to pixel shaders in general; a tour of the most important aspects of Pixel Bender for use creating shaders specifically for Flash, directions for integrating the custom shader you create into a Flash project, and example source code for two simple examples.

Some Introductions

Shaders, specifically pixel shaders.

These are programs that operate on an image on a per-pixel basis.  Per-pixel means the program operates on each pixel individually, without reference to operations carried out on other pixels.  Because shaders operate in this per-pixel manner they very well suited to parallel processing: Having many instances of the shader operating at the same time, each on their own pixel.  Operating per-pixel means that there are certain tasks that are well suited to being performed by a shader as well as many tasks that are completely unsuitable.

Filters, Effects, Blendmodes, Kernels.

There’s a lot of terminology floating around depending on where you read about stuff.  I’m going to call everything a Shader, because that’s the name of the class I end up using in Flash for everything (not the best method for naming things, but still.) To give a very rough idea to help you follow other documents: Filters and Effects are applied to just one image; Blendmodes are applied to two (or more, in theory?) images; a Kernel is the thing you make when you type code into Pixel Bender.

Pixel Bender.

The Pixel Bender Toolkit, to give it its full name, is a neat program made by Adobe to produce custom filters and effects for use in a variety of their applications.  You should be aware when reading other sources for information on Pixel Bender that many of its functions will just not function in Flash.  Pixel Bender is available for free from Adobe, look to the right of that page.  Pixel Bender was known as Hydra when it was earlier in development, so you may see that name crop up around the place too.

Here be monsters

Using Pixel Bender to make shaders for Flash is – to put it politely – prone to inconsistencies.  The Pixel Bender Toolkit lets you run your shader within it to test it.  What you see in the preview will quite often not match what comes out in Flash.  I’m attempting to build up a record of ways in which the two do not match but it’s far from complete.  Just be aware that you should test primarily in Flash and not rely on the inbuilt preview.  Since I first wrote this tutorial the Pixel Bender Toolkit now includes an option to simulate running a shader in Flash which is helpful, but I suspect still doesn’t catch everything.  Also be sure to regularly check for an updated version from Adobe – I’ve found myself stuck trying to work around some bug which has actually already been fixed if I’d just updated.

What’s the point

So Pixel Bender can make buggy programs that are limited to operating on images in a per-pixel basis.  Why am I bothering learning this?

Because it’s fast!

You might like to look at a normal mapping example I put together recently.

Flash programs are pretty slow generally, and are limited to executing in a single thread.  Consider if you want to tint an image red in Flash.  If you’re charmingly naïve you might write something like:

for (var x:int = 0; x < image.width; x++)
{
   for (var y:int = 0; y < image.height; y++)
   {
      var colour:uint = image.getPixel(x, y);
      colour = (colour | 0xff0000); //set the red channel to 255
      image.setPixel(x, y, colour);
   }
}

This will be slow.  It’ll be slow because of the BitmapData.getPixel and BitmapData.setPixel operations being called huge numbers of times, but on a more general level it’s slow because you’re visiting each pixel in turn when you could be performing that operation on all the pixels at once.  This is effectively what a custom shader from Pixel Bender does.

Your shader will not actually operate on all the pixels at once, as it’s limited by the number of threads it can execute on the machine its on.  But thinking of it as operating on every pixel at once is a useful mental model.

Pixel Bender was designed to run on GPUs, which are especially well suited to performing many parallel tasks.  When running in Flash it is actually limited to using the CPU.  This means it’s not nearly as fast as it could be, but it still has the potential to be many times faster than plain old actionscript.

To the Pixel Bender!

Let’s get right in and actually make a shader and get it working in Flash.

Download and install Pixel Bender.

Set a couple of options within Pixel Bender (Edit menu, Preferences…) to tell it you’ll be exporting to Flash:

Select New Kernel from the File menu.

You’ll now have the code for a shader that does nothing.  Excellent!

<languageVersion : 1.0;>
kernel NewFilter
<  namespace : "Your Namespace";
   vendor : "Your Vendor";
   version : 1;
   description : "your description";
>
{
   input image4 src;
   output pixel4 dst;
   void
   evaluatePixel()
   {
      dst = sampleNearest(src,outCoord());
   }
}

Oh no, this isn’t Actionscript!  Time to learn a new language.  Good news is it’s easy, and you can ignore most of it.

So what actually matters here?

input image4 src;

This declares a variable of the type image4 named src, and says that this variable will hold the input that our shader is given.  Later we’ll see how a shader can have multiple input images.

The type image4 just means that it’s an image with 4 colour channels: Red, green, blue and alpha.  There is also image3 if you want to ignore the alpha channel.

output pixel4 dst;

This declares a variable of the type pixel4 named dst, and says that whatever value we put in this variable will be the output of the shader.  pixel4 is a single pixel with 4 colour channels.

But wait!  We’re taking a whole image as input, and only giving a single pixel as output?  Shouldn’t this be producing a whole image?

I was just getting to that.

The evaulatePixel() function is the heart of our shader.  It is executed once for each pixel of the source image.  It produces just a single pixel – our output variable named dst – which is then built up to form the finished image.

void
evaluatePixel()
{
   dst = sampleNearest(src,outCoord());
}

Here’s the evaluatePixel function in full.  The only bit you ever need change is between the curly braces.  In this case it takes the result of the built-in function sampleNearest and assigns that to dst.  Note that unlike actionscript functions where you use ‘return’ to have a function produce a value, evaluatePixel just sets the value of the variable that we’ve previously defined as the shader’s output.

What does sampleNearest do?  Press F1 in Pixel Bender to get the documentation.  The majority of the documentation doesn’t apply to us (most sections end with “Flash player note: X is not supported in Flash Player.”)  But it has lists of built-in functions and data types, most of which are supported.  sampleNearest gives us the value of a single pixel from the image passed as the first parameter, which in this case is src.  Which pixel to return is determined by the second parameter, which in this case is the result of another built-in function: outCoord. This function returns a float2 value giving the position of the pixel that in this instance evaluatePixel was called on.

Vector data types

Unlike actionscript, Pixel Bender uses something called vector data types.  These store several values within one variable.  For instance a position in two-dimensional space can be specified by a float2 value, which is just two floating point values (basically the same as number in actionscript) joined together.  You can access the individual values, or operate on them as a whole.  Let’s take an example:

void
evaluatePixel()
{
   pixel4 myPixelVariable;
   myPixelVariable = sampleNearest(src, outCoord());
   dst = myPixelVariable * 2.0;
}

First new thing:

pixel4 myPixelVariable;

Simply declares a variable named myPixelVariable which is of type pixel4.

myPixelVariable = sampleNearest(src, outCoord());

As you’d expect we can assign a value to it.

dst = myPixelVariable * 2.0;

Then we can multiply the whole thing by 2, and assign that to our output variable.

This is exactly the same as performing:

dst.r = myPixelVariable.r * 2.0;
dst.g = myPixelVariable.g * 2.0;
dst.b = myPixelVariable.b * 2.0;
dst.a = myPixelVariable.a * 2.0;

Isn’t that neat.

myPixelVariable.r is accessing one of the four elements that make up the variable.  The elements can also be accessed by:

myPixelVariable[0], myPixelVariable[1], myPixelVariable[2], myPixelVariable[3]

myPixelVariable.x, myPixelVariable.y, myPixelVariable.z, myPixelVariable.w

myPixelVariable.s, myPixelVariable.t, myPixelVariable.u, myPixelVariable.v

The suffixes [0], .r, .x and .s are all equivalent.

You can access several elements of a vector-type variable at once.

myPixelVariable.rg *= 2.0;

Will multiply just the red and green channels by two.

You can also assign values between elements of a vector-type:

myPixelVariable.rgb = myPixelVariable.brg;

This assigns the blue value of the pixel to be the red value, the red to the green and the green to the blue.

You can specify each element of a vector-type like so:

myPixelVariable.rgb = float3(0.5, 0.2, 0.8);

Fussy about types

Notice that in the examples above we multiply by 2.0 rather than just 2.  This is because unlike actionscript, Pixel Bender doesn’t do implicit conversion.  What that means is that it will interpret 2 as an integer, and not know how to multiply that by the float values of a pixel4 variable.  Writing the value as 2.0 makes it clear that you want it treated as a float. You could also use float(2) to explicitly convert the integer into a float.

Slightly weirdly there seems to be little difference between pixel4 and float4.  I think there’s a difference in how accurately the values are stored, but they are largely interchangeable.

A working shader

Let’s pull that together into a simple working shader so we can move on to implementing it in Flash:

kernel NewFilter
<    namespace : "Your Namespace";
     vendor : "Your Vendor";
     version : 1;
     description : "your description";
>
{
    input image4 src;
    output pixel4 dst;

    void
    evaluatePixel()
    {
        dst = sampleNearest(src,outCoord());
        dst.r *= 2.0;
    }
}

This will just take each pixel from an image and double the red channel.

We can give this a quick test within Pixel Bender.  Just Load Image 1… from the File menu and select any old photo you have laying around. Then hit the Run button (or F5) and the photo will be displayed with a red tint.

Now choose Export Filter for Flash Player… from the File menu.  This will create a .pbj file containing Pixel Bender byte code which Flash will be able to understand.  Whenever you change a shader you’re working on, remember you’ll need to re-export it to have those changes show up in Flash.

And in to Flash

Once you know how, getting the shader in to your Flash program is actually quite simple.  The basic technique is very similar to embedding an image, font or sound:

[Embed(source = 'TestShader.pbj', mimeType = 'application/octet-stream')]
private var TestShader:Class;

And that’s it embedded! To create an instance of the shader:

var myShader:Shader = new Shader(new TestShader() as ByteArray);

The shader we’ve created requires an input image, so we need to point myShader at an image to use as its input.

myShader.data.src.input = myBitmapData;

This assumes we already have an image loaded and named myBitmapData. Note that the src corresponds to the name we gave the input variable in Pixel Bender.

To actually run the shader, we make use of a ShaderJob.

myJob:ShaderJob = new ShaderJob(myShader, outputBitmapData);
myJob.start(true);

Again this assumes we already have a bitmapData object named outputBitmapData which is ready to receive the finished image (it should be created as the same size as the input image.)  The true value passed when calling start on myJob tells Flash you want to wait until the job is complete before continuing.  It’s possible to pass false instead and have the shader execute asynchronously, but that brings up a whole lot of other issues and a pretty major bug.

Putting this all together into a program that’ll actually work:

package
{
	import flash.display.Bitmap;
	import flash.display.BitmapData;
	import flash.display.Shader;
	import flash.display.ShaderJob;
	import flash.display.Sprite;
	import flash.utils.ByteArray;

	public class PrettyShader extends Sprite
	{
		//embed the shader
		[Embed(source = 'TestShader.pbj', mimeType = 'application/octet-stream')]
		private var TestShader:Class;

		//embed an image to test it on
		[Embed(source = 'SomeImage.jpg')]
		private var SomeImage:Class;

		public function PrettyShader()
		{
			//set up the input and output BitmapDatas
			var myBitmapData:BitmapData = (new SomeImage() as Bitmap).bitmapData;
			var outputBitmapData:BitmapData = new BitmapData(myBitmapData.width, myBitmapData.height);

			//set up the Shader
			var myShader:Shader = new Shader(new TestShader() as ByteArray);
			myShader.data.src.input = myBitmapData;

			//create the ShaderJob
			var myJob:ShaderJob = new ShaderJob(myShader, outputBitmapData);

			//and run it!
			myJob.start(true);

			//display the input and output.
			var before:Bitmap = new Bitmap(myBitmapData);
			addChild(before);
			var after:Bitmap = new Bitmap(outputBitmapData);
			after.x = before.width;
			addChild(after);
		}
	}
}

Remember to set your compiler to target Flash Player 10, and you should end up with something rather like this:

We’re actually almost done now!  There are two more things to cover:  multiple image inputs, and parameters.

Multiple images as inputs

The example above just takes one image and fiddles with the colour a little.  You can do much more powerful things if you work with more than one image.

In Pixel Bender all we need to do is add an extra input statement for each image we want to sample from.  For example:

kernel NewFilter
<    namespace : "Your Namespace";
     vendor : "Your Vendor";
     version : 1;
     description : "your description";
>
{
    input image4 srcOne;
    input image4 srcTwo;
    output pixel4 dst;

    void
    evaluatePixel()
    {
        float2 positionHere = outCoord();
        pixel4 fromOne = sampleNearest(srcOne, positionHere);
        pixel4 fromTwo = sampleNearest(srcTwo, positionHere);
        float crossfade = positionHere.x / 400.0;
        dst = fromOne * crossfade + fromTwo * (1.0 - crossfade);
    }
}

This will sample from both images, and mix them together depending on the horizontal position of the pixel in question. You might notice that I create a float2 variable named positionHere to store the result of outCoord(), this is simply to save calling the function multiple times when I know it’ll give the same result.  Unnecessary premature optimisation or just being tidy?  You decide!

The changes in Flash are just as simple.  Simply assign another BitmapData as the shader’s second input:

myShader.data.srcOne.input = myBitmapData;
myShader.data.srcTwo.input = myOtherBitmapData;

Parameters
Parameters are simply variables that are used within the shader and can be set from within Flash.  Like input images and the output pixel, parameters are defined outside of the evaluatePixel function.  As well as specifying the data type that the parameter should be you give minimum, maximum and default values.

Here is the above shader amended with two parameters:

kernel NewFilter
<    namespace : "Your Namespace";
     vendor : "Your Vendor";
     version : 1;
     description : "your description";
>
{
    input image4 srcOne;
    input image4 srcTwo;
    output pixel4 dst;

    parameter float middleOfFade
    <
         minValue: -1.0;
         maxValue: 1.0;
         defaultValue: 0.0;
    >;

    parameter int widthOfImage
    <
         minValue: 0;
         maxValue: 5000;
         defaultValue: 400;
    >;

    void
    evaluatePixel()
    {
        float2 positionHere = outCoord();
        pixel4 fromOne = sampleNearest(srcOne, positionHere);
        pixel4 fromTwo = sampleNearest(srcTwo, positionHere);
        float crossfade = middleOfFade + (positionHere.x / float(widthOfImage));
        if (crossfade > 1.0) crossfade = 1.0;
        if (crossfade < 0.0) crossfade = 0.0;
        dst = fromOne * crossfade + fromTwo * (1.0 - crossfade);
    }
}

In this case we use two parameters, for two quite different purposes. You might have noticed that the previous versions relied on a constant value of 400.0 in the calculation of crossfade. This was the width of the image in use, but of course our shader should be ready to accept other sized images too so the parameter widthOfImage was added.  Secondly the parameter middleOfFade is added to allow interactive changes to how the two source images are mixed.

As a side note, Pixel Bender actually includes some built-in functions for determining the size of input images, but they are not available when exporting for the Flash player.

If you test this shader in Pixel Bender it will display some sliders that let you alter your parameters on the fly and see what effect they have.

Passing values to your shader from Flash is again fairly simple, although slightly obscure.

myShader.data.middleOfFade.value = [0.2];

Notice that you use .value when setting a parameter, but .input when setting an input image.  You also have to put your value inside an array first.  It makes more sense if you need to pass values into a float3 variable:

myShader.data.someFloat3Variable.value = [0.1, 0.6, 0.3];

All together now
A working Flash program that uses a shader with two image inputs and parameters.

package
{
	import flash.display.Bitmap;
	import flash.display.BitmapData;
	import flash.display.Shader;
	import flash.display.ShaderJob;
	import flash.display.Sprite;
	import flash.events.Event;
	import flash.utils.ByteArray;

	public class PrettyShader extends Sprite
	{
		//embed the shader
		[Embed(source = 'TestShader.pbj', mimeType = 'application/octet-stream')]
		private var TestShader:Class;

		//embed the images to test it with
		[Embed(source = 'SomeImage.jpg')]
		private var SomeImage:Class;
		[Embed(source = 'SecondImage.jpg')]
		private var OtherImage:Class;

		private var myShader:Shader;
		private var outputImage:BitmapData;

		private var count:Number = 0;
		private const twoPi:Number = Math.PI * 2;

		public function PrettyShader()
		{
			//set up the input and output BitmapDatas
			var firstImage:BitmapData = (new SomeImage() as Bitmap).bitmapData;
			var secondImage:BitmapData = (new OtherImage() as Bitmap).bitmapData;
			outputImage = new BitmapData(firstImage.width, firstImage.height);

			//set up the Shader
			myShader = new Shader(new TestShader() as ByteArray);
			myShader.data.srcOne.input = firstImage;
			myShader.data.srcTwo.input = secondImage;

			//set parameters
			myShader.data.widthOfImage.value = [firstImage.width];
			myShader.data.middleOfFade.value = [0.0];

			//display the output.
			addChild(new Bitmap(outputImage));

                        //update every frame
			addEventListener(Event.ENTER_FRAME, perFrame);
		}

		private function perFrame(e:Event = null):void
		{
			//update realtime parameter
			count += 0.05;
			if (count > twoPi) count -= twoPi;
			myShader.data.middleOfFade.value = [Math.sin(count)];

			//create the ShaderJob
			var myJob:ShaderJob = new ShaderJob(myShader, outputImage);

			//and run it!
			myJob.start(true);
		}
	}
}

Here it is running:

I think that is about all for now.  There is plenty more to learn about both Pixel Bender and using shaders in Flash, but this should give you everything you need to get some cool stuff working.

Of course making those awesome shaders is another topic altogether.

As a final note, keep in mind that it is possible to use custom shaders as BlendModes or Filters directly in Flash.  I have found that doing so ended up being more complex (or at least confusing), didn’t allow for the creation of anything that couldn’t be achieved through the use of Shader and ShaderJob, and may even run slightly slower.