A thorough tutorial on the use of registers for AGAL shaders. How to go about setting their values and accessing them in the shader.

This tutorial is weighted towards theory rather than the practical writing of code. You might find it beneficial to refer to my simple rendering example to see how this all gets applied.

Registers:

Each register consists of four components, which are floating point values. A particular component is accessed by the notation [registerName].[componentLetter], where componentLetter is x, y, z, or w. For example va0.x is the first component of the first vertex attribute register. For a listing of the available registers, please see my AGAL reference page.

Vertex Buffer:

The vertex buffer is how you pass per-vertex data into your vertex shader, where it will appear in the va (vertex attribute) registers. Usually this will include the position of that vertex in the world and some data used to colour it – either by passing in a colour for the vertex, uv coordinates for texture mapping, or something more complex. Keep in mind that the vertex buffer is just a series of floating point numbers which you can use for anything you like in your shader.

To deal with the vertex buffer you first create a VertexBuffer3D object by using the function context3D.createVertexBuffer (you cannot just instantiate it yourself with new VertexBuffer3D().)  This object will store the values until you’re ready to upload them into registers and have a shader use them.

Imagine the VertexBuffer3D object as a table. When you create it with context3D.createVertexBuffer you specify the number of vertices (the number of rows in the table) and the number of data32 per vertex (the number of columns in the table). In this illustration I’ve labelled each column with what I plan to use the values for, but remember they’re just a series of meaningless numbers as far as Flash is concerned.

You upload data into the VertexBuffer3D table using vertex3D.uploadFromVector, where you supply:

• data - A Vector.<Number> of the values you’re going to upload. Must be a whole number of rows’ worth of data.
• startVertex - The row in the vertex3D table at which to start writing.
• numVertices - The number of rows of data to copy.

In this illustration see how the startVertex determines where in the VertexBuffer3D object our Vector of numbers will be placed. If numVertices was set to 1 then only one row of the table would have been filled.

You set the va registers using context3D.setVertexBufferAt. When doing so you specify:

• index - Which va register you want to upload into, e.g. 0 for va0.
• buffer - The vertexBuffer3D object to use.
• bufferOffset - Which column in the vertexBuffer3D table to start from when reading in data
• format - How many columns of the table to read in, from FLOAT_1 for one to FLOAT_4 for four. Alternatively read in a single column but interpret it as four individual bytes instead of a single float.

As seen here, bufferOffset (0 in this case) and format (FLOAT_2) determine which columns of the VertexBuffer3D object to copy. Index (0 for the va0 register) determine which register those values should be placed in.

In this second example bufferOffset is 2, format is FLOAT_4, and index is 1. This results in copying the four columns shown from the VertexBuffer3D and using them to fill the va1 register.

Keep in mind that when performing the setVertexBufferAt operation you are setting the registers for as many vertices as you have rows in the vertexBuffer3D object you’re using.

Note! Rendering will fail if you have set values for registers that are then not used by your shader. Similarly it will fail if your shader tries to access a va register that hasn’t had data uploaded to it. Make sure you have context3D.enableErrorChecking set to true so you receive these error messages.

Vertex and Fragment Constants:

The other way to get numbers into a shader is through the vertex constant and fragment constant registers. These are accessed in the shader at vc0 to vc127 and fc0 to fc27. Keep in mind that only the vertex shader can access vc registers and only the fragment shader can access fc registers. Unlike the va and v (see below for details on these Varying registers) registers, the contents of vc and fc are the same for all vertices and all fragments.

To set values in the constant registers you use the context.setProgramConstantsFromVector or context.setProgramConstantsFromMatrix functions.

When using .setProgramConstantsFromVector you supply:

• programType – Must be either Context3DProgramType.FRAGMENT or Context3DProgramType.VERTEX, use this to indicate if you want to set a fc or a vc register.
• firstRegister - Which register to set, or if you supply enough data for multiple registers which is the first you’d like to set. For example use 0 to set fc0 or vc0.
• data - The Vector.<Number> of values you want to upload. This must contain a multiple of 4 values so as to fill the x, y, z and w components of however many registers you’re writing to. Even if you only actually need to upload a single number, you are not allowed to have only some components of a constant register set.
• numRegisters - Optional. If your data has enough for 3 registers but you only want to upload into 1 then you can do that here.

When using .setProgramConstantsFromMatrix you supply:

• programType - Must be either Context3DProgramType.FRAGMENT or Context3DProgramType.VERTEX, use this to indicate if you want to set  fc or  vc registers.
• firstRegister - The first register to set with the data from your matrix.
• matrix - The Matrix3D object you want to upload into constant registers. Matrix3Ds are always 4×4 matrices so four registers are needed to store their contents.
  If you specify the firstRegister as 2 and programType as Context3DProgramType.FRAGMENT, then fc2 fc3 fc4 and fc5 will be written to.
• transposedMatrix - If set to false the matrix is written into the registers with the first register being the first row of the matrix. If set to true the matrix is written in transposed order, meaning the first register stores the first column of the matrix.

Unlike vertex attributes, setting a constant and then not using it seems to pass without error. Obviously a shader trying to access a constant that you haven’t set will fail.

Varying Registers

Varying registers (v0 to v7) allow per-vertex data to be indirectly used in your fragment shader. To access a varying register within a fragment shader your vertex shader must be feeding it values by including an operation that writes data to one of the v registers.

Each triangle that is drawn will of course have three vertices, each of which provide a value to a particular v register. Every fragment contained within that triangle will have its corresponding v register contain a value that has been interpolated between that of the three vertices in that triangle.

In this illustration we look at a single register component as it is interpolated between the three vertices of a triangle. The image is shaded to indicate what value v0.x has at each point on the triangle. The value increases at positions closer to the vertex which has it set to 1.

Using Registers Within a Shader

Now that we have assigned values to the various registers we can deal with actually using them in a shader. To do anything in a shader you must use operations, which you can find outlined on my quick reference page.

Here we will deal with how to use registers in general rather than how to use specific operations. This mainly comes down to understanding how the components which make up each register work.

Recall that every register consists of four values, which are accessed through the a dot-suffix notation. For instance fc3.x is the first component of the register “fragment constant 3″. The components are referred to as .x .y .z and .w, you can also refer to them as .r .g .b and .a. Keep in mind that fc3.x is exactly the same thing as fc3.r; for this reason I prefer to stick with using the xyzw notation throughout my shaders even when I’m dealing with colours.

AGAL operations almost all act in a “component-wise” fashion, meaning we can supply them with more than a single component and they will apply the operation to all components at once. Some operations require multiple components be given, namely those meant to operate on vectors. To specify multiple components from a register simply use multiple component letters after the register name, for instance fc3.xyz gives the first three components of the “fragment constant 3″ register.

Note! If you specify just a register name then AGAL assumes you want to use all four components. So fc3.xyzw is exactly the same as fc3.

How Component-wise Operations Work

add vt0 va0 va1
or
add vt0.xyzw va0.xyzw va1.xyzw

before:
 vt0: ? ? ? ?
va0: 2 7 4 2
va1: 1 4 9 0

after:
 vt0: 3 11 13 2
va0 and va1 unchanged

Each component of va0 is put through the addition operation with the matching component of va1 and the result is put in the matching component of vt0.

In the above example:
vt0.x = va0.x + va0.x
vt0.y = va0.y + va0.y
vt0.z = va0.z + va0.z
vt0.w = va0.w + va0.w

Warning! I’ve encountered some situations where component-wise multiplication doesn’t work as I would expect. Specifically the operation:
mul vt0.xy va0.xy vc0.xy

Gives a result which is different from that of performing these two operations:
mul vt0.x va0.x vc0.x
mul vt0.y va0.y vc0.y

I’ve yet to work out exactly what’s going on. Expect this section to be edited.

Limited Component Operations

add vt0.xy va0.xy va1.xy

before:
 vt0: ? ? ? ?
va0: 2 7 4 2
va1: 1 4 9 0

after:
 vt0: 3 11 ? ?
va0 and va1 unchanged

Does as you’d expect by adding the values on a component-by component basis. As the .zw components of vt0 are not included in the target they are left untouched by the operation.

Mismatched Component Counts

add vt0 va0.xy va1.xy

Here we’re assigning the result of a two-component operation to a four-component target (remember that vt0 is the same as saying vt0.xyzw). In this case I believe it will put the result of the addition in vt0.xy, but your code isn’t clear so it is much better to specify where you want the result to go.

Swizzled Components

mov vt0.xy va0.zw

before:
vt0: ? ? ? ?
va1: 1 2 3 4

after:
 vt0: 3 4 ? ?
va1: 1 2 3 4

The fact that the source and destination components are not the same doesn’t matter. So far as AGAL is concerned it’s just a group of numbers. You can also use the same components but in a different order such as:

mov vt0.xyz va0.zyx

before:
vt0: ? ? ? ?
va1: 1 2 3 4

after:
 vt0: 3 2 1 ?
va1: 1 2 3 4

Finally, you can even use the same component multiple times.

mov vt0.xyz va0.xxx

before:
vt0: ? ? ? ?
va1: 1 2 3 4

after:
 vt0: 1 1 1 ?
va1: 1 2 3 4

Next Time

That’s quite enough theory. Next tutorial will definitely be something more direct and practical. Probably stepping through the process of creating a shader, from initial idea to final code.

The power of being given access to GPU acceleration for graphics isn’t just in getting images to be drawn really fast. If we use vertex shaders to make things move around the world then we can offload a significant amount of per-frame calculations over to the graphics card, which frees up the CPU for more interesting tasks like AI and physics.

As an example of the CPU time saved by using graphics hardware, here is virtually the same program embedded twice. The first version attempts to use your graphics hardware while the second is forced to use the fall-back software implementation. If you’re viewing this on a device that isn’t currently supported by Flash’s hardware acceleration then both versions will be using software rendering. You will of course need Flash Player 11 to see the demo.

If your results are anything like mine then the difference in performance even with this relatively small scale example will be dramatic. That said I’ve got to take a moment to say how impressed I am by the performance of the software implementation. It multi-threads and spreads itself across my CPU cores beautifully, and most importantly produces output which is so far as I can tell completely identical to that coming out of the graphics hardware version.

Perhaps more interesting than the raw increase in speed is the difference in CPU use. For me the hardware accelerated version’s main loop is measured at taking 0ms, and as you would expect the software version’s is much higher at around 20ms. Not only is the hardware accelerated version pumping out frames faster but it’s barely touching the CPU, which is left free for running the rest of the game.

Implementation

Writing the shader for these particles was good fun. If stripped of comments the final code is 54 lines (of a maximum 200 allowed) of virtually indecipherable opcodes and registers, but the process it represents is simple enough.

The classic way to handle moving objects in game programming is to adjust their position each update cycle by adding on a value representing their velocity. This technique doesn’t work for objects that are to be handled entirely through AGAL shaders as there is simply no place where their position could be updated by the shader and stored between frames.

Instead the shader needs to be able to produce the particle’s position when provided with the current time. So the process of writing the shader starts with finding an equation in terms of time that gives position (or rotation, colour, scale, or any other property of your particles that you want to have change). If the particles are moving at a constant velocity this is simple enough:

position = startPosition + startVelocity * time

The variables like startPosition and startVelocity will be passed into the shader through the vertex buffer (as they will be different for each particle), while time will be passed in through the shader constants as it is the same for all particles being drawn that frame. If you want the particles to undergo acceleration (such as due to gravity) then you can just add on a new term to account for that.

position = startPosition + startVelocity * time
           + constantAcceleration * time^2

Effects like the particles bouncing once when reaching a certain y-value are a little more complex but it still all comes down to making an equation that gives the position of the particle in terms of time.

Bonus Content!

Not particularly related to particle systems, but I decided to reproduce an old example of normal mapping I made back when Pixel Bender was the closest to hardware acceleration Flash had available. This version is of course running entirely on the GPU.

]]> http://www.saltgames.com/2011/accelerated-particle-effects/feed/ 2 http://www.saltgames.com/2011/simple-annotated-stage3d/ http://www.saltgames.com/2011/simple-annotated-stage3d/#comments Wed, 05 Oct 2011 17:08:03 +0000 Sam Driver http://www.saltgames.com/?p=539 Continue reading →]]> The first step is often the hardest. Here I provide a very simple example of rendering using Stage3D, complete with extensive comments.

All this does is draw a single multicoloured triangle on the screen. Not much, but it’s a good simple place to start.

This example makes use of the MiniAGALAssembler, which is a neat little class allowing the creation of AGAL shader code from within our Actionscript. It is available from the original source (bundled with a couple of other things), or mirrored on this site.

My apologises for the slightly ugly way the code gets displayed on this page. If you prefer it is also available as a FlashDevelop project, with the included Actionscript files of course readable in any text editor.

Main.as

package
{
import flash.display.Sprite;
import flash.display.Stage3D;
import flash.display3D.Context3D;
import flash.display3D.Context3DProgramType;
import flash.display3D.Context3DVertexBufferFormat;
import flash.display3D.Program3D;
import flash.events.Event;
import flash.geom.Matrix3D;

/**
* @author Salt
* www.saltgames.com
*/
public class Main extends Sprite
{
  private const renderAreaWidth:int = 600;
  private const renderAreaHeight:int = 400;

  private var myContext3D:Context3D;
  private var myStage3D:Stage3D;

  private var aListOfBundlesToRender:Vector.<BundleToRender>;

  public function Main():void
  {
   if (stage) init();
   else addEventListener(Event.ADDED_TO_STAGE, init);
  }

  private function init(e:Event = null):void
  {
   removeEventListener(Event.ADDED_TO_STAGE, init);

   // Start here <<

   // to do any rendering a Context3D is needed, and they can only be made by a Stage3D object
   // fortunately the Stage already has a list of Stage3D objects ready for us to use

   myStage3D = stage.stage3Ds[0];
   // creating a Context3D is not instant so we have to wait for an event telling us when it is ready
   myStage3D.addEventListener(Event.CONTEXT3D_CREATE, context3DCreated);
   myStage3D.requestContext3D();
  }

  private function context3DCreated(e:Event):void
  {
   // as the CONTEXT3D_CREATE event has triggered, we know that myStage3D has finished creating its Context3D
   myContext3D = myStage3D.context3D;

   // set up the back buffer of this Context3D, this is the 'canvas' which will be displayed.
   var antiAliasLevel:int = 0;	// valid values are 0, 2, 4, 8, 16
   var enableDepthAndStencilBuffers:Boolean = true;
   myContext3D.configureBackBuffer(renderAreaWidth, renderAreaHeight, antiAliasLevel, enableDepthAndStencilBuffers);

   // if you want the rendered area to be offset from the top-left corner, the position of myStage3D can be changed
   myStage3D.x = 0;
   myStage3D.y = 0;

   // myContext3D is now ready for rendering to take place on it
   addEventListener(Event.ENTER_FRAME, perFrame);

   // create a test item to render
   aListOfBundlesToRender = new Vector.<BundleToRender>;
   aListOfBundlesToRender.push(createTestBundleOfTrianglesToRender());
  }

  private function perFrame(e:Event):void
  {
   performRender();
  }

  private function performRender():void
  {
   // the back buffer must be cleared before we start trying to draw new stuff on it
   // here you can specify what it should be cleared to - including the depth and stencil buffers
   myContext3D.clear();

   // if you want your view of the world to be changed, simply translate the cameraMatrix
   var cameraMatrix:Matrix3D = new Matrix3D();

   // the cameraMatrix and orthographicProjectionMatrix are combined to give a complete transformation matrix
   // this matrix describes how each vertex in the world should be transformed (moved) to appear in the correct place on the screen
   var completeProjectionMatrix:Matrix3D = new Matrix3D();
   completeProjectionMatrix.append(cameraMatrix);
   completeProjectionMatrix.append(createOrthographicProjectionMatrix(renderAreaWidth, renderAreaHeight, 0, 1));

   // now that the transformation matrix has been created, it is loaded into the vertex constant register vc0,
   // ready to be accessed by our vertex shader program.
   myContext3D.setProgramConstantsFromMatrix(Context3DProgramType.VERTEX, 0, completeProjectionMatrix, true);

   for each (var bundleToRender:BundleToRender in aListOfBundlesToRender)
   {
    // set the shader program
    var shaderProgram:Program3D = myContext3D.createProgram();
    shaderProgram.upload(bundleToRender.vertexShaderAssembler.agalcode, bundleToRender.fragmentShaderAssembler.agalcode);
    myContext3D.setProgram(shaderProgram);

    // load vertex position into the vertex registers
    myContext3D.setVertexBufferAt(0, bundleToRender.vertexBuffer, 0, Context3DVertexBufferFormat.FLOAT_2);
    //                                                                                            ^ load a pair of values for each vertex - x and y position
    //                            ^ load into the va0 (vertex attribute zero) register

    // load vertex colour into the vertex registers
    myContext3D.setVertexBufferAt(1, bundleToRender.vertexBuffer, 2, Context3DVertexBufferFormat.FLOAT_3);
    //                                                                                            ^ load three values for each vertex - r,g,b to define a colour
    //                                                            ^ skip over the first two values for each vertex (those two values are the x and y coordinates)
    //                            ^ load into the va1 (vertex attribute one) register

    // actually perform the rendering operation
    myContext3D.drawTriangles(bundleToRender.indexBuffer, 0, bundleToRender.numberOfTriangles);
   }

   // once all the rendering is complete, present the back buffer so that the user actually sees it
   myContext3D.present();
  }

  private function createOrthographicProjectionMatrix(viewWidth:Number, viewHeight:Number, near:Number, far:Number):Matrix3D
  {
   // this is a projection matrix that gives us an orthographic view of the world (meaning there's no perspective effect)
   // the view is defined with (0,0) being in the middle,
   //	(-viewWidth / 2, -viewHeight / 2) at the top left,
   // 	(viewWidth / 2, viewHeight / 2) at the bottom right,
   //	and 'near' and 'far' giving limits to the range of z values for objects to appear.
   return new Matrix3D(Vector.<Number>
   ([
    2/viewWidth, 0, 0, 0,
    0, -2/viewHeight, 0, 0,
    0, 0, 1/(far-near), -near/(far-near),
    0, 0, 0, 1
   ]));
  }

  private function createTestBundleOfTrianglesToRender():BundleToRender
  {
   var testBundle:BundleToRender = new BundleToRender();

   // generate the vertex and index buffers for a single triangle
   // in this case the vertex buffer will include both the position and colour data
   // if we were to be using texturing, uv coordinates would be loaded in the vertex buffer too

   // first prepare the vertex buffer, specifying its dimensions
   var verticesForATriangle:int = 3;
   var numberOfTriangles:int = 1;
   var dataCountForEachVertex:int = 5; // for x, y, red, green, blue
   testBundle.vertexBuffer = myContext3D.createVertexBuffer(verticesForATriangle * numberOfTriangles, dataCountForEachVertex);

   // similarly prepare the index buffer
   var numberOfIndicesForATriangle:int = 3;
   testBundle.indexBuffer = myContext3D.createIndexBuffer(numberOfIndicesForATriangle * numberOfTriangles);

   // create the vertex data that will be loaded into the buffer
   var vertexData:Vector.<Number> = new Vector.<Number>();
   //                x     y   r  g  b
   vertexData.push(   0, -100, 1, 0, 0);
   vertexData.push(-150,  150, 0, 1, 0);
   vertexData.push( 200,   50, 0, 0, 1);

   // load the vertex data to the buffer
   testBundle.vertexBuffer.uploadFromVector(vertexData, 0, verticesForATriangle * numberOfTriangles);

   // create the index data to be loaded in
   var indexData:Vector.<uint> = new Vector.<uint>();
   indexData.push(0, 1, 2);

   testBundle.numberOfTriangles = numberOfTriangles;

   // load the index data to the buffer
   testBundle.indexBuffer.uploadFromVector(indexData, 0, numberOfIndicesForATriangle * numberOfTriangles);

   // create both the vertex and fragment shader programs to be used when rendering this bundle of triangles
   // the vertex data created above will be available to these shaders through registers:
   // the first 2 values for each vertex (the x and y position) will appear in va0
   //	the remaining 3 values for each vertex (r, g, b for colour) will appear in va1
   testBundle.setVertexShaderCode(
    "m44 op, va0, vc0 \n" + // apply the matrix transform to va0 (vertex position), putting result in op (the position output)
    "mov v0, va1"           // move va1 (vertex colour) to v0 where the fragment shader will be able to access it
    );

   testBundle.setFragmentShaderCode(
    "mov oc, v0"				// simply put the vertex colour directly into oc (the colour output)
    );

   return testBundle;
  }
}
}

BundleToRender.as

package
{
import flash.display3D.Context3DProgramType;
import flash.display3D.IndexBuffer3D;
import flash.display3D.VertexBuffer3D;
/**
* @author Salt
* www.saltgames.com
*/
public class BundleToRender
{
  // I use the not entirely technical term of "bundle" for a group of triangles being rendered all in one batch.

  public var vertexShaderAssembler:AGALMiniAssembler;
  public var fragmentShaderAssembler:AGALMiniAssembler;

  // the vertexBuffer stores all the per-vertex data
  public var vertexBuffer:VertexBuffer3D;

  // the indexBuffer is used to determine how the vertices should be connected to form triangles
  public var indexBuffer:IndexBuffer3D;

  // the number of triangles for which we have data, and we want to draw.
  public var numberOfTriangles:int;

  public function BundleToRender()
  {

  }

  public function setVertexShaderCode(vertexCode:String):void
  {
   vertexShaderAssembler = new AGALMiniAssembler();
   vertexShaderAssembler.assemble(Context3DProgramType.VERTEX, vertexCode);
  }

  public function setFragmentShaderCode(fragmentCode:String):void
  {
   fragmentShaderAssembler = new AGALMiniAssembler();
   fragmentShaderAssembler.assemble(Context3DProgramType.FRAGMENT, fragmentCode);
  }
}
}

]]> http://www.saltgames.com/2011/simple-annotated-stage3d/feed/ 3 http://www.saltgames.com/2011/stage-3d-shader-cheatsheet/ http://www.saltgames.com/2011/stage-3d-shader-cheatsheet/#comments Wed, 05 Oct 2011 15:08:44 +0000 Sam Driver http://www.saltgames.com/?p=523 Continue reading →]]>

A quick reference for working with AGAL, the new shader language for use with Stage3D introduced with Flash Player 11.

I’ve been working on a series of tutorials for Stage3D which aren’t yet ready, but with the recent release of Flash Player 11 now seems a good time to get at least some information out. If you’re new to Stage3D then this may not make much sense until you’ve read some other resources, but I hope it serves as a useful reference as you experiment and learn.

Registers:
Each register consists of four components, which are floating point values. These components are accessed by registerName.x, registerName.y, registerName.z, registerName.w. They’re named for dealing with 3D positions – with the ‘w’ for rotation in the style of quaternions – but they can just as well be used to hold a colour (in fact you can also access the components with .r .g. b. a) or any other values you want to use.

There are several registers of each type available. For instance you might have va0 giving the 3D position of a vertex in space, and va1 giving the uv mapping coordinate for that vertex. The nice thing about having registers made up of components is you can do things like perform a basic addition operation va0 and vc0, and the addition will be performed correctly on each component.

Registers for Vertex Shaders

• va[0 to 7] Vertex Attribute. The contents of the vertex buffer, as set with context3D.setVertexBufferAt. Each vertex has its own space in the vertex buffer which only it can access.
• vc[0 to 127] Vertex Constant. Passed into the shader with context3D.setProgramConstantsFromVector or context3D.setProgramConstantsFromMatrix. These registers can be read by all vertices, but cannot be written to by the shader.
• vt[0 to 7] Vertex Temporary. A handy temporary register where you can put values during a calculation.
• op Vertex Output or “Output Position”. The output: op.x and op.y is where in the 2D space of the screen this vertex will be drawn. The op.z value is used for depth checking and writing to the depth buffer if you have those enabled. So far as I know the op.w value is not actually used.

• v[0 to 7] Varying. A magical (not actually magical) register that allows you to pass values from the vertex shader to the fragment shader. The value that arrives in the fragment shader will be interpolated between the value of the three v registers of the vertices which make up the triangle in which the fragment falls.
  Note!  The fragment shader cannot directly access the vertex buffer, so anything it needs from there has to be passed through the v registers. For instance if you’re using texture mapping this register will need to pass the uv coordinates to the fragment shader.

Registers for Fragment Shaders

• fc[0 to 27] Fragment Constant. Much like vc for the vertex shader, this register is set by context.setProgramConstantsFromVector or context.setProgramConstantsFromMatrix, can be read by each fragment, and not written to by the shader.
• ft[0 to 7] Fragment Temporary. Again just like vt for the vertex shader, this is a temporary store useful for performing calculations.
• fs[0 to 7] Texture Sampler. This is where the fragment shader is able to access whatever texture(s) were bound using context3D.setTextureAt.
• oc Fragment Output or “Output Colour”. The output: oc.x oc.y oc.z oc.w are the red, green, blue and alpha values respectively for the fragment to be drawn.

Operations:
Shaders are made up of a series of operations, with one operation on each line. First the operation to perform is identified by a three letter opcode such as “add”, then the parameters for that operation are given. The parameters are (almost) always specified as registers. If you want to use a number in your shader, it should be supplied through the vc or fc registers.

In the parameters the target register is always specified first. The target register is where the result of the operation is placed. No change is made to a register other than the target register. Some operations require two further parameters, others just one. The tex operation used for texture sampling is a special case that has six parameters, three of which are given as strings rather than registers. tex is a pretty wild guy.

At first look the mess of opcodes and registers names that make up the AGAL code for a shader can look intimidating but they’re actually quite simple. Just remember that each operation does exactly one thing, and writes to exactly one register. AGAL doesn’t allow for conditional statements like if then or any form of looping, so following along with what a shader is doing is extremely easy: it always just proceeds to the next operation.

With that said, AGAL code is not nearly as intuitive to glance at and understand what it does as (well written) AS3 code is. Taking a minute to type out some comments for the AGAL code you write is a very good idea.

You are limited to 200 operations in a single AGAL shader.

m44 op, va0, vc0
mov v0, va1

tex oc, v0, fs0 <2d,clamp,linear>

A practical test of Pixel Bender powered normal mapped lighting in a game setting, in contrast to a less practical but more pretty example.

W, A, S and D to move the player ship, mouse to shoot. Press Q to spawn a group of enemies.  Be aware – there is sound in this example!

A single light source follow’s the player, and additional light sources are created for each enemy destroyed. All light sources illuminate a generated layer of textured and bump mapped rock.

To attempt to match the various performance requirements of a finished game, there is a simple particle animation system, enemy AI and parallax background. A circular distortion effect also powered by Pixel Bender can be seen if an enemy collides with the player, or the player detonates a bomb using Spacebar.

The lighting effect can be disabled from the menu accessible by right-clicking while the game is running.

The source code for this simple example is available here. Although be aware it uses the Box2D physics library for handling the shapes, so some of the source may be confusing if you’re not familiar with that library.

A demonstration of the 2D shadow effects that I discuss in a tutorial on the TIGSource forums that I wrote a while ago.  Just a series of rooms that you can walk through. No source code for this one because frankly it’s a bit of a mess. It’s the same technique as used in the first example, just applied to a more overtly game-like setting.

W, A, S and D to move, space bar to toggle the light on your faerie companion.

A quite detailed tutorial on creating custom shaders in Pixel Bender and using them in your Flash projects.

This is quite a lengthy tutorial.  It includes a brief introduction to pixel shaders in general; a tour of the most important aspects of Pixel Bender for use creating shaders specifically for Flash, directions for integrating the custom shader you create into a Flash project, and example source code for two simple examples.

Some Introductions

Shaders, specifically pixel shaders.

These are programs that operate on an image on a per-pixel basis.  Per-pixel means the program operates on each pixel individually, without reference to operations carried out on other pixels.  Because shaders operate in this per-pixel manner they very well suited to parallel processing: Having many instances of the shader operating at the same time, each on their own pixel.  Operating per-pixel means that there are certain tasks that are well suited to being performed by a shader as well as many tasks that are completely unsuitable.

Filters, Effects, Blendmodes, Kernels.

There’s a lot of terminology floating around depending on where you read about stuff.  I’m going to call everything a Shader, because that’s the name of the class I end up using in Flash for everything (not the best method for naming things, but still.) To give a very rough idea to help you follow other documents: Filters and Effects are applied to just one image; Blendmodes are applied to two (or more, in theory?) images; a Kernel is the thing you make when you type code into Pixel Bender.

Pixel Bender.

The Pixel Bender Toolkit, to give it its full name, is a neat program made by Adobe to produce custom filters and effects for use in a variety of their applications.  You should be aware when reading other sources for information on Pixel Bender that many of its functions will just not function in Flash.  Pixel Bender is available for free from Adobe, look to the right of that page.  Pixel Bender was known as Hydra when it was earlier in development, so you may see that name crop up around the place too.

Here be monsters

Using Pixel Bender to make shaders for Flash is – to put it politely – prone to inconsistencies.  The Pixel Bender Toolkit lets you run your shader within it to test it.  What you see in the preview will quite often not match what comes out in Flash.  I’m attempting to build up a record of ways in which the two do not match but it’s far from complete.  Just be aware that you should test primarily in Flash and not rely on the inbuilt preview.  Since I first wrote this tutorial the Pixel Bender Toolkit now includes an option to simulate running a shader in Flash which is helpful, but I suspect still doesn’t catch everything.  Also be sure to regularly check for an updated version from Adobe – I’ve found myself stuck trying to work around some bug which has actually already been fixed if I’d just updated.

What’s the point

So Pixel Bender can make buggy programs that are limited to operating on images in a per-pixel basis.  Why am I bothering learning this?

Because it’s fast!

You might like to look at a normal mapping example I put together recently.

Flash programs are pretty slow generally, and are limited to executing in a single thread.  Consider if you want to tint an image red in Flash.  If you’re charmingly naïve you might write something like:

for (var x:int = 0; x < image.width; x++)
{
   for (var y:int = 0; y < image.height; y++)
   {
      var colour:uint = image.getPixel(x, y);
      colour = (colour | 0xff0000); //set the red channel to 255
      image.setPixel(x, y, colour);
   }
}

This will be slow.  It’ll be slow because of the BitmapData.getPixel and BitmapData.setPixel operations being called huge numbers of times, but on a more general level it’s slow because you’re visiting each pixel in turn when you could be performing that operation on all the pixels at once.  This is effectively what a custom shader from Pixel Bender does.

Your shader will not actually operate on all the pixels at once, as it’s limited by the number of threads it can execute on the machine its on.  But thinking of it as operating on every pixel at once is a useful mental model.

Pixel Bender was designed to run on GPUs, which are especially well suited to performing many parallel tasks.  When running in Flash it is actually limited to using the CPU.  This means it’s not nearly as fast as it could be, but it still has the potential to be many times faster than plain old actionscript.

To the Pixel Bender!

Let’s get right in and actually make a shader and get it working in Flash.

Download and install Pixel Bender.

Set a couple of options within Pixel Bender (Edit menu, Preferences…) to tell it you’ll be exporting to Flash:

Select New Kernel from the File menu.

You’ll now have the code for a shader that does nothing.  Excellent!

<languageVersion : 1.0;>
kernel NewFilter
<  namespace : "Your Namespace";
   vendor : "Your Vendor";
   version : 1;
   description : "your description";
>
{
   input image4 src;
   output pixel4 dst;
   void
   evaluatePixel()
   {
      dst = sampleNearest(src,outCoord());
   }
}

Oh no, this isn’t Actionscript!  Time to learn a new language.  Good news is it’s easy, and you can ignore most of it.

So what actually matters here?

input image4 src;

This declares a variable of the type image4 named src, and says that this variable will hold the input that our shader is given.  Later we’ll see how a shader can have multiple input images.

The type image4 just means that it’s an image with 4 colour channels: Red, green, blue and alpha.  There is also image3 if you want to ignore the alpha channel.

output pixel4 dst;

This declares a variable of the type pixel4 named dst, and says that whatever value we put in this variable will be the output of the shader.  pixel4 is a single pixel with 4 colour channels.

But wait!  We’re taking a whole image as input, and only giving a single pixel as output?  Shouldn’t this be producing a whole image?

I was just getting to that.

The evaulatePixel() function is the heart of our shader.  It is executed once for each pixel of the source image.  It produces just a single pixel – our output variable named dst – which is then built up to form the finished image.

void
evaluatePixel()
{
   dst = sampleNearest(src,outCoord());
}

Here’s the evaluatePixel function in full.  The only bit you ever need change is between the curly braces.  In this case it takes the result of the built-in function sampleNearest and assigns that to dst.  Note that unlike actionscript functions where you use ‘return’ to have a function produce a value, evaluatePixel just sets the value of the variable that we’ve previously defined as the shader’s output.

What does sampleNearest do?  Press F1 in Pixel Bender to get the documentation.  The majority of the documentation doesn’t apply to us (most sections end with “Flash player note: X is not supported in Flash Player.”)  But it has lists of built-in functions and data types, most of which are supported.  sampleNearest gives us the value of a single pixel from the image passed as the first parameter, which in this case is src.  Which pixel to return is determined by the second parameter, which in this case is the result of another built-in function: outCoord. This function returns a float2 value giving the position of the pixel that in this instance evaluatePixel was called on.

Vector data types

Unlike actionscript, Pixel Bender uses something called vector data types.  These store several values within one variable.  For instance a position in two-dimensional space can be specified by a float2 value, which is just two floating point values (basically the same as number in actionscript) joined together.  You can access the individual values, or operate on them as a whole.  Let’s take an example:

void
evaluatePixel()
{
   pixel4 myPixelVariable;
   myPixelVariable = sampleNearest(src, outCoord());
   dst = myPixelVariable * 2.0;
}

First new thing:

pixel4 myPixelVariable;

Simply declares a variable named myPixelVariable which is of type pixel4.

myPixelVariable = sampleNearest(src, outCoord());

As you’d expect we can assign a value to it.

dst = myPixelVariable * 2.0;

Then we can multiply the whole thing by 2, and assign that to our output variable.

This is exactly the same as performing:

dst.r = myPixelVariable.r * 2.0;
dst.g = myPixelVariable.g * 2.0;
dst.b = myPixelVariable.b * 2.0;
dst.a = myPixelVariable.a * 2.0;

Isn’t that neat.

myPixelVariable.r is accessing one of the four elements that make up the variable.  The elements can also be accessed by:

myPixelVariable[0], myPixelVariable[1], myPixelVariable[2], myPixelVariable[3]

myPixelVariable.x, myPixelVariable.y, myPixelVariable.z, myPixelVariable.w

myPixelVariable.s, myPixelVariable.t, myPixelVariable.u, myPixelVariable.v

The suffixes [0], .r, .x and .s are all equivalent.

You can access several elements of a vector-type variable at once.

myPixelVariable.rg *= 2.0;

Will multiply just the red and green channels by two.

You can also assign values between elements of a vector-type:

myPixelVariable.rgb = myPixelVariable.brg;

This assigns the blue value of the pixel to be the red value, the red to the green and the green to the blue.

You can specify each element of a vector-type like so:

myPixelVariable.rgb = float3(0.5, 0.2, 0.8);

Fussy about types

Notice that in the examples above we multiply by 2.0 rather than just 2.  This is because unlike actionscript, Pixel Bender doesn’t do implicit conversion.  What that means is that it will interpret 2 as an integer, and not know how to multiply that by the float values of a pixel4 variable.  Writing the value as 2.0 makes it clear that you want it treated as a float. You could also use float(2) to explicitly convert the integer into a float.

Slightly weirdly there seems to be little difference between pixel4 and float4.  I think there’s a difference in how accurately the values are stored, but they are largely interchangeable.

A working shader

Let’s pull that together into a simple working shader so we can move on to implementing it in Flash:

kernel NewFilter
<    namespace : "Your Namespace";
     vendor : "Your Vendor";
     version : 1;
     description : "your description";
>
{
    input image4 src;
    output pixel4 dst;

    void
    evaluatePixel()
    {
        dst = sampleNearest(src,outCoord());
        dst.r *= 2.0;
    }
}

This will just take each pixel from an image and double the red channel.

We can give this a quick test within Pixel Bender.  Just Load Image 1… from the File menu and select any old photo you have laying around. Then hit the Run button (or F5) and the photo will be displayed with a red tint.

Now choose Export Filter for Flash Player… from the File menu.  This will create a .pbj file containing Pixel Bender byte code which Flash will be able to understand.  Whenever you change a shader you’re working on, remember you’ll need to re-export it to have those changes show up in Flash.

And in to Flash

Once you know how, getting the shader in to your Flash program is actually quite simple.  The basic technique is very similar to embedding an image, font or sound:

[Embed(source = 'TestShader.pbj', mimeType = 'application/octet-stream')]
private var TestShader:Class;

And that’s it embedded! To create an instance of the shader:

var myShader:Shader = new Shader(new TestShader() as ByteArray);

The shader we’ve created requires an input image, so we need to point myShader at an image to use as its input.

myShader.data.src.input = myBitmapData;

This assumes we already have an image loaded and named myBitmapData. Note that the src corresponds to the name we gave the input variable in Pixel Bender.

To actually run the shader, we make use of a ShaderJob.

myJob:ShaderJob = new ShaderJob(myShader, outputBitmapData);
myJob.start(true);

Again this assumes we already have a bitmapData object named outputBitmapData which is ready to receive the finished image (it should be created as the same size as the input image.)  The true value passed when calling start on myJob tells Flash you want to wait until the job is complete before continuing.  It’s possible to pass false instead and have the shader execute asynchronously, but that brings up a whole lot of other issues and a pretty major bug.

Putting this all together into a program that’ll actually work:

package
{
	import flash.display.Bitmap;
	import flash.display.BitmapData;
	import flash.display.Shader;
	import flash.display.ShaderJob;
	import flash.display.Sprite;
	import flash.utils.ByteArray;

	public class PrettyShader extends Sprite
	{
		//embed the shader
		[Embed(source = 'TestShader.pbj', mimeType = 'application/octet-stream')]
		private var TestShader:Class;

		//embed an image to test it on
		[Embed(source = 'SomeImage.jpg')]
		private var SomeImage:Class;

		public function PrettyShader()
		{
			//set up the input and output BitmapDatas
			var myBitmapData:BitmapData = (new SomeImage() as Bitmap).bitmapData;
			var outputBitmapData:BitmapData = new BitmapData(myBitmapData.width, myBitmapData.height);

			//set up the Shader
			var myShader:Shader = new Shader(new TestShader() as ByteArray);
			myShader.data.src.input = myBitmapData;

			//create the ShaderJob
			var myJob:ShaderJob = new ShaderJob(myShader, outputBitmapData);

			//and run it!
			myJob.start(true);

			//display the input and output.
			var before:Bitmap = new Bitmap(myBitmapData);
			addChild(before);
			var after:Bitmap = new Bitmap(outputBitmapData);
			after.x = before.width;
			addChild(after);
		}
	}
}

Remember to set your compiler to target Flash Player 10, and you should end up with something rather like this:

We’re actually almost done now!  There are two more things to cover:  multiple image inputs, and parameters.

Multiple images as inputs

The example above just takes one image and fiddles with the colour a little.  You can do much more powerful things if you work with more than one image.

In Pixel Bender all we need to do is add an extra input statement for each image we want to sample from.  For example:

kernel NewFilter
<    namespace : "Your Namespace";
     vendor : "Your Vendor";
     version : 1;
     description : "your description";
>
{
    input image4 srcOne;
    input image4 srcTwo;
    output pixel4 dst;

    void
    evaluatePixel()
    {
        float2 positionHere = outCoord();
        pixel4 fromOne = sampleNearest(srcOne, positionHere);
        pixel4 fromTwo = sampleNearest(srcTwo, positionHere);
        float crossfade = positionHere.x / 400.0;
        dst = fromOne * crossfade + fromTwo * (1.0 - crossfade);
    }
}

This will sample from both images, and mix them together depending on the horizontal position of the pixel in question. You might notice that I create a float2 variable named positionHere to store the result of outCoord(), this is simply to save calling the function multiple times when I know it’ll give the same result.  Unnecessary premature optimisation or just being tidy?  You decide!

The changes in Flash are just as simple.  Simply assign another BitmapData as the shader’s second input:

myShader.data.srcOne.input = myBitmapData;
myShader.data.srcTwo.input = myOtherBitmapData;

Parameters
Parameters are simply variables that are used within the shader and can be set from within Flash.  Like input images and the output pixel, parameters are defined outside of the evaluatePixel function.  As well as specifying the data type that the parameter should be you give minimum, maximum and default values.

Here is the above shader amended with two parameters:

kernel NewFilter
<    namespace : "Your Namespace";
     vendor : "Your Vendor";
     version : 1;
     description : "your description";
>
{
    input image4 srcOne;
    input image4 srcTwo;
    output pixel4 dst;

    parameter float middleOfFade
    <
         minValue: -1.0;
         maxValue: 1.0;
         defaultValue: 0.0;
    >;

    parameter int widthOfImage
    <
         minValue: 0;
         maxValue: 5000;
         defaultValue: 400;
    >;

    void
    evaluatePixel()
    {
        float2 positionHere = outCoord();
        pixel4 fromOne = sampleNearest(srcOne, positionHere);
        pixel4 fromTwo = sampleNearest(srcTwo, positionHere);
        float crossfade = middleOfFade + (positionHere.x / float(widthOfImage));
        if (crossfade > 1.0) crossfade = 1.0;
        if (crossfade < 0.0) crossfade = 0.0;
        dst = fromOne * crossfade + fromTwo * (1.0 - crossfade);
    }
}

In this case we use two parameters, for two quite different purposes. You might have noticed that the previous versions relied on a constant value of 400.0 in the calculation of crossfade. This was the width of the image in use, but of course our shader should be ready to accept other sized images too so the parameter widthOfImage was added.  Secondly the parameter middleOfFade is added to allow interactive changes to how the two source images are mixed.

As a side note, Pixel Bender actually includes some built-in functions for determining the size of input images, but they are not available when exporting for the Flash player.

If you test this shader in Pixel Bender it will display some sliders that let you alter your parameters on the fly and see what effect they have.

Passing values to your shader from Flash is again fairly simple, although slightly obscure.

myShader.data.middleOfFade.value = [0.2];

Notice that you use .value when setting a parameter, but .input when setting an input image.  You also have to put your value inside an array first.  It makes more sense if you need to pass values into a float3 variable:

myShader.data.someFloat3Variable.value = [0.1, 0.6, 0.3];

All together now
A working Flash program that uses a shader with two image inputs and parameters.

package
{
	import flash.display.Bitmap;
	import flash.display.BitmapData;
	import flash.display.Shader;
	import flash.display.ShaderJob;
	import flash.display.Sprite;
	import flash.events.Event;
	import flash.utils.ByteArray;

	public class PrettyShader extends Sprite
	{
		//embed the shader
		[Embed(source = 'TestShader.pbj', mimeType = 'application/octet-stream')]
		private var TestShader:Class;

		//embed the images to test it with
		[Embed(source = 'SomeImage.jpg')]
		private var SomeImage:Class;
		[Embed(source = 'SecondImage.jpg')]
		private var OtherImage:Class;

		private var myShader:Shader;
		private var outputImage:BitmapData;

		private var count:Number = 0;
		private const twoPi:Number = Math.PI * 2;

		public function PrettyShader()
		{
			//set up the input and output BitmapDatas
			var firstImage:BitmapData = (new SomeImage() as Bitmap).bitmapData;
			var secondImage:BitmapData = (new OtherImage() as Bitmap).bitmapData;
			outputImage = new BitmapData(firstImage.width, firstImage.height);

			//set up the Shader
			myShader = new Shader(new TestShader() as ByteArray);
			myShader.data.srcOne.input = firstImage;
			myShader.data.srcTwo.input = secondImage;

			//set parameters
			myShader.data.widthOfImage.value = [firstImage.width];
			myShader.data.middleOfFade.value = [0.0];

			//display the output.
			addChild(new Bitmap(outputImage));

                        //update every frame
			addEventListener(Event.ENTER_FRAME, perFrame);
		}

		private function perFrame(e:Event = null):void
		{
			//update realtime parameter
			count += 0.05;
			if (count > twoPi) count -= twoPi;
			myShader.data.middleOfFade.value = [Math.sin(count)];

			//create the ShaderJob
			var myJob:ShaderJob = new ShaderJob(myShader, outputImage);

			//and run it!
			myJob.start(true);
		}
	}
}

Here it is running:

I think that is about all for now.  There is plenty more to learn about both Pixel Bender and using shaders in Flash, but this should give you everything you need to get some cool stuff working.

Of course making those awesome shaders is another topic altogether.

As a final note, keep in mind that it is possible to use custom shaders as BlendModes or Filters directly in Flash.  I have found that doing so ended up being more complex (or at least confusing), didn’t allow for the creation of anything that couldn’t be achieved through the use of Shader and ShaderJob, and may even run slightly slower.

Use the mouse to position the lightsource.  This demo’s normal map was made by following an excellent tutorial on producing normal maps from photographs by Ryan Clark.  The shader is of my own design but is nothing special at all, and I think it’s slightly buggy still working correctly.

On my machine this runs at a capped 50fps, whilst trying to do the exact same pixel calculations ‘manually’ with bitmapData methods manages 2 to 3 fps.  I would post a SWF of the manual version too but it has a decent chance of timing out the flash plugin.

This is all thanks to the magic and wonder of Pixel Bender, which lets you write your own shaders/filters/blendmodes for use in a range of Adobe’s products – including Flash.  Basically it lets you write an operation that will be executed on every pixel in a given image.  The good bit is that the execution of that operation will take advantage of the parallel nature of the task and split its operation into threads, so showing a huge increase in performance from the single-thread bound Flash Player.  You may also read about how Pixel Bender pushes its calculations on to the GPU, but it actually doesn’t do that when executing in Flash.

Sadly a significant amount of its features are disabled for Flash, and there are still plenty of oddities and inconsistencies that even in my brief testing I’ve come up against.  Despite that, I think it’s worthy of investigation simply because of the huge increase in performance it can give – it’s also free and works fine without needing to use the Flash IDE.

Once I’m a little more certain that I’m doing things right I’ll write up an introduction tutorial for using Pixel Bender in Flash projects.

Edit: Here it is.

A technique for automatically selecting the correct tile from a tilemap.

I originally wrote this as a reply to a question on TIGSource and I figure it’s worthy of expanding and repeating here.

The problem: We have generated a neat platformer level, and want to be able to automatically lay neighbour-sensitive tiles over it so that they fit properly.

Neighbour Sensitive Tiles

Super Mario’s tiles are not neighbour sensitive;  the stone block always looks the same if it’s floating alone or if it’s part of a wall.

That’s fine for many games, but can look out of place if you’re trying to create a more organic feel in a level, or just wanting to cut down on repeated patterns.  Neighbour sensitive tiles overcome this by matching their appearance to the presence of their neighbours.

A 1-bit Map

Let’s imagine that with some clever techniques we have generated a level layout for a platformer game that consists of either stone or empty air at each grid location.  This can be represented as a 1-bit image – that is each pixel’s state will be determined by a single bit, so either on indicating rock or off to indicate open air.  Here’s a section of such a map, greatly enlarged and with grid lines added:

A Tileset

A tile set is simply a collection of the graphics that may be used to fill in a map.  Mario’s tileset is quite small, consisting of just a few different types of block and scenery, whilst ours will have many graphics for the same type of tile:

Valuing Neighbours

To determine which tile should go at each spot on the map, we shall examine that spot’s immediate neighbours (for now we’ll ignore diagonal neighbours.)  Rather than writing a huge if/else-if statement to handle all the possible combinations of neighbours, we will use a system that assigns values to each direction.

A value is found for each location by looking at its neighbours and adding up the values for whichever ones have stone present.  So if the spot we’re examining has a neighbour above it that is also filled with stone then it is given the value 1.  If it has stone-filled neighbours above and below then the value is 1 + 4, so 5.

You may notice that the values assigned to the directions are the same as the values of the positions in a binary number and this is no surprise; both are ways of representing possible combinations of 4 units which can each be in an on or off (stone or air) state.

Here is the map segment with the value of each tile filled in.  You might like to manually work out the value for a couple of the tiles yourself so you’re clear on how it works.

Adding the Tiles

The order in which the tileset is laid out above is not arbitrary; it is laid out so that each tile matches what would be expected of a tile in the map that would be assigned that value.  Once we have a value assigned for each spot on the map we just need to look up that number in the tileset and place the correct tile there:

Wonderful!

Taking it Further

Part One – Removing Air

Whilst the above works very neatly for platforms floating in air, it doesn’t yet fully handle two tile types.

Instead of a platformer imagine we’re working on a top-down 2D RTS with two tile types of grass and water.  In such a case we would expect every spot on the map to have a tile graphic present; there’s no open spaces like in our platformer.  This means that every spot on the map will need to have a value generated from its neighbours to determine which tile fits there.

We can use exactly the same neighbour valuing system as before, but we now need a way to differentiate between there being grass or water at the spot being examined itself.  This is actually very simply done – just add an extra value for the location itself, following the same 2 to the power of n pattern from the other values:

Let’s decide that the presence of water means you do add on that position’s value, whilst the presence of grass means you do not.  So a grass tile surrounded on all sides by grass would be numbered at 0.  A grass tile with water to the top and left would be 1 + 8 = 9.  A water tile surrounded on all sides by grass would be 16.  A water tile with water on all sides would be 1 + 2 + 4 + 8 + 16 = 31

Part Two – Greater Variety

How do we handle adding additional types of terrain?

Let’s say we have three types of terrain in our top-down game: water, grass, and forest.  As well as water and grass meeting, we must now also deal with water and forest meeting as well as grass and forest meeting.

Previously there was two possibilities for each neighbour position (grass or water) so we used a binary counting system.  There’s now three possibilities so we shall use a trinary counting system.  Our neighbour valuing system needs to be adjusted to match the new counting system:

Just as the binary counting system followed the pattern of 2 to the power of n, this follows the pattern of 3 to the power of n.

Using a trinary system each position has three possible states: grass, water, forest; or 0, 1, 2.  When there is grass in a given position, we ignore the value (multiply it by 0).  When there is water in the position we add on the value given (multiply it by 1).  When there is forest in the we add on twice the value (multiply it by 2.)

So if we had a tile that was forest with water above it, water to the right, forest below and grass to the left: 81 * 3 + 1 * 2 + 3 * 1 + 9 * 3 + 27 * 0 = 275

You might notice at this point that you’ll need to draw 324 tile graphics to cover all the possible combinations in a three-terrain map.  This is a lot of tiles to draw by hand.  I would strongly advise looking into at least partially automated methods of producing the multitude of tile combinations.

Of course the system can be extended in just the same way for greater numbers of terrain types, but the number of tile graphics you would need would be even more huge.  Instead I would recommend placing some limits on what tiles are allowed to neighbour one-another.  For instance if forest and water tiles are never allowed to directly touch then the above example will need several hundred less tile graphics.

The outline for a method to incorporate lock-and-key style progression in a generated game world.

Firstly, a short explanation of what I’m trying to do:  There are plenty of examples of procedurally generated (or “randomly generated”) level content for games, but they often lack a certain type of structure.  Generally they are used to produce very open worlds, where the player is free to explore as they like.  I am trying to make something that will create worlds with gated progression.  At its simplest this is Doom with a locked door that needs the red key which is off on a side path protected by monsters.  Less obviously lock-and-key examples would be games like Zelda and Metroid, where you need to gain certain abilities or special items to reach areas.

Initially I thought to generate a simple world and then develop techniques for adding on progression gates over it.  My first breakthrough came with giving up this idea. Instead I worked on generating the gating mechanisms at the same time as I generate the layout of the world.

The second important factor was to forget about the physical layout of the world.  What actually mattered was the connections of possible-movement between game areas.  So instead of working on a map of the world, I now work on a graph with the nodes as areas of the world and the edges as pathways between those areas that the player can move along.

Here’s a graph representing a very simple world:

The player starts in the node at the red arrow, and wants to get to the node with the chequered flag.  But the pathway is blocked with a lock, so the player must first travel to the node with the key.  Not at all complex, but now the world has some progression structure to it.  This graph’s layout was generated by a general purpose “add a goal, and lock it away” function – before that function was called, the graph consisted just of the starting node.  The neat thing is that the same function can be used to place the key, so creating:

When choosing where to add the node containing the key, the algorithm limits itself to picking amongst those nodes that it knows are accessible to a player who can access the lock.  This means that there was no possibility for the cyan key to be locked behind the pink door, as the cyan lock is accessible to a player with no keys at all and so the cyan key must be too.  The placing of the cyan lock now means that the pink lock/key no longer satisfy these requirements, but so long as they satisfied them when they were placed, and all locks placed afterwards satisfy them, it is guaranteed that with some work the player will be able to open the pink lock.

Here’s where my world-graph generator is at currently.  The player starts at the red node, and wants to explore the world (there are actually multiple specific ‘goal’ nodes in the map, but I figure it has enough confusing colours already.)  The player can freely move along the grey lines, but must have the correctly coloured key to travel along the coloured lines.  The nodes that hold keys are coloured indicating what key they contain. Unlike the above examples the generator produces dummy nodes so that not every node is a key or goal, and may join dummy nodes together if they share the same key requirements.