Joshua's Coding Blog

Graphics and Game Programming

Adding a Lock to the Transform Component in Unity's Inspector

Sometimes I load a scene in Unity3D and I accidentally drag something out of place only to get a bug report much later that something's not aligned correctly, and I have to go fix it. This sucks.

The folks at Unity should just add a lock option to the transform component. But until they do here's how you can add one yourself.

 

The first thing we need is a lock transform component, that if we try to update the transform it just sets it back to what it was.

using UnityEngine;
using System.Collections;

[ExecuteInEditMode]
[HideInInspector]
public class LockTransform : MonoBehaviour {
    private bool prevLocked;

    private Vector3 lockedPosition;

    public bool DestoryNextOpportunity { get; set; }

    // Use this for initialization
	void Start () {
        this.hideFlags = HideFlags.NotEditable;
        lockedPosition = transform.position;

        if (Application.isPlaying)
            Destroy(this);       
	}
	
	// Update is called once per frame
	void Update () {
        transform.position = lockedPosition;
        if (DestoryNextOpportunity)
            DestroyImmediate(this); // Only called from editor
	}
}

So whenever update is called, it puts our transform back, you will also notice that on launching the game this component is destroyed immediately- That's because we might actually want to update our transform at runtime, we just don't want it to change during edit time. So now we can add this component and it will lock our transform.

But how about instead we have a checkbox on the TransformComponent like in the image above?

Well, first we have to overwrite the default transformEditor and build our own.

using UnityEngine;
using System.Collections;
using UnityEditor;

#if UNITY_EDITOR
[CustomEditor(typeof(Transform))]
public class TransformInspector : Editor  {
    public bool locked;
    void OnEnable()  {
        var t = (Transform)target;
        locked = t.GetComponent<LockTransform>() != null;
    }

	public override void OnInspectorGUI()  {
		var t = (Transform)target;
 
		// Reproduce the default look
		EditorGUIUtility.LookLikeControls();
		EditorGUI.indentLevel = 0;
		var position = EditorGUILayout.Vector3Field("Position", t.localPosition);
		var eulerAngles = EditorGUILayout.Vector3Field("Rotation", t.localEulerAngles);
		var scale = EditorGUILayout.Vector3Field("Scale", t.localScale);
                
        // Add custom lock
        locked = EditorGUILayout.Toggle("Lock", locked, new GUILayoutOption[]{});

		if (GUI.changed)  {
			Undo.RecordObject(t, "Transform Change");
 
			t.localPosition = FixIfNaN(position);
			t.localEulerAngles = FixIfNaN(eulerAngles);
			t.localScale = FixIfNaN(scale);

            if (locked)  {
                var lockTransform = t.gameObject.GetComponent<LockTransform>();
                if (lockTransform == null)
                    t.gameObject.AddComponent(typeof(LockTransform));
            }
            else {
                var lockTransform = t.gameObject.GetComponent<LockTransform>();
                if (lockTransform != null)
                    lockTransform.DestoryNextOpportunity = true;
            }
		}
	}
 
	private Vector3 FixIfNaN(Vector3 v) {
		if (float.IsNaN(v.x))
			v.x = 0;
		if (float.IsNaN(v.y))
			v.y = 0;
		if (float.IsNaN(v.z))
			v.z = 0;
		return v;
	}
}
#endif

Now whenever that checkbox is checked or unchecked we will automatically add or remove our lockcomponent.

Special Thanks to the UnityWiki for getting me started with it's reverse engineered TransformInspector code.

Automatically Generating Persistent Unique Id's in Unity3D

This was something that took me by surprise, so I thought I'd write up my solution for it. I kind of assumed this feature would be built in to Unity3D and if it is, my Google-Fu failed me.

It's pretty common in games to require somekind of uniqueId for objects or entities so they can be located at runtime. A monster might have an Id and a sword might have another Id. The key to is they can't ever both have the same UniqueId. Just imagine if Driver License numbers weren't unique? I wouldn't want to get other peoples speeding tickets!

There are many ways to create uniqueIds, but a common method is a GUID. Guids are long, ugly and hard to remember. For example: "531ad467-5728-453c-a86f-ef33f9052c71"

So we might choose to use a GUID for our UniqueIdentifer - This is easy enough to generate in code.

var MyGuid = Guid.NewGuid();

And we have ourselves a unique Id - Yay!

But wait, when do we generate this Id? And here lies another problem. Once an Id has been generated, it should never change, otherwise if we have some savegame data refering to the fact that a character has a Sword with a particular Id and then that Sword's Id goes and changes suddenly our character has lost their sword and all we have is a long, ugly and hard to remember string that no longer has any meaning.

So lets avoid manually manipulating the ids at all. How about when our script loads, if it doesn't have an Id assigned to it, lets assign one.

public string UniqueId;
void Start () { if (String.IsNullOrEmpty(this.UniqueId)) UniqueId = Guid.NewGuid().ToString(); }

So far so good. We have a unique Id automatically generated for us. But wait, what if we duplicate the object in the unity editor? - Oh no! Because our UniqueId has been serialised, We have two objects with the same Id - Disaster! This is the tricky part.

Now Unity3D has a built-in InstanceId which you can access with:

this.GetInstanceID();

Which is guarenteed to be Unique, however it will not persist and changes everytime you run the application, so we can't use it.

What do we need?

  1. A unique Identifier for a our script.
  2. The Id once created never changes otherwise external references will break. (gamesaves!)
  3. The Id needs to persist between instances of running the application
  4. If the gameobject is copied we need to generate a new id.

 

The solution I came up with was to create a static UniqueIdentiferRegistry where scripts are registered on start, deregistered on destruction and we use the fact that this.GetInstanceID() returns a unique result at runtime. In this registry we create a mapping between UniqueIds and InstanceIds, and during the update phase we check if our game object's instanceId matches the one in the registry. If it doesn't we need to generate a new Id as it's been copied.

public static class UniqueIdRegistry
{
    public static Dictionary<String, Int32> Mapping = new Dictionary<String, int>();

    public static void Deregister(String id)
    {
        Mapping.Remove(id);
    }

    public static void Register(String id, Int32 value)
    {
        if (!Mapping.ContainsKey(id))
            Mapping.Add(id, value);
    }

    public static Int32 GetInstanceId(string id)
    {
        return Mapping[id];
    }
}


And our component class

[ExecuteInEditMode]
public class MyBehaviour: MonoBehaviour
{   [UniqueIdentifier] // Treat this special in the editor.
    public string UniqueId; // A String representing our Guid

	void Start () {
#if UNITY_EDITOR
	    if (String.IsNullOrEmpty(this.UniqueId))
            UniqueId = Guid.NewGuid().ToString();
	    UniqueIdRegistry.Register(this.UniqueId, this.GetInstanceID());
#endif
	}

    void OnDestroy() {
#if UNITY_EDITOR
        UniqueIdRegistry.Deregister(this.UniqueId);
#endif
    }

    void Update() {
#if UNITY_EDITOR
        if (this.GetInstanceID() != UniqueIdRegistry.GetInstanceId(this.UniqueId))    {
            UniqueId = Guid.NewGuid().ToString();
            UniqueIdRegistry.Register(this.UniqueId, this.GetInstanceID());
        }
#endif
    }

Note the usage of the [ExecuteInEditMode] so the code runs in the editor and the #if UNITY_EDITOR so this code is compiled out when running the game.

But why so hard?

If we could somehow have an OnGameObjectCopied() event to hook into we could just generate a new UniqueId and most of this code would be unnecessary. Note: This isn't the same as OnSerialize() and OnDeserialize() because we only want a new Id when the object is copied, not when it has been persisted to disk.

Final Tweaks

The last part is we don't want our Id to be human editable so lets modifiy the Unity Editor's GUI to display the Guid, but not allow for edits.

UniqueId.cs

public class UniqueIdentifierAttribute : PropertyAttribute { }

public class UniqueId : MonoBehaviour {
    [UniqueIdentifier]
    public string uniqueId;
}

UniqueIdDrawer.cs

[CustomPropertyDrawer(typeof(UniqueIdentifierAttribute))]
public class UniqueIdentifierDrawer : PropertyDrawer
{
    public override void OnGUI(Rect position, SerializedProperty prop, GUIContent label)    {
        if (prop.stringValue == "") {
            Guid guid = Guid.NewGuid();
            prop.stringValue = guid.ToString();
        }

        Rect textFieldPosition = position;
        textFieldPosition.height = 16;
        DrawLabelField(textFieldPosition, prop, label);
    }

    void DrawLabelField(Rect position, SerializedProperty prop, GUIContent label)    {
        EditorGUI.LabelField(position, label, new GUIContent(prop.stringValue));
    }
}

A simple Auto Load-Balancing Multi-Threaded Task Scheduler in C#

Note: This article applies mostly to games, or any application running in a tight loop. But it could also be applicable to a web server or a database system. For more general implementations of the same idea .Net 4.0 introduced the Parallel Task Libraray. Which might be worth looking at if you like this kind of stuff. What's extra cool is that this implementation is only ~150 lines of code.

Remember: Before you make any assumptions about performance, you should always measure before and after. A lot of the time assumptions turn out to be totally incorrect and performance bottlenecks can turn up anywhere and everywhere and especially where you least expect them!

When I was looking at a high level overview of my game's code, there seemed to be lots of update calls which, when looked at a little closer where actually independent of one another. What I wanted was a simple way of scheduling these update calls as Tasks and then a TaskScheduler could distribute the work depending on the number of cores on the machine and with some additional smarts, begin to load balance such that no thread would be over saturated with work compared to any other thread.

A made up example of a game update loop might look something like this:

UpdateInput();
UpdateCamera();
UpdatePlayer();
UpdateEnemies();
UpdateParticleEffects();
UpdateScripts();
UpdateEvents();

Some of these Updates may be able to run in parallel.

Making your game take advantage of multi-threading can be tricky business. But an easy way to get started is to think of your update loop as a series of tasks that need to be completed and then deciding which dependencies each task has.

In the example code dependencies cannot be resolved automatically, so it is up to the programmer to determine if a task can be run in parallel or not. In our made up example lets say that UpdateInput(); UpdateCamera(); and UpdatePlayer(); must be processed in order because perhaps tasks in the future depend on these updates. (Such as the results of the camera's projection matrix in graphics calculations or the location of the player in enemy AI)

But say then UpdateEnemies(); UpdateParticleEffects(); and UpdateScripts(); could be processed in parallel. Finally finishing with UpdateEvents() which we have decided must be run last as any of the other updates could potentially create events which will need to be processed at the end of the frame.

So in pseudo code our tasks would look something like:

UpdateInput();
UpdateCamera();
UpdatePlayer();

taskScheduler.Add(UpdateEnemies);
taskScheduler.Add(UpdateParticleEffects);
taskScheduler.Add(UpdateScripts);

taskScheduler.ExecuteAll();

UpdateEvents();

 Here it is in diagram form: (Each T represents some task)

So what's our strategy for writing code to do all this? First of all we need some concept of a Task. A task will have an Id (given by the scheduler) an Action<GameTime> Which is basically a function pointer (If you come from C++ land) with a GameTime as it's argument.

Note: I have kept the concept of a task rather simple. Basically any function that looks like void Update(GameTime gameTime); can be converted into a task.

You can get a lot more generic with this, but because this article is largely about games an Action<GameTime> should be sufficient. Again you might want to look at the ParallelTask Library for a more general case.

    class Task
    {
        public Int32 Id;
        public Action<GameTime> Action;
        public long LastKnownExecutionTime { get; set; }
    }

Then we need the idea of a TaskBatch. Which is basically a list of tasks that have been assigned to a thread by the TaskScheduler.

    class TaskBatch
    {
        public GameTime GameTime { get; set; }

        public List<Task> Tasks
        {
            get { return m_Tasks; }
            set { m_Tasks = value; }
        }

        private List<Task> m_Tasks = new List<Task>(42);
        public long TotalEstimatedExecutionTime()
        {
            long rv = 0;
            for (int i = 0; i < m_Tasks.Count; i++)
                rv += m_Tasks[i].LastKnownExecutionTime;

            return rv;
        }
    }

LastKnownExecutionTime and TotalEstimatedExecutionTime are important for load balancing. When we first run a task we have no idea how long it will take. But over time we can estimate how long each task will take thus helping us when batching tasks.

The Task Scheduler

This is the main part of the code.

Overview:

  1. Keep Track of all tasks that have ever been queued.
  2. If a task is queued put it into the active list.
  3. When it comes time to execute the tasks distribute it amongst the task batches.
  4. Execute each batch. (And record timings for each task for future load balancing)
  5. Wait until all batches have finished.
  6. Clear the active list

 

In order to distribute the tasks:

  1. Sort the active list by Last known execution time. Most expensive to least expensive.
  2. Find batch with the lowest total cost
  3. Add task to batch with lowest total cost

 

Here is the Scheduler code:

    public class TaskScheduler
    {
        private Dictionary<Action<GameTime>, Task> m_AllTasks;
        private List<Task> m_ActiveTasks;
        private TaskBatch[] m_TaskBatches; 
        private Int32 m_NumThreads;

        private int m_Count;        // Used for syncronization
        private int m_NextTaskId;   // Used to assign Ids to tasks

        public TaskScheduler()
        {
            Initalize(Environment.ProcessorCount);
        }

        public TaskScheduler(Int32 numberOfThreads)
        {
            Initalize(numberOfThreads);
        }

        private void Initalize(Int32 numberOfThreads)
        {
            if (numberOfThreads <= 0)
                throw new ArgumentException("Number of threads must be greater than 0");

            m_NumThreads = numberOfThreads;
            m_AllTasks = new Dictionary<Action<GameTime>, Task>();
            m_ActiveTasks = new List<Task>(42);
            m_TaskBatches = new TaskBatch[m_NumThreads];

            for (int i = 0; i < m_TaskBatches.Length; i++)
                m_TaskBatches[i] = new TaskBatch();
        }

        public void QueueTask(Action<GameTime> action)
        {
            lock (this) // Check if already in the task list, Otherwise create the task
            {
                if (m_AllTasks.ContainsKey(action)) 
                {
                    m_ActiveTasks.Add(m_AllTasks[action]);
                }
                else
                {
                    var task = new Task { Action = action };
                    m_NextTaskId++;
                    task.Id = m_NextTaskId;
                    m_AllTasks.Add(action, task);
                    m_ActiveTasks.Add(task);
                }
            }
        }

        public void ExecuteSerial(GameTime gameTime)
        {
            foreach (var task in m_ActiveTasks)
                task.Action.Invoke(gameTime);

            Reset();
        }

        public void ExecuteAll(GameTime gameTime)
        {
            if (m_NumThreads == 1)
            {
                ExecuteSerial(gameTime);
                return;
            }

            ExecuteParallel(gameTime);
        }

        private void ExecuteParallel(GameTime gameTime)
        {
            // Sort Active tasks by last known execution time
            m_ActiveTasks.Sort((a, b) => b.LastKnownExecutionTime.CompareTo(a.LastKnownExecutionTime));

            // Assign all tasks to batches. Assign each task to a batch with the least estimated execution time.
            for (int i = 0; i < m_ActiveTasks.Count; i++)
            {
                var currentBatch = m_TaskBatches[0];
                for (int j = 1; j < m_TaskBatches.Length; j++)
                {
                    if (m_TaskBatches[j].TotalEstimatedExecutionTime() < currentBatch.TotalEstimatedExecutionTime())
                        currentBatch = m_TaskBatches[j];
                }
                currentBatch.Tasks.Add(m_ActiveTasks[i]);
            }

            // Execute all batches
            for (int i = 0; i < m_TaskBatches.Length; i++)
            {
                m_TaskBatches[i].GameTime = gameTime;
                if (m_TaskBatches[i].Tasks.Count > 0)
                {
                    var x = i;
                    Interlocked.Increment(ref m_Count);
                    ThreadPool.QueueUserWorkItem(o => Execute(m_TaskBatches[x]));
                }
            }

            while (m_Count > 0) { } // Wait for threads to finish

            Reset();
        }

        private void Reset()
        {
            m_ActiveTasks.Clear();
            for (int j = 0; j < m_TaskBatches.Length; j++)
                m_TaskBatches[j].Tasks.Clear();
        }

        private void Execute(Object obj)
        {
            var batch = (TaskBatch) obj;
            for (int i = 0; i < batch.Tasks.Count; i++)
            {
                var startTime = Stopwatch.GetTimestamp();
                batch.Tasks[i].Action.Invoke(batch.GameTime);
                var time = Stopwatch.GetTimestamp() - startTime;
                batch.Tasks[i].LastKnownExecutionTime = time;
            }
            Interlocked.Decrement(ref m_Count);
        }
    }

In a unit test I get the following results on my 4 core laptop

First Run Time Taken:24ms
Serial Time taken:20ms
Parallel Time taken:5ms
Parallel Time taken:5ms
Parallel Time taken:4ms
Parallel Time taken:5ms

Ignoring the first run, which start-up assembly loading and JITing all contribute to. The Parallel time becomes pretty much exactly our maximum theoretical improvement of 4x.

 

Here is the unit test in question:

    [TestFixture]
    public class SchedulerTests
    {
        [Test]
        public void Test()
        {
            var scheduler = new TaskScheduler(4);
            ScheuleTasks(scheduler);

            var sw = new Stopwatch();
            sw.Start();
            scheduler.ExecuteAll(new GameTime());
            sw.Stop();

            Console.WriteLine("First Run Time Taken:{0}ms",sw.ElapsedMilliseconds);
            ScheuleTasks(scheduler);
            sw.Restart();
            scheduler.ExecuteSerial(new GameTime());
            sw.Stop();
            Console.WriteLine("Serial Time taken:{0}ms", sw.ElapsedMilliseconds);

            for (int i = 0; i < 4; i++)
            {
                ScheuleTasks(scheduler);

                sw.Restart();
                scheduler.ExecuteAll(new GameTime());
                sw.Stop();
                Console.WriteLine("Parallel Time taken:{0}ms", sw.ElapsedMilliseconds);
            }
        }

        private void ScheuleTasks(TaskScheduler scheduler)
        {
            scheduler.QueueTask(T1);
            scheduler.QueueTask(T2);
            scheduler.QueueTask(T3);
            scheduler.QueueTask(T4);
            scheduler.QueueTask(T5);
            scheduler.QueueTask(T6);
            scheduler.QueueTask(T7);
            scheduler.QueueTask(T8);
        }

        public void T1(GameTime gameTime)
        {
            Thread.Sleep(1);
        }

        public void T2(GameTime gameTime)
        {
            Thread.Sleep(2);
        }

        public void T3(GameTime gameTime)
        {
            Thread.Sleep(3);
        }

        public void T4(GameTime gameTime)
        {
            Thread.Sleep(4);
        }

        public void T5(GameTime gameTime)
        {
            Thread.Sleep(1);
        }

        public void T6(GameTime gameTime)
        {
            Thread.Sleep(2);
        }

        public void T7(GameTime gameTime)
        {
            Thread.Sleep(3);
        }

        public void T8(GameTime gameTime)
        {
            Thread.Sleep(4);
        }
    }

 

Further Improvements:

  • A priority queue might be better when adding tasks. Which would mean inserts at O(logn) but no sort would be required.
  • I'm not entirely sure I'm using the best synchronization method with while (m_Count > 0) { } Maybe AutoResetEvents would be better?
  • Note: I have tried to avoid using Linq as that can create some extra garbage and I'm trying to minimize that.

 

Downsides:

While this makes getting started with multi-threading your game code pretty easy, there are still a large number of difficulties. It's probably good idea to read through Potential Pitfalls in Data and Task Parallelism

  1. Tasks must be identified as being independent and be thread safe. Lots of shared state will clobber any performance improvements as threads enter critical sections.
  2. A chain is only as strong as it's weakest link. If a task takes 80% of the time when processed serially the max theoretical speed up will only be a 20% improvement. Not the 2x or 4x you where hoping for. See Amdahl's Law.
  3. If you do get the performance improvements there will be a greater delta between your low end devices and your high end devices. So you might have to degrade features for low end devices while only cosmetically improving your high-end devices.
  4. If your game is like mine rendering is by far your largest bottle neck, so performance improvements in the Update() section might be minor.
  5. If your code is I/O bound. Disk/Network etc... then this method probably won't help you out much. So maybe it wouldn't be that great for a web server? You'd have to test it out.

 

Overall this was a pretty fun experiment to write. I have not used this code in production yet. So be warned!

 

 

Rumble, Earthquake and RGB Shake Shader for concussion or glitch

This weeks Pixel Shader is pretty simple. A small rumble or quake can add some urgency to your game, or make the impact of explosions or hits feel that much more visceral - Especially if you combine it with sound effects and vibration or feedback in the controller.

Rumble / Earthquake

whether we want a small rumble or a larger quake only depends on the maximum amplitude parameter. What we want to do is to start randomly displacing the screen every frame by a random vector and gradually decrease the intensity as the duration of the quake comes to an end. Most of this logic will be handled in our game code and we will just pass the necessary parameters to the Pixel Shader.

Note: You can also do this effect by applying a transform to the Vertex Shader, but we will do it in the Pixel Shader as a demonstration that leads on to the RGB Separator Effect.

texture Texture;
sampler TextureSampler = sampler_state
{
    Texture = <Texture>;
};

float2 RumbleVector;

struct VertexShaderOutput
{
	float4 Color : COLOR0;
	float2 TextureCoordinate : TEXCOORD0;
};

float4 Rumble(VertexShaderOutput input) : COLOR0
{
    float4 color = tex2D(TextureSampler, input.TextureCoordinate + RumbleVector);
	return color;
}

technique Technique1
{
    pass Pass1
    {
        PixelShader = compile ps_2_0 Rumble();
    }
}

As you can see when we perform our texture lookup we are going to offset by the RumbleVector parameter.

C# code

 var rX = m_Random.Next(0, m_Amplitude) * (m_CurrentRumbleTime / (float)TotalRumbleTime);
var rY = m_Random.Next(0, m_Amplitude) * (m_CurrentRumbleTime / (float)TotalRumbleTime);

m_Effect.Parameters["RumbleVector"].SetValue(new Vector2(rX / (float)m_GraphicsDevice.Viewport.Width, rY / (float)m_GraphicsDevice.Viewport.Height));


We call this code every frame in the Update() loop and set the RumbleVector by some random x,y but then multiply it by the effects duration. This way the effect starts at maximum amplitude and then calms down as the effect beings to finish.

RGB Separation

This gives you a bit more of a colourful shake and I've seen it used for concussion effects or computer screen/glitch effects. The method is the same, except this time instead of randomly generating one RumbleVector we will generate 3 Vectors and use one for each of the colour channels. 

The Pixel Shader

float4 RgbRumble(VertexShaderOutput input) : COLOR0
{
    float3 colorR = tex2D(TextureSampler, input.TextureCoordinate + RumbleVectorR);
	float3 colorG = tex2D(TextureSampler, input.TextureCoordinate + RumbleVectorG);
    float3 colorB = tex2D(TextureSampler, input.TextureCoordinate + RumbleVectorB);

	float4 returnColor;
	returnColor.r = colorR.r;
	returnColor.g = colorG.g;
	returnColor.b = colorB.b;
	returnColor.a = 1.0f;
	return returnColor;
}

 

Visual Studio 2010 Solution: (After building press space to change the effects)

04_Rumble_b.zip (279.80 kb)

Vingette Post Effect

 

Vignettes provide nice looking borders around your scene and are actually quite easy to create.

When I was first thinking about adding Vignettes to my game, the original method I used was to generate them in a paint program and then simply overlay the image. This works fine, except you don't have any dynamic control and if you want many different styles then you'll have to make (and manage) many different images. Better to generate the Vignette in the Pixel Shader instead.

 

Render to Texture

Firstly I am describing this method as a 'Post Effect' or an effect that is applied after your scene is rendered. Our previous effects have also been Post Effects, but because I have only been rendering a single image I haven't been taking the extra step to render the scene to a texture. Usually you will be rendering more than one image or object! To render to texture in XNA we use Render Targets. (In OpenGL these are referred to as FrameBufferObjects.)

We will create a simple PostEffect class to handle Rendering to a texture:

    public class PostEffect
    {
        private readonly GraphicsDevice m_GraphicsDevice;
        private Effect m_Effect;
        private readonly RenderTarget2D m_RenderCapture;

        public PostEffect(GraphicsDevice graphicsDevice)
        {
            m_GraphicsDevice = graphicsDevice;
            m_RenderCapture = new RenderTarget2D(m_GraphicsDevice, m_GraphicsDevice.Viewport.Width, m_GraphicsDevice.Viewport.Height);
        }

        public void LoadContent(ContentManager content)
        {
            m_Effect = content.Load<Effect>("Vingette");
        }

        public void BeginCapture()
        {
            m_GraphicsDevice.SetRenderTarget(m_RenderCapture);
        }

        public void EndCapture()
        {
            m_GraphicsDevice.SetRenderTarget(null);
        }

        public void Draw(SpriteBatch spriteBatch)
        {
            spriteBatch.Begin(SpriteSortMode.Deferred, BlendState.NonPremultiplied, SamplerState.PointClamp, DepthStencilState.Default, RasterizerState.CullNone, m_Effect);
                spriteBatch.Draw(m_RenderCapture, Vector2.Zero, Color.White);
            spriteBatch.End();
        }
    }

 

Now in Game1.cs when we render our scene it will look like the following:

protected override void Draw(GameTime gameTime)
{
  GraphicsDevice.Clear(Color.CornflowerBlue);

  // Start Rendering to Texture
  m_PostEffect.BeginCapture();
    GraphicsDevice.Clear(Color.CornflowerBlue);
      spriteBatch.Begin();
        // Render our scene as normal
        DrawScene();
      spriteBatch.End();
  m_PostEffect.EndCapture();

  // Now render our post effect to the screen
  m_PostEffect.Draw(spriteBatch);
}

Our First Vignette Effect

Effect.fx

float R = 1.0f; // Radius of vingette
float4 VingetteShader(VertexShaderOutput input) : COLOR0
{
	float4 color = tex2D(TextureSampler, input.TextureCoordinate);

	float dX = abs(input.TextureCoordinate.x - 0.5)*R;
	float dY = abs(input.TextureCoordinate.y - 0.5)*R;
	
	float p = dX*dX + dY*dY;			// Squared distance
    float d = 1 - 0.2f*p - 0.2*p*p;		// Fall off function

	color.rgb = saturate(color.rgb*d);

    return color;
}

So what's this doing? First of all it is grabbing the pixel from the texture and then it is working out the distance (dX, dY) between the screen co-ordinates and the centre of the screen (0.5, 0.5) We are using squared distance, rather than distance to save a sqrt() call - If this doesn't make much sense try to recall Pythagoras Theorem from High School.

Then we apply a falloff function which determines how much the colour is going to fall off as it moves further away from the centre. Here I am using a simple parabola. But exponentials or log functions also look good. Linear functions look pretty horrible, but you might find a use for them?

The falloff function I am using looks like this:

 

Horizontal Lines Only

This is a pretty simple modification. Instead of calculating dX and dY just calculate dY and use that as the distance in your falloff function. See the accompanying code for a working example. Likewise you can also just have vertical bars only.

 

Circular Vignette

You might notice that our Vignette is elliptical rather than circular. To make it circular we need to pass in modifiers to dX and dY to handle for the aspect radio of our scene. Because our sample uses a resolution of 1024x768 we need to multiply our dY value by the aspect ratio or dY *= 1.3333

If you want to make aspect radio handling a little more robust, passing it in a a parameter for both dX and dY is a good idea.

 

Follow Spot

The Vignette does not have to use (0.5, 0.5) as the centre point. In the sample I have also included a mode where the Vignette is rather small and follows your mouse, emulating a kind of follow spot. With a little modification, this could be used for several different effects such as: A flashlight in a hidden object game, a torch in a top down RPG over the player, a telescope in a spy game, or with a cross hair, a scope for a sniper rifle.

 

Additional Modifications

There are many more styles of Vignetting that you can use to achieve more and varied effects. In this example I have simply used it to fall off into black. But you could use the result d as a lerp parameter into a different colour, or into any other effect.

Play. See what you come up with.

 

Here is the VS2010 solution file:

03_Vingettes.zip (298.90 kb)

 

Introduction to Pixel Shaders with XNA and Monogame: Part2 - SepiaTone, Desaturation, Scanlines and More.

 

In Part One I showed how to draw an image to the screen with SpriteBatch and use an effect to produce a greyscale image.

In this tutorial I'm going to show how to produce some other commonly used effects. Remember everything that Photoshop can do you can probably find a way to do it too!

Note: All the effects here assume the Vertex shader output is coming from SpriteBatch. Here is the Vertex Shader output definition again.

// This data comes from the sprite batch vertex shader
struct VertexShaderOutput
{
    float2 Position : TEXCOORD0;
	float4 Color : COLOR0;
	float2 TextureCordinate : TEXCOORD0;
};

SepiaTone

Sepia is a brownish pigment that makes an image look like an old photo. (And according to Wikipedia is made from cuttlefish) Perfect for flashbacks or a vintage look. (And quite striking with a Vignette, which I will cover in a later post.)

Here is the PixelShader for a Sepia Effect:

float4 Sepia(VertexShaderOutput input) : COLOR0
{
	float4 color = tex2D(TextureSampler, input.TextureCordinate);
	
	float3x3 sepia ={0.393, 0.349, 0.272,
					0.769, 0.686, 0.534 ,
					0.189, 0.168, 0.131};

    float4 result;
	result.rgb = mul(color.rgb, sepia);
	result.a = 1.0f;

	return result;
}

The sepia effect is very similar to the Greyscale effect in the previous tutorial. This time however instead of multiplying a set of RGB values to produce a single grey color. We are using a 3x3 Matrix to transform our RGB colours into sepia colors. Each column in the matrix represents the amount of Red, Green and Blue to apply against our source pixel. Finally we set the alpha to 1.0 before returning the color.

Desaturation

Desaturation is simply an enhanced version of our greyscale effect. Instead of simply obtaining a new colour we are going to Linearly Interpolate (or Lerp for short) between our source colour and our grey value.

float DesaturationAmount;
float4 Desaturate(VertexShaderOutput input) : COLOR0
{
	float4 color = tex2D(TextureSampler, input.TextureCordinate);

	// Desaturate
	float desaturatedColor = dot(color,float3(0.3, 0.59, 0.11));
	color.rgb = lerp(color, desaturatedColor, DesaturationAmount);
	
	return color;
}

Where DesaturationAmount is a value between 0 and 1.

To pass a parameter to a shader from C# code, we perform the following.

m_Effect.Parameters["DesaturationAmount"].SetValue(1.0f);

Being able to pass parameters to the Shaders allow for interesting effects. For example we could fade from full colour to grey if the scene was becoming more sad. Or if the player was starting to become low on health. Or if it was an overcast day we might just desaturate the scene a little bit.

 

Fade to Black (or any colour you want)

float Pulse;
float4 FadeToColor(VertexShaderOutput input) : COLOR0 { float4 color = tex2D(TextureSampler, input.TextureCordinate); float4 destColor = float4(0.0,0.0f,0.0f,1.0f); // Black color = lerp(color, destColor, Pulse); return color; }

Similar to the desaturation effect above - We have a destination color (black) and a source color (Taken from the texture) And we want to lerp between the two values by some parameter called Pulse. I've called it Pulse because in the example code, the parameter modulates between 0.0 and 1.0 making the scene fade to black and then fade in again. With some modifications you could create flickering lights or you might use something like this if the player is losing consciousness, or power to the building is being compromised.

 

Scanlines (Or Old TV)

 

Scanlines can make it look like the image is being displayed on an old tv screen. For every 4th line we draw 2 two horizontal black lines. Because x and y coordinates for position in our Pixel Shader are between 0 and 1 we need to know the height of the image before using this effect. Which we pass as a parameter to the Shader.

Here is the code:

int ImageHeight;
float4 ScanLine(VertexShaderOutput input) : COLOR0
{
	float4 color = tex2D(TextureSampler, input.TextureCordinate);
	int a = saturate((input.Position.y * ImageHeight) % 4);
	int b = saturate((input.Position.y * ImageHeight+1) % 4);
	float m = min(a,b);

	color.rgb *= m;

	return color;
}

In the above code saturate() will clamp the value between 0 and 1. The min(a,b) function is there because we want to draw every 4th line as a black line as well as every 4th+1 (In order to make the scan lines thicker.

Brightness and Contrast

Most people should be familiar with brightness and contrast. Sometimes it's nice to be able to just touch it up a bit.

float Contrast;
float Brightness;
float4 BrightnessContrast(VertexShaderOutput input) : COLOR0
{
	float4 color = tex2D(TextureSampler, input.TextureCordinate);

	 // Apply contrast.
	float factor = (1.1 * (Contrast + 1.0)) / (1.0 * (1.1 - Contrast));
	float4 outcolor = factor * (color - 0.5) + 0.5;
	outcolor += Brightness;

	outcolor.a = 1.0f;
	return saturate(outcolor);
}

Contrast and brightness values are between -1 and 1 (Note: If you change the order of applying brightness before contrast you will get a different result. I'm not sure which order they are supposed to be applied.) Note we are using the saturate() command again. So our output values stay between 0 and 1.

That's it for this edition: Download the code and you can see all these techniques in a single effect file. Pressing spacebar will switch between them.

Download a VS2010 Project

02_Techniques.zip (488.97 kb)

 

 

Introduction to Pixel Shaders with XNA and Monogame: Part 1 - The Greyscale Effect.

 

Prerequisites

This series assumes that you have at least some familiarity with C# and XNA 4.0.

Really all that is required is knowing how to draw something to the screen using SpriteBatch. See Introduction to XNA Loading an Image and rendering it with Spritebatch if you are unfamiliar with how to do this.

Motivation Behind This Series

Shaders are cool and can add a lot of ambience and visual eye candy. I also feel that game programmers who are just starting out, or those who mainly program 2D games largely ignore Shaders because they might appear more complicated than they actually are. They also give you a lot more real time tuning options - Want to desaturate the entire scene for a flashback? or add a vignette? With a Pixel Shader it's easy!

XNA also provides an easy way to get started with Shader programming and even though it will no longer be updated by Microsoft, it lives on in the form of Monogame a cross-platform implementation that runs on almost all of the devices such as iOS, Windows, OSX, Android and More!

Modern Computer Graphics

Polygons, Triangles and Vertices, Oh My!

Most scenes in current computer graphics are made up of many many polygons. A polygon, if you remember from Math class is a shape that is made up of 3 or more points. 2 points make a line, and 1 point, well that makes a point. The points are referred to as Vertices and a polygon with 3 Vertices is commonly called a triangle.

The graphics card in your computing device likes triangles, as all polygons can be reduced to many triangles. Infact, a quad is usually represented by two triangles.

Modern graphics also means that we can write programs, called 'Shaders' that run on the graphics card and manipulate pixels and polygons at various stages in the rendering pipeline.

The Vertex Shader:

This is a program that takes vertices from polygons and transforms them from the game world space into your screen space, it does some other things too, like contain the texture co-ordinates (Commonly referred to as uvs) or define a color for the vertex (and many other things.) For now we will not concern ourselves too much with Vertex Shaders and just let SpriteBatch take care of performing this transformation for us.

The Pixel Shader:

Once your vertices have been transformed into Screen Space they want to be rendered as pixels. This is were the Pixel Shader comes it. It can take data from the Vertex Shader (and elsewhere) and decide what color pixels should be. This is what we will be writing in this tutorial. 

Geometry Shaders:

These are the newest type of Shaders which we will be seeing more of on the new generation of consoles (PS4 and XboxOne) which are largely responsible for creating new geometry on the fly. XNA does not have access to geometry Shaders so we will not concern ourselves with them.

See the Wikipedia entry on Shaders for more information.

Effect Files

Xna manages Shaders through effect files, which are stored in the content project. Simply right click and add an effect file and you will see what the default effect file looks like.

Our First Pixel Shader

XNA's SpriteBatch is really quite clever as it hides a significant amount of complexity behind what appears to be a simple 2D drawing API. Actually however it uses both Vertex and Pixel Shaders under the hood. Because SpriteBatch provides a default Vertex Shader we can get started by taking the output from SpriteBatch and just worry about our coding our first Pixel Shader. This means that when we call .Begin() on our SpriteBatch we can pass it an effect that contains our Pixel Shader.

Steps:

Add a new effect file to the content project and call it "myEffect"

Add a member variable in Game1.cs

private Effect m_Effect;

In LoadContent()

m_Effect = Content.Load<Effect>("myEffect");

In Draw() - Notice the last parameter in .Begin() is the effect we want to use.

m_SpriteBatch.Begin(SpriteSortMode.Deferred, BlendState.NonPremultiplied, SamplerState.PointClamp, DepthStencilState.Default, RasterizerState.CullNone, m_Effect);        
  m_SpriteBatch.Draw(m_background, Vector2.Zero, Color.White);
m_SpriteBatch.End();

The Default Effect Shader

By default when we create a new effect we get both a Vertex and a Pixel Shader. Because we are only concerned about Pixel Shaders for the moment, delete all the code in the newly created effect file and use the following instead:

// This data comes from the SpriteBatch's default Vertex Shader
struct VertexShaderOutput
{
    float4 Position : : TEXCOORD0;
	float4 Color : COLOR0;
	float2 TextureCordinate : TEXCOORD0;
};

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0
{
    return float4(1, 0, 0, 1);
}

technique Technique1
{
    pass Pass1
    {
        PixelShader = compile ps_2_0 PixelShaderFunction();
    }
}

If you try to use the Shader above what you will see is an entirely red screen.

Let's try to break down this effect file shall we?

First of all is a struct is declared called VertexShaderOutput which specifies the data that is coming from SpriteBatch's Vertex Shader. We don't actually do anything with the data in this effect file, but we will in future effect files, so it's good to know what is going on.

Basically, there are 4 floats representing a Vertex's Position, namely x,y,z and w (Yes it's 4-dimensional space, but don't worry about that too much.) 4 floats to represent a Vertex's Color. In this case amounts of Red, Green, Blue and Alpha (or Transparency) each value lies somewhere between 0 (No Color) and 1 (Maximum Color) - For example yellow would be (1.0f, 1.0f, 0.0f, 1.0f) or Red + Green. Then there are 2 floats representing texture co-ordinates. We will use these when we want to retrieve pixels from a texture and put them on the screen.

Then we have our PixelShaderFunction which is the pixel shader itself and determines what color each pixel should be when rendered. In this case red. Finally effect files can have multiple techniques and techniques can have multiple passes. In this case we have one technique with one pass and it is a Pixel Shader (using Pixel Shader version 2.0) called PixelShaderFunction() We will expand on techniques in the future.

 

Adding a texture sampler

Just rendering red to the screen is not very useful. What we would like to do is pull pixels from our texture and use them instead. Here is our effect file with a texture sampler.

// Our texture sampler
texture Texture;
sampler TextureSampler = sampler_state
{
    Texture = <Texture>;
};

// This data comes from the SpriteBatch Vertex Shader
struct VertexShaderOutput
{
    float4 Position : TEXCOORD0;
	float4 Color : COLOR0;
	float2 TextureCordinate : TEXCOORD0;
};

// Our Pixel Shader
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0
{
	float4 color = tex2D(TextureSampler, input.TextureCordinate);
	return color;
}

// Compile our shader
technique Technique1
{
    pass Pass1
    {
        PixelShader = compile ps_2_0 PixelShaderFunction();
    }
}

 

We have created a new parameter to the Shader called Texture. Which is actually set by SpriteBatch. (We will set custom parameters via C# code in future tutorials.) A texture sampler is than created for our texture. We will go into more detail into texture samplers in the future as they themselves can be customised. You will then notice instead of simply returning a color we are using the tex2D function which takes a texture sampler and set of texture coordinates and returns the color at those coordinates in the texture. If you use the Shader above with SpriteBatch you will get what you'd expect to see. The image that you wanted to draw.

Example:

 

Let's make it Greyscale!

This is a simple modification to the effect above. Instead of just reading the RGBA values from the texture, we are going to perform some modifications first. The formula to convert a color pixel to a greyscale pixel is greypixel = 0.299*color.r + 0.587*color.g + 0.114*color.b so we are going to apply this formula to every pixel on the screen.

 The result!

 

 

Complete Greyscale Effect Code Listing:

// Our texture sampler
texture Texture;
sampler TextureSampler = sampler_state
{
    Texture = <Texture>;
};

// This data comes from the sprite batch vertex shader
struct VertexShaderOutput
{
    float4 Position : TEXCOORD0;
	float4 Color : COLOR0;
	float2 TextureCordinate : TEXCOORD0;
};

// Our pixel shader
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0
{
	float4 color = tex2D(TextureSampler, input.TextureCordinate);

	float value = 0.299*color.r +  0.587*color.g + 0.114*color.b;
	color.r = value;
	color.g = value;
	color.b = value;
	color.a = 1.0f;

	return color;
}

// Compile our shader
technique Technique1
{
    pass Pass1
    {
        PixelShader = compile ps_2_0 PixelShaderFunction();
    }
}

 Download VS2010 Project

01_Greyscale_c.zip (327.67 kb)

Introduction to XNA and Monogame : Loading an Image and drawing it with Spritebatch

  1. Open Visual Studio and start a new XNA 4.0 windows game project.
  2. Find a suitable image to add to the content project (Right-Click Add->Existing Item).
  3. Load our image (See code listing below)
  4. Draw our image (See code listing below)
  5. Remember F5 compiles and runs our program.

 

With any luck you should have something like this:

 

 

Here is the Complete Code Listing for game1.cs

    public class Game1 : Microsoft.Xna.Framework.Game
    {
        private GraphicsDeviceManager m_GraphicsDeviceManager;
        private SpriteBatch m_SpriteBatch;
        
        private Texture2D m_background;

        public Game1()
        {
            m_GraphicsDeviceManager = new GraphicsDeviceManager(this);
            m_GraphicsDeviceManager.PreferredBackBufferWidth = 1024;
            m_GraphicsDeviceManager.PreferredBackBufferHeight = 768;
            Content.RootDirectory = "Content";
        }

        protected override void LoadContent()
        {
            m_SpriteBatch = new SpriteBatch(GraphicsDevice);
            m_background = Content.Load<Texture2D>("myImage");
        }

        protected override void Update(GameTime gameTime)
        {
            if (GamePad.GetState(PlayerIndex.One).Buttons.Back == ButtonState.Pressed)
                this.Exit();

            base.Update(gameTime);
        }

        protected override void Draw(GameTime gameTime)
        {
            GraphicsDevice.Clear(Color.CornflowerBlue);

                m_SpriteBatch.Begin(SpriteSortMode.Deferred, BlendState.NonPremultiplied);
                    m_SpriteBatch.Draw(m_background, Vector2.Zero, Color.White);
                m_SpriteBatch.End();

            base.Draw(gameTime);
        }
    }

 Download the VS2010 Project XNA_01.zip (272.13 kb)