Silverlight, as a programming environment, is awesome. C# and .NET running at full speed in a web plugin? Sign me up!
You would think that this would be a huge boon for game developers, but in fact Silverlight can be very unwieldy to use, especially where rendering performance is concerned. Everything in Silverlight must be rendered using its internal scenegraph – you are at its mercy, when drawing complicated dynamic scenes.
So it is important to understand what your options are for rendering, and what their limitations are and how to overcome them.
Plug: The other problem with Silverlight, for game developers, is that keeping the Silverlight scenegraph in sync with your game state is tedious and error-prone – we’d much prefer an immediate-mode renderer. I have solved both of these problems in ExEn – a high-performance implementation of XNA on Silverlight. I am currently crowd-funding a project to make ExEn open source, over at RocketHub. If this is something you would find useful, please go there and contribute!
There are four basic methods for rendering in Silverlight, roughly in order from slowest to fastest:
- Pixel Shaders
- Software Rendering
- Hardware Rendering
The WriteableBitmap class is a BitmapSource (you can use it to provide the picture to be displayed by an Image element), that was introduced in Silverlight 3.
There are two methods for drawing onto a WriteableBitmap. First of all you can have Silverlight render elements onto it using its software renderer, which comes in handy for caching the result of expensive rendering operations.
The other method is to rasterise directly pixel-by-pixel. This is slow. That is not to say that the implementation is bad – it is as fast as you could expect using JITted C#. However it is slower than all of the other methods mentioned here.
(Plus, you must write all the rasterising code yourself – Yuck! The WriteableBitmapEx library provides a good starting point, but it’s still not as fast or as good as Silverlight’s internal renderer.)
The most surprising thing about pixel shaders in Silverlight 3 and 4 is that they do not run on the GPU! You still write them in HLSL and then compile them as if to run them on the GPU, using the compiler from the DirectX SDK. But instead Silverlight JITs them to x86 code and runs them on the CPU.
So they are still fast – and much faster than using WriteableBitmap directly. But they are much, much slower than you might expect, especially if you have come from a game development background.
It is also worth pointing out that pixel shaders are transformative only. They are designed to take an input element and apply an effect to it. They are only marginally usable for rendering things (and wouldn’t be faster than the software renderer anyway).
Silverlight Software Renderer
This is the primary drawing mechanism in Silverlight. It takes your object tree – your scene graph – and it renders it.
It is very fast, and it does some very clever things like keeping track of what objects have changed (are “dirty”) and only rendering the parts of the screen that actually change.
Of most interest to game developers is the Image element, which can take a BitmapSource (a BitmapImage or a WriteableBitmap) and draw it to the screen – like a sprite. But you may also find other Silverlight elements useful in games. For example using the Polyline element to draw lines, or the MediaElement to play video.
Silverlight Hardware Renderer
Now the software renderer is very fast – fast enough to draw simple and even moderately complicated games. But for really complicated games running at high frame-rates, you want to use the hardware renderer. It blows the performance of the software renderer out of the water and will drastically reduce the CPU usage of your Silverlight game.
The downside to hardware rendering is that it only works in a very limited set of cases:
- You must explicitly tell Silverlight what elements to cache into hardware
- These elements and their parents can only have a limited set of operations applied to them
- You must explicitly enable hardware acceleration in the HTML file
To enable an element to be rendered in hardware, you must set the CacheMode on the element to a new instance of BitmapCache. The element being cached is then rendered to a surface which is copied to the GPU. Then when it comes time to render the scene, the GPU will simply draw that surface with the appropriate transformations applied to it – which is extremely fast!
Every time the cached element (or its children) change, that element and all its children are rendered again and the rendering is copied to hardware. So in general you should only cache elements that do not change frequently, usually this will mean setting caching on the leaf nodes of your tree.
(Performance tip: if you must modify a cached element, try to avoid changing its size. Reallocating the surface at a different size is a huge performance hit!)
Silverlight will automatically disable the hardware caching of an element if you apply an incompatible operation to it or any of its parent elements. Operations that are incompatible with hardware rendering include: pixel shaders, opacity masks, most clipping, nested caching, and perspective transforms in Silverlight 3.
Here’s what is compatible:
- Positioning (eg: Canvas.Top and Canvas.Left)
- Projection (3D/perspective transforms) in Silverlight 4 only
This is enough to move, rotate, scale and fade sprites – the bulk of what most 2D games do, graphically.
You can also do oriented, rectangular clipping in hardware. But this is mostly designed to support scrolling lists (a window of content). It isn’t generally suitable for what we’d like to do with clipping in games: sprite sheets.
Any elements that are not cached will still be rendered by the software renderer. When the hardware renderer is in use, any software rendering will be done onto special “software” surfaces (the size of the plugin window) which are copied to the GPU whenever their contents change. These software surfaces are interspersed with the hardware surfaces by Z-order, as the scene is composited by the hardware renderer.
The renderer is clever enough to group all consecutive-in-Z-order software rendering onto a single software surface. But if it encounters a hardware surface in the Z-order it must stop using that software surface, draw it, draw the hardware surface on top, and then start with a new software surface for any further software rendering. It is therefore important to keep the number of transitions between software and hardware in the Z-order to a minimum.
Finally, hardware rendering is disabled by default. You must enable it in the HTML file by setting the Silverlight object’s “EnableGPUAcceleration” parameter to true. Additionally, on the Mac, hardware rendering is only available in full-screen.
There are a few properties you can set on the Silverlight plugin that will help you inspect what Silverlight is rendering:
EnableFrameRateCounter: This will display a frame-rate counter in the status bar. (Note that browser security settings usually prevent plugins from modifying the contents of the status bar – you will have to modify them.)
Additionally, if hardware rendering is enabled (EnableGPUAcceleration) then four counters will appear in the top-left of the plugin window. These counters are: frames per second, GPU memory usage in KB, hardware surface count, software surface count.
EnableCacheVisualization: Only works in hardware mode (EnableGPUAcceleration). This lets you inspect which elements are being cached in hardware, and which are being rendered in software. Software surfaces will be tinted with a colour (the first colour is red).
EnableRedrawRegions: Will apply a tint to everything the software renderer renders, changing colour each frame. This is most useful for visualising where the dirty regions are – in particular to see if you are causing regions to become dirty when they do not actually need redrawing.
So, that was an overview of the “tools” that Silverlight provides for rendering, which are the fastest, and how to use them.
In my next article I will write about how I use them in ExEn to implement all of the operations provided for by XNA’s SpriteBatch.