The Scavengers Guide to ExEn
After my announcement that I am essentially retiring from ExEn development, I’ve invited the MonoGame team to “steal” the ExEn code for their own project.
They invited me to submit a patch. But that won’t work. I don’t have the time to produce one. And, given that ExEn was made with different priorities to MonoGame, I don’t want to go in and wholesale replace large chunks of code. An integration of ExEn code probably needs to be done with more sensitivity than that – something I have even less time to do.
(While the wholesale replacing of large chunks of MonoGame is a possibility that I would gladly support myself, or even using ExEn as a new base for new MonoGame features, such action is a complicated process that would require the buy-in of all the major MonoGame developers, and great care in ensuring that none of their newer features break in the process.)
So it will be up to the MonoGame developers to scavenge what they find useful.
I’ve been chatting on the MonoGame IRC channel about their code, in an effort to reach a shared understanding of what in ExEn could be useful for MonoGame. Rather than have the knowledge be lost in the chat log, I’m going to write this brief guide to what features ExEn contains.
(I’m also rather sick today, so this gives me a nice low-energy-but-useful thing to do, while I hang out on IRC.)
If you have any questions about any of these, I would be thrilled to answer, and also to update this document to clarify anything.
~
In no particular order…
Project Organisation: (I don’t think my project organisation is ideal – MonoGame’s is just as good. I just mention it here to help you find your way around.)
ExEn is divided up into the “core” classes (ExEnCore) that are not dependent on anything else. This is usually maths stuff and data structures. The “common” code (ExEnCommon), which contains code that is common to all platforms that has dependencies such that it can’t be put in ExEnCore (often in the form of partial classes containing the platform-independent parts of a given class). And finally the platform-specific code for various platforms (ExEnSilver, ExEnEmTouch and ExEnAndroid).
ExEn ReflectAndCompare: This is a tool I made that uses reflection to test compatibility between XNA and the ExEn library. Not only does it test the classes, properties, methods and fields match, but it also pushes a whole lot of data through the methods on each class to check that the output matches. This is an internal tool, not really suitable for mass consumption, but it is extremely useful for checking implementation accuracy. It is available on CodePlex here, outside of the main ExEn repository.
Maths Types in ExEnCore: I’m not sure how different these are to the types in MonoGame – MonoGame is likely to have a more complete implementation of the XNA maths types. But the types and methods that are in ExEn have been extensively tested with the aforementioned ReflectAndCompare tool.
ExEn Types Don’t Allocate Memory When They Shouldn’t: In MonoGame KeyboardState allocates an array where ExEn correctly uses a bit-field (ExEnCore/Input/KeyboardState.cs). In MonoGame TouchCollection is a reference type deriving from List (so it allocates) whereas ExEn correctly implements IList on value-type fields (ExEnCore/Input/Touch/TouchCollection.cs).
If a user of MonoGame were to copy one of these types, their game would behave incorrectly. In ExEn their behaviour matches XNA perfectly.
This is an excellent opportunity to just drop the ExEn types into MonoGame. It’s low-impact and the ExEn code is unarguably better.
I suspect that there may well be other types in MonoGame that are exhibiting allocation behaviour that does not match XNA.
TouchPanel bug fixes: As well as the above, ExEn implements correct thread safety and allocation behaviour for TouchPanel (ExEnCommon/Input/Touch/TouchPanel.cs).
Drift-Smoothing ExEn Game Loop: ExEn has a shared GameLoop type that smooths out incoming ticks from the operating system before they go to Game.Update and Game.Draw. This is similar to the behaviour of XNA (although does not match exactly). This prevents dropped or doubled frames that happen due to timer drift. (See ExEnCore/GameLoop.cs.)
Automagic Retina Display Handling: One neat feature of ExEn is its ability to allow games to take advantage of the iOS retina display, without having to modify any code (just provide higher resolution assets). There are a number of components that go into this:
First of all, the ExEnScaler (ExEnCore/ExEnScaler.cs) is responsible for all of the transformation maths. It uses fractional maths (ExEnCore/ExEnFractionMaths.cs) so that it is always pixel perfect – it doesn’t suffer from floating-point precision issues. (This could be expanded to allow arbitrary “virtual” backbuffer sizes – something I never got around to.)
Then the View and View Controller (ExEnEmTouch/ExEnEmTouchGameView.cs) and the iOS GraphicsDeviceManager are responsible for setting up the backbuffer and ExEnScaler to the correct sizes. The code flow is a bit complicated because it goes through iPhoneOSGameView. But there are lots of comments, “Find All References” is your friend, and I can help via email if need be.
Finally, the Texture2D and SpriteFont loaders on iOS (ExEnEmTouch/BuiltInLoaders.cs) will load up “@2x” versions of those assets when required. To the user these Texture2Ds and SpriteFonts appear to be of normal size, but under-the-hood they provide retina-sized data.
Faster Fonts and Automatic Retina Font Generation: ExEn font data is generated with the ExEnFontShim XNA content pipeline extension. This generates PNG and custom binary metrics files for sprite fonts, as well as automatically creating “@2x” versions of the fonts suitable for retina displays.
These fonts can be loaded on the device much, much, much faster than trying to decode an XNB font. Especially if the fonts are DXT3 compressed (which XNB fonts are by default).
Seamless Orientation Handling: The ExEnScaler is also capable of handling orientation changes. Although this is only used on iOS, and only because ExEn was written to support iOS < 4.2 where it was necessary to handle orientation yourself for performance reasons (ExEn blocks iOS from modifying the view orientation).
ExEn implements orientation as-per XNA on WP7. Most of the magic happens in ExEnEmTouchGameView.cs and ExEnAndroidSurfaceView.cs on iOS and Android respectively. It has special handling for the upside-down orientation on iPad.
The ExEn Orientation sample demonstrates orientation changes working correctly. In particular it handles transitioning from the iOS splash screen in any orientation at startup without any flicker (try restarting the app in different orientations).
Custom SurfaceView on Android that Doesn’t Crash: When I was making the Android version of ExEn, Mono for Android had a bug where the built-in game view in its OpenTK implementation would crash. From memory it is because it didn’t handle the activity pausing correctly (amongst other things). ExEn on Android provides a complete and correct implementation of OpenGL initialisation on Surface View (ExEnAndroid/ExEnAndroidSurfaceView.cs), covering all the OpenGL and multi-threading complexities.
Backgrounding Doesn’t Crash on iOS: It seems like MonoGame will crash on iOS (on the device) when the game gets backgrounded. ExEn doesn’t crash because it correctly pauses the game loop without touching the frame buffer. (ExEnEmTouch/ExEnEmTouchApplication.cs is a good starting point, also look at how isPausing is used in ExEnEmTouchGameView.cs.)
ExEn exposes a custom API (Game.OnEnterBackground) for users who want to respond to backgrounding (say, by saving their data). Although it would be nicer to convert this to match the WP7 tomb-stoning API. (ExEn was started before WP7 came out.)
And ExEn also correctly implements activation/deactivation events (not to be confused with backgrounding). Not sure if MonoGame’s implementation is correct – but it does exist.
Total refactoring of the game and graphics stuff: ExEn better matches the way XNA uses classes like GraphicsDeviceManager, GraphicsDevice, GameWindow, etc. It also allows better sharing of code between platforms by putting common code into ExEnCommon.
The most important change is to turn GameWindow into a thin cross-platform class, moving platform-specific stuff into separate classes (ExEnEmTouchGameView.cs and ExEnAndroidSurfaceView.cs), which have then themselves been significantly improved.
The way ExEn uses GraphicsDeviceManager as the platform-specific way of handling graphics startup better matches the behaviour of XNA. Also ExEn’s order-of-method-calls correctly matches the behaviour of XNA – including rarely used (but still important) things like calling BeginDraw and EndDraw (and using the result).
ExEn has its own implementation of the Game Components code. It’s fully tested to ensure it matches the method-call order of XNA. (I think MonoGame has some bugs here still.)
It’s important to get the calling order of these methods correct – otherwise it can introduce subtle bugs into the library user’s game.
I know this isn’t an easy one to shift into MonoGame, as it would replace a lot of code. But perhaps it would be worth it, as these classes coordinate some of the advanced features in ExEn. And they would themselves bring in a lot of the fixes mentioned in this document.
Hardware Accelerated Music on iOS: ExEn does all its audio through the AudioQueue API, whereas MonoGame uses OpenAL.
For sound effects, ExEn’s code works ok (except that I never implemented panning). But I think OpenAL is a better choice for MonoGame – especially as their OpenAL code will work cross-platform.
(Not sure if MonoGame supports this, but ExEn has handling for restarting looped sound effects and music when returning from the background.)
However for music playback I would strongly advocate lifting that code from ExEn.
ExEn supports hardware accelerated music decoding – allowing compressed music to be streamed directly from flash memory to the hardware decoder. This will free up a lot of CPU and RAM for more useful things (software decoding compressed audio is slooooooow).
It also correctly supports the game going to and from background on iOS. It supports playing nicely with other music-playing applications (eg: the iPod app). It’s also properly thread-safe, unlike MonoGame’s implementation (especially the MediaPlayer class).
(In ExEnEmTouch, see SoundEffect.cs and SoundEffectInstance.cs for the nuts-and-bolts of decoding and playing audio. See AudioSessionManager.cs for properly setting audio categories. See MediaPlayer.cs for correct implementation of locking plus various minor bug fixes. Finally Song.cs – there’s not much to it.)
Audio Improvements on Android: Similar to ExEn on iOS. ExEn on Android uses SoundPool for sound effects and Android Media Player for hardware-accelerated music. It correctly handles stopping and restarting audio in progress when the activity is paused (sent to the background). It also is correctly thread-safe.
(In ExEnAndroid: See SoundEffect.cs and SoundEffectInstance.cs for SoundPool playback. See MediaPlayer.cs and Song.cs for Android Media Player playback. See ExEnAndroidActivity.cs for where pausing/resuming handling begins.)
CatGirls Testing Application: This is a series of graphical interactive unit tests for various ExEn features. It allows easy verification of features across each supported platform. It’s an XNA game so it should be easy enough bring it into MonoGame as a sample.
The Orientation sample is also worth bringing in to verify orientation behaviour, particularly at startup (see the section on Orientation, above).
Cross-Platform Pluggable ContentManager: The logic of the ContentManager class (ExEnCore/Content/ContentManager.cs) has been made cross-platform. Actual content loaders are pluggable. ExEn uses this for the platform-specific business of actually loading content files (e.g.: ExEnEmTouch/Content/BuiltInLoaders.cs).
End-users can use this as a simple way to add platform-specific methods of loading non-XNB content. (ExEn does not include an XNB reader, so other methods of getting Content Pipeline content into ExEn had to be devised.)
Various Cleanup: Many of the classes in ExEn have been neatly formatted. The named colours are split out of the main Color type (ExEnCore/Graphics/NamedColors.cs). Maths types are organised into cohesive sections.
Platform-agnostic parts of various classes are partial classes in the ExEnCommon area (removing a lot of the convolution required to share code between platforms).
For example: the common parts of the Game class make up 259 lines of code. The platform specific parts of the Game class on iOS are only 86 lines. On Android: 49.
Take a look at the files (here, here and here) to get a full appreciation of this.
ExEn seems to implement IDisposable more consistently and correctly than MonoGame.
Other Stuff: Perhaps the rendering code in ExEn could be ignored in favour of MonoGame’s rendering code. I hear you have made serious improvements in this area. Still it might be worth looking at my SpriteBatch and Texture2D classes for some ideas.
~
Have fun with my code! If you have any questions about it – please don’t hesitate to contact me!
Comments
Dan
I’ve been following ExEn for more than half a year, in anticipation of using it while I’ve been developing a game part-time. You are a very talented developer, and as a developer myself, I would hate to see your code not being utilized.
I understand that maintaining ExEn is not viable, financially, but I would be willing to contribute to a fundraiser (as you initially did for ExEn developement) in order to get your code into MonoGame. I assume others, especially those who are currently using ExEn, could be willing as well.
I understand if this doesn’t sound like an option to consider, but I thought I would give it a try!
Dan (again)
Ok, sorry, disregard my comment above. I noticed you replied to the same type of comment in the previous blog post, regarding funding.
Andrew Russell
That’s ok 🙂 Thanks for your comment anyway, Dan.