Android Game Development Blog

A blog on Android game development by Douglas Tanner

Ogg Vorbis audio decoding on Android NDK



    Why Ogg Vorbis?
    
  • MP3 technology is encumbered by patents and I don't want to have to worry about paying royalties or being sued sometime in the future.
  • Vorbis is variable bitrate by default, and in listening tests the quality is at least as good as MP3.
  • Vorbis also supports gapless encoding, which is a requirement for seamless looping (possible with MP3, but hard).
  • There's a very efficient ARM-optimized New-BSD-licensed (free to use) decoder called Tremor (there is also Tremolo, which is even faster).

    Compiling an open-source C++ project for Android NDK
    
The first step is to download the source code and save it to a directory on your hard drive. Then we'll open up a Cygwin prompt by running C:\cygwin\Cygwin.bat and navigate to that folder. Then execute the following command:



This creates a config.h file that tells the open-source project a bit about the destination architecture and environment.

Note: I don't know if those parameters above are exactly right. If you do know the optimal parameters, answer my question on Stack Overflow.

configure.png
Now, you can either do things the hard way and use the Makefile to compile an Android library (.so) and link that into your project. Or just drag-and-drop the required files into Visual Studio:

tremor_vs.png
Note: Before embedding an open-source library into Visual Studio instead of linking a library or loading a .DLL, check the license terms of the project to make sure this is ok.


    Enable optimizations and fix compile problems

First off, I had some compile errors which I fixed by adding the following to os.h:



Then I wanted to enable the ARM-specific assembler optimizations (which sped up the decoding by about 100%), so I added the following to asm_arm.h:




    Convert Ogg/Vorbis to PCM
    
The Tremor library is designed to work with FILE* input, but we're able to create our own callbacks to duplicate this behaviour while reading a memory buffer:



Now we can send it an Ogg/Vorbis buffer already resident in memory:



Saving data to internal storage on Android



    Saving to Internal Storage

The easiest place on Android to store small files tracking persistent data from your game is probably in the internal storage. You don't have to worry about SD cards being mounted or not, or needing any extra permissions in your manifest. The files can be created in a location un-accessible to other programs and the user, they will also be deleted automatically when your program is un-installed. Some things you might want to store: High Scores, Character Progression, Unlockables, etc...



Here is the code that you can use to return a Java array from JNI:



    Loading from internal storage

    
Loading is a bit trickier because you have to get the file size separately from the input stream. getFilesDir() returns the directory that contains the files created with openFileOutput().



When you need to send the byte array to JNI you can do the following:



    Verify your data

   
As always when saving/loading data from a physical storage medium, it's a good idea to verify that the data has not become corrupted (or been modified). You can use the code below to generate a CRC32 hash of your data and then append it to the end of your buffer.



Next time I will show you Ogg Vorbis audio decoding with the Android NDK.

Streaming Android AudioTracks from JNI/C++



    Playing a sample with an AudioTrack (Final version)

With the simple implementation of a static AudioTrack I showed you last time, as soon as the audioTrack variable goes out of scope it has the chance of being garbage collected. There is also no obvious place to insert this code and have it interface nicely with JNI. So what I did was put all my static AudioTrack processing in its own thread (streaming audio will get its own thread as well).

One very important thing to realize about an AudioTrack is that getPlayState() will always return PLAYSTATE_PLAYING unless you manually stop the track, even if the track is finished. To get around this rather amateurish design decision you have to compare the getPlaybackHeadPosition to a stored value for the number of samples you wrote to the AudioTrack to know when the tracks are finished (so that you can release their pointers and let them be garbage collected).



Here is the JNI code I use to create a new Java array and return it:




    Streaming into an AudioTrack
   
When streaming audio into an AudioTrack we follow the opposite sequence of calls from a static AudioTrack; we call play() and then write() into it. The way I have it set up below, C++ always returns a buffer every time the AudioTrack requires more data, but if no music is playing it writes silence into that buffer. This way I don't have to deal with pausing/unpausing the AudioTrack which was causing me a whole slew of problems for me.

My choice of 4x the size of the minimum buffer size for the streaming buffer was purely arbitrary, ~50K bytes is a good buffer size from my experiences with audio on other platforms (the minimum size on my test phone was 12288 bytes).
   


Here is the JNI code I use to copy into the Java short array:




    Starting and stopping the threads

   
At the moment I'm calling these in onResume() and onPause().
   




Next time I will show you how to store files in Android internal storage.

Playing a .wav PCM sample with AudioTrack



Unfortunately in Android 2.2, OpenSL has not yet been implemented for native audio playback in C++ so I will be using the Java audio functions for now.


Loading a RIFF format PCM .wav

If you are packing your audio data yourself like I am, the first step is to load up a .wav file.



This isn't the safest code and I don't think it covers all cases (a 'list' chunk can sometimes be after the data chunk), but it worked for me with .wav saved by both Windows Sound Recorder and Audacity 1.3.13. If you find some .wav that are giving you problems, loading and then exporting them back to .wav with something like Audacity ought to do the trick.

The fields that are important to us are the number of channels (numOfChan), the sample size (bitsPerSample), and the number of bytes per second (samplesPerSec). The PCM data we loaded into pvDataPCM can be sent directly to the AudioTrack below as a short[].


JNI Interface

I've decided to embed my audio samples into my already existing packed data file, and load the them the same way I load my OpenGL textures (in C++). This is the best way to go I feel, in preparation for 2.3 when we'll be able to play audio directly from C++ with OpenSL. So for now we need a way to get the information out of the C++ code and into the Java code.

On the Java side:



And on the C++ side:





Playing the sample with an AudioTrack





Next time I'll show you how to stream music into an AudioTrack as well as create two seperate threads, one for static samples and one for streaming samples.

Finding Free 2D Game Art



    Understanding licenses

Although all of the art on the following sites is 'free', some of the licenses attached to the art mandate certain restrictions. For example some of the authors have chosen licenses that force your entire project to use the same license as the single piece of artwork, others oblige you to open-source your source-code and include it with your game. Of course, you can try to contact the author to have these restrictions waived on a case-by-case basis, or perhaps offer to purchase a license to the art.

One very acceptable condition that some licenses attach is an attribution clause, where you have to acknowledge the creator of the art (most likely in your credits). Another acceptable (but very annoying) condition is that you need to include the original source of the art you are using. I'm honestly not sure how that would work with an Android package though, I don't know if an uncompressed .png included in the package would be sufficient?

The good licenses (truly free, or attribution):


The annoying licenses (not sure exactly how to handle them):


The bad licenses (author forces his personal philosophy on your entire project):



    The Sites

Open Game Art
    
Directly targeted towards game art, with a great search interface where you can specify exactly the licenses you want. Seems mostly geared towards RPG sprite artwork however, and very few of the sprites are animated. If you're looking to make a Dragon's Quest style RPG with static fight scenes this place is heaven on earth. For other types of games, not too useful.

Free Art Search

A pretty good image search which seems to have a bit more selection than Open Game Art, with the trade-off that most of the licenses are 'unknown' and you would have to track down the original author to verify that you can use it. And again, almost none of the images are animated.

Reiner`s Tilesets

Completely free to use 2D and 3D artwork done in an Ultima Online art style.

Open Clip Art

Although not at all game focused, this site is actually a lot more useful to me than the above ones. Obviously being clip-art, nothing is animated, but every single image is released under the CC0 Public Domain Dedication, so anything I find I'm guaranteed to be able to use. Also the art styles are more similar to what I envision for my project than the pixel art from the two previous sites. The search works beautifully and there is a huge selection to chose from.

Icon Finder

A great search engine for small images and GUI elements. It is very easy to filter the search to exclude non-commercial-allowed images.

   
    The Collections
   
GfxLib-Fuzed

A great collection of platformer game graphics, including tilesets, sprites and backgrounds. Professionally done with full animations on the sprites. Artwork is royalty free and can be used commercially.

SpriteLib

A big collection of 80s-style arcade sprites. Contains lots of airplane and tank graphics, as well as some more generic platformer sprites. A welcome change from all the RPG-heavy graphics I've found elsewhere. Free to use for any use.

Circle Textures

Beautiful terrain tiles, could conceivably be used for the terrain of an RTS game. Free to use by anyone, with attribution.

Airplane Sprite Pack and Airplane and Tank Sprite Pack

Fucking cool looking planes and tanks. Free to use with attribution and notification of the author. Makes me wish I was working on an airplane game, these sprites are awesome.


    Conclusion
   
Based on a quick look around on the above sites I'd say that they're not useful for much more than placeholder art and perhaps GUI elements in your final game. You'd be better off creating a game based on the art you find rather than trying to find art for an existing game concept. The quantity simply isn't there for you to be able to find something in particular that you're looking for. Looks like I'll be looking to hire an artist some time in the future.


Next time I will show you how to play .wav files with AudioTrack.

Android ATITC (ATC) texture encoding and performance



As I discussed before in my blog post about PVRTC texture encoding and performance, every Android OpenGL ES 2.0 phone supports at least two texture formats: ETC1 and one of the three proprietary formats (PVRTC, ATITC and S3TC). I just got myself a cheap second-hand Nexus One (with an Adreno 200 graphics chip), so I wanted to test out the ATI-specific texture format and see what the trade-offs are vs. ETC1 texture performance.


    Get the Adreno SDK
    
Get adreno-sdk-2-2-00.zip from the Qualcomm developer website. Free registration is required (WARNING: They send you your own password in plaintext). Once installed it will be placed in C:\Program Files (x86)\Qualcomm\AdrenoSDK\.


    Encoding ATC textures

Unlike with PVRTC, the SDK does not contain a stand-alone executable, it contains a library that has to be directly linked into your packing program. The documentation explaining in detail how to do that is located in AdrenoSDK\Tools\Texture Converter\docs. After including the appropriate header file (TextureConverter.h) and linking with the library (TextureConverter.lib), your code will look something like the following (this is directly based on AdrenoSDK\Tools\Texture Converter\samples\Convert.c).

The first step is to initialize the source image (in my case a .tga file):



Then we setup the destination image (either ATITC_RGB or ATITC_RGBA):



Finally we do the actual conversion from .tga to ATC:




    Loading ATC textures

Nothing special needs to be done to load ATC textures, just make a regular call to glCompressedTexImage2D with the right constant (substitute GL_ATC_RGBA_EXPLICIT_ALPHA_AMD for ATC alpha textures).




    ATC texture performance

For my test case setup, read blog post on ETC1 texture performance. The only difference is that these results are from a Nexus One which has an Adreno 200 instead of the PowerVR SGX on my Milestone. These are the results I got:



These results are very interesting, they show that the graphics architecture of the Nexus One is vastly different from that of the Milestone. There is absolutely no performance hit to using uncompressed textures, compared to a setup that was giving me a 13-33 fps difference between compressed and uncompressed on my Milestone. It's definitely worth using compressed textures for the file size difference, but unless your game has some crazy-complex textures you'll probably be fine with the more compatible ETC1 format.

However, one thing I will mention is that the ATC texture format is subjectively 'better looking' than the ETC1 format (to my eyes anyway). This is possibly due to better hardware decoding/interpolation support. When I manage to get my hands on a phone that supports S3TC I think I'll do an in-depth visual quality comparison between the four formats.

Next time I will show you how to find free 2D game art.

POSIX threads on Android



    Threading setup for an Android OpenGL Game
    
I'm not going to go into too much detail about how to setup your thread organization other than to say that for performance reasons you need an update thread separate from your render thread, because eglSwapBuffers() is a blocking call on the render thread. For an in-depth discussion of how to structure things check out Rendering With Two Threads.

My basic plan is to have an update thread running at 60 fps (to minimize input lag) and my render thread running at 30 fps. My render thread will grab the latest snapshot from the update thread when it begins and then use this information to build the graphics primitives and upload them to the GPU. I may also have a separate thread to handle audio updates and streaming, but I'll deal with that later when I've actually implemented audio.


    Creating POSIX threads



You'll notice right away that I'm doing something pretty weird; I'm creating/destroying my threads in OnResume/OnPause. If you look at the application lifecycle graph you'd assume the best place to create and destroy a thread would be in onCreate/onDestroy. But, there is no guarantee that either onStop or onDestroy is ever called! There are some sneaky little lines on the left side of the graph that lead directly to "Process is killed". This is really atrocious design, why even have states that can randomly never be called? What code would you possibly want to put in them? I'm sure there's a good reason for the way this is set up, but for now it eludes me... So anyway, I decided to create/kill my update thread in onResume/onPause because I am guaranteed that these states will be called every time my application is run. The initial state of the threads is set up in onCreate.


    Mutual exclusion (critical section)




    Thread communication (semaphores)




    Thread local storage


 

Next time I will explain ATITC (ATC) texture encoding and performance.

Input - Touch Screen, Hardware Keyboard, and Back/Menu/Home/Search



What is a game without interaction? It's time to start accepting input!

    Touch Screen
   
This is the important one! Because the trackball and keyboard are not supported by every phone, your game needs to be playable and navigable using only the touchscreen.


   



    Intercept key-presses (back/menu/search)
   
Every Android phone has 4 physical buttons on the front: back, menu, home, and search (although not necessarily in that order). For a game it's a good idea to override the back button and insert an "Are you sure you want to quit?" dialog box. And obviously the menu button should be there to expose more advanced options to your power users (basic options can be put directly in the front-screen menu). For now I'm just using the physical buttons for debug input, I'll probably cover their actual in-game use in more detail later when I actually have a menu and proper back handling.




    Override home button?

You can't! Unless you're writing a full home replacement (like AWD launcher for example). It's very important for a user to have a 'safety button' that can always return him to his home screen, otherwise it would be possible for a malicious application to completely take over a user's phone.

    Hardware Keyboard

   
The first thing you will want to do for phones with hardware keyboards is to stop your application from restarting when the user opens the hardware keyboard. For regular applications this is important to properly re-set the visual formatting, but for a game that has a fixed landscape orientation you definitely don't want this to occur.



Next time I'll explain how to use POSIX threads on Android.


    Delta Time
    
To profile the framerate of your game, you'll have to query the high resolution timer supported by your hardware. On Android you can use the clock_gettime call to get the current time:



This call returns the number of hardware ticks since the start of your application. By itself this number is useless of course, you need to compare it to the time from the previous frame:



Because a timespec is a 64-bit type, if you just compared the number of nanoseconds from frame to frame, tv_nsec would eventually overflow and you one frame you'd have a large negative time delta.


    Full-screen and forced landscape orientation

   
Use the following code to fully full-screen your game, and also to force landscape orientation.




    Prevent application restart on orientation change

Warning: The following code doesn't seem to play well with OpenGL ES 2.0 and the GLSurfaceView (API version 2.2). On my Nexus One the OpenGL surface does not get re-created and it causes the renderer to blit garbage onto the screen.

Add the following android:configChanges to your AndroidManifest.xml



Override onConfigurationChanged in your Activity to prevent application restart




    A random error message

After having my workflow working fairly well for over a month, I randomly started receiving the following error message: "The Program can't start because AdbWinApi.dll is missing from your computer"

I'm just reproducing the solution here in case anyone ever Googles around for the same problem. My solution was to add the following to the end of my "Path" environment variable:



Next time I will show you how to use the various types of input available to Android devices.
 

Android PVRTC texture encoding and performance



Every Android OpenGL ES 2.0 phone supports at least two texture formats: ETC1 and one of the three proprietary formats (PVRTC, ATITC and S3TC). My phone supports PVRTC, so I wanted to see what the trade-offs were vs. ETC1 texture performance.

Bear in mind, if you want to support one of the proprietary formats, you'll have to support the other two as well and hence deal with packing all three formats in your data and correctly detecting which type to use at run-time.


    Encoding PVRTC textures

The first step is to download and install PVRTexTool_2.08.28.0634.msi from the Imagination website (requires free registration). The command line tool PVRTexTool.exe can be found in C:\Program Files (x86)\Imagination Technologies\POWERVR SDK\PVRTexTool_2.08.28.0634\PVRTexToolCL\Windows_x86_32. Here is how to use the compressor:



The first argument selects the format of the texture, either 2 bits-per-pixel or 4 bits-per-pixel. The second argument prevents it from automatically flipping the texture upside down (I had to do this to bring it in line with the texture orientation output by etc1tool.exe). And obviously the next two arguments are the input and output filenames.

The encoder automatically detects the alpha layer in your .png file and encodes the .pvr with or without the alpha channel. One thing to note is that the alpha layer is very low precision, which makes the edges of the texture very jagged.

The encoded file has the following header:




    Loading PVRTC textures in OpenGL

Loading is super simple, it works exactly like it does with ETC1 textures. Simply substitute GL_COMPRESSED_RGB_PVRTC_4BPPV1_IMG, GL_COMPRESSED_RGBA_PVRTC_2BPPV1_IMG or GL_COMPRESSED_RGBA_PVRTC_4BPPV1_IMG as needed:
    


    PVRTC texture performance


For my test case setup, read my last blog post on ETC1 texture performance. These are the results I got:




    Conclusion

This confirms what I suspected, my phone has hardware support for decoding both ETC1 and PVRCT formats, and because I'm not texture bandwidth limited the extra alpha textures have a negligible impact. Using 2 bits-per-pixel PVRTC is not worth the reduced visual quality, and the small loading time gain with 4 bits-per-pixel is not worth the reduced alpha quality. I'll be sticking with ETC1 for this project.

Next time I'll show you how to force full-screen landscape orientation use the high-resolution timer (to measure framerate and delta time accurately), as well as other miscellaneous tips.