Libmad on Android with the NDK

So i was porting all the decoders i had build for the onset detection tutorial to C++, using libmad as the mp3 decoder of choice. After getting that to work on the desktop properly i had to make it work on Android too. Now, there’s no build of libmad in the NDK for obvious reasons, so i had to build that myself. As the autotools configure script of libmad is not useable with the NDK toolchain i used the config.h file from http://gitorious.org/rowboat/external-libmad/blobs/raw/master/android/config.h, which has all the settings needed for building libmad on Android. Compiling libmad is then a simple matter of creating a proper Android.mk and Application.mk file. The Android.mk file looks like this:

LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)

LOCAL_MODULE    := audio-tools
LOCAL_ARM_MODE := arm
LOCAL_SRC_FILES := NativeWaveDecoder.cpp NativeMP3Decoder.cpp mad/bit.c mad/decoder.c mad/fixed.c mad/frame.c mad/huffman.c mad/layer12.c mad/layer3.c mad/stream.c mad/synth.c mad/timer.c mad/version.c
LOCAL_CFLAGS := -DHAVE_CONFIG_H -DFPM_ARM -ffast-math -O3

include $(BUILD_SHARED_LIBRARY)

Now there’s a couple of things that initially bogged down the performance of this. I tested it with the song “Schism” by tool which is a 6:47min long song, encoded at 192kbps. The file weights in at 9.31mb, pretty big for an mp3 imo. NativeMP3Decoder is just a libmad based implementation of the MP3Decoder in the onset detection tutorial framework. So it has a simple NativeMP3Decoder.readSamples method which will fill a float array with as many samples as there are elements in the float array. If the input file is in stereo the channels get mixed down to mono by averaging. The NativeMP3Decoder.readSamples() method internally calls a native method with a similar signature. Instead of a float array i pass in a direct ByteBuffer that has enough storage to hold all the samples requested. The native wrapper then writes the samples to this direct buffer which in turn then gets copied to the float array passed into the NativeMP3Decoder.readSamples() method. It looks something like this:

public int readSamples(float[] samples) 
{	
   if( buffer == null || buffer.capacity() != samples.length )
   {
      ByteBuffer byteBuffer = ByteBuffer.allocateDirect( samples.length * Float.SIZE / 8 );
      byteBuffer.order(ByteOrder.nativeOrder());
      buffer = byteBuffer.asFloatBuffer();
   }
		
   int readSamples = readSamples( handle, buffer, samples.length );
   if( readSamples == 0 )
   {
      closeFile( handle );
      return 0;
   }
	
   buffer.position(0);
   buffer.get( samples );
	
   return samples.length;
}

The call to buffer.get( samples ) kills it all. Without any optimizations (thumb code, -O0, -DFPM_DEFAULT == standard fixed point math in libmad, no arm assembler optimized fp math) decoding the complete files takes 184 seconds on my Milestone. Holy shit, batman! If i eliminate the buffer.get( samples ) call that gets down to 44 seconds! Incredible. Now i still thought that is way to slow so i started adding optimizations. The first thing i did was compiling to straight arm instead of thumb code. You can tell the NDK toolchain to do so by placing this in the Android.mk file:

LOCAL_ARM_MODE := arm

With this enabled decoding takes 36 seconds. The next thing i did was agressive optimization via -O3 as a CFLAG. That shaved off only 2 more seconds, so nothing to write home about. The last optimization is libmad specific. The config.h file i linked to above does not define the fixed point math mode libmad should use. Now, when you have a look at fixed.h of libmad you can see quiet some options for fixed point math there. There’s also a dedicated option for arm processors that uses some nice little arm assembler code to do the heavy lifting. You can enable this by passing -DFPM_ARM as a CFLAG. Now that did wonders! i’m now down to 20 seconds for decoding 407 seconds of mp3 encoded audio. That’s roughly 20x real-time which is totally ok with me. The song i chose is at the extreme end of the song length spectrum i will have to handle in my next audio game project. A song a user uses will be processed once and waiting for that 20 seconds is ok in my book.

I’m afraid i won’t release the source of the ported audio framework as it’s a bit of a mess and would need some work to clean up. What i can give you is the plain source for the native side of the NativeMP3Decoder class if you guarantee me not to laugh. My C days are long over so there’s probably a shitload of don’ts in there. The “handle” system is also kind of creative but good enough for my needs. I learned how to use the low level libmad api by looking at the code here. I actually like doing it this way more than with the shitty callback high-level API. Your mileage may vary. So here it is, be afraid:

#include "NativeMP3Decoder.h"
#include "mad/mad.h"
#include 
#include 

#define SHRT_MAX (32767)
#define INPUT_BUFFER_SIZE	(5*8192)
#define OUTPUT_BUFFER_SIZE	8192 /* Must be an integer multiple of 4. */

/**
 * Struct holding the pointer to a wave file.
 */
struct MP3FileHandle
{
	int size;
	FILE* file;
	mad_stream stream;
	mad_frame frame;
	mad_synth synth;
	mad_timer_t timer;
	int leftSamples;
	int offset;
	unsigned char inputBuffer[INPUT_BUFFER_SIZE];
};

/** static WaveFileHandle array **/
static MP3FileHandle* handles[100];

/**
 * Seeks a free handle in the handles array and returns its index or -1 if no handle could be found
 */
static int findFreeHandle( )
{
	for( int i = 0; i < 100; i++ )
	{
		if( handles[i] == 0 )
			return i;
	}

	return -1;
}

static inline void closeHandle( MP3FileHandle* handle )
{
	fclose( handle->file );
	mad_synth_finish(&handle->synth);
	mad_frame_finish(&handle->frame);
	mad_stream_finish(&handle->stream);
	delete handle;
}

static inline signed short fixedToShort(mad_fixed_t Fixed)
{
	if(Fixed>=MAD_F_ONE)
		return(SHRT_MAX);
	if(Fixed<=-MAD_F_ONE)
		return(-SHRT_MAX);

	Fixed=Fixed>>(MAD_F_FRACBITS-15);
	return((signed short)Fixed);
}


JNIEXPORT jint JNICALL Java_com_badlogic_audio_io_NativeMP3Decoder_openFile(JNIEnv *env, jobject obj, jstring file)
{
	int index = findFreeHandle( );

	if( index == -1 )
		return -1;

	const char* fileString = env->GetStringUTFChars(file, NULL);
	FILE* fileHandle = fopen( fileString, "rb" );
	env->ReleaseStringUTFChars(file, fileString);
	if( fileHandle == 0 )
		return -1;

	MP3FileHandle* mp3Handle = new MP3FileHandle( );
	mp3Handle->file = fileHandle;
	fseek( fileHandle, 0, SEEK_END);
	mp3Handle->size = ftell( fileHandle );
	rewind( fileHandle );

	mad_stream_init(&mp3Handle->stream);
	mad_frame_init(&mp3Handle->frame);
	mad_synth_init(&mp3Handle->synth);
	mad_timer_reset(&mp3Handle->timer);

	handles[index] = mp3Handle;
	return index;
}

static inline int readNextFrame( MP3FileHandle* mp3 )
{
	do
	{
		if( mp3->stream.buffer == 0 || mp3->stream.error == MAD_ERROR_BUFLEN )
		{
			int inputBufferSize = 0;
			if( mp3->stream.next_frame != 0 )
			{
				int leftOver = mp3->stream.bufend - mp3->stream.next_frame;
				for( int i = 0; i < leftOver; i++ )
					mp3->inputBuffer[i] = mp3->stream.next_frame[i];
				int readBytes = fread( mp3->inputBuffer + leftOver, 1, INPUT_BUFFER_SIZE - leftOver, mp3->file );
				if( readBytes == 0 )
					return 0;
				inputBufferSize = leftOver + readBytes;
			}
			else
			{
				int readBytes = fread( mp3->inputBuffer, 1, INPUT_BUFFER_SIZE, mp3->file );
				if( readBytes == 0 )
					return 0;
				inputBufferSize = readBytes;
			}

			mad_stream_buffer( &mp3->stream, mp3->inputBuffer, inputBufferSize );
			mp3->stream.error = MAD_ERROR_NONE;
		}

		if( mad_frame_decode( &mp3->frame, &mp3->stream ) )
		{
			if( mp3->stream.error == MAD_ERROR_BUFLEN ||(MAD_RECOVERABLE(mp3->stream.error)))
				continue;
			else
				return 0;
		}
		else
			break;
	} while( true );

	mad_timer_add( &mp3->timer, mp3->frame.header.duration );
	mad_synth_frame( &mp3->synth, &mp3->frame );
	mp3->leftSamples = mp3->synth.pcm.length;
	mp3->offset = 0;

	return -1;
}

JNIEXPORT jint JNICALL Java_com_badlogic_audio_io_NativeMP3Decoder_readSamples__ILjava_nio_FloatBuffer_2I(JNIEnv *env, jobject obj, jint handle, jobject buffer, jint size)
{
	MP3FileHandle* mp3 = handles[handle];
	float* target = (float*)env->GetDirectBufferAddress(buffer);

	int idx = 0;
	while( idx != size )
	{
		if( mp3->leftSamples > 0 )
		{
			for( ; idx < size && mp3->offset < mp3->synth.pcm.length; mp3->leftSamples--, mp3->offset++ )
			{
				int value = fixedToShort(mp3->synth.pcm.samples[0][mp3->offset]);

				if( MAD_NCHANNELS(&mp3->frame.header) == 2 )
				{
					value += fixedToShort(mp3->synth.pcm.samples[1][mp3->offset]);
					value /= 2;
				}

				target[idx++] = value / (float)SHRT_MAX;
			}
		}
		else
		{
			int result = readNextFrame( mp3 );
			if( result == 0 )
				return 0;
		}

	}
	if( idx > size )
		return 0;

	return size;
}

JNIEXPORT jint JNICALL Java_com_badlogic_audio_io_NativeMP3Decoder_readSamples__ILjava_nio_ShortBuffer_2I(JNIEnv *env, jobject obj, jint handle, jobject buffer, jint size)
{
	MP3FileHandle* mp3 = handles[handle];
	short* target = (short*)env->GetDirectBufferAddress(buffer);

	int idx = 0;
	while( idx != size )
	{
		if( mp3->leftSamples > 0 )
		{
			for( ; idx < size && mp3->offset < mp3->synth.pcm.length; mp3->leftSamples--, mp3->offset++ )
			{
				int value = fixedToShort(mp3->synth.pcm.samples[0][mp3->offset]);

				if( MAD_NCHANNELS(&mp3->frame.header) == 2 )
				{
					value += fixedToShort(mp3->synth.pcm.samples[1][mp3->offset]);
					value /= 2;
				}

				target[idx++] = value;
			}
		}
		else
		{
			int result = readNextFrame( mp3 );
			if( result == 0 )
				return 0;
		}

	}
	if( idx > size )
		return 0;

	return size;
}

JNIEXPORT void JNICALL Java_com_badlogic_audio_io_NativeMP3Decoder_closeFile(JNIEnv *env, jobject obj, jint handle)
{
	if( handles[handle] != 0 )
	{
		closeHandle( handles[handle] );
		handles[handle] = 0;
	}
}

To compile that for Android all you have to do is download libmad and put the source files into your Android project’s jni folder along with the code above. Then use the Android.mk from above et voila you got yourself a native mp3 decoder for Android. You can use it in combination with the AndroidAudioDevice class of the last post. If you feel adventureous you could even extend it to return stereo data.

12 thoughts on “Libmad on Android with the NDK

  1. Man, this is awesome! With your article I ported my sound mixer component very fast. Many many thanks!

  2. libmad sucks license wise. i gave it up in favor of libmpg123, it’s lgpl without all the other license crap that comes with libmad. For something similar like the above for libmpg123 check out the libgdx sources. All my code is Apache License 2.0 so you can basically do what you want as long as you leave the license headers in the files.

  3. I tried libmad like you did but it’s strange that I got external/qemu/tcg/tcg.c:1363 problem after a few frame being decoded and the emulator switched off. Have you faced the same problem and know how to solve it?

  4. Hm, i’m afraid i didn’t experience that problem. I’d suggest looking into libmpg123 which has a better license anyway. You’d have to license libmad for anything commercial.

  5. I tried libmpg123 also. But I got an output file which is smaller than the input one and when I tried to “play -t cdr output” then I heard only noise ( I used your code ). I really don’t know what happened. Any idea?

  6. Hi Mario. I downloaded your source code for libmg123. It worked but for some file it could not decode the whole file. I did something like this :

    do{
    int index = readSample(…,buffer,….);
    ….write buffer to file….
    if(index == 0)
    break;
    }while (1)

    Do you have any idea?

  7. I debugged and I got “Trouble with mpg123: Error reading the stream. (code 18)” while decoding file. Still don’t know why this happened to me :)

  8. Hi Mario,

    is it possible to obtain raw stream from android audio jack . actually I want to read sensor data . I hope you can help me .

    Regards

  9. Hi Mario!

    I’ve tried using libmad as well for my Android apps and I followed your codes, but somehow it took 1 minute to decode 4:45 mp3 file. I’ve been wondering why it is so slow.
    Any tips?

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>