PLEASE NOTE THAT Howie Library is deprecated: AAudio API is the new API and available from Android-O Developer Preview 1+ ( DP1 and above). To know more about AAudio, please refer to:

This codelab is built on a starter project, which is a simple Android JNI application with

Let's get it and build:

  1. Get starter project from github
git clone https://github.com/googlesamples/android-audio-high-performance.git
ndk.dir=/your/local/ndk/path
  1. Click the Gradle sync button.
  2. Click Build and Run. You should see

Touch "START ECHO", you should see the button toggles to "STOP ECHO". In the logcat pane of Android Studio you should see the following messages:

I/AUDIO-ECHO: I/AUDIO-ECHO: start : 0
I/AUDIO-ECHO: I/AUDIO-ECHO: stop : 0

Frequently Asked Questions

Our application depends on the Howie audio library, let's get it into our application:

  1. Add Howie engine into dependency list
  1. Add Howie dependency for JNI module
    open app/build.gradle, add following inside model scope:
model {
...
// new code starts
  android.sources {
     main {
         jni {
             dependencies {
                 project ":howie"
             }
         }
     }
  }
// new code ends
...
}
  1. Import Howie library into EchoMainActivity in EchoMainActivity.java:
import android.widget.ToggleButton;

import com.example.android.howie.HowieEngine;  // <=== new code
  1. Initialize Howie library with HowieEngine.init(this) in EchoMainActivity.java, EchoMainActivity class, function onCreate(...):
onCreate(Bundle savedInstanceState) {
     // ...
    System.loadLibrary("audio-echo");

     HowieEngine.init(this);  // <=== new code
  1. Request RECORD_AUDIO permission in app/src/main/AndroidManifest.xml:
    <uses-permission android:name="android.permission.RECORD_AUDIO"/>
  1. Sync , Build and Run application
    you should see the same UI as step "Build the starter app" on your android device

    checkpoint -- your samples/audio-echo/app/src/main/AndroidManifest.xml should now look like this:
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
   package="com.google.sample.audio_echo" >

   <application
       android:allowBackup="true"
       android:icon="@mipmap/ic_launcher"
       android:label="@string/app_name"
       android:supportsRtl="true"
       android:theme="@style/AppTheme" >
       <activity android:name=".EchoMainActivity" android:screenOrientation="portrait">
           <intent-filter>
               <action android:name="android.intent.action.MAIN" />

               <category android:name="android.intent.category.LAUNCHER" />
           </intent-filter>
       </activity>
   </application>
   <uses-permission android:name="android.permission.RECORD_AUDIO"/>

</manifest>

checkpoint -- your .../sample/audio_echo/EchoMainActivity.java should look like:

package com.google.sample.audio_echo;

import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.widget.CompoundButton;
import android.widget.ToggleButton;

import com.example.android.howie.HowieEngine;

public class EchoMainActivity extends AppCompatActivity {
   private   long streamId;
   private   ToggleButton toggle;

   @Override
   protected void onCreate(Bundle savedInstanceState) {

       super.onCreate(savedInstanceState);
       setContentView(R.layout.activity_echo_main);

       System.loadLibrary("audio-echo");
       HowieEngine.init(this);

       toggle = (ToggleButton) findViewById(R.id.toggleEcho);
           toggle.setOnCheckedChangeListener(new CompoundButton.OnCheckedChangeListener() {
                   public void onCheckedChanged(CompoundButton buttonView, boolean isChecked) {
                       startEcho(streamId, isChecked ? true : false);
                   }
               });
   }

   @Override
   protected void onPause() {
       toggle.setChecked(false);
       destroyStream(streamId);
       super.onPause();
   }

   @Override
   protected void onResume() {
       super.onResume();
       streamId = createStream();
   }

   native public long createStream();
   native public void destroyStream(long streamId);
   native public void startEcho(long stream, boolean start);
}

Howie is an efficient native library for interacting with openSL ES library, let's build a loopback Howie stream:

  1. Implement OnDeviceChangedCallback() and OnCleanupCallback() in jni/echo_main.cpp:
    We do not plan to keep any state in our app at this step, so just implement them as pass through functions:
#include <android/log.h>
#include "echo_main.h"

//new code begins ...
#include <cstring>
#include <howie.h> 

HowieError OnDeviceChangedCallback(
        const HowieDeviceCharacteristics *characteristics,
        const HowieBuffer *state,
        const HowieBuffer *params) {
    return HOWIE_SUCCESS;
}

HowieError OnCleanupCallback(
        const HowieStream *stream,
        const HowieBuffer *state ) {
    return HOWIE_SUCCESS;
}
  1. Implement OnProcessCallback() in jni/echo_main.cpp:
    when Howie needs an audio buffer to playback, it calls this function for new audio data, and also provides us with newly recorded PCM audio samples; we simply copy the input stream to given output stream to create a loopback audio stream:
HowieError OnProcessCallback(
        const HowieStream *stream,
        const HowieBuffer *input,
        const HowieBuffer *output,
        const HowieBuffer *state,
        const HowieBuffer *params)
{
    // loopback audio
    memcpy(output->data, input->data, output->byteCount);
    return HOWIE_SUCCESS;
}

// new code ends
  1. Create a Howie stream with playback and recording in function: Java_com_google_sample_audio_1echo_EchoMainActivity_createStream
JNIEXPORT jlong JNICALL
Java_com_google_sample_audio_1echo_EchoMainActivity_createStream(
(JNIEnv *env, jobject instance) {
    HowieStreamCreationParams hscp = {
            sizeof(HowieStreamCreationParams),
            HOWIE_STREAM_DIRECTION_BOTH,
            OnDeviceChangedCallback,
            OnProcessCallback,
            OnCleanupCallback,
            0,
            0,
            HOWIE_STREAM_STATE_STOPPED
    };

    HowieStream *stream = nullptr;
    HowieStreamCreate(&hscp, &stream);
    return reinterpret_cast<jlong>(stream);
}
  1. Destroy the stream before closing application with:
JNIEXPORT void JNICALL
Java_com_google_sample_audio_1echo_EchoMainActivity_destroyStream( 
   JNIEnv *env, 
   jobject instance,
   jlong streamId) {
    HowieStream* stream = reinterpret_cast<HowieStream*>(streamId);
    HowieStreamDestroy(stream);
}
  1. Call HowieStreamSetState() to start/stop loopback stream:
JNIEXPORT void JNICALL
Java_com_google_sample_audio_1echo_EchoMainActivity_startEcho(
       JNIEnv *env,
       jobject instance,
       jlong stream,
       jboolean start) {
    HowieStream* stream = reinterpret_cast<HowieStream*>(streamId);
    HowieStreamSetState(stream, (start == JNI_TRUE) ?
                                HOWIE_STREAM_STATE_PLAYING :
                                HOWIE_STREAM_STATE_STOPPED);
}
  1. Sync , Build and Run application

Your echo_main.cpp file should now look like this:

#include <android/log.h>
#include "echo_main.h"
#include <cstring>
#include <howie.h>

const char * MODULE_NAME="AUDIO-ECHO";

HowieError OnDeviceChangedCallback(
       const HowieDeviceCharacteristics *characteristics,
       const HowieBuffer *state,
       const HowieBuffer *params) {
   return HOWIE_SUCCESS;
}

HowieError OnCleanupCallback(
       const HowieStream *stream,
       const HowieBuffer *state ) {
   return HOWIE_SUCCESS;
}

HowieError OnProcessCallback(
       const HowieStream *stream,
       const HowieBuffer *input,
       const HowieBuffer *output,
       const HowieBuffer *state,
       const HowieBuffer *params)
{
   // loopback audio
   memcpy(output->data, input->data, output->byteCount);
   return HOWIE_SUCCESS;
}

JNIEXPORT jlong JNICALL
Java_com_google_sample_audio_1echo_EchoMainActivity_createStream(
       JNIEnv *env,
       jobject instance) {
   HowieStreamCreationParams hscp = {
           sizeof(HowieStreamCreationParams),
           HOWIE_STREAM_DIRECTION_BOTH,
           OnDeviceChangedCallback,
           OnProcessCallback,
           OnCleanupCallback,
           0,
           0,
           HOWIE_STREAM_STATE_STOPPED
   };

   HowieStream *stream = nullptr;
   HowieStreamCreate(&hscp, &stream);
   return reinterpret_cast<jlong>(stream);
}

JNIEXPORT void JNICALL
Java_com_google_sample_audio_1echo_EchoMainActivity_destroyStream(
       JNIEnv *env,
       jobject instance,
       jlong streamId) {
   HowieStream* stream = reinterpret_cast<HowieStream*>(streamId);
   HowieStreamDestroy(stream);
}

JNIEXPORT void JNICALL
Java_com_google_sample_audio_1echo_EchoMainActivity_startEcho(
       JNIEnv *env,
       jobject instance,
       jlong streamId,
       jboolean start) {
   HowieStream* stream = reinterpret_cast<HowieStream*>(streamId);
   HowieStreamSetState(stream, (start == JNI_TRUE) ?
                       HOWIE_STREAM_STATE_PLAYING : HOWIE_STREAM_STATE_STOPPED);
}

To simulate an echo effect, we will record audio into a delay buffer and mix it with new live audio in file echo_main.cpp:

  1. Define echo delay:
    We delay 75 ms for our echo stream then blend it with incoming live audio stream
// delay time in milliseconds
#define ECHO_DELAY 75
  1. Create echo state:
    Recorded audio samples need to be saved for later blending, we will use a queue to hold saved audio sample buffers, and put the queue into stream state (Howie provides us a pointer to our state in callbacks). Let's declare our state in echo_main.cpp:
#include <cstring>
#include <howie.h>   // <== existing code

// New code begins
#include <queue>

struct EchoState {
   std::queue<char*> *echoQueue_;
   char*             buf_;
};
  1. Initialize stream state in function OnDeviceChangedCallback():
    We need create small buffers to hold the given amount of audio samples for our chosen delay time of 75 ms; the size of each buffer is the same as the one Howie uses for recording and playback. After Howie allocates its internal objects (player/recorder, states and parms), it notifies us via OnDeviceChangedCallback(), this is the place we initialize stream state:
// init our stream states after howie creates device
HowieError OnDeviceChangedCallback(
       const HowieDeviceCharacteristics *characteristics,
       const HowieBuffer *state,
       const HowieBuffer *params) {
   EchoState *echoState = reinterpret_cast<EchoState*>(state->data);
   echoState->echoQueue_ = new std::queue<char*>;
   int BufSize  = (characteristics->sampleRate * ECHO_DELAY) / 1000;
   BufSize -= BufSize % characteristics->framesPerPeriod;
   BufSize *= characteristics->bytesPerSample *
              characteristics->channelCount;

   int bytesPerBuf = characteristics->bytesPerSample *
                     characteristics->channelCount *
                     characteristics->framesPerPeriod;
   // Allocate one contiguous memory chunk
   echoState->buf_ = new char [BufSize];

   // init memory with silent audio
   memset(echoState->buf_, 0, BufSize);

   // use memory as small buffers
   int offset = 0;
   while (offset < BufSize) {
       echoState->echoQueue_->push(&echoState->buf_[offset]);
       offset += bytesPerBuf;
   }

   return HOWIE_SUCCESS;
}

  1. Cleanup our stream state when Howie notifies us that Howie is about to disappear:
// release state after howie destroys stream
HowieError OnCleanupCallback(
       const HowieStream *stream,
       const HowieBuffer *state ) {
   EchoState *echoState = reinterpret_cast<EchoState*>(state->data);
   delete echoState->echoQueue_;
   delete [] echoState->buf_;
   return HOWIE_SUCCESS;
}
  1. Blend live audio in OnProcessCallback():
    When Howie needs a new buffer to play, we blend newly recorded audio sample with delayed audio sample in the queue; at the same time save newly recorded audio to the tail of saved audio samples.
HowieError OnProcessCallback(
        const HowieStream *stream,
        const HowieBuffer *input,
        const HowieBuffer *output,
        const HowieBuffer *state,
        const HowieBuffer *params)
{
      EchoState *echoState = reinterpret_cast<EchoState*>(state->data);
      char* echoBuf = echoState->echoQueue_->front();
      short *echo = reinterpret_cast<short*>(echoBuf);
      short *src = reinterpret_cast<short*>(input->data);
      short *dst = reinterpret_cast<short*>(output->data);

      int samples = output->byteCount / 2;

      while (samples--) {
          *dst++ = *src / 2 + *echo / 2;
          *echo++ = *src++;
      }
      echoState->echoQueue_->pop();
      echoState->echoQueue_->push(echoBuf);
      return HOWIE_SUCCESS;
}
  1. Inform Howie to create state block for our stream:
    We tell Howie our stream state size (in bytes) when we create a Howie steam; Howie will allocate a buffer for our stream state.
JNIEXPORT jlong JNICALL
Java_com_google_sample_audio_1echo_EchoMainActivity_createStream
(JNIEnv *env, jobject instance) {

   HowieStreamCreationParams hscp = {
           sizeof(HowieStreamCreationParams),
           HOWIE_STREAM_DIRECTION_BOTH,
           OnDeviceChangedCallback,
           OnProcessCallback,
           OnCleanupCallback,
           sizeof(EchoState), // ⇐ new change
           0,
           HOWIE_STREAM_STATE_STOPPED
   };
    // ...
}
  1. Click menu Build > Clean Project, then Build and Run
  1. Experience different echo value
// delay time in milliseconds
#define ECHO_DELAY 200
  1. Experience different echo effect
// delay time in milliseconds
#define ECHO_DELAY 1000
*dst++ = *src / 2 + *echo / 2;
*echo++ = *src++;
*dst = *src / 2 + *echo / 4;
*echo++ = *dst++;
src++;
  1. Add parameter to control blending coefficients (Bonus point exercise, code in this section is for reference)
    Howie allows user to adjust stream blending parameters at run time via HowieStreamSendParameters() API. This steps demonstrate how to use it to control echo attenuation
struct EchoParms {
   float attenuation_;
};
EchoParms *echoParms = reinterpret_cast<EchoParms*>(params->data);
echoParms->attenuation_ = 1.0f;
EchoParms *echoParms = reinterpret_cast<EchoParms*>(params->data);
...

// find and change this
// *echo++ = *src++;
// to
*echo++ = (short)(*src++ * echoParms->attenuation_);
JNIEXPORT jlong JNICALL
Java_com_google_sample_audio_1echo_EchoMainActivity_createStream
(JNIEnv *env, jobject instance) {

   HowieStreamCreationParams hscp = {
           sizeof(HowieStreamCreationParams),
           HOWIE_STREAM_DIRECTION_BOTH,
           OnDeviceChangedCallback,
           OnProcessCallback,
           OnCleanupCallback,
           sizeof(EchoState),
           sizeof(EchoParms), // ⇐ new change
           HOWIE_STREAM_STATE_STOPPED
   };
    // ...
}
JNIEXPORT void JNICALL
Java_com_google_sample_audio_1echo_EchoMainActivity_sendEchoAttenuation(JNIEnv *env,                                                                 jobject instance, jlong streamId, jfloat attenuation) {
   HowieStream* stream = reinterpret_cast<HowieStream*>(streamId);
   EchoParms  parms;
   parms.attenuation_ = attenuation;
   HowieStreamSendParameters(stream, &parms, sizeof(parms), 5);
}

What we've covered

Learn More

How to help

Tell us how we did