Add sound and music to your Flutter game

1. Before you begin

Games are audiovisual experiences. Flutter is a great tool for building beautiful visuals and solid UI, so it gets you far on the visual side of things. The missing ingredient that's left is audio. In this codelab, you learn how to use the flutter_soloud plugin to introduce low-latency sound and music to your project. You start with a basic scaffold so that you can jump straight to the interesting parts.

You can, of course, use what you learn here to add audio to your apps, not just games. But while almost all games require sound and music, most apps don't, so this codelab focuses on games.

Note: The flutter_soloud package supports all platforms, from Android and iOS through Windows, Linux, and macOS to the web (through Web Assembly).

Prerequisites

• Basic familiarity with Flutter.
• Knowledge of how to run and debug Flutter apps.

What you learn

• How to play one-shot sounds.
• How to play and customize gapless music loops.
• How to fade sounds in and out.
• How to apply environmental effects to sounds.
• How to deal with exceptions.
• How to encapsulate all of these features into a single audio controller.

What you need

• The Flutter SDK
• A code editor of your choice

2. Set up

1. Download the following files. If you have a slow connection, don't worry. You need the actual files later, so you can let them download while you work.

• Download looped-song.ogg
• Download pew1.mp3
• Download pew2.mp3
• Download pew3.mp3

2. Create a Flutter project with a name of your choice.

Note: For simplicity, you won't build a proper game in this codelab. You learn how to write an audio controller with methods, such as the initialize() and startMusic() methods. It's easy to bring such a controller to your own game.

3. Create a lib/audio/audio_controller.dart file in the project.
4. In the file, enter the following code:

lib/audio/audio_controller.dart

import 'dart:async';

import 'package:logging/logging.dart';

class AudioController {
  static final Logger _log = Logger('AudioController');

  Future<void> initialize() async {
    // TODO
  }

  void dispose() {
    // TODO
  }

  Future<void> playSound(String assetKey) async {
    _log.warning('Not implemented yet.');
  }

  Future<void> startMusic() async {
    _log.warning('Not implemented yet.');
  }

  void fadeOutMusic() {
    _log.warning('Not implemented yet.');
  }

  void applyFilter() {
    // TODO
  }

  void removeFilter() {
    // TODO
  }
}

As you can see, this is only a skeleton for future functionality. We will implement it all during this codelab.

5. Next, open the lib/main.dart file and then replace its contents with the following code:

lib/main.dart

import 'dart:developer' as dev;

import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:logging/logging.dart';

import 'audio/audio_controller.dart';

void main() async {
  // The `flutter_soloud` package logs everything
  // (from severe warnings to fine debug messages)
  // using the standard `package:logging`.
  // You can listen to the logs as shown below.
  Logger.root.level = kDebugMode ? Level.FINE : Level.INFO;
  Logger.root.onRecord.listen((record) {
    dev.log(
      record.message,
      time: record.time,
      level: record.level.value,
      name: record.loggerName,
      zone: record.zone,
      error: record.error,
      stackTrace: record.stackTrace,
    );
  });

  WidgetsFlutterBinding.ensureInitialized();

  final audioController = AudioController();
  await audioController.initialize();

  runApp(
    MyApp(audioController: audioController),
  );
}

class MyApp extends StatelessWidget {
  const MyApp({required this.audioController, super.key});

  final AudioController audioController;

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter SoLoud Demo',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.brown),
        useMaterial3: true,
      ),
      home: MyHomePage(audioController: audioController),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.audioController});

  final AudioController audioController;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  static const _gap = SizedBox(height: 16);

  bool filterApplied = false;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: const Text('Flutter SoLoud Demo')),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            OutlinedButton(
              onPressed: () {
                widget.audioController.playSound('assets/sounds/pew1.mp3');
              },
              child: const Text('Play Sound'),
            ),
            _gap,
            OutlinedButton(
              onPressed: () {
                widget.audioController.startMusic();
              },
              child: const Text('Start Music'),
            ),
            _gap,
            OutlinedButton(
              onPressed: () {
                widget.audioController.fadeOutMusic();
              },
              child: const Text('Fade Out Music'),
            ),
            _gap,
            Row(
              mainAxisSize: MainAxisSize.min,
              children: [
                const Text('Apply Filter'),
                Checkbox(
                  value: filterApplied,
                  onChanged: (value) {
                    setState(() {
                      filterApplied = value!;
                    });
                    if (filterApplied) {
                      widget.audioController.applyFilter();
                    } else {
                      widget.audioController.removeFilter();
                    }
                  },
                ),
              ],
            ),
          ],
        ),
      ),
    );
  }
}

Note: When you make the change above, the widget test in the tests/widget_test.dart file shows a compile-time error because you changed the MyApp() constructor signature. Feel free to delete that file because you won't need it during this codelab.

6. After the audio files are downloaded, create a directory in the root of your project called assets.
7. In the assets directory, create two subdirectories, one called music and the other called sounds.
8. Move the downloaded files to your project so that the song file is in the assets/music/looped-song.ogg file and the pew sounds are in the following files:

• assets/sounds/pew1.mp3
• assets/sounds/pew2.mp3
• assets/sounds/pew3.mp3

Your project structure should now look something like this:

Now that the files are there, you need to tell Flutter about them.

9. Open the pubspec.yaml file and then replace the flutter: section at the bottom of the file with the following:

pubspec.yaml

...

flutter:
  uses-material-design: true

  assets:
    - assets/music/
    - assets/sounds/

10. Add a dependency on the flutter_soloud package and the logging package.

pubspec.yaml

...

dependencies:
  flutter:
    sdk: flutter

  flutter_soloud: ^2.0.0
  logging: ^1.2.0

...

11. Run the project. Nothing works yet because you add the functionality in the following sections.

Caution: The first time that you run your project with a dependency on the flutter_soloud package, you might see compile warnings such as this one:

/flutter_soloud/src/filters/filters.cpp:21:24: warning: implicit conversion loses integer precision: 'decltype(__x.base() - __y.base())' (aka 'long') to 'int' [-Wshorten-64-to-32];

These come from the underlying SoLoud C++ library. They don't have any effect on functionality and can be safely ignored.

3. Initialize and shut down

To play audio, you use the flutter_soloud plugin. This plugin is based on the SoLoud project, a C++ audio engine for games that's used—among others—by Nintendo SNES Classic.

To initialize the SoLoud audio engine, follow these steps:

1. In the audio_controller.dart file, import the flutter_soloud package and add a private _soloud field to the class.

lib/audio/audio_controller.dart

import 'dart:ui';

import 'package:flutter_soloud/flutter_soloud.dart';  // ← Add this...
import 'package:logging/logging.dart';

class AudioController {
  static final Logger _log = Logger('AudioController');

  SoLoud? _soloud;                                    // ← ... and this.

  Future<void> initialize() async {
    // TODO
  }

  ...

The audio controller manages the underlying SoLoud engine through this field and will forward all calls to it.

Note: This is called the façade pattern. You could call SoLoud methods directly from anywhere in your code, but it would quickly become unwieldy. Which parts of your code play which sounds? Who loads them? How do you know that they are already loaded? Which part of your code changes music tracks and how do you know that it's time to play something soothing? Last, but not least, who is responsible for initializing the engine and dealing with errors?

It's much easier to keep the audio functionality and all related responsibilities in one class and provide a limited API for the rest of your project.

2. In the initialize() method, enter the following code:

lib/audio/audio_controller.dart

...

  Future<void> initialize() async {
    _soloud = SoLoud.instance;
    await _soloud!.init();
  }

...

This populates the _soloud field and awaits initialization. Note the following:

• SoLoud provides a singleton instance field. There is no way to instantiate several SoLoud instances. This is not something the C++ engine allows, so it's not allowed by the Dart plugin either.
• Initialization of the plugin is asynchronous and isn't finished until the init() method returns.
• For brevity in this example, you're not catching errors in a try/catch block. In production code, you want to do so and report any errors to the user.

3. In the dispose() method, enter the following code:

lib/audio/audio_controller.dart

...

  void dispose() {
    _soloud?.deinit();
  }

...

Shutting down SoLoud on app exit is good practice, although everything should work fine even if you neglect to do so.

4. Notice that the AudioController.initialize() method is already called from the main() function. This means that hot-restarting the project initializes SoLoud in the background, but it won't do you any good before you actually play some sounds.

4. Play one-shot sounds

Load an asset and play it

Now that you know that SoLoud is initialized on startup, you can ask it to play sounds.

SoLoud differentiates between an audio source, which is the data and metadata used to describe a sound, and its "sound instances", which are the sounds actually played. An example of an audio source can be an mp3 file loaded into memory, ready to be played, and represented by an instance of the AudioSource class. Every time you play this audio source, SoLoud creates a "sound instance" which is represented by the SoundHandle type.

You get an AudioSource instance by loading it. For example, if you have an mp3 file in your assets, you can load it to get an AudioSource. Then, you tell SoLoud to play this AudioSource. You can play it many times, even simultaneously.

When you're done with an audio source, you dispose of it with the SoLoud.disposeSource() method.

To load an asset and play it, follow these steps:

1. In the playSound() method of the AudioController class, enter the following code:

lib/audio/audio_controller.dart

  ...

  Future<void> playSound(String assetKey) async {
    final source = await _soloud!.loadAsset(assetKey);
    await _soloud!.play(source);
  }

  ...

2. Save the file, hot reload, and then select Play sound. You should hear a silly pew sound. Note the following:

• The provided assetKey argument is something like assets/sounds/pew1.mp3—the same string that you'd provide to any other asset-loading Flutter API, such as the Image.asset() widget.
• The SoLoud instance provides a loadAsset() method that asynchronously loads an audio file from the Flutter project's assets and returns an instance of the AudioSource class. There are equivalent methods to load a file from the filesystem (the loadFile() method), and to load over the network from a URL (the loadUrl() method).
• The newly acquired AudioSource instance is then passed to SoLoud's play() method. This method returns an instance of the SoundHandle type that represents the newly playing sound. This handle can, in turn, be passed to other SoLoud methods to do things like pause, stop, or modify the volume of the sound.
• Although play() is an asynchronous method, playback starts basically instantaneously. The flutter_soloud package uses Dart's foreign function interface (FFI) to call C code directly and synchronously. The usual messaging back and forth between Dart code and platform code that is characteristic to most Flutter plugins is nowhere to be found. The only reason that some methods are asynchronous is that some of the plugin's code runs in its own isolate and communication between Dart isolates is asynchronous.
• You simply assert that the _soloud field isn't null with _soloud!. This is, again, for brevity. Production code should gracefully deal with the situation when the developer tries to play a sound before the audio controller has had the chance to fully initialize.

Note: If you click Play sound more than once, you get a warning in the development console like this:

[flutter_soloud.SoLoud] Sound '/path/to/sound' was already loaded. Prefer loading only once, and reusing the loaded sound when playing.

For now, you can safely ignore this, but better game code would load all the required sounds on startup (or during level loading) and have them prepared for immediate playback.

Deal with exceptions

You might have noticed that you're, once again, ignoring possible exceptions. Let's fix that for this particular method for learning purposes. (For brevity, the codelab returns to ignoring exceptions after this section.)

• To deal with exceptions in this case, wrap the two lines of the playSound() method in a try/catch block and only catch instances of SoLoudException.

lib/audio/audio_controller.dart

  ...

  Future<void> playSound(String assetKey) async {
    try {
      final source = await _soloud!.loadAsset(assetKey);
      await _soloud!.play(source);
    } on SoLoudException catch (e) {
      _log.severe("Cannot play sound '$assetKey'. Ignoring.", e);
    }
  }

  ...

SoLoud throws various exceptions, such as the SoLoudNotInitializedException or SoLoudTemporaryFolderFailedException exceptions. Each method's API docs lists the kinds of exceptions that might be thrown.

SoLoud also provides a parent class to all its exceptions, the SoLoudException exception, so that you can catch all errors related to the audio engine's functionality. This is especially helpful in cases where playing audio is not critical. For example, when you don't want to crash the player's game session only because one of the pew-pew sounds couldn't load.

As you would probably expect, the loadAsset() method can also throw a FlutterError error if you provide an asset key that doesn't exist. Trying to load assets that aren't bundled with the game is generally something you should address, therefore it's an error.

Play different sounds

You might have noticed that you only play the pew1.mp3 file, but there are two other versions of the sound in the assets directory. It often sounds more natural when games have several versions of the same sound, and play the different versions in a random fashion or on a rotating basis. This prevents, for example, footsteps and gunshots from sounding too uniform and therefore fake.

• As an optional exercise, modify the code to play a different pew sound each time that the button is tapped.

5. Play music loops

Manage longer-running sounds

Some audio is meant to play for extended periods of time. Music is the obvious example, but many games also play ambience, such as the wind howling through corridors, the distant chanting of monks, the creaking of centuries-old metal, or the distant coughs of patients.

These are audio sources with playtimes that can be measured in minutes. You need to keep track of them so that you can pause or stop them when needed. They are also often backed by big files and can consume a lot of memory, so another reason to track them is so that you can dispose of the AudioSource instance when it's no longer needed.

For that reason, you will introduce a new private field to AudioController. It is a handle for the currently playing song, if any. Add the following line:

lib/audio/audio_controller.dart

...

class AudioController {
  static final Logger _log = Logger('AudioController');

  SoLoud? _soloud;

  SoundHandle? _musicHandle;    // ← Add this.

  ...

Start music

In essence, playing music is no different from playing a one-shot sound. You still first need to load the assets/music/looped-song.ogg file as an instance of the AudioSource class, then use SoLoud's play() method to play it.

This time, though, you take a hold of the sound handle that the play() method returns to manipulate the audio while it's playing.

• If you want, implement the AudioController.startMusic() method on your own. It's okay if you don't get some of the details right. The important thing is that the music starts when you select Start music.

Here's a reference implementation:

lib/audio/audio_controller.dart

...

  Future<void> startMusic() async {
    if (_musicHandle != null) {
      if (_soloud!.getIsValidVoiceHandle(_musicHandle!)) {
        _log.info('Music is already playing. Stopping first.');
        await _soloud!.stop(_musicHandle!);
      }
    }
    final musicSource = await _soloud!
        .loadAsset('assets/music/looped-song.ogg', mode: LoadMode.disk);
    _musicHandle = await _soloud!.play(musicSource);
  }

...

Notice that you load the music file in disk mode (the LoadMode.disk enum). This simply means that the file is only loaded in chunks as needed. For longer-running audio, it is generally best to load in disk mode. For short sound effects, it makes more sense to load and decompress them into memory (the default LoadMode.memory enum).

You have a couple of problems, though. First, the music is too loud, overpowering the sounds. In most games, music is in the background most of the time, giving center stage to the more informative audio, such as speech and sound effects. This is easy to fix using the volume parameter of the play method. You can, for example, try _soloud!.play(musicSource, volume: 0.6) to play the song at 60% volume. Alternatively, you can set the volume at any later point with something like _soloud!.setVolume(_musicHandle, 0.6).

The second problem is that the song stops abruptly. This is because it's a song that is supposed to be played in a loop and the starting point of the loop is not the beginning of the audio file.

This is a popular choice for game music because it means the song starts with a natural intro and then plays as long as needed without an obvious loop point. When the game needs to transition out of the currently playing song, it simply fades the song.

Thankfully, SoLoud provides ways to play looping audio. The play() method takes a boolean value for the looping parameter, and also the value for the loop's starting point as the loopingStartAt parameter. The resulting code looks like this:

lib/audio/audio_controller.dart

...

_musicHandle = await _soloud!.play(
  musicSource,
  volume: 0.6,
  looping: true,
  // ↓ The exact timestamp of the start of the loop.
  loopingStartAt: const Duration(seconds: 25, milliseconds: 43),
);

...

If you don't set the loopingStartAt parameter, it defaults to Duration.zero (in other words, the start of the audio file). If you have a music track that's a perfect loop without any introduction, this is what you want.

Caution: There's one caveat around looping that's specific to MP3 files. The MP3 compression format doesn't make it easy to quickly jump (using the seek() method) to an arbitrary position. If you load a large MP3 file in disk mode (using the LoadMode.disk enum, as explained above), your loops will likely have gaps in them. You have two options: either load the MP3 file in memory mode (the LoadMode.memory enum), or use a different supported audio format, such as .ogg or .wav.

• To ensure that the audio source is properly disposed of once it finishes playing, listen to the allInstancesFinished stream that each audio source provides. With added log calls, the startMusic() method then looks like this:

lib/audio/audio_controller.dart

...

  Future<void> startMusic() async {
    if (_musicHandle != null) {
      if (_soloud!.getIsValidVoiceHandle(_musicHandle!)) {
        _log.info('Music is already playing. Stopping first.');
        await _soloud!.stop(_musicHandle!);
      }
    }
    _log.info('Loading music');
    final musicSource = await _soloud!
        .loadAsset('assets/music/looped-song.ogg', mode: LoadMode.disk);
    musicSource.allInstancesFinished.first.then((_) {
      _soloud!.disposeSource(musicSource);
      _log.info('Music source disposed');
      _musicHandle = null;
    });

    _log.info('Playing music');
    _musicHandle = await _soloud!.play(
      musicSource,
      volume: 0.6,
      looping: true,
      loopingStartAt: const Duration(seconds: 25, milliseconds: 43),
    );
  }

...

Fade sound

Your next problem is that the music never ends. Let's implement a fade.

One way that you could implement the fade would be to have some kind of a function that is called several times a second, such as a Ticker or Timer.periodic, and lower the volume of the music by small decrements. This would work, but it's a lot of work.

Thankfully, SoLoud provides convenient fire-and-forget methods that do this for you. Here's how you can fade the music over the course of five seconds and then stop the sound instance so that it doesn't consume CPU resources unnecessarily. Replace the fadeOutMusic() method with this code:

lib/audio/audio_controller.dart

...

  void fadeOutMusic() {
    if (_musicHandle == null) {
      _log.info('Nothing to fade out');
      return;
    }
    const length = Duration(seconds: 5);
    _soloud!.fadeVolume(_musicHandle!, 0, length);
    _soloud!.scheduleStop(_musicHandle!, length);
  }

...

6. Apply effects

One huge advantage of having a proper audio engine at your disposal is that you can do audio processing, such as routing some sounds through a reverb, an equalizer, or a low-pass filter.

In games, this can be used for auditory differentiation of locations. For example, a clap sounds differently in a forest than in a concrete bunker. While a forest helps dissipate and absorb the sound, the bare walls of a bunker reflect the soundwaves back, leading to reverb. Similarly, people's voices sound different when heard through a wall. The higher frequencies of those sounds are more easily attenuated as they travel through the solid medium, resulting in a low-pass filter effect.

SoLoud provides several different audio effects, which you can apply to audio.

• To make it sound like your player is in a large room, like a cathedral or a cave, use the SoLoud.filters field:

lib/audio/audio_controller.dart

...

  void applyFilter() {
    _soloud!.filters.freeverbFilter.activate();
    _soloud!.filters.freeverbFilter.wet.value = 0.2;
    _soloud!.filters.freeverbFilter.roomSize.value = 0.9;
  }

  void removeFilter() {
    _soloud!.filters.freeverbFilter.deactivate();
  }

...

The SoLoud.filters field gives you access to all filter types and their parameters. Every parameter also has built-in functionalities like gradual fading and oscillation.

Note: _soloud!.filters exposes global filters. If you want to apply filters to a single source, please use the counterpart AudioSource.filters which act the same.

Note: The SoLoud.filters field exposes global filters. These will be applied to the whole sound mix (including music, for example). If you want to apply a filter selectively, to a single source, use the AudioSource.filters field. It works exactly the same.

With the previous code, you do the following:

• Enable the freeverb filter globally.
• Set the Wet parameter to 0.2, which means that the resulting audio will be 80% original and 20% the reverb effect's output. If you set this parameter to 1.0, it would be like hearing only the sound waves that come back to you from the distant walls of the room and none of the original audio.
• Set the Room Size parameter to 0.9. You can tweak this parameter to your liking or even change it dynamically. 1.0 is a massive cavern while 0.0 is a bathroom.

Note: Hot reload and hot restart mostly work when debugging a game or app that uses the flutter_soloud package, but it is possible to put the C++ side of things into an inconsistent state during a hot restart. For example, a filter might be applied from the previous run. In that case, you might need to stop and restart the whole program.

It always pays to code defensively, too. You can use the filters.freeverbFilter.isActive() method to check before trying to add a filter that might already be there or remove a filter that isn't there.

• If you're up to it, change the code and apply one of the following filters or a combination of the following filters:
• biquadFilter (can be used as a low pass filter)
• pitchShiftFilter
• equalizerFilter
• echoFilter
• lofiFilter
• flangerFilter
• bassboostFilter
• waveShaperFilter
• robotizeFilter

7. Congratulations

You implemented an audio controller that plays sounds, loops music, and applies effects.

Learn more

• Try taking the audio controller further with features such as preloading sounds on startup, playing songs in a sequence, or applying a filter gradually over time.
• Read the flutter_soloud's package documentation.
• Read the underlying C++ library's homepage.
• Read more about Dart FFI, the technology used to interface with the C++ library.
• Watch Guy Somberg's talk about game audio programming for inspiration. (There's also a longer one.) When Guy talks about "middleware", he means libraries like SoLoud and FMOD. The rest of the code tends to be specific to each game.
• Build your game and release it.