Streaming sound wave

For Voice Activity Detection (VAD), refer to this page.

A streaming sound wave is a type of imported sound wave that supports adding audio data dynamically, even during playback. It offers the same functionality as an imported sound wave, e.g. rewinding, and can be used in SoundCues, etc.

Creating a streaming sound wave

First you should create a streaming sound wave. Please note that you should treat it as a strong reference to prevent premature destruction (e.g. by assigning it to a separate variable in Blueprints or using UPROPERTY() in C++).

Blueprint

C++

UStreamingSoundWave* StreamingSoundWave = UStreamingSoundWave::CreateStreamingSoundWave();

Playing the sound wave

You can then play back that sound wave. However, this is not necessary to do now, you can start playing the sound wave later.

Pre-allocating audio data

Optionally, you can pre-allocate audio data (bytes) to avoid reallocating the entire PCM buffer each time new audio data is appended.

Blueprint

C++

// Assuming StreamingSoundWave is a UE reference to a UStreamingSoundWave object (or its derived type, such as UCapturableSoundWave)
StreamingSoundWave->PreAllocateAudioData(12582912, FOnPreAllocateAudioDataResultNative::CreateWeakLambda(this, [StreamingSoundWave](bool bSucceeded)
{
   // Handle the result
}));

Appending audio data

To add audio data to the end of the existing buffer, use the appropriate functions for dynamically appending audio data. The playback will follow the queue sequence of these appends.

Blueprint

C++

// Assuming StreamingSoundWave is a UE reference to a UStreamingSoundWave object (or its derived type, such as UCapturableSoundWave)

// Example of appending encoded audio data
TArray<uint8> AudioData = ...; // Fill with audio data
StreamingSoundWave->AppendAudioDataFromEncoded(AudioData, ERuntimeAudioFormat::Auto);

// Or, if you have raw audio data
TArray<uint8> RawAudioData = ...; // Fill with raw audio data
StreamingSoundWave->AppendAudioDataFromRAW(RawAudioData, ERuntimeRAWAudioFormat::Float32, 44100, 2);

Avoiding accelerated audio playback

Often, when you need to stream audio data and play simultaneously, you should add a slight delay before the playback to avoid accelerated audio due to too fast buffer population. A delay of around half a second is typically recommended.

Example usage

Finally, your implementation might look like this:

Blueprint

C++

This is a basic code example for appending audio data to a streaming sound wave.

The example uses the AppendAudioExample function located in the UAppendAudioClassExample class within the EXAMPLEMODULE module.

Note: To successfully run the example, make sure to add the RuntimeAudioImporter module to either PublicDependencyModuleNames or PrivateDependencyModuleNames in the .Build.cs file, as well as to your project's .uproject file.

Header file (.h)

#pragma once

#include "CoreMinimal.h"
#include "UObject/Object.h"
#include "AppendAudioClassExample.generated.h"

UCLASS(BlueprintType)
class EXAMPLEMODULE_API UAppendAudioClassExample : public UObject
{
	GENERATED_BODY()

public:
	UFUNCTION(BlueprintCallable)
	void AppendAudioExample();

private:
	// Please pay attention to making the StreamingSoundWave a hard reference, such as using UPROPERTY(), to prevent it from being prematurely garbage collected
	UPROPERTY()
	class UStreamingSoundWave* StreamingSoundWave;
};

Source file (.cpp)

#include "AppendAudioClassExample.h"
#include "Sound/StreamingSoundWave.h"

void UAppendAudioClassExample::AppendAudioExample()
{
    // Create a streaming sound wave
	StreamingSoundWave = UStreamingSoundWave::CreateStreamingSoundWave();

    // Append audio data
    TArray<uint8> StreamedAudioDataToAdd = ...; // Fill with audio data
    StreamingSoundWave->AppendAudioDataFromEncoded(StreamedAudioDataToAdd, ERuntimeAudioFormat::Auto);

    // Play the sound wave
    UGameplayStatics::PlaySound2D(GetWorld(), StreamingSoundWave);
}