Streaming Sound Wave

For Voice Activity Detection (VAD), refer to this page.

A streaming sound wave is a type of imported sound wave that supports adding audio data dynamically, even during playback. It offers the same functionality as an imported sound wave, e.g. rewinding, and can be used in SoundCues, etc.

Creating a streaming sound wave

First you should create a streaming sound wave. Please note that you should treat it as a strong reference to prevent premature destruction (e.g. by assigning it to a separate variable in Blueprints or using UPROPERTY() in C++).

Blueprint

C++

UStreamingSoundWave* StreamingSoundWave = UStreamingSoundWave::CreateStreamingSoundWave();

Playing the sound wave

You can then play back that sound wave. However, this is not necessary to do now, you can start playing the sound wave later.

Pre-allocating audio data

Optionally, you can pre-allocate audio data (bytes) to avoid reallocating the entire PCM buffer each time new audio data is appended.

Blueprint

C++

// Assuming StreamingSoundWave is a UE reference to a UStreamingSoundWave object (or its derived type, such as UCapturableSoundWave)
StreamingSoundWave->PreAllocateAudioData(12582912, FOnPreAllocateAudioDataResultNative::CreateWeakLambda(this, [StreamingSoundWave](bool bSucceeded)
{
   // Handle the result
}));

Appending audio data

To add audio data to the end of the existing buffer, use the appropriate functions for dynamically appending audio data. The playback will follow the queue sequence of these appends.

Blueprint

C++

// Assuming StreamingSoundWave is a UE reference to a UStreamingSoundWave object (or its derived type, such as UCapturableSoundWave)

// Example of appending encoded audio data
TArray<uint8> AudioData = ...; // Fill with audio data
StreamingSoundWave->AppendAudioDataFromEncoded(AudioData, ERuntimeAudioFormat::Auto);

// Or, if you have raw audio data
TArray<uint8> RawAudioData = ...; // Fill with raw audio data
StreamingSoundWave->AppendAudioDataFromRAW(RawAudioData, ERuntimeRAWAudioFormat::Float32, 44100, 2);

Avoiding accelerated or distorted audio playback

When streaming audio data while the sound wave is playing, you may experience accelerated or distorted audio playback in specific scenarios. This issue typically occurs when:

1. The sound wave playback is near/at the end of the current buffer
2. New audio data is being continuously queued into the streaming sound wave
3. The playback catches up to the incoming data stream

You can stop populating audio data at any time without issues. However, for scenarios where you need continuous streaming (such as real-time audio streaming), there are two approaches depending on your audio data streaming reliability:

For reliable, consistent streaming: Use the OnPopulateAudioState delegate to start playback immediately after the first chunk is received.

For unreliable streaming (network issues, intermittent data): Add an additional delay even after OnPopulateAudioState is triggered to build up a larger buffer before starting playback.

Blueprint

C++

// Approach 1: For reliable streaming - start immediately after first chunk
bool bHasStartedPlayback = false;

StreamingSoundWave->OnPopulateAudioStateNative.AddWeakLambda(this, [this, &bHasStartedPlayback]()
{
   if (!bHasStartedPlayback)
   {
       UGameplayStatics::PlaySound2D(GetWorld(), StreamingSoundWave);
       bHasStartedPlayback = true;
   }
});

// Approach 2: For unreliable streaming - add delay after first chunk
bool bHasStartedPlayback = false;

StreamingSoundWave->OnPopulateAudioStateNative.AddWeakLambda(this, [this, &bHasStartedPlayback]()
{
   if (!bHasStartedPlayback)
   {
       // Add delay to build up buffer for unreliable streaming
       GetWorld()->GetTimerManager().SetTimer(PlaybackDelayTimer, [this]()
       {
           UGameplayStatics::PlaySound2D(GetWorld(), StreamingSoundWave);
       }, 0.5f, false); // Half-second delay
       
       bHasStartedPlayback = true;
   }
});

Note: You can stop populating audio data at any time without causing playback issues. The exact approach depends on your audio data streaming reliability - use immediate playback for consistent streams, or add additional buffering delay for unreliable streams (e.g., network connectivity issues).

Example usage

Finally, your implementation might look like this:

Blueprint

C++

This is a basic code example for appending audio data to a streaming sound wave.

The example uses the AppendAudioExample function located in the UAppendAudioClassExample class within the EXAMPLEMODULE module.

Note: To successfully run the example, make sure to add the RuntimeAudioImporter module to either PublicDependencyModuleNames or PrivateDependencyModuleNames in the .Build.cs file, as well as to your project's .uproject file.

Header file (.h)

#pragma once

#include "CoreMinimal.h"
#include "UObject/Object.h"
#include "AppendAudioClassExample.generated.h"

UCLASS(BlueprintType)
class EXAMPLEMODULE_API UAppendAudioClassExample : public UObject
{
	GENERATED_BODY()

public:
	UFUNCTION(BlueprintCallable)
	void AppendAudioExample();

private:
	// Please pay attention to making the StreamingSoundWave a hard reference, such as using UPROPERTY(), to prevent it from being prematurely garbage collected
	UPROPERTY()
	class UStreamingSoundWave* StreamingSoundWave;
};

Source file (.cpp)

#include "AppendAudioClassExample.h"
#include "Sound/StreamingSoundWave.h"

void UAppendAudioClassExample::AppendAudioExample()
{
    // Create a streaming sound wave
	StreamingSoundWave = UStreamingSoundWave::CreateStreamingSoundWave();

	// Append audio data
	TArray<uint8> StreamedAudioDataToAdd = ...; // Fill with audio data
	StreamingSoundWave->AppendAudioDataFromEncoded(StreamedAudioDataToAdd, ERuntimeAudioFormat::Auto);

	// Play the sound wave
	UGameplayStatics::PlaySound2D(GetWorld(), StreamingSoundWave);
}

On Populate Audio State

The OnPopulateAudioState delegate functions similarly to OnPopulateAudioData but doesn't broadcast the populated audio data. This can be useful when you want to track when the audio data is populated without passing an array of the populated audio data, which can improve performance.

Blueprint

C++

// Assuming StreamingSoundWave is a UE reference to a UStreamingSoundWave object (or its derived type, such as UCapturableSoundWave)

StreamingSoundWave->OnPopulateAudioStateNative.AddWeakLambda(this, [this]()
{
   // Handle the result
});

Working with PCM Data

For real-time access to PCM data during playback, see PCM Data Handling.