Capturable sound wave

Overview

A capturable sound wave is derived from the streaming sound wave. It supports capturing audio data from input devices like a microphone and playing it back, with the same capabilities as an imported sound wave (including rewinding, usage in SoundCues, etc). You can even capture and play audio simultaneously without any limitations.

tip

For Voice Activity Detection (VAD), refer to this page.

note

The capturable sound wave automatically requests microphone permissions on iOS and Android once the capture is started.

Basic Usage

Creating a Capturable Sound Wave

First you should create a capturable sound wave.

Blueprint

C++

UCapturableSoundWave* CapturableSoundWave = UCapturableSoundWave::CreateCapturableSoundWave();

warning

You should treat the capturable sound wave as a strong reference to prevent premature destruction (e.g. by assigning it to a separate variable in Blueprints or using UPROPERTY() in C++).

Getting Available Input Devices

To use this feature, you need to first get all available audio input devices that can be used for capturing.

Blueprint

C++

UCapturableSoundWave::GetAvailableAudioInputDevices(FOnGetAvailableAudioInputDevicesResultNative::CreateWeakLambda(this, [](const TArray<FRuntimeAudioInputDeviceInfo>& AvailableDevices)
{
   // Handle the result
}));

Starting and Stopping Capture

After getting the available devices, you can start capturing audio data from the input device and stop it when needed.

Blueprint

C++

// Assuming CapturableSoundWave is a UE reference to a UCapturableSoundWave object (or its derived type)

// Start capturing audio data from the input device with the specified device ID (0 in this case)
CapturableSoundWave->StartCapture(0);

// Stop capturing audio data
CapturableSoundWave->StopCapture();

info

DeviceId is the index of the device in the array of available devices obtained from the previous step.

Advanced Features

Muting and Unmuting Capture

You can choose to mute or unmute sound capture to prevent the accumulation of audio data without interrupting the capturing process.

Blueprint

C++

// Assuming CapturableSoundWave is a UE reference to a UCapturableSoundWave object (or its derived type)
CapturableSoundWave->ToggleMute(true);

Performance Optimization

This is useful when switching capturing frequently since starting recording through StartCapture may cause minor hitches on the engine's side. You should first start the capture itself, and only then mute/unmute it.

If your aim is to eliminate a slight freeze, you could consider starting the capture during a time when this freeze might be negligible. For example, you could start the capture during the loading screen of your game, mute the capturable sound wave right after that, and then unmute the capturable sound wave when needed for capturing audio data without any hitches.

Complete Example

Here's a complete implementation example of using a capturable sound wave:

Blueprint

C++

This is a basic code example for capturing audio data from an input device (microphone) using a capturable sound wave.

The example uses the CaptureAudioExample function located in the UCaptureAudioClassExample class within the EXAMPLEMODULE module.

note

To successfully run the example, make sure to add the RuntimeAudioImporter module to either PublicDependencyModuleNames or PrivateDependencyModuleNames in the .Build.cs file, as well as to your project's .uproject file.

Header file (.h)

#pragma once

#include "CoreMinimal.h"
#include "UObject/Object.h"
#include "CaptureAudioClassExample.generated.h"

UCLASS(BlueprintType)
class EXAMPLEMODULE_API UCaptureAudioClassExample : public UObject
{
	GENERATED_BODY()

public:
	UFUNCTION(BlueprintCallable)
	void CaptureAudioExample();

private:
	// Please pay attention to making the StreamingSoundWave a hard reference, such as using UPROPERTY(), to prevent it from being prematurely garbage collected
	UPROPERTY()
	class UCapturableSoundWave* CapturableSoundWave;
};

Source file (.cpp)

#include "CaptureAudioClassExample.h"
#include "Sound/StreamingSoundWave.h"

void UCaptureAudioClassExample::CaptureAudioExample()
{
	// Create a capturable sound wave
	CapturableSoundWave = UCapturableSoundWave::CreateCapturableSoundWave();

	// Capture audio data from the input device with the specified device ID (0 in this case)
	CapturableSoundWave->StartCapture(0);

	// Delay for 5 seconds, just for demonstration purposes
	FTimerHandle TimerHandle;
	GetWorld()->GetTimerManager().SetTimer(TimerHandle, [this]()
	{
		// Stop capturing audio data
		CapturableSoundWave->StopCapture();

		// Play the sound wave
		UGameplayStatics::PlaySound2D(GetWorld(), CapturableSoundWave);
	}, 5, false);
}