Capturable sound wave

For Voice Activity Detection (VAD), refer to this page.

There is a capturable sound wave derived from the streaming sound wave. It supports capturing audio data from input devices like a microphone and playing it back, with the same capabilities as an imported sound wave (including rewinding, usage in SoundCues, etc). You can even capture and play audio simultaneously without any limitations.

It also automatically requests microphone permissions on iOS and Android once the capture is started.

First you should create a capturable sound wave. Please note that you should treat it as a strong reference to prevent premature destruction (e.g. by assigning it to a separate variable in Blueprints or using UPROPERTY() in C++).

Blueprint

C++

UCapturableSoundWave* CapturableSoundWave = UCapturableSoundWave::CreateCapturableSoundWave();

---

To use this feature, you need to first get all available audio input devices that can be used for capturing by calling the corresponding function.

Blueprint

C++

UCapturableSoundWave::GetAvailableAudioInputDevices(FOnGetAvailableAudioInputDevicesResultNative::CreateWeakLambda(this, [](const TArray<FRuntimeAudioInputDeviceInfo>& AvailableDevices)
{
   // Handle the result
}));

---

After that, you can start capturing audio data from the input device and stop it when needed. DeviceId is the index of the device in the array of available devices obtained from the previous step.

Blueprint

C++

// Assuming CapturableSoundWave is a UE reference to a UCapturableSoundWave object (or its derived type)

// Start capturing audio data from the input device with the specified device ID (0 in this case)
CapturableSoundWave->StartCapture(0);

// Stop capturing audio data
CapturableSoundWave->StopCapture();

---

Additionally, you can choose to mute or unmute sound capture to prevent the accumulation of audio data without interrupting the capturing process. This is useful when switching capturing frequently since starting recording through StartCapture may cause minor hitches on the engine's side. You should first start the capture itself, and only then mute/unmute it.

If your aim is to eliminate a slight freeze, you could consider starting the capture during a time when this freeze might be negligible. For example, you could start the capture during the loading screen of your game, mute the capturable sound wave right after that, and then unmute the capturable sound wave when needed for capturing audio data without any hitches.

Blueprint

C++

// Assuming CapturableSoundWave is a UE reference to a UCapturableSoundWave object (or its derived type)
CapturableSoundWave->ToggleMute(true);

---

Finally, your implementation might look like this:

Blueprint

C++

This is a basic code example for capturing audio data from an input device (microphone) using a capturable sound wave.

The example uses the CaptureAudioExample function located in the UCaptureAudioClassExample class within the EXAMPLEMODULE module.

Note: To successfully run the example, make sure to add the RuntimeAudioImporter module to either PublicDependencyModuleNames or PrivateDependencyModuleNames in the .Build.cs file, as well as to your project's .uproject file.

Header file (.h)

#pragma once

#include "CoreMinimal.h"
#include "UObject/Object.h"
#include "CaptureAudioClassExample.generated.h"

UCLASS(BlueprintType)
class EXAMPLEMODULE_API UCaptureAudioClassExample : public UObject
{
	GENERATED_BODY()

public:
	UFUNCTION(BlueprintCallable)
	void CaptureAudioExample();

private:
	// Please pay attention to making the StreamingSoundWave a hard reference, such as using UPROPERTY(), to prevent it from being prematurely garbage collected
	UPROPERTY()
	class UCapturableSoundWave* CapturableSoundWave;
};

Source file (.cpp)

#include "CaptureAudioClassExample.h"
#include "Sound/StreamingSoundWave.h"

void UCaptureAudioClassExample::CaptureAudioExample()
{
	// Create a capturable sound wave
	CapturableSoundWave = UCapturableSoundWave::CreateCapturableSoundWave();

	// Capture audio data from the input device with the specified device ID (0 in this case)
	CapturableSoundWave->StartCapture(0);

	// Delay for 5 seconds, just for demonstration purposes
	FTimerHandle TimerHandle;
	GetWorld()->GetTimerManager().SetTimer(TimerHandle, [this]()
	{
		// Stop capturing audio data
		CapturableSoundWave->StopCapture();

		// Play the sound wave
		UGameplayStatics::PlaySound2D(GetWorld(), CapturableSoundWave);
	}, 5, false);
}