Runtime Speech Recognizer

Documentation for the Runtime Speech Recognizer plugin.

Get it on Fab
Download Demo (Windows)
Discord support server
Video tutorial
Custom Development: [email protected] (tailored solutions for teams & organizations)

The Runtime Speech Recognizer plugin is designed to recognize words from incoming audio data. It uses a slightly modified version of whisper.cpp to work with the engine. To use the plugin, follow these steps:

📄️ How to use language models

Selecting, downloading and packaging models

📄️ Recognition parameter list

These parameters can only be set while the recognizer is not running.

📄️ Supported languages

This is the complete list of languages supported by the available language models.

📄️ Commands recognition

Compute Levenshtein Similarity

📄️ Minimizing freezes

This guide addresses two common sources of freezing in the RuntimeSpeechRecognizer plugin and provides practical solutions to mitigate the performance impact.

📄️ Troubleshooting

Most of the issues are related to the language model staging, which can cause problems in the packaged build. Namely, you might encounter the following logs:

📄️ Demo project

Packaged Demo Project for Windows.

📄️ How to improve performance

Windows platforms use Vulkan for GPU acceleration, which significantly speeds up the recognition process. On other platforms, the plugin uses the CPU + intrinsics for acceleration. However, you can further improve the performance of the plugin by following the recommendations below: