Overview
Runtime Speech Recognizer is a cross-platform plugin that enables real-time, offline speech recognition. Based on Whisper OpenAI technology, particularly whisper.cpp library, and ships with a catalog of pre-trained language models you can download from the plugin's settings, with automatic language detection when using multilingual models.
Installing language models
On first run, the editor prompts you to download a language model automatically. See How to use language models for details on selecting, downloading, and packaging models.
Basic description
This plugin provides real-time speech recognition using advanced algorithms based on whisper.cpp library, which is available under the permissive MIT license. It matches incoming audio data, provided as a stream or non-stream input (such as a file or buffer of audio data), against pre-trained language models. When using multilingual models, the plugin can automatically detect the spoken language and provide this information alongside the recognized text.
The plugin uses different GPU acceleration methods depending on the platform:
- Windows and Linux: Uses Vulkan for GPU acceleration, which significantly speeds up the recognition process
- Mac and iOS: Uses Metal for GPU acceleration, delivering performance that's comparable to Windows or Linux Vulkan acceleration, if not faster
- Other platforms: Uses CPU + intrinsics for acceleration (may be slower, such as on Android or Meta Quest, when running natively). On these platforms, consider the Vosk extension as a lighter alternative provider.
Additional Resources
- Get it on Fab
- Product website
- Download Demo (Windows)
- Video tutorial (older video)
- Plugin Support & Custom Development: [email protected] (tailored solutions for teams & organizations)