Demo project
The documentation you are currently viewing is for a plugin that has not yet been released. Content may be incomplete or subject to change. Please check back once the plugin is officially available on the Fab marketplace.
In the Demo folder within the Content directory of the plugin, you will find assets created for demonstration purposes.
If you can't see the plugin directory, open the Content Browser, click on the Settings button in the top-right corner, and enable Show Engine Content and Show Plugin Content. The demo project should be located in the Engine → Plugins → Runtime Local LLM Content directory.
These assets are provided to demonstrate the implementation of on-device LLM inference. The demo includes:
- A chat interface where you can send messages and receive streaming token-by-token responses from the LLM
- Pre-downloaded models ready to use out of the box
- The ability to download additional models via URL at runtime
- A settings menu to configure inference parameters (temperature, max tokens, context size, etc.)
The demo is implemented using Blueprints and UMG widgets for simplicity. It supports UE 4.27 and later versions.
To test the demo, open the RLLM_Demo level located in the Demo folder and run it in the editor. This will display the LLM chat widgets on your screen, which you can interact with. You can also package the project for your target device. To set the RLLM_Demo level as the default in your packaged project, go to the project settings and set RLLM_Demo as the Game Default Map.