Managing models in the editor

Unreleased Plugin

The documentation you are currently viewing is for a plugin that has not yet been released. Content may be incomplete or subject to change. Please check back once the plugin is officially available on the Fab marketplace.

The plugin provides a settings panel inside the Unreal Engine editor where you can browse, download, import, delete, and test LLM models.

Accessing the Plugin Settings

1. Open Edit → Project Settings
2. Navigate to Plugins → Runtime Local LLM

The settings panel has two main sections:

• Downloaded Models: Models currently on disk and ready to use
• Available Models: Pre-defined models from the built-in catalog that you can download

Downloading Models

1. In the Available Models section, find the model family and variant you want
2. Click the Download button next to the variant
3. A progress bar shows download status
4. Once complete, the model moves to the Downloaded Models section

You can download multiple models simultaneously. To cancel an in-progress download, click the Cancel button next to it.

To delete any downloaded models, click the Delete button next to the model you want to remove.

All downloaded models will be packaged with your project, so to reduce the project size, delete any models you no longer need.

Choosing a Model

For mobile/VR platforms, start with a small model (1B–3B parameters) at Q4_K_M quantization. For desktop, 7B models at Q4_K_M or Q5_K_M offer a good balance of quality and performance.

Importing Custom Models

You can import any GGUF file from your local file system or a URL:

1. Click the Import Custom Model button at the top of the settings panel
2. In the dialog that appears, choose a mode:
   • Import Local File: Browse for a .gguf file on disk
   • Download from URL: Enter a direct URL to a .gguf file
3. Fill in the model name and optional description / parameter count
4. Click Confirm to import or start the download

When importing a local file, the plugin copies it into the Content/RuntimeLocalLLM/Models directory and creates a metadata sidecar file. When importing from URL, the plugin downloads the file directly to that directory.

Custom models are treated the same as downloaded models and will be packaged with your project.

Testing Models in the Editor

The plugin includes a test window for quick inference testing without running your game:

1. Click the Test LLM button at the top of the settings panel
2. In the test window, select a model from the dropdown of models on disk
3. Configure inference parameters (temperature, max tokens, system prompt, etc.)
4. Enter a prompt and click Send
5. Watch the response stream in real time
6. Click Stop to cancel generation, or Clear to reset the conversation

This is useful for verifying that a model works correctly before integrating it into your game logic.

Packaging Settings

When you download or import a model, the plugin automatically updates your project's packaging configuration to include the RuntimeLocalLLM/Models directory in Additional Non-Asset Directories To Copy (DirectoriesToAlwaysStageAsNonUFS). This ensures all model files ship with your packaged build. You do not need to configure this manually.