Suggested Text Models
Sometimes people assume due to the scale of Yozakura, generating memories and managing relationships between dozens of characters, that an especially strong model is required. Fortunately, that's not the case, since characters effectively only interact with a few other characters at a time (usually just one other character, depending on your group chat settings). Users have reported Yozakura working well with models as small as 4B. I've done most of my testing with local 12B models, though I've since upgraded.
Smaller models are of course "stupider" and more prone to hallucination, misattribution, forgetfulness, etc, so the experience is qualitatively different from using larger models. But you can:
- Embrace hallucinations, which can add interesting spontaneity if you let them
- Or, you can get hands on and edit conversations and memories to clean up manually if you like being in the driver's seat and guiding the plot
Below are some local models I've tried or have heard reports about (and if you find a particularly good model, please share your report too, you can contact via the links page)
| Model | Size | Notes |
|---|---|---|
| Gemma 4 26B A4B | 26B (but see notes) | This is my new daily driver now and I find it radically smarter than 12B models. This model runs exceptionally well with layers offloaded to CPU, so it can run on weaker hardware than you might expect. I run the K_4_M quant on 12GB of VRAM with only 17 layers on the GPU, and get close to 20 tokens per second. That can feel a little slow at the memory generation stage, but I find it's worth it. I run an SDXL image model alongside this, so if you're not doing that, you can probably do this on 10GB VRAM or maybe even 8. If you're in that range, try it out, you might be surprised. |
| Gemma 4 12B IT | 12B | It's not as strong as the 26B model, which is to be expected, but it might be the best 12B model for coherence. At the same time, it might be a little less creative and entertaining than some other 12B models. |
| NemoMix Unleashed 12B | 12B | This was my clear favorite 12B model before Gemma 4, providing a good mixture of believable character behavior and memory correctness. It can be funnier than Gemma 4. If you're looking for a 12B model, my recommendation would be to give both Gemma4 and NemoMix Unleashed a shot and see which you prefer. |
| Mag Mell R1 | 12B | Another 12B model that's a long-time favorite in the RP community. It can be pretty aggressive and I find it a bit much, but it can be hilarious sometimes in a way that other models don't match. Worth a shot if you value spontaneity, but hard to recommend broadly. |
| Llama 3.1 8B Instruct | 8B | A workhorse model. A little plain, but does a good job mechanically. There are a lot of finetunes out there that might give a more interesting experience, but some of the RP-specialized ones do poorly on generating memories and sometimes fail to understand that they're supposed to be generating memories and instead try to continue the conversation or something. |
| Gemma 4 E2B | 5B | Yes you can use this tiny model for Yozakura and it's surprisingly good. From what I saw in my limited time with it, I think it's at least as good as Llama 3.1 8B, and may even approach 12B models. You may not even need a GPU for this model, many machines can get acceptable performance purely from CPU inference. |
The only cloud model I've done much with is deepseek/deepseek-chat-v3 which is excellent.