Skip to main content

Memory System

Memories are what make Yozakura characters feel like they live in a persistent world. The memory system is designed for continuity and narrative momentum: characters remember what happened, form intentions about what they want to happen next, and gossip travels between characters who weren't even in the room. All of it is generated by your LLM and stored on the characters and their relationships, ready to be injected into future prompts.

This article explains the data flow and each type of memory and what it's for.

The big picture​

After every conversation ends, each non-user participant gets a round of memory updates, generated from their own point of view. Two characters can walk away from the same conversation with very different memories of it. The user character is the exception: it doesn't generate memories (that's your job).

For each non-user participant, the flow looks like this:

  1. Summarize the conversation from that character's perspective. The summary is appended to the character's rolling list of conversation summaries.
  2. Update pairwise memories towards each other participant: append the conversation summary to the list of conversations they've had with that participant, generate a new goal for their next conversation, rewrite their consolidated memory of that character, update the relationship descriptor, and bump their familiarity score.
  3. Handle offscreen mentions: if a character who wasn't present got talked about during the chat, extract what was learned about them secondhand and fold it into the listener's memories of that character. Offscreen mentions are designed to produce more limited, but still meaningful, memory updates compared to actual conversations with that character.
  4. Rewrite the character's global memory: the personal memories, beliefs, and goals they carry into every conversation, which is another way for information to cross-polinate between different sets of characters.

Each of these steps is its own LLM call with its own editable prompt template (see the prompt template system).

post-conversation memories image

Memory types​

Conversation summaries​

Each character keeps a rolling list of brief summaries of their recent conversations, written from their own perspective. Each relationship also keeps a rolling list of summaries of conversations between just those two characters. By default only the most recent ten summaries are kept in each list. These are the raw material that the consolidated memories below get distilled from, so older summaries fall off the end.

Pairwise memory​

Each relationship has a single consolidated memory: what the "from" character remembers about the "to" character. After each conversation, the LLM rewrites it from scratch using the rolling conversation summaries, any secondhand information, and the previous consolidated memory. Instead of an ever-growing log, memories get continuously compressed and rewritten, so old details fade naturally unless they keep mattering. The length of the rolling information log can be adjusted in settings, and more powerful models may be able to handle longer memories.

Next conversation goal​

Alongside the memory, each relationship stores a goal: something the "from" character wants out of their next conversation with the "to" character. This is what creates narrative momentum. It gets injected into the system prompt when the two characters next talk one-on-one, so conversations pick up threads instead of starting cold. The default prompts encourage characters to keep things moving, though YMMV depending on your model.

Relationship descriptor​

A short label (usually 1-3 words) for what the "to" character means to the "from" character: "best friend", "rival", "distant acquaintance". It's regenerated after each interaction and you can watch relationships evolve through it.

Global memory​

Each character also carries a single block of personal memories, beliefs, and goals that follows them into every conversation, regardless of who they're talking to. It's rewritten by the LLM after each conversation, using the same rolling-compression approach as pairwise memory, but includes their recent conversations with all characters.

Familiarity​

A numeric score for how well two characters know each other character. Unlike the other fields, this one isn't LLM-generated: it goes up by a fixed amount per interaction and decays a little each turn the characters don't interact. Familiarity influences how likely characters are to speak to each other. Notably, familiarity has nothing to do with whether two characters like each other. They may hate each other. Regardless, as they interact more, their chance to interact even more goes higher, but with diminishing returns.

Offscreen learned information​

Characters can learn about other characters who aren't present in a chat. While a chat is running, Yozakura scans each message for the names of non-participant characters. When one is mentioned, two things happen:

  • During the chat, a hidden system message is inserted reminding the speaker of their existing memories of the mentioned character, so they can react in character based on what they know about the mentioned character.
  • After the chat, each listener gets an extra LLM pass that extracts what they learned secondhand about the mentioned character. If anything was learned, it's appended to the relationship's rolling list of offscreen learned information, and the listener's consolidated memory and relationship descriptor are updated to account for it. Their next conversation goal towards the mentioned character may also be updated, but by default the LLM is given the option of retaining the old one, as it might be more important.

This is how gossip works. Alice tells Bob something embarrassing about Carol, and the next time Bob runs into Carol, he remembers it, even though Carol was never there.

How memories get into prompts​

When a character speaks, their system prompt (by default) includes:

  • Their global memory.
  • Their consolidated pairwise memory of each other participant.
  • Their next conversation goal towards the other participant (in one-on-one conversations, the default template omits this for group chats).
  • Optionally, their memory of a "gossip target". At the start of a chat, the system sometimes chooses a non-participating character, and injects into the participating characters' system prompts their memories about the gossip target, gently encouraging them to talk about the gossip target. The rate at which this happens can be controlled via settings, and it's intended to create more information sharing between characters about other characters.

Watching it happen​

After each conversation, all of the memory updates are recorded in the conversation log, so you can expand them and see exactly how the conversation changed each character's head. You can also edit memories by hand in the character overview.

Customizing it​

Every LLM-driven step here is a prompt template you can edit, which can change how the system works significantly (though there are no custom memory fields - yet). See the prompt template system for more info.