Rick Vu

How AI NPCs work in games in 2026, from persona to voice

,

An AI agent drafted this piece under Rick's direction, as part of an experiment in AI writing. Every figure links to its source.

ONE TURN

An AI NPC in 2026 is a language model running inside rules that the game owns. On each turn, the game builds a prompt from a written persona, the character’s memories, and the current game state. It then asks the model what the character says and which allowed action it takes, checks the answer in code, and speaks the line through a voice model.

The rest of this article walks through each part of that loop, what it costs in time, and what released and demonstrated games have done with it. Rick Vu, who directed this piece, builds Red Dog Studio, a game studio building AI-native NPCs. Nothing below describes Red Dog’s own stack.

One turn of an AI NPCThe game builds a prompt from a written persona, the character's memories, and the current game state. A model answers with a line and an action. Code checks the answer before the character speaks and acts.WHAT THE GAME KNOWSONE TURNTEXT AND A REQUESTED ACTIONPERSONAWRITTEN BYA PERSONMEMORYEVENTS,REFLECTIONSGAMESTATEWHAT ISTRUE NOWPROMPTASSEMBLED BY THE GAME EACH TURNMODELCLOUD OR ON THE DEVICECHECKS IN CODEFILTERS, ALLOWED ACTIONS, FALLBACK LINELINE AND ACTIONVOICE, ANIMATION, GAME EFFECT
Fig 1. One NPC turn. The model only ever sees what the game puts in the prompt, and nothing reaches the player until code has checked it.

One turn of dialogue, step by step

A typical turn with a spoken AI NPC runs through six steps:

  1. A speech recognition model turns the player’s voice into text. Typed input skips this step.
  2. The game builds the prompt from the persona, the recent conversation, relevant memories, and facts about the world right now.
  3. The language model writes a reply that contains dialogue, a tool call, or both.
  4. Game code checks the reply against rules and filters.
  5. The game runs any approved action and sends the dialogue to a text-to-speech model.
  6. The animation system plays the audio on the character’s face and body.

Every step adds delay, and every step can fail. Steps 2 and 4 hold most of the design work, and both are ordinary game code.

The persona is written by people

The persona is a block of text, usually the system prompt, that tells the model who the character is. It covers personality, backstory, what the character knows, what it refuses to discuss, and how it talks. The model reads it on every turn.

Writers still own this part. Ubisoft’s NEO NPC prototype at GDC 2024 used Inworld and NVIDIA technology, but writers authored the personas. Epic followed the same pattern with the Persona Device for Unreal Editor for Fortnite. The tool lets creators build AI characters, and it grew out of the game’s conversational Darth Vader NPC.

A second approach moves the persona into the model’s weights. The paper Fixed-Persona SLMs with Modular Memory fine-tunes a small language model for each persona on synthetic data and swaps memory modules in at runtime. The authors benchmarked it on consumer-grade hardware. A baked-in persona needs fewer prompt tokens on each turn. Changing the character means training the model again.

Memory comes in three layers

Short-term memory is the recent conversation, placed directly in the prompt. It is the cheapest layer and the first to run out, because every model has a context limit and every extra token adds latency.

Long-term memory is a store outside the model. The game writes events into it and pulls back the few that matter for the current turn. The reference design comes from the Generative Agents paper by researchers at Stanford and Google, which ran 25 agents in a Sims-like town called Smallville. Each agent records its experiences in natural language. Retrieval scores every memory on three measures and puts the top results in the prompt. Recency decays with game time. The model itself rates importance. Relevance is the embedding similarity between the memory and the current situation.

Reflection is the third layer. In the same paper, the agent reflects when the summed importance of recent events passes a threshold. It asks the model to draw higher-level conclusions from those events and stores the conclusions as new memories. A line like “the player lied to me about the key” is worth more in a prompt than twelve raw lines of dialogue. The paper’s ablation found that observation, planning, and reflection each mattered to how believable the agents were.

Released games use simpler versions of the same idea. In inZOI, NVIDIA says Smart Zoi characters adjust their daily schedules at the end of each day based on their experiences. That is a reflection step tied to the game clock instead of an importance score.

For memories about the player, store the facts the game must enforce as structured fields. Names, promises, debts, and quest choices belong there. Keep free-text summaries for tone, where a small mistake costs less.

Ground the model in game state

A language model knows nothing about the current match unless the prompt tells it. Grounding means the game writes the relevant state into the prompt on each turn. That state includes where the character stands, what it carries, the quest stage, the time of day, and who is nearby.

The game stays the source of truth. If the model says the door is open, the door stays shut until game code opens it. The AI Native Games survey names the central design problem as “organizing semantic openness into stable gameplay.” It also lists goals, rules, state, feedback, pacing, and player agency as the parts that AI-native design depends on. In practice, the model writes words and picks from a menu of actions, and the engine decides what happens.

Tool calls let the character act

A tool call is how the model asks the game to do something. The developer exposes a short list of functions, such as give_item, follow_player, or set_attitude, each with typed arguments. The model returns a structured request, and game code validates it before anything runs. A merchant cannot give away an item it does not have, because the inventory check runs in code.

Tool calls turn a chat partner into a teammate. NVIDIA describes PUBG Ally as a co-playable character that shares loot, drives vehicles, and fights other players, running on the Mistral-Nemo-Minitron-8B-128k-instruct small language model. The same NVIDIA ACE announcement says NetEase’s AI teammates in NARAKA BLADEPOINT find items, swap gear, and suggest skills. Each of those behaviors depends on an action the engine already supports. The model chooses among them and cannot invent new ones.

Guardrails keep the character on script

No single guardrail is enough, so production systems stack them:

Platform policy sits on top of those layers. Epic released the Persona Device as an experimental feature called Conversations in April 2026, according to Outlook Respawn’s report on the UEFN launch. That report says Epic’s rules ban NPCs that give medical or mental health advice or act as romantic partners. The same report says creators could not yet publish those islands to public servers.

A language model can always produce something unexpected. Guardrails lower how often that happens, and they do not bring it to zero. Test with players who try to break the character, and log every turn that ends in a fallback line.

The latency budget decides where inference runs

Spoken conversation leaves little room for delay. As a rough estimate, a reply that starts more than a second or two after the player stops talking feels like lag. That window has to cover speech recognition, the model’s first tokens, and the first chunk of synthesized audio. Streaming at every stage lets the character start speaking before the model finishes the sentence.

Cloud inference gives you larger models and puts no load on the player’s hardware. It also adds a network round trip, a per-token bill for every player, and a dependency on a server staying up. Fortnite’s Conversations takes the cloud route. Outlook Respawn’s Conversations report says it uses Google’s Gemini 3.1 Flash-Lite for the character and ElevenLabs for the voice, and both are hosted services.

On-device inference removes the network and the per-token bill, but the model has to fit beside the game. NVIDIA’s In-Game Inferencing SDK runs models inside the game process and schedules AI work next to rendering on the same GPU to limit the frame-rate cost. It also supports cloud workflows, so one game can use both. The shipped on-device examples are small. NVIDIA’s inZOI and NARAKA launch post says Smart Zoi run on an on-device Mistral NeMo Minitron model with about half a billion parameters. The same post quotes NetEase describing NARAKA’s AI teammates as running on the device locally.

A hybrid split is a sensible default. Run speech recognition and short reactive lines on the device, and send long conversations or rare high-stakes scenes to the cloud. The AI NPC cost per player article works through what each side costs.

Voice needs more models

Voice adds speech recognition on the way in and text-to-speech on the way out. The In-Game Inferencing SDK includes local Whisper speech recognition, and Fortnite’s Conversations use ElevenLabs for synthesized speech. Facial animation is a further model. In the Ubisoft NEO NPC press release, Ubisoft says it used NVIDIA Audio2Face for real-time facial animation.

Each voice model adds its own delay, so short lines work better than long speeches. Voice also raises the cost of a guardrail failure. Players can skim a bad line of text, but a bad line spoken aloud in a known character’s voice is hard to miss.

What has shipped and what was a demo

The public examples so far fall into a short list:

Questions

How do AI NPCs remember the player?

The game saves events about the player in a memory store outside the model and adds the most relevant ones to the prompt on each turn. Many designs follow the Generative Agents paper, which ranks memories by recency, importance, and relevance and turns batches of them into higher-level reflections.

Do AI NPCs run on the device or in the cloud?

Both approaches have shipped. inZOI and NARAKA BLADEPOINT run small NVIDIA ACE models on the player's own hardware, while Fortnite's conversational NPCs call cloud models. On-device inference avoids network delay and per-player API bills, and cloud inference allows larger models.

How do you stop an AI NPC from going off script?

Stack several checks. A human-written persona sets the limits, filters screen what the player says and what the model writes, game code validates every action the model requests, and a written fallback line plays when a check fails. None of these removes the risk entirely, so teams also test with players who try to break the character.

Which released games use AI NPCs?

inZOI launched its Smart Zoi characters as an experimental setting in Early Access in March 2025, and the PC version of NARAKA BLADEPOINT MOBILE added on-device AI teammates the same month. Fortnite opened AI-powered conversations to creators as an experimental tool in April 2026. Ubisoft's NEO NPC was a GDC 2024 prototype.