Introducing AKA-1: Akapulu Lab's Flagship Real-Time Avatar Model

Today we're introducing AKA-1, Akapulu's flagship real-time avatar model. It converts text into a synchronized, photorealistic talking avatar fast enough to sustain live, two-way conversation.

Getting there is harder than it looks. Real-time audio-driven avatar synthesis is one of the more punishing problems in applied generative AI, and most systems that look good in a demo reel quietly fail the moment they have to run live, uninterrupted, for the length of a real conversation.

Below is a tour of where these systems break, and what AKA-1 does differently.

Diffusion: high fidelity, and way too slow

Why full video diffusion can't run live

End-to-end video diffusion models produce the highest visual fidelity available today, but they generate pixels through hundreds of iterative denoising steps, each one a full forward pass over a high-dimensional spatiotemporal latent.

• Cost scales with sampling steps × latent grid size
• With transformer backbones, attention scales quadratically in spatial-temporal tokens
• A few seconds of video can take tens of seconds of compute, a non-starter when conversational AI needs a response within 1 second
• Smaller models or fewer steps trade away exactly the realism that made diffusion worth using

AKA-1's approach: diffuse over motion, not pixels

AKA-1 sidesteps the cost curve by pushing diffusion to its extreme: we do not diffuse over pixels, or even over spatial latent patches, but over a compact vector of facial control parameters.

1. Factor identity from dynamics: lock the speaker's appearance into a fixed reference
2. Denoise kinematic coefficients: low-dimensional values governing face, expression, head pose, and articulation, conditioned on incoming audio
3. Single-shot render: a synthesis network expands coefficients into full-resolution pixels in one feed-forward pass

Because appearance is held constant, the generator only solves the audio-to-dynamics problem rather than re-synthesizing the entire face at every step. The diffusion target is a few hundred parameters per frame instead of the millions of values in a spatial latent, collapsing per-step cost by orders of magnitude and allowing very few sampling steps with no visible quality loss.

Small errors = big drift

The streaming feedback loop

To run live, a model has to generate in a streaming, chunk-by-chunk fashion, emitting short windows of motion as audio arrives and conditioning each new segment on its own past output. The model's predictions become the context for its next predictions.

Any autoregressive system of this form is subject to exposure bias: the mismatch between clean ground-truth context seen during training and imperfect, self-generated context at inference. Small errors in early frames are fed back in and, because the mapping is recursive, compound rather than wash out.

What goes wrong over a long session

• Identity drift: the reconstructed face gradually diverges from the original speaker as residual error accumulates across windows
• Temporal jitter: motion loses smoothness at chunk seams, producing abrupt transitions the eye reads instantly as fake

How AKA-1 prevents drift

• Fixed identity anchor: every frame is reconstructed against an immutable source representation, not a degrading copy of the previous frame. No recursive appearance path for drift to travel down.
• Bounded, overlapped context: temporal conditioning is explicitly bounded with boundary-aware blending across chunk seams. Historical state informs continuity but cannot snowball.

An avatar that looks identical in minute thirty as it did in second one, with smooth, seam-free motion across unbounded multi-turn dialogue.

Who you are vs. how you move

The entanglement problem

Many pipelines drive each new frame from features pulled out of previous frames, which entangles two fundamentally different signals:

• Identity & appearance: what this person looks like
• Motion: how they are currently moving (expression, articulation, pose)

Because both ride in the same shared representation, the network has no clean way to vary one without perturbing the other. When those signals bleed together:

• Flickering between frames
• Smearing around the mouth as lips move
• A face that subtly warps as it talks, with appearance re-estimated from motion-contaminated features every step

Separate, decoupled representations

AKA-1 keeps appearance and motion apart end to end:

1. Extract a fixed identity code once per session; appearance locked for the entire conversation
2. Generate motion independently, conditioned only on audio
3. Combine at render time: motion applied on top of the frozen identity, never folded back into it

Clean, stable, phoneme-accurate lip-sync and full-face animation, with coordinated motion across jaw, cheeks, eyes, and brow, instead of a moving mouth pasted onto a drifting head.

Voice + video in one pipeline

A real-time conversational avatar breaks down into four sub-stages, chained back to back:

• STT (speech-to-text): transcribing what the user says into text
• LLM (reasoning): deciding what to say back
• TTS (text-to-speech): turning that response into a spoken audio waveform
• Avatar rendering: animating a face that speaks that audio in sync

Best-in-class partners for STT & LLM

• Deepgram: fast, accurate, low-latency speech recognition
• OpenAI: bring your favorite LLM and configure avatar behavior however you need

AKA-1 owns the hard part: voice + face

The last two stages (speech synthesis and avatar rendering) are where the real-time problem lives. AKA-1 takes the LLM's text output and produces both the lifelike voice and the synchronized, photorealistic talking face:

• Text-to-speech: LLM tokens → audio waveform
• Audio-to-motion-to-video: waveform → moving face

Run naively, these stages are sequential and latencies compound. AKA-1 instead runs them as a single streaming pipeline, emitting audio in small chunks and handing each to the renderer as it is produced, so the face starts animating before the sentence finishes. Because the stages run concurrently, their costs hide behind each other instead of summing.

Non-autoregressive speech synthesis

Our speech model is non-autoregressive and style-conditioned: a single style vector drives both timbre (who the speaker is) and prosody (how they speak), feeding a neural vocoder that produces the waveform in one pass.

Unlike token-by-token speech models that must generate each slice of audio conditioned on the one before it, our synthesizer predicts full prosody and duration up front, then decodes the entire waveform in a single parallel pass, with no sequential sampling loop to bottleneck on.

One GPU, one container

The entire back end runs inside a single container on one GPU. The voice model is non-autoregressive and the motion model diffuses over a handful of facial coefficients rather than pixels, so both stages fit in the same memory budget and execute back to back on the same device, with tensors passed directly from speech to render and no network hop.

• No cross-service latency
• No idle GPU waiting on the other stage
• No stacked per-vendor markup
• A fraction of the cost of a multi-vendor pipeline → per-minute pricing

Voice from a single short recording

Every Akapulu avatar is built from a real human actor. From one short reference recording we fine-tune a dedicated voice model, but fine-tuning the whole network overshoots fast, pushing audio out of distribution into gargly artifacts. Instead, speaker identity routes through a tightly constrained timbre adapter while the rest of the model stays frozen.

The result: a voice close to the actor, stable, clear, and natural, fit from a single short recording and consistent across thousands of conversations.

Interruptible by design

Because work flows in small chunks, an incoming user turn can barge in immediately: the avatar stops and re-engages like a person instead of talking over you. Output is sustained at a continuous 25 FPS with seamless idle-to-speaking transitions.

Generic models produce generic faces

Most systems animate a single shared model across every identity, conditioning on a reference image at inference time. Deployment stays simple, but subtle, person-specific detail gets regressed toward the mean, because the network's weights encode a population average of faces and motion.

Per-avatar fine-tuning

From an actor's reference material (a short, 1 minute clip), AKA-1 fits dedicated weights for that individual:

• Voice profile: timbre and prosody
• Motion profile: idiosyncratic dynamics: how their mouth shapes phonemes, brow and eye movement, resting pose, micro-expressions
• Appearance model: high-fidelity capture of their face

Those weights compile into an optimized inference engine for the target hardware, so personalization adds fidelity without adding latency. The identity stays consistent across thousands of conversations because it is baked into the model, not approximated from a reference at run time.

Text, then voice, now face

The first wave of conversational AI was text. Now, we're starting to see voice-to-voice applications flourish. We believe face-to-face interactions are the next step, and the hardest one to get right.

A response can be fast and still not feel present. A face can look good in a demo and still break once a conversation runs long, gets interrupted, or needs to hold a specific identity across thousands of turns.

AKA-1 exists to solve that layer: real-time face-to-face presence, not just animated output. It is the model behind every live avatar on Akapulu today, and the foundation for the next generation of live, face-to-face AI experiences.