AKA-1.5: New photorealistic avatars

Today we're introducing AKA-1.5, a new generation of photorealistic avatars on Akapulu.

The problem

We used to train live avatars on real human video. That looks like the right distribution. In practice it is a hard one to learn from. Every actor brought a different setup:

• Distance to camera — the same face filled a different fraction of the frame
• Talking speed — the same phoneme occupied a different number of frames
• Bounding box — as they talked, head aspect ratio changed, the box grew and shrank, and the motion coefficients had to adapt to a geometry that was never stable

The model spent its capacity becoming robust to all of that, and the cost showed up on the person: lips smear while the head breathes, and the mouth never quite lands where a real speaker would put it.

Image, training video, realtime engines

AKA-1.5 trains on footage we generate, from a locked identity still. Three steps:

1. Image

Each identity starts as a photorealistic still. We generate that still with pose control: a shared skeleton so every person sits at the same camera distance, with the same head size, in the same place in the frame. Appearance changes. The body configuration does not. That is the input the rest of the pipeline is built on.

2. Training video

We run that still through our own static video API — the same API that renders scripted clips. It is a video diffusion model. It takes the still, the pose, and audio, and outputs talking upper-body video.

It is too slow for real-time inference. A live call has to answer in well under a second. This API takes much longer than that, so it never sits on the conversation. It only builds the training set.

The mouth needs more resolution than a wide shot can give it, so the API renders the face and the upper body as two passes and composites them. That video is what we fine-tune on.

3. Finetuned realtime engines

We take that generated video and fine-tune a live model per identity, then compile it into realtime engines. Those engines are what run on a call.

Each of the eight public avatars below was built this way: a pose-controlled still, generated training video from the static video API, then a fine-tuned realtime engine for live conversation.

Clara

Edda

Hollis

James

Mara

Marcus

William

Wren

You can try AKA-1.5 on a live call. Talk to Clara on akapulu.com, or pick someone else from the set.