Persona

Persona receives ARKit face data streamed by a phone app over the local network. Each frame is mapped to Live2D parameters or a VRM avatar’s expressions and bones. The phone performs the face capture

The phone must meet the chosen tracking app’s device requirements and connect to the same network as the computer

A webcam on the computer can track the face, hands and body instead, and a microphone can drive the mouth

Several feeds can run at once. Each one is a tracking source, and you decide which avatars in the scene it drives — so several people can each hold a phone and each drive an avatar, on one machine or across a VPN

Tracking Sources

Tracking appears in the Control Panel’s left column, independently of the selected layer. Tracking sources are application settings and remain available when you switch scenes. From top to bottom it holds Face Tracking, Pose Tracking, the Local IP Address card, Webcam Tracking, Lip Sync and Controller. Face Tracking and Pose Tracking each have a master switch, aggregate status and source list. Each source is a card showing its app icon, name, status and switch. Persona includes a VTube Studio App (3rd Party UDP) face source and a VMC Protocol pose source by default

Clicking a card expands its editor:

Add Face Source… and Add Pose Source…, cards beside each section's source cards, add any protocol supported by that channel; Remove Source at the foot of the editor deletes one

Enable Face Tracking and Enable Pose Tracking control their respective channels' network sources, with aggregate status shown beside each switch; the webcam has a switch of its own. Pose tracking takes VMC and mocopi sources, supports VRM only and displays a notice when the scene has no VRM avatars. The Local IP Address card below these sections provides the destination address for phone apps

Assigning Avatars to a Source

Assign Avatars opens a multi-select: tick the avatars this source should drive, or take them all with Select All. It only counts avatars this source's channel can drive — a pose source lists VRMs only

The webcam source's card has Face, Body and Hands buttons instead, one per channel — see Webcam Tracking

Each avatar can be assigned to one source per channel. Avatars already assigned elsewhere show Tracked by ‹name›. Selecting one opens a Reassign Tracking dialog listing the affected avatars; the assignment changes only after confirmation

One source can drive multiple avatars, sending the same data to each. For example, one phone can give two avatars the same expressions

Each avatar's own detail also has a Tracking section for its per-avatar settings: Controller Movement, and Microphone Lip Sync, which picks when the microphone drives this avatar's mouth — see Lip Sync. Which source drives it is set on the source cards

Master and Source Switches

A row's status answers to two things: the source's own switch, and its channel's master switch (Enable Face Tracking or Enable Pose Tracking). With either off, the row reads off. Only with both on does it show its own live status

So you can stop one source for a moment, or cut a whole channel at once, without editing sources one by one

Supported Protocols

The following pages explain how to set up each protocol and how it transmits data:

To use multiple sources of the same protocol, enter each phone’s address in its source’s Tracker IP field. A source with no address accepts senders not assigned to another source of that type. For two phones driving separate avatars, configure two sources with their respective IP addresses

What Each Source Carries

All three face tracking protocols send the 52 raw ARKit blendshapes, plus head rotation and position, and support VRM perfect sync. Persona handles input derivation, blink latching, and eye-open calibration for all three. The table compares how each protocol connects and transmits that data

Quantization can become more noticeable when Persona applies gain to a channel. For example, the mouth-frown signal is amplified 5×, which can make 1% input steps visible in the model's expression

LAPLACE Persona Tracker also offers one-tap calibration that is saved between sessions, a Save Energy mode that dims the screen, and a wireframe preview instead of the camera feed. See LAPLACE Persona Tracker for details

Each of the 52 raw values is also available as an individual ARKit* input in Parameter Bindings. Live2D models can use either Persona's derived inputs or the original blendshape values captured by the phone

The webcam's face tracking computes the same head pose and blendshapes from the camera image, without tongueOut

Status

Each source's row, and the line under the tracking master switch, report what is actually happening:

A row's status is that source's own; the one under the master switch is the channel's aggregate. Read the row to find which phone dropped out, the aggregate to see whether anything is arriving at all

The webcam source uses the same words, but reads tracking whenever one of its enabled tasks detects a face, a hand or a body — see Webcam Tracking. The Face Tracking and Pose Tracking aggregates count network sources only

How Tracking Reaches the Model

Tracking data passes through three processing stages:

1. Receive — whichever source got the data parses its protocol into a neutral frame: head rotation, head position and ARKit blendshapes. The two steps below then run once per avatar assigned to that source
2. Derive — the frame is converted into VTube Studio's input vocabulary, the same signal names model authors work with: FaceAngleX, EyeOpenLeft, MouthSmile, Brows and so on. The 52 raw ARKit channels pass through untouched beside them, one ARKit* input each — no latch, no calibration, no gain
3. Map — each parameter binding sums one or more inputs at their weights, then runs the total through an input range, an output range, a response curve and per-parameter smoothing before writing a Cubism parameter

Head rotation, head lean, eye open and blink, eye gaze, eyebrows, cheek puff, mouth open, mouth shape and smile or frown are all driven. Parameters a model does not have are skipped, and any parameter a frame does not carry is released rather than frozen at its last value

Models with Their Own Mapping

If a model ships a .vtube.json, Persona uses that model's mapping — its parameter pairings, its ranges, its gains and its smoothing — rather than the built-in defaults. A rig whose author swapped the left and right eye bindings, doubled the eye-open range or renamed a parameter behaves in Persona exactly as it does in VTube Studio

Models without a .vtube.json use a built-in mapping tuned against real ARKit recordings

VBridger Parameters

VBridger is not a parameter registry but an engine for user-authored formulas: each output name carries one to three expressions over the 52 ARKit shapes. What gets called its "standard" is the preset it ships, VBridger_AdvancedARKit_V3.0, and the names in it — MouthPucker, MouthFunnel, MouthShrug, MouthPressLipOpen and the six Body* — are absent from VTube Studio's own input registry. Over there they exist only as plugin-created custom parameters, and the plugin creating them is VBridger

So Persona does not treat those names as inputs. They expand, as a .vtube.json is imported, into the same weighted sums VBridger computes them with, transcribed from its preset. MouthPucker is (mouthDimple_R + mouthDimple_L) × 2 − mouthPucker, with the pucker negative; the six Body* are the same expression as their FaceAngle*/FacePosition* counterparts. Once expanded they are ordinary weighted inputs, laid out in the editor for you to change

MouthOpen and JawOpen are two separate inputs: the first is how far the lips separate, the second how far the jaw drops — exactly the pair a VBridger rig maps as ParamMouthOpenY and ParamJawOpen. JawOpen is a verbatim copy of ARKitJawOpen, so both names read the same float and either name can be used in a binding

Model Position Movement

Leaning toward or away from the camera moves the whole model, rather than a rig parameter: it slides with your head's horizontal and vertical position and scales as you lean in and out. This mirrors VTube Studio's built-in behaviour, including its default amounts and smoothing, and is configured per model through the ModelPositionMovement section of a .vtube.json

VRM Avatars

The same face sources drive a VRM avatar; only the last step differs, because a VRM has no Cubism parameters

A model carrying all 52 ARKit blendshapes as expressions — perfect sync — skips the derived vocabulary entirely and is driven from the phone's raw ARKit values one to one, preserving the original tracking values. Names are matched case-insensitively, to accommodate different model naming conventions. Partial coverage is not enough: a model with 51 of the 52 falls back to the mapping below, and the Perfect Sync row of the info block reports the count

Without perfect sync, the derived signals map onto the model's presets:

• Mouth — mouth open, funnel, pucker and smile combine into the VRM viseme expressions (aa, ih, ou, oh), with the shapes constraining each other to avoid conflicting mouth shapes
• Eyes — per-eye blinks when the model has them, or the combined blink expression when it does not
• Head — yaw, pitch and roll, with a share distributed down the chest and spine so the whole torso follows a head turn naturally
• Gaze — eye direction through the model's own look-at, whether it is bone-driven or expression-driven

Switching Between Tracking and Idle

Active tracking takes priority over idle behaviour, pausing automatic blinking. Turning tracking off restores idle behaviour

What happens when the stream drops mid-session is set per avatar by When Tracking Is Lost in its Idle section: the default Keep Last Pose freezes on the last tracked frame rather than snapping back, while Return to Idle eases to the idle animation. Either way only the eyes are released, allowing automatic blinking until the stream returns. Live2D can also name a stand-in motion — see When Tracking Is Lost

No Calibration

There is no neutral-pose calibration, matching what VTube Studio does on the ARKit path: it trusts the absolute head pose the phone reports. Position the phone facing you at roughly eye level — a phone propped low reads as a permanently raised chin

This applies to phone sources; the webcam calibrates in Persona

Webcam Tracking

Webcam Tracking runs face, hand and body tracking from a camera on this computer, under its own Enable Webcam Tracking switch. Each avatar takes its face, body and hands from separate sources, so the webcam can add fingers to a phone and a mocap kit. See Webcam Tracking

Lip Sync

Lip Sync drives avatars' mouths from a microphone, with a voice calibration saved for each microphone. Each avatar's Microphone Lip Sync picks Off, Always or When Face Tracking Is Unavailable. See Lip Sync

Controller

The Controller section at the bottom of Tracking supports parameter bindings for Live2D and VRM, per-instance stage movement, and combinations for automations and avatar shortcuts. Controllers do not use tracking sources or avatar assignments, and cannot control the camera

Connecting a Controller

Turn Enable Controller on, then connect a controller. DualSense and DualSense Edge go through WebHID and are found without focusing a window or pressing a button first, including after the app relaunches. Other controllers go through the browser's Gamepad API: click the stage and press a button before they appear

Connected Controllers counts devices that are live right now, unassigned ones included. Connected profiles sort first; the rest stay in the collapsed Saved Profiles group with their names and bindings intact. Selecting a card only changes the live monitor, never a binding

Profiles are saved in app settings with their number, the reported product name, and a hash of the device's hardware identity — only the hash. Two identical controllers keep their own bindings when the connection order or the cables change. Unplugging one only releases its input; it deletes no profile and no binding

Xbox, other PlayStation and Switch Pro controllers keep Chromium's standard mapping, which reads physical button positions rather than the letters printed on them. Unrecognized browser layouts are listed as unsupported. Gyro, touchpad, custom remapping and merging several devices into one are outside this version

Driving Parameters

A model's Parameters editor carries one input family per saved or already-bound profile, 19 controls each: two sticks, stick clicks, the D-pad, face buttons, shoulders, triggers, options and home. Sticks and the D-pad range from −1 to 1 with positive Y upward; buttons and triggers from 0 to 1. Each binding picks which family it reads, so different controllers can drive different parameters on different avatars

Profile #1 keeps the VTS names — ControllerStickLeftX, ControllerCross — so existing mappings keep working. Profiles #2 onward insert the number after Controller: Controller2StickLeftX, Controller3Cross, Controller12TriggerRight. Those numbered names are a Persona extension, and the plugin API uses the same ones in its input targets

Controller input feeds each loaded model's own rules, independently of the face tracking switch and of source assignment. It combines with face terms before the range, curve and smoothing steps — through a dropout, held face inputs stay available while controller terms keep responding. Scrubbing an input in the editor takes precedence over the same physical input. Controller bindings share one frame-rate-independent exponential response: smoothing 0 is immediate, 100 has a 350 ms time constant

Unplugging one controller releases only its family; switching Enable Controller off releases every one. Live2D outputs that no remaining source drives reset to their Cubism defaults

VRM Targets

The same Parameters editor generates a second set of targets for a VRM avatar. These are virtual binding targets, not parameters imported from Cubism:

Angles follow the existing face-input conventions, and the renderer handles the VRM 0.x / 1.0 pitch and roll differences; expression names keep the spelling authored in the model. Bones and gaze layer over the current animation, pose tracking and face tracking, while an expression binding owns only the weight it targets. At the end of each frame the driver restores the underlying state, so offsets never accumulate and hand-set expression weights are never erased. Releasing an input or removing a rule eases that target back to whatever the layer underneath is doing; other controllers and tracking sources continue to control their assigned targets

Bindings belong to the model, stored in its existing tracking sidecar, so every instance of that model shares them. To move two copies of one model independently, use the per-instance movement settings below

Stage Movement

In an avatar's own Tracking section, turn Controller Movement on and pick a saved profile. The Enable Controller master switch has to be on too. Movement is optional and off by default — parameter bindings and shortcuts do not depend on it

Move Speed is stage heights per second on Live2D and metres per second on VRM (default 0.5, range 0–10); it also sets the scale response on Live2D and the lift speed on VRM. Turn Speed is degrees per second (default 90, range 0–720). Response Smoothing is the velocity response time in seconds (default 0.12, range 0–0.5), where zero responds immediately. Live2D scaling is proportional to the current size and keeps the normal scale limits

Movement integrates the smoothed velocity over elapsed time, so 30, 60 and 120 fps travel at the same speed. Returning the sticks to neutral eases to a stop at the new placement. Unplugging, disabling or reassigning a controller clears its velocity at once, and a reconnect never resumes stored momentum. Hidden avatars do not move. Direct placement gestures and placement transitions — automation Model Position actions and API moves — pause movement for that avatar, as do text editing, shortcut recording and controller assignment

Movement edits the instance's normal placement, and persists with the scene

Walking While Moving

VRM only. Turn Walk While Moving on and pick a Walking Animation from the animation inventory — the switch can be on before a clip is chosen, and stays inactive until one is. Imported VRMA, Mixamo FBX and MMD VMD animations use the same registered assets as idle clips

The clip loops while the avatar travels across the ground and blends back to its normal idle when it stops. Walk Cycle Speed is the clip's nominal speed in metres per second at playback speed 1; playback follows measured ground speed divided by that and by the avatar's scale, so changing movement speed or model size adjusts the gait rate. Hips X/Z translation is removed to keep scene placement authoritative, while the vertical bob remains. The walking layer omits head, gaze and expression tracks, preserving controller and face posing. One-shot animations take priority, body tracking on the hips or legs — VMC, mocopi or the webcam — suppresses walking, and hands-only VMC can run alongside it

Face Movement Direction turns the avatar toward travel at Turn Speed, whether or not a walking clip is enabled; right-stick turning takes priority. Walking follows actual planar travel, including the smooth stop — a height change alone does not trigger it. This is animation layered over direct stage movement, with no navigation, collisions or terrain following

Controller Shortcuts

The shortcut recorder — for an automation's Keyboard or Controller trigger and for avatar shortcuts alike — accepts up to three controls from one controller, stick and D-pad directions and triggers included. A recorded button identifies its profile, so the same button on #1 and #2 can run different actions; buttons held on another controller are excluded from the chord. A press fires once, and holding the combination does not repeat. Run in Background applies to controller shortcuts without needing a keyboard modifier — recording and text editing suppress activation and consume those presses, so leaving a field or closing the recorder cannot fire a still-held button

Controller #1 combinations use the canonical VTS Controller* tokens; #2 onward use the numbered Persona tokens. They never register with the operating system's global shortcuts, and mixed keyboard/controller and cross-controller combinations are unsupported. For an identical combination, an automation takes priority over avatar shortcuts

.persona.json stores every profile in its normal trigger strings. When a model uses .vtube.json, numbered controller shortcuts go in the hotkey entry's PersonaControllerTriggers extension, with the native VTS trigger slots cleared so VTS never receives unknown enum names. Persona reads the extension preferentially; assigning a keyboard or #1 shortcut, or clearing it, removes the extension

Troubleshooting

Status stays at "waiting…"

• Confirm the phone and the computer are on the same network, and that the network is not one that blocks device-to-device traffic — guest and public Wi-Fi usually do
• Check that your firewall allows Persona to receive incoming UDP
• Disconnect from any VPN. A VPN tunnel commonly captures broadcast traffic. Persona sends subnet-directed broadcasts on each physical interface specifically to avoid this, but some configurations still block them

If broadcasts cannot get through, specify the phone's address directly with an environment variable before launching Persona:

The model moves, but the wrong features respond

Check whether the model includes a .vtube.json. Persona uses the mappings in this file. Load the same model in VTube Studio to compare how the affected parameters respond

Some parameters never move

Expand Info on the selected model and check the parameter count. Models using non-standard parameter names, and older rigs using Cubism 2 naming conventions, will have no parameter for a given input to drive

Nothing responds, but the status says "tracking"

Expand the source and check its Avatars list. Incoming data only drives avatars assigned to that source. An avatar can belong to one source per channel, so also check whether it is assigned to another source

Two phones, but they move the same avatar

With Tracker IP blank on both sources, either unmatched source may receive a phone's feed. Enter the corresponding phone address for each source to route its data to the intended avatar