Persona

Persona does not capture your face itself. It is a receiver: a phone app captures ARKit face tracking and streams it over your local network, and Persona maps that stream onto the model every frame — a Live2D model's parameters, or a VRM avatar's expressions and bones

You need an iPhone with a TrueDepth camera — the same requirement as VTube Studio — and both devices on the same network

Several feeds can run at once. Each one is a tracking source, and you decide which avatars in the scene it drives — so several people can each hold a phone and each drive an avatar, on one machine or across a VPN

Tracking Sources

Tracking is a row of its own in the Control Panel's left column, there whatever you have selected — a source is an input this machine listens on, and it outlives the scene whose avatars it happens to be driving. It lists every configured source, one row each, carrying that app's icon, the source's name, its status right now, and a switch. A fresh install ships two sources: one VTube Studio App (3rd Party UDP) and one VMC Protocol

Clicking a row expands its editor:

Add Tracking Source… at the bottom adds any protocol, and Remove Source at the foot of the editor deletes one

Below the list sit the Face Tracking master switch with its aggregate status, and the Local IP Address card showing what your machine answers on — which is what a phone app asks for when it wants a destination

Assigning Avatars to a Source

Avatars is a multi-select dropdown: tick the avatars this source should drive, or take them all with Select All. It only counts avatars this source's channel can drive — a VMC source lists VRMs only

An avatar belongs to one source per channel. To take one another source already holds, its row reads Tracked by ‹name›, and ticking it raises a Reassign Tracking confirmation naming exactly what is about to move before anything does

The other direction is free: one source can drive several avatars at once, mirroring its feed to every avatar assigned to it. That is how one phone puts the same expression on two avatars

Two Layers of Switch

A row's status answers to two things: the source's own switch, and its channel's master switch (Face Tracking or Body Tracking). With either off, the row reads off. Only with both on does it show its own live status

So you can stop one source for a moment, or cut a whole channel at once, without editing sources one by one

VTube Studio App (3rd Party UDP)

The protocol VTube Studio for iOS offers to non-VTS desktop clients

1. In the VTube Studio iOS app, enable streaming to third-party PC clients
2. Add a source of this kind in Persona, assign it avatars, and enable face tracking

Persona broadcasts a tracking request to UDP port 21412 on every local subnet once per second, telling the phone which port to stream back to. There is nothing to type in

iFacialMocap / Facemotion3d

Both apps speak the same protocol, and Persona parses the v2 format they each emit today

1. Add a source of this kind in Persona, assign it avatars, and enable face tracking
2. Start iFacialMocap or Facemotion3d on the phone. If it asks for a destination address, enter your computer's local IP — the Local IP Address card in the Tracking section lists them

Persona listens on UDP 49983 and sends a periodic hello so the app starts streaming

Running several feeds of the same protocol is what Device IP is for: give each source the address of its phone and the feeds stay apart. A source left blank catches every sender no source of its kind has claimed, so two phones driving two avatars means two sources, each with an IP filled in

What Each Source Carries

Both face sources talk to the same phone and send Persona the same data: all 52 raw ARKit blendshapes, plus head rotation and position. What differs is the protocol, not the payload

Alongside the derived vocabulary, each of those 52 raw values is bindable on its own as an ARKit* input in Parameter Bindings — so a Live2D model can read the shape the phone measured rather than Persona's reconstruction of it

Status

Each source's row, and the line under the tracking master switch, report what is actually happening:

A row's status is that source's own; the one under the master switch is the channel's aggregate. Read the row to find which phone dropped out, the aggregate to see whether anything is arriving at all

How Tracking Reaches the Model

Tracking data passes through three stages before it touches a parameter:

1. Receive — whichever source got the data parses its protocol into a neutral frame: head rotation, head position and ARKit blendshapes. The two steps below then run once per avatar assigned to that source
2. Derive — the frame is converted into VTube Studio's input vocabulary, the same signal names model authors work with: FaceAngleX, EyeOpenLeft, MouthSmile, Brows and so on. The 52 raw ARKit channels pass through untouched beside them, one ARKit* input each — no latch, no calibration, no gain
3. Map — each parameter binding sums one or more inputs at their weights, then runs the total through an input range, an output range, a response curve and per-parameter smoothing before writing a Cubism parameter

Head rotation, head lean, eye open and blink, eye gaze, eyebrows, cheek puff, mouth open, mouth shape and smile or frown are all driven. Parameters a model does not have are skipped, and any parameter a frame does not carry is released rather than frozen at its last value

Models with Their Own Mapping

If a model ships a .vtube.json, Persona uses that model's mapping — its parameter pairings, its ranges, its gains and its smoothing — rather than the built-in defaults. A rig whose author deliberately crossed the eyes, doubled the eye-open range or renamed a parameter behaves in Persona exactly as it does in VTube Studio

Models without a .vtube.json use a built-in mapping tuned against real ARKit recordings

VBridger Parameters

VBridger is not a parameter registry but an engine for user-authored formulas: each output name carries one to three expressions over the 52 ARKit shapes. What gets called its "standard" is the preset it ships, VBridger_AdvancedARKit_V3.0, and the names in it — MouthPucker, MouthFunnel, MouthShrug, MouthPressLipOpen and the six Body* — are absent from VTube Studio's own input registry. Over there they exist only as plugin-created custom parameters, and the plugin creating them is VBridger

So Persona does not treat those names as inputs. They expand, as a .vtube.json is imported, into the same weighted sums VBridger computes them with, transcribed from its preset. MouthPucker is (mouthDimple_R + mouthDimple_L) × 2 − mouthPucker, with the pucker negative; the six Body* are the same expression as their FaceAngle*/FacePosition* counterparts. Once expanded they are ordinary weighted inputs, laid out in the editor for you to change

MouthOpen and JawOpen are two separate inputs: the first is how far the lips separate, the second how far the jaw drops — exactly the pair a VBridger rig maps as ParamMouthOpenY and ParamJawOpen. JawOpen is a verbatim copy of ARKitJawOpen, so both names read the same float and either one may be written

Model Position Movement

Leaning toward or away from the camera moves the whole model, rather than a rig parameter: it slides with your head's horizontal and vertical position and scales as you lean in and out. This mirrors VTube Studio's built-in behaviour, including its default amounts and smoothing, and is configured per model through the ModelPositionMovement section of a .vtube.json

VRM Avatars

The same two sources drive a VRM avatar; only the last step differs, because a VRM has no Cubism parameters

A model carrying all 52 ARKit blendshapes as expressions — perfect sync — skips the derived vocabulary entirely and is driven from the phone's raw ARKit values one to one, which is as faithful as the phone gets. Names are matched case-insensitively, since perfect-sync rigs author them in every casing. Partial coverage is not enough: a model with 51 of the 52 falls back to the mapping below, and the Perfect Sync row of the info block reports the count

Without perfect sync, the derived signals map onto the model's presets:

• Mouth — mouth open, funnel, pucker and smile combine into the VRM viseme expressions (aa, ih, ou, oh), with the shapes constraining each other so they never stack into an impossible mouth
• Eyes — per-eye blinks when the model has them, or the combined blink expression when it does not
• Head — yaw, pitch and roll, with a share distributed down the chest and spine so the whole torso follows a head turn naturally
• Gaze — eye direction through the model's own look-at, whether it is bone-driven or expression-driven

Idle Handover

While tracking is active it takes over from idle animation — look-at-cursor and automatic blinking step aside. Turning tracking off hands control straight back to idle

Losing the phone's stream mid-session is treated differently: the avatar holds its last tracked pose rather than snapping back, and only the eyes are released — so automatic blinking keeps the face alive until the stream returns

No Calibration

There is no neutral-pose calibration, matching what VTube Studio does on the ARKit path: it trusts the absolute head pose the phone reports. Position the phone facing you at roughly eye level — a phone propped low reads as a permanently raised chin

Body Tracking

A VRM has a humanoid rig, so it can also take a full-body pose from a mocap application over VMC Protocol. The Body Tracking switch appears under face tracking as soon as the scene holds a VRM — with none, it stays hidden, since Live2D has no skeleton to drive

Anything that sends VMC works: VSeeFace, Virtual Motion Capture and mocopi among them. Persona is a pure receiver — the sender does the solving, so there is nothing to calibrate here either

1. Pick a VMC Protocol source under Tracking, or make a new one with Add Tracking Source…
2. Tick the VRMs it should drive under its Avatars, then turn Body Tracking on
3. Point the sending application at your computer's local IP and at that source's Port, 39539 by default

A VMC source is identified by its port rather than by the sender's address, so several body feeds means giving each source a different port. Two sources cannot share one — one of them would fail to bind, so a duplicate reverts the field

Body tracking has its own status line, reading the same off / waiting… / tracking values as face tracking. Face tracking can run at the same time — the face keeps coming from the phone while the body comes from the mocap app, each channel through its own sources

Troubleshooting

Status stays at "waiting…"

• Confirm the phone and the computer are on the same network, and that the network is not one that blocks device-to-device traffic — guest and public Wi-Fi usually do
• Check that your firewall allows Persona to receive incoming UDP
• Disconnect from any VPN. A VPN tunnel commonly captures broadcast traffic. Persona sends subnet-directed broadcasts on each physical interface specifically to avoid this, but some configurations still swallow them

If broadcasts cannot get through, pin the phone's address directly with an environment variable before launching Persona:

The model moves, but the wrong features respond

Check whether the model ships a .vtube.json. Persona honours it, so a mapping that looks reversed is most likely the one the model's author wrote — verify by loading the same model in VTube Studio

Some parameters never move

Expand Info on the selected model and check the parameter count. Models using non-standard parameter names, and older rigs using Cubism 2 naming conventions, will have no parameter for a given input to drive

Nothing responds, but the status says "tracking"

Expand that source and check its Avatars. Data is arriving, but with no avatar assigned to the source it has nowhere to go. Check too that another source has not taken the avatar — one avatar belongs to one source per channel

Two phones, but they move the same avatar

With Device IP blank on both sources, whichever feed arrives first is caught by the unclaimed source, and it is a race. Fill in each source's phone address and the feeds separate