Persona

Persona receives ARKit facial tracking data streamed by mobile apps over the local network and maps it frame-by-frame to Live2D model parameters or VRM avatar expressions and bones. Facial tracking is computed in real time on the mobile device.

The mobile device must meet the hardware requirements of the chosen tracking app and connect to the same local network as the computer running Persona.

You can also use a computer-connected webcam to track face, hand, and body poses, or use a microphone for lip sync.

Persona supports running multiple tracking sources concurrently. Each source can be assigned independently to one or more avatars in the scene. For example, multiple performers can each hold a phone and drive separate avatars on the same computer or across a VPN connection.

Tracking SourcesCopy Anchor Link

Tracking sits in the left column of the Control Panel and doesn't depend on the selected layer. Tracking sources are app-wide settings, so they carry over when you switch scenes.

From top to bottom, the panel contains:

• Face Tracking: The Enable Face Tracking master switch and the face source list, which starts with one LAPLACE Persona Tracker source.
• Pose Tracking Applies toVRM: The Enable Pose Tracking master switch and the pose source list, which takes VMC and mocopi sources and starts with one VMC Protocol source. With no VRM avatar in the scene, the section shows a notice.
• Local IP Address: A card listing this computer's local network addresses, to enter in mobile tracking apps.
• Webcam Tracking: Face, hand, and body tracking from a computer camera, with its own switch. See Webcam Tracking.
• Lip Sync: Mouth movement from microphone audio. See Lip Sync.
• Controller: Game controller mapping. See Controllers.

Each master switch turns network data reception on or off for its channel and shows the channel's aggregate status beside it. Each source appears as a card with its app icon, name, live status, and an on/off switch. The webcam card has no switch of its own; Enable Webcam Tracking in its section does that job.

Clicking a source card expands its configuration editor:

The Add Face Source… and Add Pose Source… cards at the end of each source list add a source of any supported protocol to that channel. Remove Source at the bottom of the editor deletes the source.

Assigning Avatars to a SourceCopy Anchor Link

The Assign Avatars button on a source card opens a multi-select list. Tick the avatars this source should drive, or click Select All to tick every avatar listed. The list shows only avatars the source's channel can drive, so pose sources list only VRM models.

The webcam source card has three buttons instead (Face, Body, and Hands, one per channel), as detailed in Webcam Tracking.

Within a channel, each avatar can be assigned to only one source. Avatars already assigned to another source are labeled "Tracked by ‹name›". Ticking one opens a Reassign Tracking confirmation dialog that lists every affected avatar; nothing moves until you confirm.

A single source can drive several avatars at once, sending each the same tracking data, which is useful for driving twin avatars in sync from one phone.

Each avatar's detail panel also has a Tracking section for settings that belong to that avatar alone: Live2D Whole-Model Movement, Controller Movement, and Microphone Lip Sync. Microphone Lip Sync sets when the microphone takes over the avatar's mouth; see Lip Sync. Which network source or camera drives the avatar is set only on the source cards.

Master and Source SwitchesCopy Anchor Link

A source card's status is governed by two controls: the source's own switch and the channel's master toggle (Enable Face Tracking or Enable Pose Tracking). If either switch is turned off, the status displays off. Only when both switches are active does the card display the source's real-time connection status.

This allows temporarily muting an individual source or disabling an entire channel at once without editing sources one by one.

Supported ProtocolsCopy Anchor Link

For detailed setup guides and protocol specifications, see the respective documentation pages:

When running multiple sources of the same protocol simultaneously, enter each phone's local IP address into its source's Tracker IP field. Unconfigured sources automatically receive incoming streams from unassigned senders of that type. To drive two avatars with two phones, configure a dedicated source with an assigned IP address for each device.

What Each Source CarriesCopy Anchor Link

All three face tracking protocols stream all 52 raw ARKit blendshapes along with head rotation and translation data, and all natively support VRM perfect sync. Input derivation, blink latching, and eye-open calibration are handled uniformly by Persona. The following table highlights differences in connection flow, transport overhead, and precision across protocols:

For channels with high gain multipliers, integer quantization steps can become noticeable. For example, Persona applies a 5× gain to mouth frown signals by default, so a 1% input jump can produce visible stepping in model expressions.

LAPLACE Persona Tracker also supports persistent one-tap calibration across sessions, a Save Energy mode that dims the screen, and a wireframe preview to avoid displaying camera video. See LAPLACE Persona Tracker for details.

All 52 raw blendshape values are also exposed as independent ARKit* inputs in the Parameter Bindings editor. Live2D models can use Persona's smoothed derived inputs, bind raw phone blendshapes directly, or combine both.

Webcam face tracking derives equivalent head poses and blendshape coefficients directly from camera video frames, omitting only tongueOut.

StatusCopy Anchor Link

The status badge on each source card and the aggregate status beside each master switch report the live connection state:

Individual source statuses diagnose specific device connectivity, while the channel aggregate status reflects whether any data is arriving across the pipeline.

Webcam tracking uses identical status labels, displaying tracking whenever any enabled detection task (face, hands, or body) detects targets; see Webcam Tracking. The Face Tracking and Pose Tracking aggregate indicators count network sources only.

How Tracking Reaches the ModelCopy Anchor Link

Tracking data proceeds through three sequential processing stages before affecting model parameters:

1. Receive: The receiving source unpacks incoming protocol packets into a normalized frame containing head rotation, head translation, and 52 ARKit blendshapes. Subsequent stages run independently for each avatar assigned to that source.
2. Derive: The frame is converted into VTube Studio's standard input vocabulary, the parameter names model riggers usually author for (FaceAngleX, EyeOpenLeft, MouthSmile, Brows, etc.). Alongside them, the 52 raw ARKit channels pass through unchanged with an ARKit* prefix, with no latching, calibration, or gain.
3. Map: If the avatar has per-input Input Calibration set up, raw input values are first remapped onto their natural spans. Each parameter binding then takes a weighted sum of one or more inputs and runs it through input range remapping, output range bounds, a response curve, and per-parameter smoothing before writing the target Cubism parameter.

Head rotation, tilt, eye openness and blinking, gaze direction, eyebrows, cheek puff, mouth opening, mouth shape, and smile/frown expressions are all dynamically driven. Parameters omitted from the model rig are automatically skipped; if a frame lacks data for a given parameter, that parameter smoothly relaxes to its default value rather than freezing.

With iPhone (ARKit) face tracking, both a frown and a deliberate pout lower MouthSmile, turning the ParamMouthForm mouth shape toward −1.

Models with Their Own MappingCopy Anchor Link

Applies toLive2D

If a Live2D model directory contains a .vtube.json configuration file, Persona uses the model's custom parameter pairings, ranges, gains, and smoothing settings instead of built-in defaults. Even if a model author inverted eye bindings, doubled eye openness ranges, or renamed parameters, the model behaves in Persona identically to VTube Studio.

Models without a .vtube.json use Persona's built-in default mapping, tuned against real-world ARKit motion capture recordings.

VBridger ParametersCopy Anchor Link

Applies toLive2D

VBridger is an expression engine: authors write their own formulas, and each output parameter evaluates one to three expressions over the 52 ARKit blendshapes. The widely used community "standard" is the VBridger_AdvancedARKit_V3.0 preset that ships with it. That preset adds parameters VTube Studio doesn't have natively, such as MouthPucker, MouthFunnel, MouthShrug, MouthPressLipOpen, and six Body* inputs; the VBridger plugin injects them at runtime.

Persona therefore doesn't treat these names as inputs of their own. When you import bindings from .vtube.json, it expands each name into the weighted sum its VBridger preset defines:

• MouthPucker becomes (mouthDimple_R + mouthDimple_L) × 2 − mouthPucker, so a pucker reads as a negative value.
• The six Body* parameters use the same formulas as their FaceAngle* and FacePosition* counterparts.

Once expanded, they appear in the binding editor as ordinary weighted multi-input bindings, and every weight stays adjustable.

MouthOpen and JawOpen are separate inputs: MouthOpen measures how far the lips part, JawOpen how far the jaw drops. They match the usual VBridger pairing of ParamMouthOpenY and ParamJawOpen. JawOpen carries exactly the same values as ARKitJawOpen, so the two names are interchangeable in bindings.

Model Position MovementCopy Anchor Link

Applies toLive2D

Leaning toward or away from the camera moves the entire avatar on stage rather than adjusting internal rig parameters: the model translates horizontally and vertically with head position and scales proportionally as you move closer or farther away. This replicates VTube Studio's built-in behavior, including default displacement ratios and smoothing, and can be customized per model via the ModelPositionMovement block in .vtube.json.

VRM AvatarsCopy Anchor Link

Face tracking sources also drive VRM avatars; the pipeline differs only in the final mapping stage, where expressions and bones replace Cubism parameters.

Models that implement all 52 ARKit blendshapes (perfect sync) skip derived inputs and are driven 1:1 by the phone's raw ARKit weights, keeping fine facial detail. Expression and morph target names match case-insensitively, so different naming conventions work. If some or all ARKit shapes exist only on mesh morph targets without VRM expression presets, Persona registers the missing VRM expressions automatically at load time when all 52 shapes are covered.

Partial coverage doesn't count as perfect sync: a model with 51 of the 52 blendshapes falls back to the generic mapping below. The Perfect Sync row in the Model Info panel shows the exact matched count.

When perfect sync is unavailable, derived signals map onto the model's standard preset expressions and humanoid bones:

• Mouth: Mouth opening, funnel, pucker, and smile composite into VRM vowel visemes (aa, ih, ou, oh), with built-in mutual exclusion constraints to prevent unnatural mouth shape overlap.
• Eyes: Independent left and right eye blink expressions are used when present, falling back to a unified blink expression when separate shapes are omitted.
• Head: Yaw, pitch, and roll rotations are partially distributed across spine and chest bones to ensure natural upper-body posture follow-through.
• Gaze: Eye direction is controlled via VRM standard look-at systems, supporting both bone-driven and expression-driven gaze rigs.

Switching Between Tracking and IdleCopy Anchor Link

When tracking is active, incoming motion capture data takes precedence, automatically pausing idle auto-blinking. Disabling tracking immediately restores normal idle behavior.

What an avatar does when tracking drops out unexpectedly is set per avatar under When Tracking Is Lost in its Idle panel:

• Keep Last Pose (default): Freezes the avatar in the last valid tracking frame.
• Return to Idle: Eases back into the idle loop.

Either way, the eyes are released so automatic blinking runs until tracking resumes. Live2D models can also play a fallback motion; see When Tracking Is Lost.

No CalibrationCopy Anchor Link

Persona's mobile facial tracking pipeline requires no neutral-pose calibration, adhering to VTube Studio's ARKit philosophy: it trusts absolute head pose quaternions reported by the device. Position the phone directly in front of you at roughly eye level; placing the device too low can cause the avatar to tilt upward continuously.

This principle applies specifically to mobile sources; computer webcams can be calibrated directly within Persona, and individual tracking inputs can also be tuned to your comfortable reach via Input Calibration in the parameter bindings editor.

Webcam TrackingCopy Anchor Link

Webcam Tracking drives face, hand, and body tracking using a computer-connected camera, controlled via an independent Enable Webcam Tracking master switch. Because face, body, and hand channels can be assigned from different tracking sources, a webcam can be used to add hand tracking alongside mobile face tracking and dedicated mocap suits. See Webcam Tracking.

Lip SyncCopy Anchor Link

Lip Sync uses microphone audio input to animate the avatar's mouth, with independent audio calibration profiles saved per microphone. In an avatar's detail panel, Microphone Lip Sync can be set to Off, Always, or When Face Tracking Is Unavailable. See Lip Sync.

ControllersCopy Anchor Link

Connect standard game controllers (such as Xbox, PlayStation, or Switch Pro controllers) to map analog sticks and buttons to Live2D parameter bindings, stage navigation, shortcut triggers, and VRM bone or expression control, with support for independent profiles and multi-button chords; see Controllers.

TroubleshootingCopy Anchor Link

Status stays at "waiting…"

• Confirm the phone and computer are connected to the same local network, and that AP isolation or client blocking is disabled on the router (guest and public Wi-Fi networks typically block device-to-device traffic).
• Verify that your operating system firewall allows incoming UDP traffic for Persona.
• Disconnect from all VPNs. Virtual network adapters often intercept or discard broadcast packets. Although Persona sends subnet-directed broadcasts on each physical network interface, some VPN routing rules still block this traffic.

If discovery broadcasts cannot penetrate the network environment, hardcode the phone's IP address using environment variables before launching Persona:

A "can't use port" notice appears

Another app holds the port, or the system has reserved it (Windows reserves blocks of ports for Hyper-V, WSL2, and Docker, which netsh interface ipv4 show excludedportrange protocol=udp lists). Choose a port outside those ranges in the source card's Port field, and set the same one in your tracking app. Failing ports for one kind of source share a single notice, which is withdrawn once the port works again or no enabled source uses it.

The model moves, but the wrong features respond

Check whether the model folder contains a .vtube.json configuration file. Persona prioritizes mappings defined in this file. Load the model in VTube Studio to verify whether parameter behaviors match.

Some parameters never move

Expand the Info panel for the selected model to inspect the total parameter count. Models using non-standard parameter names or older Cubism 2 conventions may not match Persona's standard derived input names.

Nothing responds, but the status reads "tracking"

Open Assign Avatars on the source card and check that the avatar is ticked. Incoming tracking data drives a model only after it is explicitly assigned to that source. Each avatar can belong to only one source per channel, so also check that it isn't assigned to another source.

Two phones, but they move the same avatar

When Tracker IP is left blank on both sources, incoming packets are claimed by whichever source receives them first. Enter the corresponding phone's local IP address in each source card to ensure data routes to the intended avatar.