The webcam tracking source uses a camera connected to the computer running Persona to capture your facial expressions, hand gestures, and body pose. Powered locally by Google MediaPipe, it requires no mobile device or dedicated motion-capture software, and it integrates seamlessly with network sources—for example, pairing phone-based facial tracking and mocopi body tracking with webcam finger tracking. A single webcam can drive multiple Live2D and VRM avatars simultaneously.
Setup
- In the Tracking panel, navigate to the Webcam Tracking section and turn on Enable Webcam Tracking. When enabled for the first time, this creates a dedicated webcam source card named MediaPipe Webcam.
- Click the card to open its settings, select your Camera, and choose which tracking tasks to run.
- Use the Face, Body, and Hands buttons on the card to assign target avatars.
- Face forward with a neutral expression, stand upright, and click Calibrate below the card.
On macOS, the operating system prompts for camera permission the first time the camera starts. If permission was denied previously, see Troubleshooting.
Source Settings
Persona maintains a single global webcam source controlled entirely by the master switch Enable Webcam Tracking—the card itself contains no separate power switch. This switch operates independently of Enable Face Tracking and Enable Pose Tracking (which govern network-based tracking sources). Turning the master switch off preserves all task choices and avatar assignments; turning it back on restores them immediately without altering any other source assignments.
| Setting | Description |
|---|---|
| Camera | Selects the camera device to use. Default Camera follows the system default input; disconnected cameras appear as Unavailable Camera. |
| Face Tracking | Tracks facial expressions and head rotation. Enabled by default. |
| Hand Tracking | Tracks Live2D hand inputs as well as VRM arm and finger bones. Enabled by default. |
| Body Tracking | Tracks VRM torso and leg movements. Disabled by default. |
| Mirror Tracking | Enabled by default: the avatar mirrors your movements (your right hand moves its left hand). When disabled, left and right are preserved. |
| Processing | Select CPU (default) or GPU. GPU acceleration can improve tracking frame rates, but shares graphics resources with avatar rendering. |
| Hand Motion for ‹name› | Configured per VRM avatar assigned to webcam hands: choose Arms and Fingers (default) or Fingers Only. See VRM. |
Only enabled tasks are loaded into memory and executed. Switching cameras, toggling tasks, or changing Processing will restart and reinitialize the camera pipeline.
The Web Console and Plugin API modify the exact same source instance, tasks, and avatar assignments. The camera selection menu, Calibrate action, and real-time landmark preview are available exclusively in the desktop app.
Assigning Avatars
Unlike network source cards that offer a single Assign Avatars button, the webcam card provides three independent channel buttons: Face, Body, and Hands. Each opens a multi-select dialog, where Body filters to VRM avatars only. Each avatar channel can only be bound to one tracking source at a time, allowing you to combine webcam and dedicated hardware tracking across different channels.
Consequently, an avatar can receive face tracking from LAPLACE Persona Tracker, body tracking from mocopi, and finger tracking from the webcam. Selecting an avatar whose channel is already driven by another source opens the Reassign Tracking confirmation dialog. When a task is disabled, its corresponding channel stops updating on assigned avatars; Persona never falls back automatically to other sources.
As with all tracking sources, a single webcam can drive multiple avatars simultaneously, broadcasting identical tracking data to each model.
Calibration
The Calibrate button is located in the status bar below the card; after calibration, it changes to Recalibrate. The button is enabled only when the camera detects a face, hand, or body in the frame. The hint below the preview describes the target posture: look directly forward with a relaxed, neutral expression, stand upright, and click calibrate.
- Face—Records current head orientation and facial expressions as the neutral reference baseline. Head angles and depth lean are measured relative to this frame, and expressions measure displacement from your resting face. Before calibration, the first detected face serves as the temporary baseline.
- Hands—For VRM avatars, the first single-hand gesture detected after calibration establishes the reference distance and elevation relative to the camera, scaling the avatar's arm reach accordingly.
- Body—Sets the direction you face and your standing position as the origin for rotation and translation. Bent legs are not treated as a rest pose.
Restarting the camera or toggling Mirror Tracking invalidates and resets the current calibration baseline.
Preview and Frame Rate
The Webcam Tracking Preview beneath the card renders detected MediaPipe landmarks as vector outlines—including eyes, mouth contours, hand skeletons, and body pose keypoints—mirrored whenever Mirror Tracking is active. To safeguard user privacy, the preview never displays raw camera video streams, and between pipeline iterations it holds the latest detected landmarks without interpolating artificial frames. The status line above displays the current source status alongside the actual tracking rate (e.g., tracking · 30 FPS).
The tracking rate measures algorithm detection cycles completed per second, completely independent of the stage render frame rate shown by Show FPS Counter. Persona requests up to 60 fps from the camera hardware; the actual rate achieved depends on the camera hardware's physical output, the number of active tasks running sequentially on each frame, and the chosen Processing mode. A high-end GPU cannot exceed the camera's hardware capture rate, and each additional active task introduces processing overhead that may reduce the overall frame rate.
Status
The card and the label next to Enable Webcam Tracking reflect standard Status states based on real-time computer vision detection:
| Status | Description |
|---|---|
| off | Enable Webcam Tracking is switched off, or all individual tracking tasks are disabled. |
| waiting… | The camera is initializing, or Face Tracking is disabled and no hands or body are detected. |
| tracking | An enabled task successfully detects a face, hand, or body (may display tracking even if no face is in view). |
| no face | Face Tracking is enabled but no face is found, and neither hands nor body are detected. |
The status line above the preview also displays starting… while the camera and neural models load, or unavailable if initialization fails, accompanied by an error description and a Retry button. Note that the aggregate status indicators on the Face Tracking and Pose Tracking headers reflect network sources only.
Driven Parameters by Task
Live2D
- Face: Uses the standard mobile mapping pipeline, routing derived inputs such as
FaceAngleXandMouthOpenalongside standardARKit*BlendShape inputs through the model's parameter bindings. - Hands: Generates 28 standardized Hand Inputs, which drive Cubism parameters through model bindings in the same manner.
- Body: Inactive for Live2D models. Because Live2D lacks a standardized humanoid skeletal rig, Persona does not synthesize synthetic torso or limb parameters.
VRM
- Face: Drives expressions, gaze, and head rotation identically to mobile sources, with full support for Perfect Sync.
- Hands: When configured for Arms and Fingers, drives the shoulders, arms, wrists, and fingers. The detected wrist coordinate serves as an inverse kinematics (IK) target scaled to the avatar's specific limb proportions. In Fingers Only mode, only finger bone rotations are driven.
- Body: Rotates the torso and leg bones while applying estimated positional translation. Horizontal and vertical translations match screen-space motion; forward and backward depth translation is estimated from torso bounding proportions and clamped within ±0.5 m. This does not provide absolute room-scale tracking and does not pin feet to the ground plane.
Body Tracking never controls arm movements; arm gestures are driven exclusively by Hand Tracking. When an active face source drives the avatar, head and neck rotations remain assigned to that face source.
Using with VMC or mocopi
When an avatar's body is driven by VMC or mocopi, that source maintains full authority over shoulders and elbows, while webcam hand tracking layers wrist and finger rotations on top. If set to Fingers Only, wrist rotation is also retained by the primary body tracking source.
Hand Inputs
Hand tracking provides the following inputs to Live2D parameter bindings, grouped under Hands in the binding editor. Parameter naming conforms to the VTube Studio hand-tracking standard, augmented with HandLeftAngleY and HandRightAngleY; every HandLeft… input has a corresponding HandRight… counterpart:
| Input | Range | Description |
|---|---|---|
HandLeftFound | 0 – 1 | Evaluates to 1 when the corresponding hand is detected. |
HandLeftPositionX, HandLeftPositionY, HandLeftPositionZ | −10 – 10 | Wrist position: X outward, Y upward, Z toward camera (estimated from palm size). |
HandLeftAngleX, HandLeftAngleZ | −180° – 180° | Wrist Euler rotation angles. |
HandLeftAngleY | −90° – 90° | Wrist yaw rotation angle. |
HandLeftOpen | 0 – 1 | Weighted average openness across all five fingers. |
HandLeftFinger_1_Thumb … HandLeftFinger_5_Pinky | 0 – 1 | Individual finger extension (1 when fully extended). |
BothHandsFound | 0 – 1 | Evaluates to 1 when both hands are simultaneously tracked. |
HandDistance | 0 – 10 | Euclidean distance between both wrists. |
Models without custom hand bindings will automatically bind to Cubism parameters matching the hand input names, honoring the parameter's native range definition (for example, a parameter authored with a range of ±1 will not be overdriven to ±10).
Limitations
- MediaPipe facial BlendShapes adopt standard ARKit naming but do not support tongue tracking; consequently,
tongueOutremains inactive. A monocular webcam cannot replace the depth-sensing capabilities of an Apple TrueDepth sensor. - All spatial coordinates are estimated from a single RGB video frame: depth relies on palm proportions, and torso distance relies on upper-body scale. These values represent visual approximations rather than physical sensor measurements.
- Hands that are occluded, overlapping, or out of frame are dropped immediately rather than inferred through guesswork. Body tracking requires both shoulders and both hips to remain visible in frame.
- On VRM avatars, temporarily lost hand or body tracking is held for 200 ms before smoothly blending back to the underlying rest pose over approximately 0.4 s. On Live2D models, lost hands drop the
Foundinput to 0 while maintaining last known parameter values so built-in model fade-outs can complete naturally.
Privacy and Security
All tracking algorithms execute strictly on the local machine. MediaPipe models are bundled directly within the Persona distribution package, requiring no external asset downloads at runtime. Raw camera video frames never leave Persona or escape its rendering sandbox—even the control window preview displays only abstracted landmark vectors. While upstream MediaPipe packages report usage analytics to Google by default, Persona intercepts and suppresses all such telemetry requests. For full details on local data retention, see Privacy and Security.
Troubleshooting
"Camera Access Denied"
Grant Persona permission to access the camera in your operating system preferences, then restart the application. On macOS, navigate to System Settings → Privacy & Security → Camera. Newly granted permissions take effect only after Persona restarts.
"Camera Unavailable"
The camera is either disconnected or currently locked by another application. Select another Camera in the settings menu, or click Retry.
"Webcam Tracking Failed to Start"
The neural tracking runtime failed to initialize. Click Retry to reload the tracking pipeline.
Last updated on October 11, 2026