00:00 / 00:00

Persona

SDK

@laplace.live/persona-sdk is the official TypeScript and JavaScript client library for the LAPLACE Persona Plugin API, and serves as the single source of truth for its wire protocol schema. This page provides a complete type and API reference. For server configuration, API key management, and WebSocket RPC method specifications, see the Plugin API documentation.

The SDK runs across Node.js 22+, Bun, modern web browsers, and OBS browser sources. Its only runtime dependency is zod, which defines and validates all wire schemas.

npm install @laplace.live/persona-sdk

What You Can Build

The Persona Console is the official reference implementation built on this SDK. It provides a complete web-based control console in a single browser tab, letting creators manage scenes, layers, asset libraries, expressions, motions, parameter bindings, hotkeys, tracking sources, and global settings, with an interface optimized for mobile touch devices during live broadcasts.

The console contains no proprietary desktop code. It interacts with Persona exclusively through the Plugin API, demonstrating that third-party integrations can achieve equivalent control using the same SDK.

When building client applications, follow these recommended design patterns:

  • Gate on capabilities, not on model formats or version numbers: Query InstanceRuntime.capabilities before exposing controls, disabling unsupported features dynamically rather than catching unsupported-for-format errors;
  • Treat reconnections as cache invalidations: Broadcast events missed while disconnected are not replayed; cached data may diverge from the host. Flush local caches and re-query state snapshots upon reconnecting;
  • Handle terminal close codes appropriately: Status codes 4001 (key revoked) and 4002 (session disconnected by user) represent terminal closures and should stop reconnection loops; other disconnects should trigger exponential backoff retries;
  • Rely on SDK subscription management: Listeners registered via persona.on(…) are restored automatically upon reconnecting, eliminating manual subscription bookkeeping.

LLM Prompt

Provide the following prompt to an LLM or include it in your project's CLAUDE.md or AGENTS.md rules file to supply complete contextual guidance for developing Persona plugins:

You are helping me build a plugin for LAPLACE Persona, a desktop VTuber app
(https://laplace.live/persona). Persona exposes a local WebSocket server called the
Plugin API, and `@laplace.live/persona-sdk` is its typed client.

Before writing any code, fetch the reference documents listed at the end — they are
the single source of truth for method names, events, error codes and types. Do not
rely on memory and do not invent names.

## Setup

- Install `@laplace.live/persona-sdk`. Isomorphic: Node 22+, Bun, browsers and OBS
  browser sources. Its only runtime dependency is zod.
- The user enables the server in Persona under Settings → Plugin API and creates an
  API key there. Tokens look like `sk-lp-v1-…`. Default endpoint: `ws://127.0.0.1:25034`.
- Everything — values and types — is exported from the package root; there is no
  other entry point.

## Client

```ts
import { PersonaClient } from "@laplace.live/persona-sdk";

const persona = new PersonaClient({
  token: "sk-lp-v1-…", // required; everything else is optional
  clientInfo: { name: "My Plugin", version: "1.0.0" }, // display-only identity
});
await persona.connect();

const { scenes, activeSceneId } = await persona.call("scene.list");
await persona.call("scene.patch", {
  background: { mode: "color", color: "#00ff00", imageAssetId: null },
});
persona.on("motion.started", (m) => console.log("playing", m.group));

const mouth = persona.driveParameter("MouthOpen", 0.8); // parameter lease
mouth.set(0.3);
mouth.release();
```

`call(method, params?)` is fully typed — the method name determines the request and
response shapes.

Methods cover scenes, model instances, stage objects, expressions, motions, speech,
hotkeys, automations, model and asset registration, settings, tracking and pose
input, per-key storage and parameter injection. Events notify you of scene,
selection, expression, motion, speech, tracking and registry changes.

The exact method and event names, parameter shapes, error codes and limits live in
the reference below — look them up instead of guessing. Gate features on the
capabilities reported by `app.info` and `instance.list`, never on version strings.

## Reference

Fetch these before writing code. The documentation is served as raw Markdown —
append `.mdx` to any page URL (`.en.mdx` for English):

- https://laplace.live/persona/plugin-api.en.mdx — server setup, API keys, and every
  method, event, error code and limit
- https://laplace.live/persona/sdk.en.mdx — the SDK client: options, auth modes and
  every exported type

Client Reference

import { PersonaClient } from "@laplace.live/persona-sdk";

const persona = new PersonaClient({ token: "sk-lp-v1-…" });
await persona.connect();

const { scenes, activeSceneId } = await persona.call("scene.list");
await persona.call("scene.patch", {
  background: { mode: "color", color: "#00ff00", imageAssetId: null },
});
persona.on("motion.started", (m) => console.log("playing", m.group));

Instantiate PersonaClient using the options below. Only token is required:

Prop

Type

ClientInfo

clientInfo declares the client identity displayed in the desktop application under Settings → Plugin API → Connected Clients. It is optional, self-declared display metadata and does not affect access permissions. The SDK re-announces this payload upon every reconnection.

const persona = new PersonaClient({
  token,
  clientInfo: { name: "My Overlay", version: "1.2.0", developer: "You" },
});

Prop

Type

Authentication

In browser environments, the API token is passed via the ?token= query parameter due to browser WebSocket API constraints. In Node.js or Bun environments, you can supply a custom createWebSocket hook that passes the token as an Authorization: Bearer header using ws:

import WebSocket from "ws";

const persona = new PersonaClient({
  token,
  auth: "header",
  createWebSocket: (url, headers) => new WebSocket(url, { headers }),
});

WebSocketLike defines the structural interface required by createWebSocket. Both global browser WebSocket instances and the Node.js ws client satisfy this contract without type assertions.

Prop

Type

In Node.js 22+, the global WebSocket accepts handshake headers through its WebSocketInit extension. The SDK's createNodeClient helper builds a client on top of it:

  • Sets auth: 'header'
  • Uses Node's global WebSocket
  • Sets a 5-second connection timeout
  • Sends SDK warnings and terminal close notices (key revoked or force-disconnected) to the log function you pass
import { createNodeClient } from "@laplace.live/persona-sdk";

const persona = createNodeClient({
  token: "sk-lp-v1-…",
  log: console.warn,
});
await persona.connect();

Prop

Type

Parameter Leases

driveParameter writes a continuous value to a model parameter and returns an InjectionHandle. The SDK maintains lease heartbeats automatically. The lease expires approximately one second after the last write, restoring default parameter behavior safely if the client process terminates unexpectedly.

const mouth = persona.driveParameter("MouthOpen", 0.8);
mouth.set(0.3);
mouth.release();

A parameter can be leased by only one active session at a time; concurrent attempts return a conflict error.

The method accepts an optional weight multiplier and an onError callback invoked if heartbeat renewal fails in the background, removing the need to await individual updates.

Prop

Type

Connection Close Handling

onClose reports disconnects that come from the server or the network. The callback receives the raw socket close code, the reason string, and a terminal flag that says whether reconnect retries have stopped.

  • It fires after the SDK has finished its internal cleanup, so calling close() inside the handler is safe.
  • A clean close() initiated by the client does not trigger onClose.
  • Once onClose is set, terminal disconnects are reported there instead of through onWarning.

Prop

Type

PersonaClientState and Client Errors

type PersonaClientState = "closed" | "connecting" | "open" | "reconnecting";
type ClientErrorReason =
  | "connect-timeout"
  | "connection-closed"
  | "client-closed"
  | "not-connected"
  | "request-timeout";

When the client or its transport fails connect() or call(), the promise rejects with a PersonaClientError carrying reason: ClientErrorReason (one of CLIENT_ERROR_REASONS). Server error frames reject call() with PersonaApiError (code: ApiErrorCode). See Plugin API: Error Codes.

Prop

Type

Constants

ExportValueDescription
PROTOCOL_VERSION4Incremented on breaking protocol changes; reported by the server in hello
DEFAULT_API_HOST127.0.0.1Loopback address bound by default unless local network access is enabled
DEFAULT_API_PORT25034Default WebSocket port
SCENE_ACTIVATION_TIMEOUT_MS120000Default timeout for scene.activate when requestTimeoutMs is unspecified
ASSET_IMPORT_PROMPT_MS45000How long asset.import waits for user confirmation when importing a file outside plugin folders (when requestTimeoutMs is unspecified, the SDK defaults to 55000 ms for asset.import)
CLOSE_KEY_REVOKED4001Terminal close code: API key revoked by server
CLOSE_FORCE_DISCONNECTED4002Terminal close code: session terminated by user
CLOSE_SERVER_STOPPING1001Non-terminal close code: server shutting down; client will retry
INJECT_LEASE_TTL_MS1000Lease time-to-live after last write
INJECT_HEARTBEAT_MS100Heartbeat transmission interval for active leases
STORAGE_KEY_MAX_LENGTH128Maximum length of a storage key name
STORAGE_VALUE_MAX_LENGTH65536Maximum serialized length of a storage value
STORAGE_KEYS_MAX256Maximum key-value pairs stored per API key
SPEECH_URL_MAX_LENGTH8000000Maximum URL length for speech.play (accommodates ~40 seconds of Base64 WAV)

The SDK also exports these value lists and helpers:

  • API_ERROR_CODES, CLIENT_ERROR_REASONS, EVENT_NAMES, INPUT_NAMES, TIME_INPUT_NAMES, RANDOM_PULSE_INPUT_NAMES, MODIFIER_TOKENS, APP_CAPABILITIES, FPS_LIMIT_PRESETS, EFFECTS_QUALITY_LEVELS: as const arrays, so a UI picker can list every valid value
  • isApiErrorCode, isEventName, isAppCapability, isTimeInputName, isRandomPulseInputName: the matching TypeScript type guards
  • injectTargetKey: builds a normalized injection target string
  • personaWsUrl: builds a WebSocket URL from { host, port, secure } (PersonaAddress, where host must be a plain host name or IP address) or from an address string the user typed (bare hostnames, host:port, IPv4/IPv6 literals, and full ws:// or wss:// URLs); PersonaClient appends the token when connecting
  • parseHost, parsePort, isValidPort: validate a plain host name or IP address (accepting bare or bracketed IPv6 and returning null for schemes, ports, paths, or zone IDs) and a port number
  • requestTimeoutMs(method, fallbackMs): resolves the timeout for a request method (scene.activate and asset.import use their extended defaults)

Models and Assets

Models and general assets share a unified metadata structure. kind defines the asset classification, while origin identifies its source: bundled for built-in samples and content downloaded on demand from the Persona catalog, and user for locally registered files, accompanied by author attribution metadata.

Prop

Type

type ModelFormat = "live2d" | "vrm";
type ContentOrigin = "bundled" | "user";
type AssetKind =
  | "image"
  | "video"
  | "audio"
  | "prop"
  | "ibl"
  | "lut"
  | "animation"
  | "cameraMotion"
  | "lyrics";
type InventoryKind = ModelFormat | AssetKind | "effect" | "addon";

effect represents a virtual asset type without a backing disk file, used by the asset library to enumerate built-in effects; addon represents an optional application add-on (such as the Webcam Tracking Add-On) that extends the app rather than adding a scene layer.

model.list returns an array of ModelRef objects and asset.list returns an array of AssetRef objects. Both extend ContentRef with kind narrowed to ModelFormat and AssetKind respectively.

Both also carry a boolean exists flag. On an AssetRef it says whether the asset file is still on disk; a ModelRef sets it to false when the model's entry file is missing.

Prop

Type

Prop

Type

Prop

Type

CatalogItem

CatalogItem defines transport-agnostic metadata used by asset pickers. Both local registry items and remote catalog listings normalize to this type. It removes the installation-specific origin field from ContentRef and adds localized description (LocalizedText) and catalog metadata. download (CatalogDownload) describes retrieval parameters for remote items; if absent, the item exists locally.

Prop

Type

Prop

Type

Prop

Type

Catalog and localization helpers:

  • CATALOG_FORMAT, MARKETPLACE_ITEMS_PATH, CATALOG_KINDS: catalog manifest format version, the public catalog endpoint path (/api/items), and the supported catalog item kinds
  • isCatalogId, healCatalog, healCatalogItem, healCatalogRows, healSha256: validate and sanitize remote or local catalog manifests
  • payloadEntryPath, payloadEntryKey, catalogMediaFile (takes a CatalogMediaRole): resolve the extracted entry path of a catalog payload and the dot-prefixed file names for a source's rendered thumbnail and preview video (.thumbnail.avif and .preview.webm)
  • pickLocalized(text, locale), healLocalizedText: pick the display string for a UI locale from a LocalizedText object and sanitize localized text maps

Placement

2D screen space is measured in pixels from the stage center (+y down). 3D world space is measured in meters. All rotations in the wire schema use radians (converted to degrees in user interfaces).

Prop

Type

Prop

Type

Place2D and Place3D represent placement definitions for scene objects, sharing identical position fields with an added opacity channel.

Scene Items

The items array in a scene orders models and objects together according to their Z-order:

type SceneItem = SceneModelItem | SceneObjectItem;

Prop

Type

Prop

Type

Prop

Type

ObjectContent

The payload of a scene object is determined by its kind field. Except for capture (display capture), all object types can be created directly in the desktop interface, as described in Layer Management.

type ObjectContent =
  | { kind: "image"; assetId: string }
  | {
      kind: "video";
      assetId: string;
      loop: boolean;
      muted: boolean;
      volume: number;
    }
  | { kind: "prop"; assetId: string }
  | {
      kind: "web";
      url: string;
      width: number;
      height: number;
      fps: number;
      transparent: boolean;
      css: string;
      layer: WebLayer;
      shutdownWhenHidden: boolean;
    }
  | { kind: "capture"; source: CaptureKind; sourceId: string; label: string };

type ObjectSpace = "2d" | "3d";
type WebLayer = "behind" | "front";
type CaptureKind = "display" | "window";

Prop

Type

Attachment

Prop

Type

AttachAnchor

type AttachAnchor =
  | { kind: "root" }
  | { kind: "bone"; bone: string }
  | {
      kind: "artMesh";
      id: string;
      verts: [number, number, number];
      weights: [number, number, number];
    };

object.anchors returns available attachment anchors provided by a parent model as AnchorOption records, pairing anchor identifiers with human-readable labels. VRM models provide the root transform and humanoid bones; Live2D models provide the root transform and individual ArtMeshes with an order draw-order index.

AttachDepth

type AttachDepth =
  { kind: "front" } | { kind: "behind" } | { kind: "artMesh"; id: string };

depth controls the internal layering of Live2D props attached to a parent Live2D model, positioning the item behind the model, in front of the model, or directly above a specified ArtMesh. Configured via instance.attach.

Prop

Type

Attachment helpers:

  • depthStops(meshes): the available depth stops from back to front (behind the model, above each ArtMesh in draw order, then in front)
  • depthStopIndex(meshes, depth): which stop a depth sits at; an ArtMesh the model no longer has counts as the frontmost
  • pinRiders(items, instanceId): every item attached to instanceId, directly or through other items, so circular attachments can be prevented
  • pinnableParents(models, items, instanceId): the candidate parents, with invalid ones marked ridesThis
  • defaultAttach(parentInstanceId, screenSpace?): a fresh attachment (root anchor, in front, unsplit, no physics tuning). With screenSpace set (a Live2D model or 2D object), the item also follows the host's head at the default response

Prop

Type

Prop

Type

Scenes

Prop

Type

Prop

Type

Prop

Type

Background

Prop

Type

type BackgroundMode = "color" | "image";

Camera

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

cameraDistanceLimits(environment, radius) computes how close and how far the orbit camera may go. The defaults are DISTANCE_MIN (0.35 m) and DISTANCE_MAX (20 m); when a 3D environment's bounding radius differs a lot from a standard room (~10 m), the limits scale with it. radius is the bounding radius that scene.inspect returns under environment.radius; when it is null, the default limits apply.

Lighting

Prop

Type

type SceneLightType = "directional" | "point" | "ambient" | "area" | "spot";
type ShadowQuality = "off" | "low" | "medium" | "high" | "ultra";
type ShadowFilter = "pcf" | "pcss";

SCENE_LIGHT_TYPES, SHADOW_QUALITY_LEVELS, and SHADOW_FILTERS are as const arrays defining supported light types, shadow qualities, and filter modes matching interface order.

Three SceneLight fields make a light follow the camera:

  • followCamera: binds the light's position and orientation to the camera view
  • followCameraOptions: which degrees of freedom follow the camera's motion; DEFAULT_LIGHT_FOLLOW_CAMERA_OPTIONS enables all of them
  • followCameraReference: the frozen reference transform for the axes that don't follow

To turn following on or off without moving the light, call scene.setLightFollowCamera instead of editing these fields; see Lighting.

Prop

Type

Prop

Type

iblAssetId and modelAssetId set the image-based lighting (IBL) map and the 3D stage environment. Releases with the environment-map-model capability accept both at once.

  • Use environmentWithAsset() to change either one: it sets the new asset, keeps the other, and resets light overrides tied to the old model.
  • Selecting an IBL texture turns bakeFromModel off. Baking defaults to on only when a 3D model is loaded into an empty environment.
  • environmentClearPatch(environment, slot) builds a patch that clears 'model' or 'map', or both when slot is null.
  • applyEnvironmentLighting() and applyEnvironmentLook() back the panel's quick actions: they apply the environment lighting preset and an EnvironmentLook profile.

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Volumetric lighting is enabled per light through SceneLight.volumetric. Related helpers:

  • SCENE_VOLUMETRIC_LIGHTING_SPECS: the numeric ranges of the volumetric fog sliders
  • defaultSceneVolumetricLighting(): the default fog settings
  • healSceneVolumetricLighting(): clamps out-of-range values
  • sceneLightCanHaze(type): whether a light type supports volumetric scattering
  • sceneLightHazes(light): whether a light is emitting volumetric light right now

Effects

SceneEffects uses a flat schema: every toggleable post-processing effect is exposed as a top-level property, enabling generic reflection and iteration. Visual grouping in user interfaces is a presentation concern independent of the wire schema.

Prop

Type

Effect ordering in the Layer List is tracked separately in environment.effectLayers (an array of ToggleEffectKey strings). This list is orthogonal to the enabled flag: enabled activates rendering, while effectLayers controls layer item presentation. Enabling an effect automatically appends it to the layer list.

type SceneToneMapping = "none" | "neutral" | "aces" | "agx";

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Gradient provides 24 blend modes defined in EffectBlendMode, enumerated in EFFECT_BLEND_MODES.

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

rain and snow share standard effect structures but operate as 3D geometric particle systems rather than 2D screen-space post-processing passes; see Rain and Snow.

Model and object layers have their own effect pipelines. effects and effectLayers on SceneModelItem and SceneObjectItem run before the scene-wide effects and are set with instance.setEffects.

  • Layer effects support the subset in LayerEffectKey: color, levels, colorWheels, colorShift, selectColors, gradient, blur, bloom, diffusion, sketch, rim, outline, and dropShadow.
  • Bloom color selection works only at the layer level; see Layer Effects.

Transitions

transition configures the visual transition when activating a scene; see Scene Transitions (requires scene-transitions capability). SCENE_TRANSITION_TYPES lists valid transition styles, defaultSceneTransition() returns standard cuts, and healSceneTransition() validates configurations.

Prop

Type

environment.itemTransition governs entrance and exit animations for models, props, and spawned items; see Showing and Hiding. ITEM_TRANSITION_STYLES enumerates supported styles, defaultItemTransition() defaults to a 300 ms glitch effect, and ITEM_TRANSITION_MAX_MS imposes a 5000 ms ceiling.

Prop

Type

Scene Presets

presets lists the scene's saved scene presets in display order; builds without the scene-presets capability omit it. Manage the active scene's presets with preset.create, preset.update, preset.rename, preset.setTransition, preset.delete, and preset.apply; see Plugin API: Scene Presets.

  • SCENE_PRESET_TRANSITIONS: every preset transition
  • scenePresetIsCurrent(): whether the stage's current layout matches a preset; the desktop and Web Console use it to highlight that preset
  • capturePresetItems(): captures a layer layout
  • presetStep(): computes each layer's change when a preset is applied

Prop

Type

Prop

Type

Runtime State

instance.list reports dynamic capabilities for active stage items, allowing clients to disable unsupported controls without triggering RPC errors:

Prop

Type

type InstanceCapability =
  | "motions"
  | "expressions"
  | "placement-2d"
  | "placement-3d"
  | "mtoon"
  | "idle-clips"
  | "idle-motions"
  | "pose"
  | "breath"
  | "expression-weights"
  | "live2d-params"
  | "speech"
  | "post-fx";

speech capabilities are reported exclusively for loaded Live2D models supporting voice synthesis. Application-wide capabilities are reported separately during hello handshakes and via app.info. Feature gating must inspect capability flags rather than comparing version strings:

type AppCapability =
  | "storage"
  | "speech"
  | "automations"
  | "controllers"
  | "model-editing"
  | "asset-inspection"
  | "layer-effects"
  | "scene-transitions"
  | "light-area"
  | "light-spot"
  | "light-camera-follow"
  | "light-shadow-filter"
  | "environment-map-model"
  | "stage-spawn"
  | "idle-tracking-lost"
  | "motion-stop"
  | "stage-capture"
  | "object-pin-depth"
  | "object-text"
  | "camera-follow"
  | "camera-aim"
  | "camera-focus"
  | "camera-handheld"
  | "object-video-sound"
  | "object-chroma-key"
  | "model-movement"
  | "model-movement-pivot"
  | "layer-effects-2d"
  | "scene-presets"
  | "background-alpha"
  | "binding-calibration"
  | "binding-random-pulses"
  | "expression-groups"
  | "expression-groups-hide-first"
  | "model-history"
  | "automation-editing"
  | "automation-transport"
  | "asset-import";

canonicalAppCapability maps a legacy capability name to its current one. Legacy names include area-lights, spot-lights, camera-follow-lights, shadow-filters, spawn, tracking-lost, text-objects, and video-sound. PersonaClient already does this for the capability lists it receives from the server.

stage.capture (requiring the stage-capture capability) returns { dataUrl, width, height }, a PNG render of the stage with alpha transparency and its longest edge constrained to maxEdge (range 64 – 2048, default 1024). Headless automations and AI assistants use it for visual verification; see Plugin API Capabilities.

Prop

Type

files preserves raw motion index positions defined in model manifests: missing motions use empty string placeholders so subsequent indices remain stable. motionSlots(files) returns playable motions and indices. Calling motion.play on empty slots returns not-found. Live2D models also discover unreferenced .motion3.json files in the model folder, creating synthetic groups named after each file.

motion.stop (requiring motion-stop capability) ends the running one-shot motion and eases the model back to idle; if only idle is playing, it returns { stopped: false }. The oneShot flag in motion.playing and motion.started indicates whether a motion is interruptible via stop.

Prop

Type

Prop

Type

Prop

Type

Expression Groups

expression.getGroups and expression.setGroups (requiring the expression-groups capability) read and edit a model's expression groups and return ExpressionGroups; see Plugin API: Expression Groups for the rules.

Prop

Type

Prop

Type

Prop

Type

EXPRESSION_GROUP_FADE_SECONDS_MAX (2) is the maximum fade duration for a group member, and EXPRESSION_GROUP_FADE_SECONDS_DEFAULT (0.2) is the default duration for a new member. foldExpressionKey normalizes an expression key case-insensitively the way VTS compares them, so two keys refer to the same expression when their normalized forms match, and firstByExpressionKey(items, keyOf) indexes items by their normalized key while keeping the first item listed under each non-empty key (matching how VTS resolves an expression key). newVtsId generates an ID in VTS format (32 lowercase hex characters) for a new group.

Hotkeys

Prop

Type

Prop

Type

HotkeyConfig represents hotkey payloads without the registered runtime flag, matching the arguments expected by hotkey.set. registered lists the hotkeys that currently fire while Persona is in the background; because the desktop app reads key states directly rather than registering OS-level global hotkeys, any combination (including a single key) can trigger in the background. hotkeyRunsInBackground checks whether a hotkey is configured to run in the background. Because combinations fire based on the set of held inputs regardless of order, comboKey(accelerator) returns an order-independent identifier, which duplicateAccelerators uses to detect combinations bound to multiple hotkeys. acceleratorLabel(accelerator, platform, mouseLabels?) formats a combination into an array of key-cap labels in canonical modifier order (MODIFIER_TOKENS: Control, Alt, Shift, Super), using macOS symbols (⌃, ⌥, ⇧, ⌘) when hello's app.platform is 'darwin' and Ctrl, Alt, Shift, Win elsewhere (mouse triggers default to MOUSE_TRIGGER_LABELS and accept localized overrides).

Each avatar on stage has its own hotkey set:

  • hotkey.list, hotkey.set, and hotkey.trigger take an optional modelId, which defaults to the primary model.
  • hotkey.state events report updates per model.
  • hold, fadeSeconds, deactivateAfterSeconds, and stopsOnLastFrame are newer protocol fields. Fields left out of a hotkey.set call keep their current values.

The SDK provides helpers for querying hotkeys across all on-stage models: listSceneHotkeys(client) resolves the active scene's hotkeys (primary model first); listModelHotkeys(client, scene) returns an array of promises (Promise<ModelHotkeys>[]), one per on-stage model, allowing callers to settle each independently.

Prop

Type

Automations

Automations operate globally across the application rather than belonging to individual models. Each automation contains an ordered action sequence triggered by hotkeys, events, or API calls. automation.list retrieves all automations, automation.run executes an automation by ID, and automation.state broadcasts list modifications. Requires automations capability; see Automations.

Prop

Type

type AutomationActionKind =
  | "effect-toggle"
  | "effect-params"
  | "effect-clip"
  | "camera-pose"
  | "camera-reset"
  | "camera-motion-play"
  | "camera-motion-stop"
  | "camera-follow"
  | "camera-aim"
  | "camera-focus"
  | "camera-handheld"
  | "layer-visibility"
  | "preset-apply"
  | "text-set"
  | "lyrics-play"
  | "stream-mode"
  | "scene-switch"
  | "expression-toggle"
  | "motion-play"
  | "audio-play"
  | "audio-control"
  | "expression-clear"
  | "model-load"
  | "model-position";

actionKinds and triggerKinds are typed as string[] rather than closed unions, so they stay forward-compatible with newer servers. Action kinds use category-first names; the client normalizes names from older hosts automatically.

  • automationActionLabel: the English label of an action kind
  • automationLabel: an automation's summary title (title when present, otherwise its action labels joined with ·)
  • AUTOMATION_SETUP_KINDS, isAutomationSetup (and isAutomationSetupAction): identify the setup steps (scene-switch and model-load) that run before timeline playback. AUTOMATION_BOUNDARY_KINDS and isAutomationBoundary are deprecated aliases
  • isAutomationMediaAction, isAutomationSourcedAction, isAutomationSampledClip, isAutomationContentAction: classify timeline actions (media clips, clips that accept a sourceInMs trim offset, clips sampled during playhead scrubbing, and regular timeline actions)

Timeline Editing and Transport

With automation-editing, automation.get, automation.create, automation.update, and automation.delete read and write full AppAutomation timeline definitions; with automation-transport, automation.control and automation.status drive timeline playback and inspect AutomationRunStatus. See Plugin API: Automations.

Prop

Type

Prop

Type

Prop

Type

Prop

Type

  • AutomationDefinitionSchema, AppAutomationSchema, automationTimelineActionSchemas, automationTriggerSchemas: Zod schemas for timeline definitions, individual action kinds, and event trigger kinds
  • validateAutomationTimeline, automationTimelineLimit: validate timeline duration and clip timing, and compute the end of the timeline
  • AUTOMATION_TRANSPORT_COMMANDS, AUTOMATION_RUN_PHASES, AUTOMATION_STEP_STATES: as const arrays for transport commands (play, pause, stop, seek, prepare), run phases, and step states
  • MAX_AUTOMATION_DURATION_MS, MAX_AUTOMATION_EFFECT_FADE_MS, MAX_AUTOMATION_HOLD_MS, MAX_AUTOMATION_COOLDOWN_MS: maximum durations for timelines, effect clip fades, holds, and trigger cooldowns
  • AUDIO_RETRIGGER_MODES, PLAYBACK_COMMANDS, MAX_AUDIO_FADE_MS, MAX_AUDIO_REPEAT_COUNT: retrigger modes, playback commands, and audio limits used by audio-play and audio-control actions

Settings

Prop

Type

SettingsPatch is the writable subset of the settings. Its fields are all optional: lipSync, controller (only its enabled flag), window, ui, and performance. Tracking and pose parameters are changed through tracking.* and pose.* instead.

Settings and SettingsPatch are derived from the Zod schemas SettingsSchema and SettingsPatchSchema:

  • Validating a patch strips unknown and read-only properties.
  • Parsing a response tolerates extra properties.
  • PersonaClient does no client-side validation before sending.
type TrackingSourceId = "persona-ios" | "ifacialmocap" | "vts-ios";
type PoseSourceId = "vmc" | "mocopi";
type TrackingSourceKind = TrackingSourceId | PoseSourceId | "mediapipe";
type TrackingStatus = "off" | "waiting" | "tracking" | "no-face";
type PoseStatus = "off" | "waiting" | "tracking";
type EffectsQuality = "low" | "medium" | "high";

Tracking Sources

tracking.sources lists all configured input sources, including network facial tracking, body pose tracking, and webcams. Multiple sources run concurrently; model instances bind to sources by ID via faceSourceId, poseSourceId, and handSourceId on SceneModelItem.

Prop

Type

tracking.source, pose.source, and pose.port are legacy properties mirroring the primary network source in each channel for backward compatibility. New plugins should query sources directly.

The mediapipe webcam tracking source provides dedicated configuration options, with default values specified in DEFAULT_MEDIAPIPE_CONFIG.

Prop

Type

type HandTrackingMode = "arms" | "fingers";

A single source can provide multiple tracking channels simultaneously; for example, one webcam can supply face, hand, and body tracking data. sourceSupportsChannel(source, channel) checks if a source supports face, hands, or pose output in its active configuration; sourceChannelEnabled(source, channel, masters) calculates composite activation accounting for channel flags and global switches.

Lip Sync

settings.lipSync and lipSync.configure share the LipSyncConfig schema, mapping to microphone audio processing. Installations without microphone lip sync omit this property from Settings.

Prop

Type

lipSync.state and lipSync.calibrate return LipSyncState; calibration workflows are detailed in Microphone Lip Sync.

Prop

Type

lipSyncMode on SceneModelItem controls avatar-level lip sync behavior:

type LipSyncMode = "off" | "always" | "when-untracked";

Model Movement Configuration

The model.getMovement and model.setMovement methods use ModelMovementConfig, representing whole-model translation and scaling driven by tracked face position (matching VTube Studio's Movement Config). Horizontal, vertical, and depth sliders range from −50 to 50; pivot (Depth Pivot, requiring model-movement-pivot capability) specifies the scaling center in normalized Live2D canvas coordinates.

Prop

Type

Version History

The model.*Version* methods and model.restoreFiles (requiring the model-history capability) read and manage a model's saved versions, the desktop's Version History. See Plugin API: Version History for each method's parameters and errors.

Prop

Type

Prop

Type

Prop

Type

type ModelVersionKind = "first" | "saved" | "backup";
type ModelVersionUnsavedAction = "save" | "discard";
type ModelVersionChangeStatus = "added" | "modified" | "deleted" | "unchanged";
  • MODEL_VERSION_KINDS, MODEL_VERSION_UNSAVED_ACTIONS, MODEL_VERSION_CHANGE_STATUSES: as const arrays for those types
  • MODEL_VERSION_CURRENT ('current'): represents the current files on disk wherever a version ID is accepted, such as from or to in model.compareVersions
  • MODEL_VERSION_NAME_MAX_LENGTH (80), MODEL_VERSION_NOTES_MAX_LENGTH (2000): maximum character lengths for version names and notes
  • MODEL_VERSIONS_LIST_MAX (500), MODEL_VERSIONS_LIST_DEFAULT (50): maximum and default number of versions returned by model.listVersions

Storage

storage.* endpoints persist arbitrary JSON data. Capacity limits are documented under Constants, and usage patterns are covered in Plugin Storage.

type JsonValue =
  string | number | boolean | null | JsonValue[] | { [key: string]: JsonValue };

Parameter Injection

Prop

Type

type InjectTargetType = "input" | "live2d-param" | "vrm-expression";

InjectTarget is an injection target without value and weight.

Input Names

For input targets, id is one of the input names below. INPUT_NAMES holds every name in the table; the VTube Studio and ARKit names match those in .vtube.json. Injected input values reach Live2D models only.

GroupInputs
VTube StudioThe 21 classic VTS names: FaceAngleX FaceAngleY FaceAngleZ FacePositionX FacePositionY FacePositionZ EyeOpenLeft EyeOpenRight EyeLeftX EyeLeftY EyeRightX EyeRightY Brows BrowLeftY BrowRightY MouthSmile MouthOpen MouthX CheekPuff JawOpen TongueOut
ARKitThe remaining 52 channels: ARKit followed by the PascalCase blendshape name, e.g., ARKitJawOpen, ARKitMouthSmileLeft, ARKitEyeBlinkRight
ControllerBASE_CONTROLLER_INPUT_NAMES, the primary gamepad's VTS-compatible names: sticks (ControllerStickLeftX/Y, ControllerStickRightX/Y) and stick clicks, D-pad axes and four direction buttons (ControllerDPadUp/Down/Left/Right), face buttons, bumpers, triggers, Options, Home, the PlayStation touchpad click (ControllerTouchPadPress), VTS-derived thumb and index-finger positions (ControllerThumbPosLeft/Right, ControllerFingerPosLeft/Right), left/right button activity and counts (ControllerButtonsAnyLeft/Right, ControllerButtonCountLeft/Right), and controller lean (ControllerLean)
HandsHAND_INPUT_NAMES: the 26 VTS hand channels plus HandLeftAngleY and HandRightAngleY, covering hand presence, coordinates, rotations, hand open/close, individual finger flexion (e.g., HandLeftFinger_1_Thumb), BothHandsFound, and HandDistance
VoiceVOICE_INPUT_NAMES: the VTS voice channels VoiceVolume, VoiceFrequency, VoiceVolumePlusMouthOpen, VoiceFrequencyPlusMouthSmile, VoiceA through VoiceO, and VoiceSilence, plus VoiceMouthOpen and the signed VoiceMouthSpread
MouseMOUSE_INPUT_NAMES: MousePositionX and MousePositionY, the pointer position normalized to −1..1 across the screen (+x right, +y up, matching FacePositionY)
TimeTIME_INPUT_NAMES: Time24h and Time12h, spanning 0..1 (TIME_INPUT_RANGES)
RandomRANDOM_PULSE_INPUT_NAMES: RandomPulse1 through RandomPulse8, spanning 0..1 (RANDOM_PULSE_INPUT_RANGES)
  • The ARKit prefix avoids collisions with the VTS channels JawOpen, CheekPuff, and TongueOut, since .vtube.json matches input names case-insensitively. Those three form ARKIT_TWINS: each shares its float value with its ARKit channel, accepts values under either name, and maps to ARKit internally.
  • INPUT_RANGES gives the standard ranges: head and hand rotations are in degrees, ARKit channels span 0..1, and other inputs are normalized, dimensionless floats.
  • CONTROLLER_THUMB_POSITIONS, CONTROLLER_FINGER_POSITIONS, and isControllerPositionInput cover the thumb and index-finger positions. Secondary gamepads add an index after Controller (e.g., Controller2StickLeftX, Controller12Cross), so any number of controllers can be mapped. controllerInputName builds an indexed name, and getInputRange returns an input's range (Y axes positive upward).
  • Mouse inputs work independently of the face tracking switches.
  • Time inputs sample the desktop's local wall clock once per rendered frame, independently of face tracking, and drive both Live2D and VRM parameter bindings; see Time Parameter Bindings. isTimeInputName is their TypeScript type guard.
  • Random pulse inputs fire on their own schedules between 0 (at rest) and 1 (at full), independently of face tracking (see Parameter Bindings and Plugin API: Parameter Bindings). RandomPulseSchedules maps each customized pulse to its RandomPulseSchedule, while unconfigured pulses use DEFAULT_RANDOM_PULSE_SCHEDULE (0.5–4.5 s wait, 0.58 s duration, 0.29 s fade in and out, matching VTube Studio's auto-blink); RANDOM_PULSE_MIN_SECONDS (0.1) and RANDOM_PULSE_MAX_SECONDS (3600) define the timing bounds, sanitizeRandomPulse and sanitizeRandomPulses normalize schedules, and isPulseBinding, pulseRest, pulseHoldsRest, firePulseBinding, and bindingOutputRange evaluate pulse bindings.

Prop

Type

For Live2D parameter mapping configurations, refer to Parameter Bindings.

Wire Protocol Envelopes

Direct WebSocket protocol integrators can reference the underlying message envelopes below:

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

Prop

Type

ApiErrorCode defines the closed enumeration of error codes returned in ErrorMessage.code; see Error Code Reference. PersonaApiError instances thrown by call encapsulate this error code.

Last updated on November 7, 2026

Tech otakus destroy the world