@laplace.live/persona-sdk is the official TypeScript and JavaScript client library for the LAPLACE Persona Plugin API, and serves as the single source of truth for its wire protocol schema. This page provides a complete type and API reference. For server configuration, API key management, and WebSocket RPC method specifications, see the Plugin API documentation.
The SDK runs across Node.js 22+, Bun, modern web browsers, and OBS browser sources. Its only runtime dependency is zod, which defines and validates all wire schemas.
npm install @laplace.live/persona-sdkThe reference tables below mirror the TypeScript declarations exported by the SDK package root. Import any type directly via import type { … } from '@laplace.live/persona-sdk'.
What You Can Build
The Persona Console is the official reference implementation built on this SDK. It provides a complete web-based control console in a single browser tab, letting creators manage scenes, layers, asset libraries, expressions, motions, parameter bindings, hotkeys, tracking sources, and global settings, with an interface optimized for mobile touch devices during live broadcasts.
The console contains no proprietary desktop code. It interacts with Persona exclusively through the Plugin API, demonstrating that third-party integrations can achieve equivalent control using the same SDK.
When building client applications, follow these recommended design patterns:
- Gate on capabilities, not on model formats or version numbers: Query
InstanceRuntime.capabilitiesbefore exposing controls, disabling unsupported features dynamically rather than catchingunsupported-for-formaterrors; - Treat reconnections as cache invalidations: Broadcast events missed while disconnected are not replayed; cached data may diverge from the host. Flush local caches and re-query state snapshots upon reconnecting;
- Handle terminal close codes appropriately: Status codes
4001(key revoked) and4002(session disconnected by user) represent terminal closures and should stop reconnection loops; other disconnects should trigger exponential backoff retries; - Rely on SDK subscription management: Listeners registered via
persona.on(…)are restored automatically upon reconnecting, eliminating manual subscription bookkeeping.
When connecting from an HTTPS webpage over unencrypted ws://, browser
mixed-content restrictions permit connections only to 127.0.0.1 on the local
machine. To connect to an external machine on your network, serve your page
over HTTP or place a reverse proxy with TLS in front of Persona, as described
in the Web Console documentation.
LLM Prompt
Provide the following prompt to an LLM or include it in your project's CLAUDE.md or AGENTS.md rules file to supply complete contextual guidance for developing Persona plugins:
You are helping me build a plugin for LAPLACE Persona, a desktop VTuber app
(https://laplace.live/persona). Persona exposes a local WebSocket server called the
Plugin API, and `@laplace.live/persona-sdk` is its typed client.
Before writing any code, fetch the reference documents listed at the end — they are
the single source of truth for method names, events, error codes and types. Do not
rely on memory and do not invent names.
## Setup
- Install `@laplace.live/persona-sdk`. Isomorphic: Node 22+, Bun, browsers and OBS
browser sources. Its only runtime dependency is zod.
- The user enables the server in Persona under Settings → Plugin API and creates an
API key there. Tokens look like `sk-lp-v1-…`. Default endpoint: `ws://127.0.0.1:25034`.
- Everything — values and types — is exported from the package root; there is no
other entry point.
## Client
```ts
import { PersonaClient } from "@laplace.live/persona-sdk";
const persona = new PersonaClient({
token: "sk-lp-v1-…", // required; everything else is optional
clientInfo: { name: "My Plugin", version: "1.0.0" }, // display-only identity
});
await persona.connect();
const { scenes, activeSceneId } = await persona.call("scene.list");
await persona.call("scene.patch", {
background: { mode: "color", color: "#00ff00", imageAssetId: null },
});
persona.on("motion.started", (m) => console.log("playing", m.group));
const mouth = persona.driveParameter("MouthOpen", 0.8); // parameter lease
mouth.set(0.3);
mouth.release();
```
`call(method, params?)` is fully typed — the method name determines the request and
response shapes.
Methods cover scenes, model instances, stage objects, expressions, motions, speech,
hotkeys, automations, model and asset registration, settings, tracking and pose
input, per-key storage and parameter injection. Events notify you of scene,
selection, expression, motion, speech, tracking and registry changes.
The exact method and event names, parameter shapes, error codes and limits live in
the reference below — look them up instead of guessing. Gate features on the
capabilities reported by `app.info` and `instance.list`, never on version strings.
## Reference
Fetch these before writing code. The documentation is served as raw Markdown —
append `.mdx` to any page URL (`.en.mdx` for English):
- https://laplace.live/persona/plugin-api.en.mdx — server setup, API keys, and every
method, event, error code and limit
- https://laplace.live/persona/sdk.en.mdx — the SDK client: options, auth modes and
every exported typeClient Reference
import { PersonaClient } from "@laplace.live/persona-sdk";
const persona = new PersonaClient({ token: "sk-lp-v1-…" });
await persona.connect();
const { scenes, activeSceneId } = await persona.call("scene.list");
await persona.call("scene.patch", {
background: { mode: "color", color: "#00ff00", imageAssetId: null },
});
persona.on("motion.started", (m) => console.log("playing", m.group));Instantiate PersonaClient using the options below. Only token is required:
Prop
Type
ClientInfo
clientInfo declares the client identity displayed in the desktop application under Settings → Plugin API → Connected Clients. It is optional, self-declared display metadata and does not affect access permissions. The SDK re-announces this payload upon every reconnection.
const persona = new PersonaClient({
token,
clientInfo: { name: "My Overlay", version: "1.2.0", developer: "You" },
});Prop
Type
Authentication
In browser environments, the API token is passed via the ?token= query parameter due to browser WebSocket API constraints. In Node.js or Bun environments, you can supply a custom createWebSocket hook that passes the token as an Authorization: Bearer header using ws:
import WebSocket from "ws";
const persona = new PersonaClient({
token,
auth: "header",
createWebSocket: (url, headers) => new WebSocket(url, { headers }),
});WebSocketLike defines the structural interface required by createWebSocket. Both global browser WebSocket instances and the Node.js ws client satisfy this contract without type assertions.
Prop
Type
In Node.js 22+, the global WebSocket accepts handshake headers through its WebSocketInit extension. The SDK's createNodeClient helper builds a client on top of it:
- Sets
auth: 'header' - Uses Node's global
WebSocket - Sets a 5-second connection timeout
- Sends SDK warnings and terminal close notices (key revoked or force-disconnected) to the
logfunction you pass
import { createNodeClient } from "@laplace.live/persona-sdk";
const persona = createNodeClient({
token: "sk-lp-v1-…",
log: console.warn,
});
await persona.connect();Prop
Type
Parameter Leases
driveParameter writes a continuous value to a model parameter and returns an InjectionHandle. The SDK maintains lease heartbeats automatically. The lease expires approximately one second after the last write, restoring default parameter behavior safely if the client process terminates unexpectedly.
const mouth = persona.driveParameter("MouthOpen", 0.8);
mouth.set(0.3);
mouth.release();A parameter can be leased by only one active session at a time; concurrent attempts return a conflict error.
The method accepts an optional weight multiplier and an onError callback invoked if heartbeat renewal fails in the background, removing the need to await individual updates.
Prop
Type
Connection Close Handling
onClose reports disconnects that come from the server or the network. The callback receives the raw socket close code, the reason string, and a terminal flag that says whether reconnect retries have stopped.
- It fires after the SDK has finished its internal cleanup, so calling
close()inside the handler is safe. - A clean
close()initiated by the client does not triggeronClose. - Once
onCloseis set, terminal disconnects are reported there instead of throughonWarning.
Prop
Type
PersonaClientState and Client Errors
type PersonaClientState = "closed" | "connecting" | "open" | "reconnecting";
type ClientErrorReason =
| "connect-timeout"
| "connection-closed"
| "client-closed"
| "not-connected"
| "request-timeout";When the client or its transport fails connect() or call(), the promise rejects with a PersonaClientError carrying reason: ClientErrorReason (one of CLIENT_ERROR_REASONS). Server error frames reject call() with PersonaApiError (code: ApiErrorCode). See Plugin API: Error Codes.
Prop
Type
Constants
| Export | Value | Description |
|---|---|---|
PROTOCOL_VERSION | 4 | Incremented on breaking protocol changes; reported by the server in hello |
DEFAULT_API_HOST | 127.0.0.1 | Loopback address bound by default unless local network access is enabled |
DEFAULT_API_PORT | 25034 | Default WebSocket port |
SCENE_ACTIVATION_TIMEOUT_MS | 120000 | Default timeout for scene.activate when requestTimeoutMs is unspecified |
ASSET_IMPORT_PROMPT_MS | 45000 | How long asset.import waits for user confirmation when importing a file outside plugin folders (when requestTimeoutMs is unspecified, the SDK defaults to 55000 ms for asset.import) |
CLOSE_KEY_REVOKED | 4001 | Terminal close code: API key revoked by server |
CLOSE_FORCE_DISCONNECTED | 4002 | Terminal close code: session terminated by user |
CLOSE_SERVER_STOPPING | 1001 | Non-terminal close code: server shutting down; client will retry |
INJECT_LEASE_TTL_MS | 1000 | Lease time-to-live after last write |
INJECT_HEARTBEAT_MS | 100 | Heartbeat transmission interval for active leases |
STORAGE_KEY_MAX_LENGTH | 128 | Maximum length of a storage key name |
STORAGE_VALUE_MAX_LENGTH | 65536 | Maximum serialized length of a storage value |
STORAGE_KEYS_MAX | 256 | Maximum key-value pairs stored per API key |
SPEECH_URL_MAX_LENGTH | 8000000 | Maximum URL length for speech.play (accommodates ~40 seconds of Base64 WAV) |
The SDK also exports these value lists and helpers:
API_ERROR_CODES,CLIENT_ERROR_REASONS,EVENT_NAMES,INPUT_NAMES,TIME_INPUT_NAMES,RANDOM_PULSE_INPUT_NAMES,MODIFIER_TOKENS,APP_CAPABILITIES,FPS_LIMIT_PRESETS,EFFECTS_QUALITY_LEVELS:as constarrays, so a UI picker can list every valid valueisApiErrorCode,isEventName,isAppCapability,isTimeInputName,isRandomPulseInputName: the matching TypeScript type guardsinjectTargetKey: builds a normalized injection target stringpersonaWsUrl: builds a WebSocket URL from{ host, port, secure }(PersonaAddress, wherehostmust be a plain host name or IP address) or from an address string the user typed (bare hostnames,host:port, IPv4/IPv6 literals, and fullws://orwss://URLs);PersonaClientappends the token when connectingparseHost,parsePort,isValidPort: validate a plain host name or IP address (accepting bare or bracketed IPv6 and returningnullfor schemes, ports, paths, or zone IDs) and a port numberrequestTimeoutMs(method, fallbackMs): resolves the timeout for a request method (scene.activateandasset.importuse their extended defaults)
Models and Assets
Models and general assets share a unified metadata structure. kind defines the asset classification, while origin identifies its source: bundled for built-in samples and content downloaded on demand from the Persona catalog, and user for locally registered files, accompanied by author attribution metadata.
Prop
Type
type ModelFormat = "live2d" | "vrm";
type ContentOrigin = "bundled" | "user";
type AssetKind =
| "image"
| "video"
| "audio"
| "prop"
| "ibl"
| "lut"
| "animation"
| "cameraMotion"
| "lyrics";
type InventoryKind = ModelFormat | AssetKind | "effect" | "addon";effect represents a virtual asset type without a backing disk file, used by the asset library to enumerate built-in effects; addon represents an optional application add-on (such as the Webcam Tracking Add-On) that extends the app rather than adding a scene layer.
model.list returns an array of ModelRef objects and asset.list returns an array of AssetRef objects. Both extend ContentRef with kind narrowed to ModelFormat and AssetKind respectively.
Both also carry a boolean exists flag. On an AssetRef it says whether the asset file is still on disk; a ModelRef sets it to false when the model's entry file is missing.
Prop
Type
Prop
Type
Prop
Type
CatalogItem
CatalogItem defines transport-agnostic metadata used by asset pickers. Both local registry items and remote catalog listings normalize to this type. It removes the installation-specific origin field from ContentRef and adds localized description (LocalizedText) and catalog metadata. download (CatalogDownload) describes retrieval parameters for remote items; if absent, the item exists locally.
Prop
Type
Prop
Type
Prop
Type
Catalog and localization helpers:
CATALOG_FORMAT,MARKETPLACE_ITEMS_PATH,CATALOG_KINDS: catalog manifest format version, the public catalog endpoint path (/api/items), and the supported catalog item kindsisCatalogId,healCatalog,healCatalogItem,healCatalogRows,healSha256: validate and sanitize remote or local catalog manifestspayloadEntryPath,payloadEntryKey,catalogMediaFile(takes aCatalogMediaRole): resolve the extracted entry path of a catalog payload and the dot-prefixed file names for a source's rendered thumbnail and preview video (.thumbnail.avifand.preview.webm)pickLocalized(text, locale),healLocalizedText: pick the display string for a UI locale from aLocalizedTextobject and sanitize localized text maps
Placement
2D screen space is measured in pixels from the stage center (+y down). 3D world space is measured in meters. All rotations in the wire schema use radians (converted to degrees in user interfaces).
Prop
Type
Prop
Type
Place2D and Place3D represent placement definitions for scene objects, sharing identical position fields with an added opacity channel.
Scene Items
The items array in a scene orders models and objects together according to their Z-order:
type SceneItem = SceneModelItem | SceneObjectItem;Prop
Type
Prop
Type
Prop
Type
ObjectContent
The payload of a scene object is determined by its kind field. Except for capture (display capture), all object types can be created directly in the desktop interface, as described in Layer Management.
type ObjectContent =
| { kind: "image"; assetId: string }
| {
kind: "video";
assetId: string;
loop: boolean;
muted: boolean;
volume: number;
}
| { kind: "prop"; assetId: string }
| {
kind: "web";
url: string;
width: number;
height: number;
fps: number;
transparent: boolean;
css: string;
layer: WebLayer;
shutdownWhenHidden: boolean;
}
| { kind: "capture"; source: CaptureKind; sourceId: string; label: string };
type ObjectSpace = "2d" | "3d";
type WebLayer = "behind" | "front";
type CaptureKind = "display" | "window";Prop
Type
Attachment
Prop
Type
AttachAnchor
type AttachAnchor =
| { kind: "root" }
| { kind: "bone"; bone: string }
| {
kind: "artMesh";
id: string;
verts: [number, number, number];
weights: [number, number, number];
};object.anchors returns available attachment anchors provided by a parent model as AnchorOption records, pairing anchor identifiers with human-readable labels. VRM models provide the root transform and humanoid bones; Live2D models provide the root transform and individual ArtMeshes with an order draw-order index.
AttachDepth
type AttachDepth =
{ kind: "front" } | { kind: "behind" } | { kind: "artMesh"; id: string };depth controls the internal layering of Live2D props attached to a parent Live2D model, positioning the item behind the model, in front of the model, or directly above a specified ArtMesh. Configured via instance.attach.
Prop
Type
Attachment helpers:
depthStops(meshes): the available depth stops from back to front (behind the model, above each ArtMesh in draw order, then in front)depthStopIndex(meshes, depth): which stop a depth sits at; an ArtMesh the model no longer has counts as the frontmostpinRiders(items, instanceId): every item attached toinstanceId, directly or through other items, so circular attachments can be preventedpinnableParents(models, items, instanceId): the candidate parents, with invalid ones markedridesThisdefaultAttach(parentInstanceId, screenSpace?): a fresh attachment (root anchor, in front, unsplit, no physics tuning). WithscreenSpaceset (a Live2D model or 2D object), the item also follows the host's head at the default response
Prop
Type
Prop
Type
Scenes
Prop
Type
Prop
Type
Prop
Type
Background
Prop
Type
type BackgroundMode = "color" | "image";Camera
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
cameraDistanceLimits(environment, radius) computes how close and how far the orbit camera may go. The defaults are DISTANCE_MIN (0.35 m) and DISTANCE_MAX (20 m); when a 3D environment's bounding radius differs a lot from a standard room (~10 m), the limits scale with it. radius is the bounding radius that scene.inspect returns under environment.radius; when it is null, the default limits apply.
Lighting
Prop
Type
type SceneLightType = "directional" | "point" | "ambient" | "area" | "spot";
type ShadowQuality = "off" | "low" | "medium" | "high" | "ultra";
type ShadowFilter = "pcf" | "pcss";SCENE_LIGHT_TYPES, SHADOW_QUALITY_LEVELS, and SHADOW_FILTERS are as const arrays defining supported light types, shadow qualities, and filter modes matching interface order.
Three SceneLight fields make a light follow the camera:
followCamera: binds the light's position and orientation to the camera viewfollowCameraOptions: which degrees of freedom follow the camera's motion;DEFAULT_LIGHT_FOLLOW_CAMERA_OPTIONSenables all of themfollowCameraReference: the frozen reference transform for the axes that don't follow
To turn following on or off without moving the light, call scene.setLightFollowCamera instead of editing these fields; see Lighting.
Prop
Type
Prop
Type
iblAssetId and modelAssetId set the image-based lighting (IBL) map and the 3D stage environment. Releases with the environment-map-model capability accept both at once.
- Use
environmentWithAsset()to change either one: it sets the new asset, keeps the other, and resets light overrides tied to the old model. - Selecting an IBL texture turns
bakeFromModeloff. Baking defaults to on only when a 3D model is loaded into an empty environment. environmentClearPatch(environment, slot)builds a patch that clears'model'or'map', or both whenslotisnull.applyEnvironmentLighting()andapplyEnvironmentLook()back the panel's quick actions: they apply the environment lighting preset and anEnvironmentLookprofile.
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Volumetric lighting is enabled per light through SceneLight.volumetric. Related helpers:
SCENE_VOLUMETRIC_LIGHTING_SPECS: the numeric ranges of the volumetric fog slidersdefaultSceneVolumetricLighting(): the default fog settingshealSceneVolumetricLighting(): clamps out-of-range valuessceneLightCanHaze(type): whether a light type supports volumetric scatteringsceneLightHazes(light): whether a light is emitting volumetric light right now
Effects
SceneEffects uses a flat schema: every toggleable post-processing effect is exposed as a top-level property, enabling generic reflection and iteration. Visual grouping in user interfaces is a presentation concern independent of the wire schema.
Prop
Type
Effect ordering in the Layer List is tracked separately in environment.effectLayers (an array of ToggleEffectKey strings). This list is orthogonal to the enabled flag: enabled activates rendering, while effectLayers controls layer item presentation. Enabling an effect automatically appends it to the layer list.
type SceneToneMapping = "none" | "neutral" | "aces" | "agx";Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Gradient provides 24 blend modes defined in EffectBlendMode, enumerated in EFFECT_BLEND_MODES.
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
rain and snow share standard effect structures but operate as 3D geometric particle systems rather than 2D screen-space post-processing passes; see Rain and Snow.
Model and object layers have their own effect pipelines. effects and effectLayers on SceneModelItem and SceneObjectItem run before the scene-wide effects and are set with instance.setEffects.
- Layer effects support the subset in
LayerEffectKey:color,levels,colorWheels,colorShift,selectColors,gradient,blur,bloom,diffusion,sketch,rim,outline, anddropShadow. - Bloom color selection works only at the layer level; see Layer Effects.
Transitions
transition configures the visual transition when activating a scene; see Scene Transitions (requires scene-transitions capability). SCENE_TRANSITION_TYPES lists valid transition styles, defaultSceneTransition() returns standard cuts, and healSceneTransition() validates configurations.
Prop
Type
environment.itemTransition governs entrance and exit animations for models, props, and spawned items; see Showing and Hiding. ITEM_TRANSITION_STYLES enumerates supported styles, defaultItemTransition() defaults to a 300 ms glitch effect, and ITEM_TRANSITION_MAX_MS imposes a 5000 ms ceiling.
Prop
Type
Scene Presets
presets lists the scene's saved scene presets in display order; builds without the scene-presets capability omit it. Manage the active scene's presets with preset.create, preset.update, preset.rename, preset.setTransition, preset.delete, and preset.apply; see Plugin API: Scene Presets.
SCENE_PRESET_TRANSITIONS: every preset transitionscenePresetIsCurrent(): whether the stage's current layout matches a preset; the desktop and Web Console use it to highlight that presetcapturePresetItems(): captures a layer layoutpresetStep(): computes each layer's change when a preset is applied
Prop
Type
Prop
Type
Runtime State
instance.list reports dynamic capabilities for active stage items, allowing clients to disable unsupported controls without triggering RPC errors:
Prop
Type
type InstanceCapability =
| "motions"
| "expressions"
| "placement-2d"
| "placement-3d"
| "mtoon"
| "idle-clips"
| "idle-motions"
| "pose"
| "breath"
| "expression-weights"
| "live2d-params"
| "speech"
| "post-fx";speech capabilities are reported exclusively for loaded Live2D models supporting voice synthesis. Application-wide capabilities are reported separately during hello handshakes and via app.info. Feature gating must inspect capability flags rather than comparing version strings:
type AppCapability =
| "storage"
| "speech"
| "automations"
| "controllers"
| "model-editing"
| "asset-inspection"
| "layer-effects"
| "scene-transitions"
| "light-area"
| "light-spot"
| "light-camera-follow"
| "light-shadow-filter"
| "environment-map-model"
| "stage-spawn"
| "idle-tracking-lost"
| "motion-stop"
| "stage-capture"
| "object-pin-depth"
| "object-text"
| "camera-follow"
| "camera-aim"
| "camera-focus"
| "camera-handheld"
| "object-video-sound"
| "object-chroma-key"
| "model-movement"
| "model-movement-pivot"
| "layer-effects-2d"
| "scene-presets"
| "background-alpha"
| "binding-calibration"
| "binding-random-pulses"
| "expression-groups"
| "expression-groups-hide-first"
| "model-history"
| "automation-editing"
| "automation-transport"
| "asset-import";canonicalAppCapability maps a legacy capability name to its current one. Legacy names include area-lights, spot-lights, camera-follow-lights, shadow-filters, spawn, tracking-lost, text-objects, and video-sound. PersonaClient already does this for the capability lists it receives from the server.
stage.capture (requiring the stage-capture capability) returns { dataUrl, width, height }, a PNG render of the stage with alpha transparency and its longest edge constrained to maxEdge (range 64 – 2048, default 1024). Headless automations and AI assistants use it for visual verification; see Plugin API Capabilities.
Prop
Type
files preserves raw motion index positions defined in model manifests: missing motions use empty string placeholders so subsequent indices remain stable. motionSlots(files) returns playable motions and indices. Calling motion.play on empty slots returns not-found. Live2D models also discover unreferenced .motion3.json files in the model folder, creating synthetic groups named after each file.
motion.stop (requiring motion-stop capability) ends the running one-shot motion and eases the model back to idle; if only idle is playing, it returns { stopped: false }. The oneShot flag in motion.playing and motion.started indicates whether a motion is interruptible via stop.
Prop
Type
Prop
Type
Prop
Type
Expression Groups
expression.getGroups and expression.setGroups (requiring the expression-groups capability) read and edit a model's expression groups and return ExpressionGroups; see Plugin API: Expression Groups for the rules.
Prop
Type
Prop
Type
Prop
Type
EXPRESSION_GROUP_FADE_SECONDS_MAX (2) is the maximum fade duration for a group member, and EXPRESSION_GROUP_FADE_SECONDS_DEFAULT (0.2) is the default duration for a new member. foldExpressionKey normalizes an expression key case-insensitively the way VTS compares them, so two keys refer to the same expression when their normalized forms match, and firstByExpressionKey(items, keyOf) indexes items by their normalized key while keeping the first item listed under each non-empty key (matching how VTS resolves an expression key). newVtsId generates an ID in VTS format (32 lowercase hex characters) for a new group.
Hotkeys
Prop
Type
Prop
Type
HotkeyConfig represents hotkey payloads without the registered runtime flag, matching the arguments expected by hotkey.set. registered lists the hotkeys that currently fire while Persona is in the background; because the desktop app reads key states directly rather than registering OS-level global hotkeys, any combination (including a single key) can trigger in the background. hotkeyRunsInBackground checks whether a hotkey is configured to run in the background. Because combinations fire based on the set of held inputs regardless of order, comboKey(accelerator) returns an order-independent identifier, which duplicateAccelerators uses to detect combinations bound to multiple hotkeys. acceleratorLabel(accelerator, platform, mouseLabels?) formats a combination into an array of key-cap labels in canonical modifier order (MODIFIER_TOKENS: Control, Alt, Shift, Super), using macOS symbols (⌃, ⌥, ⇧, ⌘) when hello's app.platform is 'darwin' and Ctrl, Alt, Shift, Win elsewhere (mouse triggers default to MOUSE_TRIGGER_LABELS and accept localized overrides).
Each avatar on stage has its own hotkey set:
hotkey.list,hotkey.set, andhotkey.triggertake an optionalmodelId, which defaults to the primary model.hotkey.stateevents report updates per model.hold,fadeSeconds,deactivateAfterSeconds, andstopsOnLastFrameare newer protocol fields. Fields left out of ahotkey.setcall keep their current values.
The SDK provides helpers for querying hotkeys across all on-stage models: listSceneHotkeys(client) resolves the active scene's hotkeys (primary model first); listModelHotkeys(client, scene) returns an array of promises (Promise<ModelHotkeys>[]), one per on-stage model, allowing callers to settle each independently.
Prop
Type
Automations
Automations operate globally across the application rather than belonging to individual models. Each automation contains an ordered action sequence triggered by hotkeys, events, or API calls. automation.list retrieves all automations, automation.run executes an automation by ID, and automation.state broadcasts list modifications. Requires automations capability; see Automations.
Prop
Type
type AutomationActionKind =
| "effect-toggle"
| "effect-params"
| "effect-clip"
| "camera-pose"
| "camera-reset"
| "camera-motion-play"
| "camera-motion-stop"
| "camera-follow"
| "camera-aim"
| "camera-focus"
| "camera-handheld"
| "layer-visibility"
| "preset-apply"
| "text-set"
| "lyrics-play"
| "stream-mode"
| "scene-switch"
| "expression-toggle"
| "motion-play"
| "audio-play"
| "audio-control"
| "expression-clear"
| "model-load"
| "model-position";actionKinds and triggerKinds are typed as string[] rather than closed unions, so they stay forward-compatible with newer servers. Action kinds use category-first names; the client normalizes names from older hosts automatically.
automationActionLabel: the English label of an action kindautomationLabel: an automation's summary title (titlewhen present, otherwise its action labels joined with·)AUTOMATION_SETUP_KINDS,isAutomationSetup(andisAutomationSetupAction): identify the setup steps (scene-switchandmodel-load) that run before timeline playback.AUTOMATION_BOUNDARY_KINDSandisAutomationBoundaryare deprecated aliasesisAutomationMediaAction,isAutomationSourcedAction,isAutomationSampledClip,isAutomationContentAction: classify timeline actions (media clips, clips that accept asourceInMstrim offset, clips sampled during playhead scrubbing, and regular timeline actions)
Timeline Editing and Transport
With automation-editing, automation.get, automation.create, automation.update, and automation.delete read and write full AppAutomation timeline definitions; with automation-transport, automation.control and automation.status drive timeline playback and inspect AutomationRunStatus. See Plugin API: Automations.
Prop
Type
Prop
Type
Prop
Type
Prop
Type
AutomationDefinitionSchema,AppAutomationSchema,automationTimelineActionSchemas,automationTriggerSchemas: Zod schemas for timeline definitions, individual action kinds, and event trigger kindsvalidateAutomationTimeline,automationTimelineLimit: validate timeline duration and clip timing, and compute the end of the timelineAUTOMATION_TRANSPORT_COMMANDS,AUTOMATION_RUN_PHASES,AUTOMATION_STEP_STATES:as constarrays for transport commands (play,pause,stop,seek,prepare), run phases, and step statesMAX_AUTOMATION_DURATION_MS,MAX_AUTOMATION_EFFECT_FADE_MS,MAX_AUTOMATION_HOLD_MS,MAX_AUTOMATION_COOLDOWN_MS: maximum durations for timelines, effect clip fades, holds, and trigger cooldownsAUDIO_RETRIGGER_MODES,PLAYBACK_COMMANDS,MAX_AUDIO_FADE_MS,MAX_AUDIO_REPEAT_COUNT: retrigger modes, playback commands, and audio limits used byaudio-playandaudio-controlactions
Settings
Prop
Type
SettingsPatch is the writable subset of the settings. Its fields are all optional: lipSync, controller (only its enabled flag), window, ui, and performance. Tracking and pose parameters are changed through tracking.* and pose.* instead.
Settings and SettingsPatch are derived from the Zod schemas SettingsSchema and SettingsPatchSchema:
- Validating a patch strips unknown and read-only properties.
- Parsing a response tolerates extra properties.
PersonaClientdoes no client-side validation before sending.
type TrackingSourceId = "persona-ios" | "ifacialmocap" | "vts-ios";
type PoseSourceId = "vmc" | "mocopi";
type TrackingSourceKind = TrackingSourceId | PoseSourceId | "mediapipe";
type TrackingStatus = "off" | "waiting" | "tracking" | "no-face";
type PoseStatus = "off" | "waiting" | "tracking";
type EffectsQuality = "low" | "medium" | "high";Tracking Sources
tracking.sources lists all configured input sources, including network facial tracking, body pose tracking, and webcams. Multiple sources run concurrently; model instances bind to sources by ID via faceSourceId, poseSourceId, and handSourceId on SceneModelItem.
Prop
Type
tracking.source, pose.source, and pose.port are legacy properties mirroring the primary network source in each channel for backward compatibility. New plugins should query sources directly.
The mediapipe webcam tracking source provides dedicated configuration options, with default values specified in DEFAULT_MEDIAPIPE_CONFIG.
Prop
Type
type HandTrackingMode = "arms" | "fingers";A single source can provide multiple tracking channels simultaneously; for example, one webcam can supply face, hand, and body tracking data. sourceSupportsChannel(source, channel) checks if a source supports face, hands, or pose output in its active configuration; sourceChannelEnabled(source, channel, masters) calculates composite activation accounting for channel flags and global switches.
Lip Sync
settings.lipSync and lipSync.configure share the LipSyncConfig schema, mapping to microphone audio processing. Installations without microphone lip sync omit this property from Settings.
Prop
Type
lipSync.state and lipSync.calibrate return LipSyncState; calibration workflows are detailed in Microphone Lip Sync.
Prop
Type
lipSyncMode on SceneModelItem controls avatar-level lip sync behavior:
type LipSyncMode = "off" | "always" | "when-untracked";Model Movement Configuration
The model.getMovement and model.setMovement methods use ModelMovementConfig, representing whole-model translation and scaling driven by tracked face position (matching VTube Studio's Movement Config). Horizontal, vertical, and depth sliders range from −50 to 50; pivot (Depth Pivot, requiring model-movement-pivot capability) specifies the scaling center in normalized Live2D canvas coordinates.
Prop
Type
Version History
The model.*Version* methods and model.restoreFiles (requiring the model-history capability) read and manage a model's saved versions, the desktop's Version History. See Plugin API: Version History for each method's parameters and errors.
Prop
Type
Prop
Type
Prop
Type
type ModelVersionKind = "first" | "saved" | "backup";
type ModelVersionUnsavedAction = "save" | "discard";
type ModelVersionChangeStatus = "added" | "modified" | "deleted" | "unchanged";MODEL_VERSION_KINDS,MODEL_VERSION_UNSAVED_ACTIONS,MODEL_VERSION_CHANGE_STATUSES:as constarrays for those typesMODEL_VERSION_CURRENT('current'): represents the current files on disk wherever a version ID is accepted, such asfromortoinmodel.compareVersionsMODEL_VERSION_NAME_MAX_LENGTH(80),MODEL_VERSION_NOTES_MAX_LENGTH(2000): maximum character lengths for version names and notesMODEL_VERSIONS_LIST_MAX(500),MODEL_VERSIONS_LIST_DEFAULT(50): maximum and default number of versions returned bymodel.listVersions
Storage
storage.* endpoints persist arbitrary JSON data. Capacity limits are documented under Constants, and usage patterns are covered in Plugin Storage.
type JsonValue =
string | number | boolean | null | JsonValue[] | { [key: string]: JsonValue };Parameter Injection
Prop
Type
type InjectTargetType = "input" | "live2d-param" | "vrm-expression";InjectTarget is an injection target without value and weight.
Input Names
For input targets, id is one of the input names below. INPUT_NAMES holds every name in the table; the VTube Studio and ARKit names match those in .vtube.json. Injected input values reach Live2D models only.
| Group | Inputs |
|---|---|
| VTube Studio | The 21 classic VTS names: FaceAngleX FaceAngleY FaceAngleZ FacePositionX FacePositionY FacePositionZ EyeOpenLeft EyeOpenRight EyeLeftX EyeLeftY EyeRightX EyeRightY Brows BrowLeftY BrowRightY MouthSmile MouthOpen MouthX CheekPuff JawOpen TongueOut |
| ARKit | The remaining 52 channels: ARKit followed by the PascalCase blendshape name, e.g., ARKitJawOpen, ARKitMouthSmileLeft, ARKitEyeBlinkRight |
| Controller | BASE_CONTROLLER_INPUT_NAMES, the primary gamepad's VTS-compatible names: sticks (ControllerStickLeftX/Y, ControllerStickRightX/Y) and stick clicks, D-pad axes and four direction buttons (ControllerDPadUp/Down/Left/Right), face buttons, bumpers, triggers, Options, Home, the PlayStation touchpad click (ControllerTouchPadPress), VTS-derived thumb and index-finger positions (ControllerThumbPosLeft/Right, ControllerFingerPosLeft/Right), left/right button activity and counts (ControllerButtonsAnyLeft/Right, ControllerButtonCountLeft/Right), and controller lean (ControllerLean) |
| Hands | HAND_INPUT_NAMES: the 26 VTS hand channels plus HandLeftAngleY and HandRightAngleY, covering hand presence, coordinates, rotations, hand open/close, individual finger flexion (e.g., HandLeftFinger_1_Thumb), BothHandsFound, and HandDistance |
| Voice | VOICE_INPUT_NAMES: the VTS voice channels VoiceVolume, VoiceFrequency, VoiceVolumePlusMouthOpen, VoiceFrequencyPlusMouthSmile, VoiceA through VoiceO, and VoiceSilence, plus VoiceMouthOpen and the signed VoiceMouthSpread |
| Mouse | MOUSE_INPUT_NAMES: MousePositionX and MousePositionY, the pointer position normalized to −1..1 across the screen (+x right, +y up, matching FacePositionY) |
| Time | TIME_INPUT_NAMES: Time24h and Time12h, spanning 0..1 (TIME_INPUT_RANGES) |
| Random | RANDOM_PULSE_INPUT_NAMES: RandomPulse1 through RandomPulse8, spanning 0..1 (RANDOM_PULSE_INPUT_RANGES) |
- The
ARKitprefix avoids collisions with the VTS channelsJawOpen,CheekPuff, andTongueOut, since.vtube.jsonmatches input names case-insensitively. Those three formARKIT_TWINS: each shares its float value with its ARKit channel, accepts values under either name, and maps to ARKit internally. INPUT_RANGESgives the standard ranges: head and hand rotations are in degrees, ARKit channels span0..1, and other inputs are normalized, dimensionless floats.CONTROLLER_THUMB_POSITIONS,CONTROLLER_FINGER_POSITIONS, andisControllerPositionInputcover the thumb and index-finger positions. Secondary gamepads add an index afterController(e.g.,Controller2StickLeftX,Controller12Cross), so any number of controllers can be mapped.controllerInputNamebuilds an indexed name, andgetInputRangereturns an input's range (Y axes positive upward).- Mouse inputs work independently of the face tracking switches.
- Time inputs sample the desktop's local wall clock once per rendered frame, independently of face tracking, and drive both Live2D and VRM parameter bindings; see Time Parameter Bindings.
isTimeInputNameis their TypeScript type guard. - Random pulse inputs fire on their own schedules between
0(at rest) and1(at full), independently of face tracking (see Parameter Bindings and Plugin API: Parameter Bindings).RandomPulseSchedulesmaps each customized pulse to itsRandomPulseSchedule, while unconfigured pulses useDEFAULT_RANDOM_PULSE_SCHEDULE(0.5–4.5s wait,0.58s duration,0.29s fade in and out, matching VTube Studio's auto-blink);RANDOM_PULSE_MIN_SECONDS(0.1) andRANDOM_PULSE_MAX_SECONDS(3600) define the timing bounds,sanitizeRandomPulseandsanitizeRandomPulsesnormalize schedules, andisPulseBinding,pulseRest,pulseHoldsRest,firePulseBinding, andbindingOutputRangeevaluate pulse bindings.
Prop
Type
For Live2D parameter mapping configurations, refer to Parameter Bindings.
Wire Protocol Envelopes
Direct WebSocket protocol integrators can reference the underlying message envelopes below:
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
Prop
Type
ApiErrorCode defines the closed enumeration of error codes returned in ErrorMessage.code; see Error Code Reference. PersonaApiError instances thrown by call encapsulate this error code.
Last updated on November 7, 2026