a pose stream in real time
Workout motion straight into the game
A camera captures the instructor, the server turns the frames into a pose stream, and an avatar in Roblox or Unity repeats the movement in real time.
pose stream · jsonframe 1284
{
"v": 1, "frame": 1284, "timestamp_ms": 1725790412034,
"coord": "world_normalized", "joint_order": "spec_web_v1",
"joints": [[0.50, 0.95, 0.00], [0.50, 0.72, 0.02], [0.50, 0.55, 0.01], …],
"confidence": [0.98, 0.97, 0.96, …],
"tracking": {"status": "ok", "lost": false, "quality": 0.98}
}
15 skeleton jointsworld_normalizedRobloxUnity
01 / pipeline
From frame to avatar
01Capturing the instructorA camera or a phone films the instructor; the frames go to the pose server.
02Frames → pose streamEvery frame becomes a set of 15 skeleton joints with a confidence score.
03Stream into the scenePose frames are sent to the game client with minimal delay.
04Repeating the movementAn avatar in Roblox or Unity reproduces the pose in real time.
02 / pose
What goes into the game
stream frame · PoseFrame v1
{
"v": 1, "frame": 1284, "timestamp_ms": 1725790412034,
"coord": "world_normalized", "joint_order": "spec_web_v1",
"joints": [[0.50, 0.95, 0.00], [0.50, 0.72, 0.02], [0.50, 0.55, 0.01], …],
"confidence": [0.98, 0.97, 0.96, …],
"tracking": {"status": "ok", "lost": false, "quality": 0.98}
}Coordinates onlyThe game receives a stream of skeleton joints — head, chest, hips, shoulders, elbows, wrists, knees, ankles — not the camera video.
Roblox or UnityThe scene client reads the stream and applies the pose to its own rig.
03 / connection
What a session needs
01Camera and spaceThe instructor fully in frame, even light, a steady camera.
02Pose serverProcesses the frames and serves the coordinate stream over the network.
03Scene clientA script in Roblox or Unity applies the pose to the avatar.
04 / questions
In short
What does the game see — video or coordinates?
Only the stream of skeleton coordinates: the frames stay on the pose server.
Which camera is needed?
An ordinary webcam or a phone pointed at the instructor; light and full height in frame matter more.
Roblox and Unity at the same time?
There is one stream; the scene client — on the Roblox or the Unity side — applies the pose to its rig.
A workout inside the game
Camera, pose server and scene client — from there the avatar repeats the movement.