Kilter Climb Real2Sim
103 short Kilter Board climbing videos from YouTube and Instagram, each reconstructed into a replayable 4D scene: the board and its holds under a calibrated camera, the climber's SMPL-X motion, hand and foot contacts, and the motion retargeted to a Unitree G1 humanoid.
Dataset on Hugging Face →
How the scene is made
Wall and hold colours: the median frame of the video (the climber removed) is projected back onto the wall and the holds through the calibrated camera. From the source camera the scene matches the video; from other angles, surfaces the camera never saw show the board's base colour.
Hold shapes: SAM 2 segments each hold's outline in the median frame; the outline is cast along camera rays onto the board and raised into a dome whose height grows with its footprint. The shapes are approximate; positions and sizes come from the image.
Climber: an SMPL-X body skinned in the browser. Each vertex colour is the median over video frames where the vertex faces the camera and is not occluded; the side facing the wall is filled in from neighbouring vertices.
G1: retargeted onto the same wall (scaled by the robot-to-climber size ratio), with every contact placed on its own hold; meshes from the Unitree G1 model. On this page the climber plays at up to 30 fps, the G1 at 15 fps, and the source video is a small compressed copy.
Quality tiers: good when both the human and the G1 score are ≥ 0.6, fair when both are ≥ 0.4, low otherwise. The videos belong to their original uploaders.