Reading 6 papers
Egocentric data and reconstruction
Head-worn camera video shows people handling objects, but it arrives without action labels. EgoDex captures the hands while recording. MINT, HaWoR and WHOLE recover hands, objects and camera motion afterwards, in a world frame. Holi-World turns its reconstructions into training data for a vision-language model, and EgoVLA trains a robot policy on such video.
In PROEmbodied AI, Vision-language-action models, SLAM, 3D Gaussian Splatting
Oct 2026 Ropedia
Holi-World
Reconstructs each scene with visual-inertial SLAM, Depth Anything 3, and 3D Gaussian splatting, then tracks static and moving objects to build "world records." Spatial and temporal QA pairs generated from these records fine-tune a 4B vision-language model. The pipeline assumes at most one moving object at a time, and the code will not be released.
ConnectsHaWoR and WHOLE reconstruct the hands and the manipulated object instead. See HaWoR, WHOLE.
In PROEmbodied AI, Vision-language models, World models
May 2025 Apple, ICLR 2026
EgoDex
A large dexterous-manipulation dataset recorded on Apple Vision Pro, with joint-level hand tracking captured at recording time.
ConnectsThe capture-time answer: hands tracked while recording. MINT, HaWoR and WHOLE recover them from ordinary video after the fact, and EgoVLA is where such data goes. See MINT, HaWoR, WHOLE, EgoVLA.
Sep 2026
MINT
Predicts the camera trajectory and camera-frame hand states together, then converts the hands into world space with an explicit coordinate transform.
ConnectsHaWoR reaches world-space hands through SLAM instead of an explicit transform. See HaWoR.
In PROEmbodied AI
Jan 2025
HaWoR
Recovers world-space hand motion from egocentric video with an adaptive SLAM step that copes with hands filling much of the frame.
ConnectsWHOLE adds the manipulated object and a motion prior. See WHOLE, MINT.
Feb 2026
WHOLE
Reconstructs both hands and the manipulated object in a world frame from metric-SLAMed egocentric video, using a diffusion prior over hand-object motion.
ConnectsHaWoR covers the hands alone. See HaWoR.
Jul 2025
EgoVLA
Trains a vision-language-action model on egocentric human video with wrist and hand poses, then transfers it to a humanoid robot.
ConnectsWhere the data goes: a policy trained on egocentric video with hand poses, the signal EgoDex records and the reconstruction papers recover. See EgoDex, MINT.
A different, unrelated EgoX (exocentric-to-egocentric video generation, CVPR 2026) also exists, so search results can mix the two.
This direction is tracked in PRO.
PRO adds the intelligence layer: the field ranked daily and Signals on every technique. It also brings a code tutorial with every issue, the full archive and ad-free issues.