On October 12, according to an arXiv paper, researchers Wancong Zhang, Basile Terver, Michael Rabbat, Yann LeCun, and Randall Balestriero proposed H-JEPA, an end-to-end training scheme for a hierarchical action-conditioned JEPA world model, where each layer predicts further into the future within its learned latent space. Planning is conducted top-down: the top layer optimizes the goal, and each layer's predictions become sub-goals for the layer below. When factors in the data evolve at separate time scales, higher layers discard rapid, unpredictable details while retaining slower, task-relevant states. In four simulated navigation and manipulation environments, hierarchical planning outperformed flat JEPA; in the Visual AntMaze task, a three-layer structure increased the success rate from 18% to 73%, while requiring less computational power for planning. Ablation experiments indicate that the gains come from both temporal decomposition and high-level goal representation. Under inverse dynamics supervision, this method can be scaled to real robot videos in DROID.
All Comments