… to integrate vision-language models and text-to-video models to enable video language
planning (VLP), where given the current image observation and a language instruction, the …

… NaVid, a video-based large vision language model (VLM), to … an on-the-fly video stream
from a monocular RGB camera … Moreover, our video-based approach can effectively encode …

… We consider the video generator as a generalizable planner that envisions how … video
generation method that combines language and gesture instructions to create robot action plans …

… We extensively evaluate Plan-X on a challenging video generation benchmark covering
text-to-video, image-tovideo, and video continuation tasks. Across all settings, Plan-X achieves …

… While vision-language models (VLMs) and video … video planning with geometrically grounded
robot execution for zero-shot long-horizon manipulation. At the high level, a VLM planner …

… between text and video plans) and spatial consistency (ie subsequent video steps must …
We follow this alignment to generate the relevant task videos which compose the Video Plan…

… and camera motion planning in a shared 3D space, a key requirement for cinematographically
coherent video synthesis (see Fig. 1). To train the LLM motion planner, we construct a …

… Our language planner generated the following series of instructions: ”Pick and place the
purple F on the rightmost quarter,” ”Pick and place the green R on the left of the purple F ,” ”Pick …

… Long-form video understanding represents a significant challenge within computer vision, …
for long-form video understanding, we emphasize interactive reasoning and planning over the …

… planning and grounded video generation. Specifically, given a single text prompt, we first ask
our video planner LLM (GPT-4) to expand it into a ‘video plan’… this video plan, our video gen…