2024/08/21 by Xie, Zhifei, Tang, Daniel, Tan, Dingwei +3 · 13 citations
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Software Engineering (cs.SE) #TsingHua University
paper · doi:10.48550/arxiv.2408.11788
Current video generation models excel at creating short, realistic clips, but struggle with longer, multi-scene videos. We introduce DreamFactory, an LLM-based framework that tackles this challenge. DreamFactory leverages multi-agent collaboration principles and a Key Frames Iteration Design Method to ensure consistency and style across long videos. It utilizes Chain of Thought (COT) to address uncertainties inherent in large language models. DreamFactory generates long, stylistically coherent, and complex videos. Evaluating these long-form videos presents a challenge. We propose novel metrics such as Cross-Scene Face Distance Score and Cross-Scene Style Consistency Score. To further research in this area, we contribute the Multi-Scene Videos Dataset containing over 150 human-rated videos.