Motion-Conditioned Image Animation for Video Editing

November 30, 2023 · Entered Twilight · 🏛 arXiv.org

"No code URL or promise found in abstract"
"Derived repo from GitHub Pages (backfill)"

Evidence collected by the PWNC Scanner

Repo contents: CODE_OF_CONDUCT.md, CONTRIBUTING.md, LICENSE, README.md, dataset, index.html, paper, requirements.txt, videos

Authors Wilson Yan, Andrew Brown, Pieter Abbeel, Rohit Girdhar, Samaneh Azadi arXiv ID 2311.18827 Category cs.GR: Graphics Cross-listed cs.AI, cs.CV, cs.LG, cs.MM Citations 20 Venue arXiv.org Repository https://github.com/facebookresearch/MoCA ⭐ 20 Last Checked 2 months ago

Abstract

We introduce MoCA, a Motion-Conditioned Image Animation approach for video editing. It leverages a simple decomposition of the video editing problem into image editing followed by motion-conditioned image animation. Furthermore, given the lack of robust evaluation datasets for video editing, we introduce a new benchmark that measures edit capability across a wide variety of tasks, such as object replacement, background changes, style changes, and motion edits. We present a comprehensive human evaluation of the latest video editing methods along with MoCA, on our proposed benchmark. MoCA establishes a new state-of-the-art, demonstrating greater human preference win-rate, and outperforming notable recent approaches including Dreamix (63%), MasaCtrl (75%), and Tune-A-Video (72%), with especially significant improvements for motion edits.