Adobe to Launch Video Generation Function Driven by Firefly Video Model

Adobe has announced plans to launch a video generation function driven by the Adobe Firefly Video model before the end of this year. Users will be able to experience this feature in the Premiere Pro beta app and on a standalone website.

Adobe is currently testing three features internally: Generative Extend, Text to Video, and Image to Video, and will open them to public beta in the near future.

The Generate Continuation feature allows users to extend existing videos by two seconds and will be embedded in the Premiere Pro beta application later this year. The Text-to-Video and Image-to-Video models can generate five-second videos based on text prompts or input images, and this feature will also be launched on the Firefly standalone website later this year. Adobe said that the video generation time may be further extended in the future.

Generative AI models have emerged in recent years, which can generate video clips in minutes, while manual creation takes hours. However, these early tools are generally considered to have poor controllability and are difficult to use in professional creation environments.

Adobe’s Chief Technology Officer (CTO) Ely Greenfield said that Adobe believes controllability is a major advantage of its Firefly AI tool. According to the company, there is “huge demand” for Firefly AI tools that can complement or accelerate existing workflows. For example, Greenfield noted that the Firefly Generate Fill feature added to Adobe Photoshop last year is “one of the most used features we’ve launched in the last decade.”

Additionally, Adobe says Firefly’s text-to-video feature excels at spelling, which is a frequent challenge for AI video models.

Adobe is taking a cautious approach in the early stages for safety reasons. Firefly’s video model blocks the generation of content such as nudity, drugs and alcohol. Adobe’s video generation model will not be trained on public figures (such as politicians and celebrities).

Adobe has not yet disclosed pricing for these AI video features. For other Firefly tools, Adobe provides Creative Cloud users with a certain number of “generation points”, one point can usually generate one result, and higher-end packages can get more points.