Luma releases a new AI model that lets users generate a video from a start and end frame

Luma, the a16z-backed AI video and 3D modeling company, has a new model called Ray3 Modify that allows users to modify existing footage by providing character reference footage that preserves the performance of the original footage. Users can also specify a start and end frame to guide the model in generating transition images.

The company said Thursday that the Ray3 Modify model solves the problems of preserving human performance while editing or generating effects using AI for creative studios. The startup said the model better tracks input images, allowing studios to use human actors for creative or brand images. Luma said the new model preserves the actor’s original movement, timing, eyeline and emotional delivery while transforming the scene.

Ray3 Modify allows users to provide a character reference for transformation to the original footage and convert the human actor’s appearance into that character. This reference also allows creators to maintain information such as costumes, likeness, and identity throughout the shoot.

Moreover, users can also specify start and end reference frames to create a video using the new Ray3 Modify model. This is useful for creators when directing transitions or controlling the movements or behavior of characters while maintaining continuity between scenes.

“Generative video models are incredibly expressive, but also difficult to control. Today we’re introducing Ray3 Modify, which combines the real world with the expressiveness of AI while giving full control to creatives. This means creative teams can capture performances with a camera and then instantly adjust them to be in any conceivable location, change costumes, or even go back and reshoot the scene with AI, without recreating the physical shoot,” said Amit Jain, co-founder and CEO of Luma AI, in a statement.

Luma said the new model is available to users through the company’s Dream Machine platform. The company, which competes with the likes of Runway and Kling, is coming out video modification capabilities in June 2025.

The model release is on the back of a new financing round of $900 millionled by AI firm Humain, owned by Saudi Arabia’s Public Investment Fund, for the startup announced in November. Existing investors such as a16z, Amplify Partners and Matrix Partners also participated in the round. The startup also plans to build a 2GW AI cluster in Saudi Arabia together with Humain.