Text to video generation for timelapse of a potato growing. Run locally using ComfyUI and MiniMax H3, montage stitched together with Shotcut.
I started off with the idea of a timelapse of a potato growing. I started off in a “rustic farmhouse” because it was a suggested option. I eventually moved to a lab because the light staying so steady didn’t make a lot of sense in the farmhouse.
Even with a pretty simple subject, there were a lot of variables involved in this clip.
- What size is the initial potato?
- How much water gets poured in? (Hydroponics? Hydro-potato-nics?)
- How big should the vase be and how do you describe that to the video generator?
- How far down does the potato get pushed into the soil?
- What happens to the original potato?
- How many new potatoes grow?
- Why is that skillet in the background sliding across the stove?
With those variables, something always went wrong and the growth of the potatoes looked “too perfect”. Just like each clip had something wrong with them, there were some well-done components in each clip too. It would just be a matter of the stars aligning to get all of the well-done components into one video. I need a better feedback loop to instruct what elements I liked and what needed correction from the previous generation.
Montage of generated clips:
Which one was your favorite?
First Prompt#
A cinematic timelapse of a potato sprouting and growing rapidly inside a clear glass jar filled with rich, dark soil. The jar sits on a weathered, distressed wooden table in a rustic farmhouse kitchen. As the plant grows, vibrant green shoots emerge and unfurl into large, textured leaves. Soft, warm natural sunlight streams through a nearby window, creating beautiful volumetric lighting and illuminating tiny dust motes in the air. The camera performs a subtle, smooth zoom out as the plant reaches its full height. Photorealistic, 4k, highly detailed textures.
Final Prompt#
A video of a potato lifecycle in a sterile, bright, white lab setting.
A female hand wearing a blue medical glove places one potato in a rectangular glass vase, pushing it down to be covered with dirt, and lightly waters it. She then positions a sun lamp above the vase and turns it on.
Transition to timelapse: The mother potato wrinkles and shrinks as a network of white roots weaves through the dirt. A single, solitary potato stem grows upward, unfurling broad green leaves. From the roots, new potatoes slowly develop under the dirt with different start times and swell in size, becoming visible against the glass. A single stem reaches toward a sun lamp. The camera performs a subtle, slow zoom out as the plant grows. Sterile lab background, cinematic lighting, 4k, photorealistic.
