Runway vs Pika Labs — which is the best AI video tool?

Test 1: Drone footage over a forest fire

This was a simple text-to-video test on both models, using default settings and no custom camera motion. It presents a chance to test the raw output of the underlying model.

I used the prompt for this test: “Drone footage flying over a forest fire. Photorealistic. Flames leaping towards the camera.” This tests different aspects of the model including how it handles the camera motion from the drone, the flames moving and the visuals of trees and fire.

Runway was an easy and clear winner on this test. The flames were more natural and it felt closest to a real video. Pika seemed to struggle with realistic flames and it threw the video off.

Test 2: A Yeti walking in snow

This was a chance to see how well each of the two AI video tools handled animation character movement within a scene. Again it was text-to-video with default settings but with a 4-second extension to see how the models handle consistency. 

The prompt was created by ChatGPT as I wanted to see how well one AI could instruct another, suggesting they visualize: “An imposing, majestic Yeti, towering at eight feet tall with a thick, shaggy coat of white fur that glistens in the sun, is captured mid-stride as it traverses a narrow, snow-covered mountain pass in the heart of the Himalayas during mid-winter.”

The overall scenery for the Runway creation was more photorealistic, but the character animation from Pika Labs was better. I think overall Pika Labs wins this round.

Test 3: A future city from an image

For this test, I pulled in an image as the prompt rather than using just text. It was a text and image to video but all other settings kept on default. The image was generated in Night Cafe Studio using the SDXL 1.0 model from StabilityAI.

The prompt for the video: “The camera moves through the city, showcasing towering skyscrapers with holographic advertisements, flying vehicles zooming past, and a bustling, technologically advanced urban landscape.”

Neither provided an outright victory in this round but I’m giving it to Pika Labs as it produced animation closer to the description and the point is a test of motion. It was close though as Runway generated a crisper video closer to the original image.

Test 4: Multiple characters in a frame

One of the areas all generative video models struggle with is multiple characters moving within a single view. For this experiment, I once again used both text and image but ramped up the motion level on both by two points. I didn’t change any other settings.

The prompt for both image and video: “Norman knights charge against Saxon shield wall. The camera pans over clashing swords and spears, focusing on William the Conqueror leading the charge, with Harold Godwinson defending.”

Neither won this round. It’s not because the output is similar, as it is surprisingly different given both come from the same source image and text. The issue is that as with all AI video models, neither coped well with multiple characters. Runway would have won if I were using their multi-motion brush feature as it allows you to set motion by region.

Test 5: Fish swimming in a clear sea

For this test, it was all about the text. A well-formed prompt designed to see how well each of the models handled complex, but less cluttered motion against a simple environment. 

The prompt: “A vibrant coral reef teeming with marine life. The scene is from the perspective of slowly gliding through the water, with colorful fish darting in and out of the coral, and a gentle current causing the sea plants to sway.”

I liked both approaches here and they were distinct from one another. Both also did a great job on diversity of motion, with Pika Labs going down the more diverse but simpler lane and Runway adding considerably more fish into the scene.

It was a close call but I gave it to Pika Labs because the Runway video was less consistent in its character motion. Some of the fish merge and others seem to move backwards.

Test 6: Unique feature test

For the sixth test, I once again turned to image-to-video. Specifically, I had Leonardo.ai generate an image of an alien creature overlooking a vast expanse. 

The goal of this test wasn’t so much how it handled the motion, as the type of image is normally relatively easy for AI video tools — minimal movement, large background and few characters.

The test here was one of the features unique to each of the models and which proved more useful. For Runway it was Multi Motion Brush and for Pika Labs I turned to Modify Region.

While I think Modify Region is an impressive feature, especially if you only want to change a small aspect of a source image, Runway’s multi-motion brush is a game changer and that is why I’ve given this round to Runway.

Test 7: Video-to-video

This final test is one of the video-to-video features available from each model. Runway has a slight edge going into the test as it has a dedicated video-to-video model that includes a range of style options including from a text, image or preset prompt.

Both models work by overlaying the uploaded source video with the style from your prompt. For Pika Labs this is only available from a test prompt, although you could use Modify Region to set a specific part of the frame to be changed.

As I had the Yeti on my mind from an earlier test I gave it the prompt: “Yeti as a YouTuber,” with a video of me talking to the camera as the source material. Neither did a great job so, to be fair to the test I tried again with the prompt: “Futuristic android character as a YouTuber.”

I’ve used video-to-video models before and they’ve managed to make the mouth move in time to the words being spoken but this time around Runway struggled. However, visually it fit the brief well and mirrored the source video movement. 

Pika Labs failed to create a well-formed character, kept flipping me upside down and didn’t change the overall view by much. Runway was the easy winner in this round.