Turning Photos into 2.5 Parallax Animations with Machine Learning

For years, filmmakers have used 2.5D parallax to make static photos feel more dynamic, as in The Kid Stays In The Picture, the 2002 documentary about film producer Robert Evans that popularized the technique.

Traditionally, a video editor would use Photoshop to isolate the photo elements on separate layers, fill in the removed objects to complete the background, and animate the layers in a tool like After Effects. Here’s a typical tutorial, showing the time-consuming and tedious process.

Last September, a team at Adobe Research released a paper and video demonstrating a new technique for animating a still image with a virtual camera and zoom, adding parallax to create what they call “the 3D Ken Burns effect.”

This new technique uses deep learning and computer vision to turn a static photo into a 2.5D parallax animation in seconds, using a neural network to estimate depths, render a virtual camera through space, and fill in the missing areas.

On Monday night, researcher Simon Niklaus finally got permission to release the code, posting it to Github with a CC-NC license, allowing anyone to experiment with it for themselves.

Sample Animations

It’s incredibly fun to play with. I ran some famous images through it, and then put a call out to Twitter for more ideas. Here are the results. Click anywhere on the video to play it.

John Rooney/Associated Press, Ali vs. Liston (1965)
Alfred Eisenstaedt, V-J Day in Times Square (1945)
Elvis Presley and Richard Nixon (1970)
Pete Souza, Situation Room
Matt McClain/Washington Post, Gordon Sondland testifies
Disaster Girl
The Unexplainable Picture

Surprisingly, it even works on illustrations and paintings.

Martin Handford, Where’s Waldo
Georges Seurat, A Sunday Afternoon on the Island of La Grande Jatte

Try It Yourself

Unlike the Spleeter library, the Ken Burns 3D library requires using PyTorch with an Nvidia GPU with CUDA drivers. Sadly, Apple phased out CUDA support in Mojave, but there’s an even easier way to play around with it.

I created a Google Colab notebook here, which will let you process images on Google’s GPUs entirely in your browser.

If you’re unfamiliar with Colab, it can be a bit intimidating. Try this, and let me know if you get stuck.

  1. Open the Google Colab notebook.
  2. Click File->Open In Playground Mode to run it yourself.
  3. Click “Connect” to connect to a hosted runtime, a temporary Google server where the commands will run.
  4. From the “Runtime” menu, click “Run All.” A warning will pop up, you can click “Run Anyway.”
  5. On the left-hand side of the window, click the tab to open the Files sidebar.
  6. The final command processes the “doublestrike.jpg” sample image, and generates a new file in the /images directory called “autozoom.mp4.”
  7. Upload your own images by right-clicking the “images” folder and clicking Upload. Change the input/output filenames, and click the play button to run the final command again.

Good luck!

Update: This Google Colab notebook by Manuel Romero is much faster and easier to use, with a handy widget to upload files, process images in bulk, and download all the finished animations.


    Thank you for your article- and also for realisation as Colab notebook! The possibilities are so wonderful, even if sometimes a little bit buggy.
    I’ll write a blog about it!

    Can I run this on my computer? I have a RTX 2060 graphics card and 16GB of RAM. I don’t know anything about coding though.

    You don’t need to know how to code, but Mac won’t run CUDA drivers and installing all the prerequisites in Windows 10 can be a bit tricky. There’s one tutorial here, which will set you up for the instructions in the Ken Burns 3D Github. If this all sounds overwhelming, you might want to just stick with the Google Colab notebook.

    Thanks for the response. This is beyond me so I’ll stick with the Colab. How would I run the depth estimate function in Colab? The command is listed as [python depthestim.py –in ./images/doublestrike.jpg –out ./depthestim.npy]

    Starting with Manuel’s Colab notebook, just click the “+Code” button to add a new cell and paste in the command preceded with an exclamation point. I just tried running a cell with “!python depthestim.py –in ./images/doublestrike.jpg –out ./depthestim.npy” and it generated an .npy and .png file in the 3d-ken-burns folder with the depth map and estimate.

    In both notebooks, I get error: urllib.error.HTTPError: HTTP Error 403: Forbidden
    Can you verify that these notebooks still work?

    I’m getting this error

    Traceback (most recent call last):
    File “autozoom.py”, line 11, in
    import gevent

    Hi Andy – your instructions are very helpful. I’m with you up to “The final command processes the “doublestrike.jpg” sample image, and generates a new file in the /images directory called “autozoom.mp4.””

    What is “the final command”? Thanks!

    The final command in the Colab notebook is “!python autozoom.py –in ./images/doublestrike.jpg –out ./images/autozoom.mp4”. When that runs, it will generate the .MP4 file.

    Gottit. And (sorry to be a complete ignoramous, I have zero code knowledge) in which fields do I change the input/output filenames?


    Did I do something wrong. I didn’t get the mp4. There is also an error about a gevent module missing. What to do?

    Traceback (most recent call last):
    File “autozoom.py”, line 11, in
    import gevent
    ModuleNotFoundError: No module named ‘gevent’

    Its working and it looks very cool! Is the a way to modify the zoom length, direction and speed? I’d like to see if I can use it in a documentary I’m helping to make.

Leave a Reply

Your email address will not be published. Required fields are marked *