Ivan Larionov's profile

Animation using Neural Networks

Animation (by neural networks) of paintings by Ivan Aivazovsky
2020

I love the work of Ivan Aivazovsky. And I really wanted to rethink his work with the help of technologies created in recent times. Seems, this video can be considered my first experience in motion design.
The Deep Dream neural network is a tool with great potential for use in design. Not only can it be used to create images that are completely different visually and identical in structure, but also and you can pass two images through the network and get an interesting overlay or texture replacement at the output. The more I studied the technical side of the issue, the more I wanted to try.

To begin with, I made a cover to test and understand in what directions I will move in this project and what techniques I will use when linking the original and processed pictures. It was necessary to use something that would look appropriate and show both the original image and the final result.

As a result of several tests, I came to the choice of inverting and dividing the image into sectors according to the rule of thirds, where each sector was displayed
would be with successively increasing opacity.

After such preliminary work, all I had to do was pass the images through the neural network, animate them and combine them into a single video. Neural network processing and image animation is a fairly simple thing, but the final assembly - not at all. What were the difficulties?

The first and most important issue is the music. I really wanted not to simply switch pictures as in a slide show, but to convey the atmosphere and sensations of the vast expanses of both the seas and oceans, and technologies. About that time
At the same time, I found a wonderful composition "HAEVN - We are" in an orchestral performance. The track is incredible, inspiring and uplifting. But it does not convey the feeling of "modernity", manufacturability and conditional cyberpunk.

So I decided to write my composition. I have some experience in writing music: I released one small music album and wrote music for two game projects. When writing my composition, I tried to capture the general mood with the track mentioned above, as well as add microsound and glitch at the moment when processed images began to appear in the video.

The result was a video that conveys all the emotions that I experienced when I heard the phrase "endless expanses" and the sound of white "sea" noise. And, of course, showing my delight from Deep Dream.
Animation using Neural Networks
Published:

Owner

Animation using Neural Networks

Published: