Creating a stop motion video

My colleague, a talented photographer and editor named Amit Zinman recently created a spectacular stop motion music video. I asked him to write an article on how he did the filming and editing. The production was somewhat challenging, as reflected in this article by Amit.

Here’s a look behind the scenes on preparing the following clip

People always ask me what comes first, the song or the idea for the music video. Truth is that I usually have a great answer ready though it might not really be related to what actually happened. At the end of a long and complex project it is difficult to remember which came first. What I do recall is that I was using, almost exclusively, Adobe applications to make this. Sure, there are possibly better tools for creating stop motion. However, instead of learning new tools and interfaces and spending more money on them, I decided to use the tools I already had and knew how to work with. Since Adobe’s tools tend to be really diverse, I first had to decide which tool I was going to use to create the animation itself. Though Premiere and Photoshop can create stop motion animation, I eventually decided to use After Effects as my main animation tool, and let the other apps do what they were originally intended for, editing and photo-retouching.

The first tool I was using was of course, my head. The main inspiration for doing the music video by Laura Marling song. It is part of a series of unofficial music video done by a group called music and muffins. They used childlike drawing to express songs of a few artists. Something in this particular music video caused to go back to it over and over, and I knew I wanted do something similar. Unfortunately, drawing is not really one of my strong suits.

Furthermore, there was something a little bit static in the format that I thought would not fit an official music video. I knew I gad to find a way to make it more dynamic. What I went on Sivan Abelson’s myspace and heard “I’m waiting for him”, I knew I found a wonderful song that I could develop my idea for. The song inspired me to combine painting on a naked exposed body with naïve painting. It seemed like the right solution for expressing the words of a song describing a young woman’s desire for naïve sort of romantic love.

I started looking for someone to create the body at work for the music video. I got high recommendations on Moran Newman, who just started studying in the Bezalel academy of arts and design, so had a sort of a tight schedule for doing this. I also started working on a storyboard, visual representation of how the music video should ultimately show.

The storyboard for the music video using the Windows Paint, though I am perfectly proficient with the more suitable Photoshop. There was no technical reason for this. I simply wanted to use the most basic tool that was at my disposal and focus on thought rather than technique. I created with my limited drawing capabilities what I wanted for the shots in the main parts of the music video to present. I created various jpeg files, each one showing the action happening in a shot. I printed a page with all the shots which we used during shooting to make sure we had done everything according to plan. Though you might find some resemblance to the finished music video, we’ve made a lot changes to the initial plan and added a few more shots when we discovered that my original plan for the final part of the music video was not going to work. I then named the jpeg files after the shot numbers and imported them into Premiere Pro, edited them with the song to see if they were working with the beat and the song’s mood. Then I exported this initial version and sent it to Sivan and Moran for approval. I also used this version as a baseline for editing the final shots into the sequence timeline.

In order to shoot the stop motion frames I used the Sony A580 DSLR camera, shooting in RAW mode. I used a Manfroto tripod and took most of the shots with 35mm 1.4 Samyang lens. We did use Moran’s Canon camera for some of the shots because I had some technical problems with my camera on one of the shoots.  I cannot stress this enough: you should always have backup plans for all the components of a production. I also can recommend cleaning the lenses really well when doing stop motion, so you won’t find yourself later retouching all multiple frames in Photoshop if you don’t have to. Trust me on this, we are talking of cleanup for thousands of frames. Most of the shots were taken over a body lying down so the quality of the tripod was really crucial. Of course, having some upper grip for holding my camera could have been nice, but would have been difficult to setup. It would also require an external monitor connection and might have proven less flexible when I had  to change angles and positions. Instead, our model was down on a mattress covered in a black sheet and the camera was placed above her, perched on a spread tripod over the mattress.

As for lighting, we used fluorescent as key light and a 800 watt light to bounce light of wall or floor to light the other side. I used a regular halogen reading lamp placed near the limb shot to add some additional lighting. Moran drew each shot, sometimes using a cloth to erase the current drawing and then re-draw the next “step” in the shot. One of the critical things to remember when shooting is to use the camera in manual-only mode. I set my camera to M mode, but soon discovered that I also had to change some other options that the camera was using to enhance the dynamic range because it caused shots to look different, even when I wasn’t changing the lighting, aperture or iris.

After wrapping up the shoot I transferred the files to my compositing machine for ingest using Lightroom.

Lightroom allows you to change the look of all the frames in the shot, leveraging the RAW capabilities of stills cameras. In the example above you could use the Lightroom “Develop” module to increase the blacks value, effectively reducing the detail in the black area while preserving the legs and drawings. You could also delete frames that you do not think will be useful. While the stills RAW format is really flexible and has better quality that most video cameras, I would not recommend creating animation from RAW sequences as they tend to be particularly heavy. Furthermore, the resolution of a typical DSLR camera is much higher than the one needed for video. The stills shown in the screenshot above are 5184 by 3456 while the final movie was to be 1280 by 720 (720p). I could have chosen 1920 by 1080 (1080p) but I believe that at the current stage of technological evolution, 720p is good enough for home use, especially for viewing clips online.

After some deliberations regarding the work format with I decided on using PSD sequences. This format preserves 16-bit color space (most video cameras shoot 8-bit) and allows adding layers to individual frames. This was a critical feature form me as I had to add elements to the animation or touch up the frames quite a bit, while still retaining the ability to roll back the changes if they did not work as planned in the animation. If you don’t have Light room. converting RAW sequence from RAW to PSD can also be done using Bridge. Bridge actually launches Photoshop to do the conversion, so it is of course much slower. After exporting from Lightroom I got a separate folder for each PSD sequence. Before committing it for use in the animation, I decided to preview the animation using the freeware Irfanview graphic viewer. Pressing the previews the animation and allows for easiy deletion of frames that are unneeded for the animation.

Inside After Effects, I imported the PSD files as separate layers rather than a video sequence. I dragged them into the create new composition icon like this:

Once the composition was created I changed the frame rate to 12 frames per second and changed the length of the layers to a single frame.

From the Animation menu, I selected Animation > Keyframe animation > Sequence Layers to have the layers display one after the other.

At this stage the animations is ready to be played inside After Effects to see if it “works”. The improvements in the most recent versions of After Effects allows for easy and fast preview by pressing the spacebar key. If it is not as responsive as you might hope you could try and reduce the preview resolution.

One of the issues I encountered in most of the shots that my canvas, that is the female body, was not a stationary one, as models tend to move their body and not stay frozen in space. This problem was especially evident in shots where the model was sitting or standing. In order to create a working shot that was not too dynamic, I aligned the shots according to the drawing or the limb as necessary. To do that I took a single reference frame, changed the frame’s layer to an opacity value of 50 percent and dragged the right side of the layer so that it would cover the length of the composition, and locked it. This enabled me to move the rest of the frames so that they would match as much as possible the reference frame.

For example, in the following shot I used the location of the legs in the first frame to stabilize the shot. It wasn’t perfect but enough not to distract the viewers while watching the music video.

Once the animation was looking good, I placed the initial composition into a new 720p 25fps comp, keeping in mind that at some point I would export it to Premiere Pro to edit it into the music video. This composition was also the one where I performed time-remapping, sometimes creating new interim frames by using frame blending I f the animation was going to quickly. As you can see in the above screenshot, while the sheet that the legs are on is black, it still lighted by the same light that is used to light the legs, looking like a gray uneven mass.  I was going for more of of a unified matte look as can be seen thorough the music video. We also had the problem of tough-to-clean white smudges appearing on the black sheet.

I feared that I would have to use Photoshop or Rotobrush to clean this up, both options taking a lot of time. Luckily for me, I discovered that a simple black solid with a dynamic mask and feathering of around 80 pixels solved most of the problems in the black areas of the shot, whilst creating some nice 3D tanning to the side of the body.

As can be seen in the screen shot above, the right solid layer is switched off, so you can see the grayness of the sheet creases whilst on the left side we see an opaque uniform surface as the left mask is on. Note that the mask doesn’t really have to be accurate to work as long as you have enough feathering. It is important to note that not all corrections can be done in After Effects. For example, the first model we shoot had a birth mark on her back looking like a spot on the lens. As her body was moving along the shot, the birth mark changed its position, making it difficult to use After Effects masking techniques to eliminate it. I did try to use the After Effects clone brush but dropped it as it did look convincing enough. Also, I used Photoshop to fix frames that didn’t work as planned in terms of the animation, and to create some new frames that were missing.

I did use After Effects to do most of the color correction work to smooth the skin tone, while preserving the black lines of the animation. This was done by duplicating layers and using various keying techniques that are outside the scope of this post. Once a shot was ready, I rendered it out to the MPEG-2 format and imported it to Premiere Pro. Dynamic Link can be but for this but as I was using massive color correction on high resolution 16-bit still frames, this might have proved to be too intensive on the CPU. I did use the “Replace Clip with After Effect Composition” in Premiere Pro quite a lot, every time I had a new version come out of After Effects. I also used some color correction tools in Premiere to make shots look more alike. In hindsight, I should have probably decided on a more homogeneous skin tone throughout the music video. That said,  I am happy to report that  these problems and imperfections did not harm the enjoyment of viewers of the music video.