AVATAR Fusion Camera System

Here is a good video showing the 3D camera used to film Avatar:

I also found this image of some other 3D cameras with some weird old people

PRIMITIVE HMC150 Camera / AVCHD Pipeline

Recently I’ve upgraded my Standard Def DVX100B to its HD big brother, the HMC150. I bought the HMC150 brand new for $3300 and I plan to resell the DVX100B on Ebay. Here is what my camera rig looks like now with my Letus35 Depth of Field adapter:

HMC150 Features

I really love this camera and would highly recommend it to any independent filmmaker. Here are my favorite features:

• First, The HMC150 does NOT have a rolling shutter. As you may know, cameras with a rolling shutter are completely useless for VFX work, especially when matchmoving is involved.

• The camera records onto tiny SDHC cards. These are the same size as the SD cards in your regular still camera. The HC stands for High Capacity. At the highest quality, an 8GB SDHC card can hold up to an hour of 1080P footage. I realize it’s highly compressed, but still, how do they do that?! Make sure to get Class 6 or above SDHC cards and always format the cards in camera.

• The recordings are file-based, which means no more “capturing” from tape. Just copy the files to your hard drive to start working.

• The focus assist feature is awesome. It zooms the image in the viewfinder to 100% so you can focus without having an external monitor around.

• There is also a built-in waveform monitor allowing you to set exposure without having an external waveform monitor around.

AVCHD Codec to Image Sequence Pipeline

First of all there are a couple of ways to copy the files off the camera onto your computer. My preferred way is to use an SD card reader, kinda like the one in this picture:

The HMC150 records in a custom AVCHD codec. Not all editing software can recognize it. Because of that sometimes the AVCHD files need to be converted to a different file format before they can be used in the editing package.  The idea for a VFX pipeline is to convert the AVCHD files to image sequences. (I prefer PNG image sequences because each image is lossless compressed.)

The first thing I tried was the command line tool ffmpeg. This is a great tool which can read and write most codecs, including AVCHD. However, I found that when converting the AVCHD files to PNG image sequences, there was a slight gamma shift in the image. I tried converting to other file formats and the gamma shift was still there. So, don’t use ffmpeg for this.

Adobe Premiere Pro CS4 caused the same weird gamma shift.

Finally, I tried using Sony Vegas Pro, which supports AVCHD files natively. Exporting image sequences out of Sony Vegas Pro worked perfectly and did not cause a gamma shift. However, there was still a small problem. The image sequences were named incorrectly, like:

myImage_1.png, myImage_2.png…

instead of:

myImage.0001.png, myImage.0002.png…

So I wrote a quick Python script to rename the image files frame padding properly.

In conclusion, the best pipeline I found for converting AVCHD files to image sequences is:

AVCHD file > Sony Vegas Pro > image sequence > framePaddingRenamer.py

If you’ve got an easier pipeline, please post it in the comments!

AVATAR

What can I say about Avatar? Go see it! Gretta has compared it to the Vegas acrobatic show Le Reve, which is another “life changing and humbling experience”. Here are my notes:

• Do not under any circumstances go see Avatar in 2D. Watching it in 2D is like watching a color film in black and white. Go see it in 3D the way it was meant to be seen even if you have to fly to the theater!

• What about the facial rigging and animation? It’s very good!

• There are sections of James Horner’s score lifted directly from Sergei Prokofiev’s score for Alexander Nevsky (1938)? And you thought no one would notice, James

• The most “unrealistic” thing about the movie is the ending. But I guess that is to be expected. I just hope if I ever get to make films that I won’t cop-out like that!

• Will Avatar change filmmaking forever? Let me put it this way: Welcome to 21st century filmmaking.

• Michael Bay, you fucking idiot, how does it feel to get schooled like that?

Gretta and I went to see Avatar with Paul Debevec and friends at the Pacific Culver Stadium. Appropriately, we took a 3D picture of everyone with my W1. Here it is! (Use the mirror method to view.)

PRIMITIVE 3D Scanning

We just got back the data from a recent 3D scanning session with our lead actor, Bill Oberst. The scans were done by the good people at Mimic Studios. We did a full body scan and a closeup facial scan. These scans are crucial to the PRIMITIVE VFX pipeline because they will become the basis for our Neanderthal model and rig. A big thanks to everyone who made these scans possible!

Here is a screenshot of the full body scan (1 million polys):

La Placita Olvera

A few 3D pictures from Olvera Street and Union Station in Downtown Los Angeles.

My digital 3D camera!

Some of you know that I am a stereoscopy enthusiast. I’ve been taking 3D pictures for over 10 years. Unfortunately, stereoscopy went out of style in the 1950s and almost no consumer stereo cameras have been produced since then. The digital photography age definitely left stereoscopy behind…until now.

Finally, Fujifilm has created the world’s first consumer digital 3D camera, the Finepix Real 3D W1. And guess who’s got one! As soon as I heard about it, I slapped my $600 down and ordered it straight from Japan. (My birthday present to myself.)

I’m not going to do another review of the W1, but I will show you my pipeline for taking, processing, and viewing 3D images.

Taking 3D pictures with the W1 is as easy as using any other point and shoot camera. However, there are a few cool features of the W1 worth noting:

• The LCD screen on the W1 is actually a lenticular 3D display! It shows a real time 3D preview before you take the picture and can play the pictures you’ve taken in 3D as well.
• If you press the shutter-release button half way before taking the picture, the W1 will automatically adjust convergence for you! That is, it will align the subjects in both left and right images to be on top of each other. Doing so sets the subject “to window”, which is what you want for comfortable 3D viewing. Adjusting convergence is normally a post process, but the W1 does it before you even take the picture!
• The W1 can also record 640×480 3D videos at 30 fps.

I do all my 3D picture processing in a wonderful free application called StereoPhoto Maker. You can download it here. This little program allows you to do anything you’ll ever want to do with 3D photos, including editing, viewing, and batch processing.

I mostly use this program to batch convert 3D photos saved in W1’s proprietary MPO format to side-by-side JPG images with the left view mirrored (more on this later). Here are the settings I like to use under File > Multi Conversion:

There are several methods of viewing 3D and some methods are better than others. Anaglyph (red/cyan) images is one of the worst ways to view 3D. Other methods require you to cross your eyes or buy expensive shutter glasses. Fujifilm will have you spending $6 per print for lenticular prints from the W1! No thanks.

In my opinion, the BEST WAY to view 3D images on a computer screen is the mirror method. The mirror method allows you to enjoy full color 3D images with no ghosting. You can use the mirror method with a single monitor, but if you have dual monitors it works even better!

To use the mirror method, all you need is a small mirror. You can use a handheld makeup mirror (one that doesn’t enlarge the reflection). I prefer a mirror with a base I can set down on the table. Position the mirror in the middle of the stereo pair and perpendicular to it. In all my stereo pairs the left image is mirrored, so the mirror should be facing left. Finally, get your eyes close to the mirror and look at the right image. If all goes well, you should be able to perceive the 3D effect because your right eye is seeing the right image and your left eye is seeing the left (mirrored) image. You may need to rotate the mirror about the base ever so slightly to achieve comfortable convergence–doing this is normal.

Here is a 3D image of me showing the setup with dual monitors. I stretched/maximized my image viewer/browser across both screens and I’m looking at the right screen. Notice that the monitors are slightly angled together–this helps to see more of the left image in the mirror. The setup with a single monitor is the same, but you’ll only be able to use half of your screen to see the 3D effect.

I hope I’ve inspired you to experiment with stereoscopy. There are many 3D galleries to enjoy online. Here are some more 3D pictures I’ve taken with the W1. I’ve made them big enough for dual monitor viewing. Many more to come!!

For those of you interested in the future of 3D filmmaking, I found this list of James Cameron’s 10 rules for good stereo.

Digital Emily TED Talk

My friend Paul Debevec gave a TED talk about the Emily Project! Paul is the only person I know who could condense all this information into a 6 minute talk. Brilliant!

More of Paul’s exciting work: http://www.debevec.org/

More about the Emily Project: http://gl.ict.usc.edu/Research/DigitalEmily/

PRIMITIVE Costume Design

I’m a man of many talents; Costume Design is not one of them. That’s why I hired my talented wife, Gretta, to be the Costume Designer on PRIMITIVE. Her job is to design and create the costumes for the Neanderthal and the 2 Cro Magnon boys.

At first we only knew the time and place: Europe, forty thousand years ago. Unfortunately there is no fashion record going back that far, so any designs we’d come up with would be pure artistic speculation. But we did have a few constraints to get us started:

• No metal.
• No woven materials.
• The Neanderthal and Cro Magnon costumes had to look significantly different. I wanted the Cro Magnon costumes to be more colorful than the Neanderthal’s.

Naturally the first place we looked for inspiration is photographs of tribal people. Gretta found several good images, books, and websites:

• http://www.dailymail.co.uk/femail/article-516490/Out-Africa-The-incredible-tribal-fashion-inspired-Mother-Nature.html
• http://www.amazon.com/gp/product/0500288054/ref=ox_ya_os_product
• http://www.amazon.com/gp/product/0224075144/ref=ox_ya_os_product

For the boys, we immediately fell in love with the photographs of Hans Silvester (first and second link). We decided that the boys would be camouflaged with plants, vines, and face paint. In other words, they would wear nothing made from animals.

On the other hand, the Neanderthal’s costume would be made entirely from animal materials. Or as William Lambeth puts it: “From shit he’s killed.” Whenever I think of a Neanderthal, I think of Frank Frazetta’s version–an absolute badass:

Gretta drew several iterations of the costume designs. The latest are posted below. (Yes, that is a bear’s head on the Neanderthal’s crotch.)

Her earlier designs are posted below. We had an idea that the Neanderthal would be covered with Maori tattoos. This ideas seems unlikely at this point because the Maori patterns are too sophisticated for 40,000 years ago. However, we haven’t ruled out simpler scarification designs.

Please post your comments!

PRIMITIVE Greenscreen Test

Originally I planned to have set builders create a realistic “cliff set” for the actors to interact with and then do matte painted set extensions in comp. The cliff set would be arduously sculpted and painted to look like a cliff and would take a significant portion of the budget. However, the following greenscreen test proves that I can get away with using a simple “green set” for the actors to interact with. This green set would be just a plywood set painted green. It would be easier to construct and cost much less.

Here is the final result of the test:

Now let’s break it down. Why was I so afraid of having a simple green set in the first place? Take a look at the plate below. The subject is touching the screen. Which means that the subject is casting a shadow on the greenscreen and the greenscreen is bouncing light on the subject! This is basically the absolute worst case scenario for greenscreen work.

So the idea here is to extract the subject element and the shadow element and comp them over some background. The first thing I did was pull two keys: one for the lighter portion of the greenscreen and one for the shadow portion.

I color corrected the plate using Lab curves to suppress the green spill on the subject’s forehead and arm. I multiplied the two greenscreen keys together; this gave me an alpha for the subject. Then I premultiplied the color corrected plate by this alpha:

Now for the shadow. I multiplied the shadow key over the background at 90%, but it looked really flat and boring. So I decided to use an ambient occlusion pass; this was just a threshold of the darkest areas of the plate. I multiplied the ambient occlusion with the shadow and the shadow immediately came to life.

Finally, I comped the subject over the shadow. The edges were looking really crunchy so I blurred the edges masked by a contour pass (derived from the subject’s alpha).

While the final result still needs a lot of roto/paint cleanup, I’m happy with the overall look. What do you think?

PRIMITIVE Auditions

We held our auditions recently for the roles of the Neanderthal and the two Cro Magnon boys.

We saw about 8 actors for the Neanderthal. Out of those, only 4 were candidates. But in the end, the clear winner was Bill Oberst. He is a tremendous actor with the perfect body and face for this role. We are really lucky to work with him!

We also saw 3 boys. Only 1 was good, so we will have another audition to find more boys soon.

True story: On the day of the auditions we were looping the PRIMITIVE previs on a laptop in the actor’s waiting room. During one of the boys’ auditions, I asked the boy (who was around 10 years old): “Did you see the movie playing outside? Did you like it?” The boy said: “Yeah! It’s so funny! My favorite part is when they hit him with a stick!”