Brylka – TooDee – PanicPost

Compression (Nuke)-Part 1

I work in commercials and every day there is the need to send a previs, a demo, a work in progress or a final spot to directors/DPs/agency or clients. A common way is to use compressed files like Quicktime, MP4 or AVI. They should usually be not too big for easy downloads and be playable even on weaker hardware. The compressed clips should show what I see on my reference monitor in the best way possible.

Why can’t a 10 MB compressed file look like my 10 GB uncompressed media?

The answer is clear, but the goal should be to stay as true as possible to the original media to avoid discussions about the image/sound you want to present.

In my case I need to compress uncompressed media from systems like Flame/Nuke/AE to a good size file format for agency/client approval while staying true with your color/gamma/contrast and image framing/resolution.

I am using OS/X and the tools from Apple (mainly Compressor) to do so and ran into some strange gamma and color shift problems. Is this related to the often so called the quicktime gamma bug? – I search the net and started to investigate a little bit.

What I found out so far is that it seems there was an old (pre Snow Leopard) gamma 1.8 (MAC) vs. 2.2 (PC) shift and there is something else – ColorSync in OS/X.

The movie is a 960×540 H264 version of the Nuke script output.

I created my own color chart in Nuke and looked for the best image format to keep most of the data in my image while being compatible with most programs used in professional environments nowadays.

So… what is the best uncompressed file format that is easy viewable on any MAC/PC and supported by the programs I mentioned above? It should be OpenEXR, but I find TIFF16 the best format while testing.

Only these two formats are allowing me to export the color chart from Nuke and load it in again without any changes of color values in the image.

So my master format is TIFF16. And my image resolution is HD 1920×1080.

color_bar_v03_0001
(Download here)

This test chart has 3 colored rectangles right to the border of the frame – to check for unwanted cropping.

The 50 frame sequence has some RGB color noise (values from 0-1) for the Compressor to chew on.

Full Black and white areas, a greyscale ramp with very low dark values and another ramp that goes from black to white.

The same ramps but smaller are also there in red, green and blue pure colors.

The image shown here is a JPG, but click on it and get the 16-bit TIFF. Use „Preview“ and the „DigitalColor Meter“ programs from OS/X to verify the 8-bit values. The float value can be measured easily in the Nuke viewer for example.

The compression tool is Compressor4 – my preferred file formats are MP4 with AAC sound for sending over the internet and ProRes4444 for working in tools like FCP-X. The compressed files I review in QuicktimeX and Windows Media Player as both programs play MP4 files.

I started the test using the TIFF16 sequence from Nuke and imported that in Compressor4 and use the standard output profile called “Broadband (high) 5MBit/s” and changed some settings:

Video bitrate 20 MBit/s, 25 fps, audio stereo 48 kHz – quality high with 256 KBit/s (although I don’t test any audio here) and use 100% of original image resolution in the geometry tab.

IMAGE 2 PNG

The compressed MP4 is opened in QuicktimeX and the DigitalColor Meter let me check what happens.

The image here is a screenshot from QTx in full screen mode.

The result is brighter than the original.

The dark grey values are now: 0, 42, 63, 74, 83, 92, 98, 104, 110, 144 and 255.

The next row changes as well: 0, 119, 153, 174, 192, 206, 218, 229, 239, 249 and 255.

The full red is now (255, 0, 29); full green (0, 255, 57) and full blue (48, 0, 251).

The luminance changes are not just a gamma shift and the color shifts might come from a colorspace conversion.

In Nuke the float values are lifted as well.

Although the resulting file is changed dramatically, the preview window in Compressor 4 is actually showing always the original luminance and color values! This is not really helping! Why is that like this?

This doesn’t work like it should! Or maybe it does and I just don’t get why it has to look like this.

But for sure it is not helpful to show a graded commercial to the director/DP and expect them to “buy”

this for approval steps from a MP4 file.

Now the same again but with a PRORES4444 as a result from Compressor4.

And it will get spooky!

QTx-MP4_20Mbit.png

The PRORES4444 snapshot from Compressor4.

The result is brighter than the original, but not as bright as the MP4.

The dark grey values are now: 0, 33, 51, 63, 72, 81, 87, 94, 99, 104 and 254-255.

The second gray ramp: 0, 109, 143, 167, 185, 201, 215, 226, 237, 247 and 254-255.

The full red is now (250, 18, 19); full green (35, 254, 51) and full blue (54, 23, 252).

Now…Nuke… set the read node to sRGB and the grey levels match!

Just with some very little error. The color values changed again and full red, green and blue are not that pure anymore.

Why is that now? QuicktimeX shows a brighter result but in fact as least the grey values are the same in Nuke?

There might be two effects happening here. One is an unwanted colorspace conversion, the other should be ColorSync. Check ColorSync on Wikipedia.

Although the wiki page doesn’t say a lot about video, the apple tech specs of FCP-X, Motion5 and Compressor4 feature colorsync support. It seams to me that compressor guesses a colorspace when there is no metadata present and chooses a destination colorspace depending what the resulting format should normally have.

Okay, next round. This time I set up a HD project in Motion5 and load in the Nuke TIFF16 sequence. Then choose “Export via Compressor” and use the same output file templates again to create the MP4 and the PRORES444.

Oh and don‘t do the mistake and use the DigitalColor Meter now and measure the values inside the Motion viewer window. The color values are already altered (visually) but the luma values match the original. I repeated the same with EXR files from Nuke and the same happened. Even a full red square in Motion added to the image has the values 1,0,0 (in float) but the measured color from OS/X is different. Not to mention that this frame exported as EXR and reimported in Nuke has the right color values but the luma values are off! On an Open EXR image – wow! This is &%$§ up!

two images

The result is that for both files the luminance levels are matching 99% to the original TIFF16, but the colors are again not as bright and pure than the original. The colors don’t match the results without using Motion5 in the “middle” but they are close.

A TIFF16 from Nuke doesn’t have a ColorSync profile in their metadata, a TIFF16 from Motion5 does.

But even passing this file through Compressor4 doesn’t get the same result.

My conclusion so far:

I did a lot more tests writing out other formats from Nuke, passing through Motion5 or not but I am not able

to get a 100% matching result for both luminance and chrominance levels in the compressed formats.

And I am not sure if I should actually be able to!!!!

Until now I was not able to load my master TIFF16 or OpenEXR test chart into Compressor/Motion without adding some sort of unwanted color treatment!

I will continue to investigate…