Understanding Nuke's Layers and Channel system

One of Nuke’s unique features is its ‘Multi-channel, high dynamic range compositing system’. What this means is that Nuke can handle over a thousand ‘Channels’ of data; red, green, blue and alpha would be counted as 4 ‘Channels’ out of a possible 1024 available. ‘Channels’ are stored in ‘Layers’ which may contain up to eight ‘Channels’ each. To help understand what’s happening take a look at the diagrams below.

layerChannelShuffle_01

A SINGLE LAYER CONTAINING 4 CHANNELS

layerChannelShuffle_02

256 LAYERS, EACH HOLDING 4 CHANNELS, MAKES 1024 CHANNELS OF DATA

This ‘Layer’ and ‘Channel’ system is unique to Nuke and gives the artist huge flexibility both working directly in Nuke and in using multi-pass files rendered from a 3D package. Let’s deal with 3D first.

Benefits Coming from 3D
Today many 3D packages and studios support the use of multi-pass rendering. What this means is that instead of rendering your images flattened, you render them in layers based on colour and light information. e.g. a render could be broken down into Ambient, Diffuse, Specular, Reflection and Shadow passes. These are then combined back together in Nuke (or any Compositing package) to create the look of the original flattened version.

layerChannelShuffle_04

FROM TOP LEFT TO BOTTOM RIGHT: BEAUTY (Flattened), EMPTY PASS, DIFFUSE, REFLECTION, SHADOW, SPECULAR

This gives the compositor a vast amount of freedom in deciding how the final image should look. A colour correction could be made to the Specular pass, for instance, or a blur added to the reflection before the passes are composited together. This is a much faster and more efficient way of working, saving both time and money because the renders do not have to be returned to 3D for re-rendering which is often a long and expensive process.

layerChannelShuffle_05

Left, ORIGINAL RENDER. Right, COLOUR CORRECTED SPECULAR & BLURRED REFLECTION PASSES

Nuke’s ‘Layer’ and ‘Channel’ system gives you the ability to work directly with file formats that can hold all this extra data. Ambient, Diffuse, Specular and Reflection passes are all colour passes which need red, green and blue to make up each pass. Shadow, on the other hand, is often just a single colour and hence needs just a single channel. So, for example, combining together Ambient, Diffuse, Specular and Reflection passes (with 3 channels each) and a single channel Shadow pass would give you a total of 13 channels. Many file formats are unable to handle this many channels, e.g. JPEG supports 3, TARGA supports 4, TIFF only 5 or so. Some file formats though have been specially designed to handle more channels, and the one most companies have now adopted is Industrial Light and Magic’s in-house format openEXR (.exr), Currently at Version 2.0. This file format can support up to 1024 channels of data and has the added benefit of supporting 16-bit floating-point, 32-bit floating-point, and 32-bit integer colour depths.

Benefits Inside Nuke
Nuke’s unique ‘Layer’ and ‘Channel’ system enables us not only to view all these different render passes, but also to reorganise, split-out and work with them in multiple ways. When the above example is loaded into Nuke via the Read node and viewed, the Viewer shows the rgba ‘Layer’ and RGB ‘Channels’ as below. In your 3D package this would be your flattened render (Beauty pass).

layerChannelShuffle_06

THE LAYER AND CHANNEL BEING SHOWN IN THE VIEWER
(ie The ‘Layer’ called RGBA and the collective ‘Channels’ Red, Green and Blue)

Now both the ‘Layer’ control and ‘Channel’ control can be changed to show you different information in the Viewer.

layerChannelShuffle_07

So in the example below the Viewer has been changed to show the ‘Layer’ Reflection and then only the 1st or Red ‘Channel’

layerChannelShuffle_08

Now the reason I say the “1st” ‘Channel’ as well as “Red” is that the Viewer can only describe the information as Red, Green, Blue or Alpha. For example a file could contain a zDepth pass, a Point Pass, a UV Pass, a Normal Pass, a Motion Vector Pass, or various Mattes, etc. All these utility or AOV (Arbitrary Output Variables) passes are not really colour passes but rather data passes. Unfortunately Nuke does not allow us to view this information using its proper name but always calls it R, G, B and A. So if you had a Motion Vector pass containing U and V information, it would be shown in the Viewer as R and G.

Before we move on from looking at the Viewer, it’s worth talking about the middle option between ‘Layers’ and ‘Channels’ which by default is set to ‘rgba.alpha’. This box only becomes active when the ‘Channel’ selection box is set to ‘Matte Overlay’ (Mat). It gives you the ability to specify a matte overlay channel and view it together with your colour channels in one view by showing it as a red semi-transparent shape. Although it would commonly be used to view the alpha channel or any other related Matte over the colour information, the overlay can in fact be any channel you specify in this middle box, giving you the ability to define and view a custom Matte of your choice.

The example below illustrates this. A Bezier node which has been drawn round three tiles on the floor. The output of this Bezier has been assigned to a custom ‘Layer’ called “floor” and a single custom ‘Channel’ called “Bezier1” – more about these custom settings later. (Note that in the Viewer, this will be called R not bezier1). In the Viewer, the ‘Layer’ has been set to rgba and the ‘Channel’ has been set to the Matte Overlay (Mat) option. The middle box has then been changed from ‘rgba.alpha’ to ‘floor.bezier1’ in order to show the Bezier1 custom Matte overlayed with the colour channel.

layerChannelShuffle_09

Shuffle and Shuffle Copy
The Shuffle and Shuffle Copy nodes in Nuke enable you to organise and rearrange the ‘Layers’ and ‘Channels’ and to create new custom ones if needed. At first they can appear slightly daunting but once you understand their layout, things are much simpler. The first thing to note is the UI for both Shuffle and Shuffle copy are almost identical, which is one of the confusing things.

layerChannelShuffle_10

Left, SHUFFLE, Single Input (Note UI change for ease of understanding). Right, SHUFFLE COPY, Double Input

The main difference between Shuffle and Shuffle Copy nodes are the number of input – single versus double. Shuffle lets you rearrange the channels from a single image and then output the result to the next node in your composite, whereas Shuffle Copy lets you rearrange channels from two images into a single output.

Shuffle
The diagram below shows the flow of data (‘Layers’ and ‘Channels’) through the shuffle node.

layerChannelShuffle_11

find the easiest way to read the Shuffle node is backwards, ie. “What do I want to get OUT in the end and therefore what do I need to feed IN to achieve this?”.

In this example the data is not changed. We want to end up with the ‘Layer’ rgba containing the ‘Channels’ Red, Green, Blue and Alpha. To do this we first set the output ‘Layer’ to rgba (pre-set already). We then need to choose which Input ‘Channels’ are going to be fed into which Output ‘Channels’. We want Red into Red, Green into Green, Blue in Blue and Alpha into Alpha. In order to achieve this we simply check the box where the DATA IN meets the DATA OUT as shown above.

Now let’s change the order. We’ll put the Red input into the Green output, the Green input into the Blue output, and the Blue input into the Red output. We’ll leave the Alpha channel where it is for the time being.

layerChannelShuffle_12

Left, ORIGINAL IMAGE. Middle, SETTINGS. Right, SHUFFLED IMAGE

Now let’s talk about Alpha. You can of course Shuffle any ‘Channel’ into the Alpha channel as well as Shuffling Alpha into the r, g, b channels, but you also have the option of replacing the Alpha with black (removing the Alpha channel) or with white (making it solid). You do this by using the 0 and 1 columns.

layerChannelShuffle_13

Left, ORIGINAL ALPHA. Middle, SETTINGS. Right, SHUFFLED ALPHA

The bottom section of the Shuffle node works just like the top and enables you to add data into more than the standard four (r, g, b, a) ‘Channels’ we’ve so far talked about. Say you have an Alpha channel that you want to make solid but you still want to keep that data, the bottom section of the Shuffle node can help you do this. You can move the Alpha channel data to a new (or existing) ‘Channel’ and then overwrite the original Alpha with a solid white (1).

In this example we’re make a new custom ‘Layer’ and ‘Channel’ to hold our existing Alpha data. To do this, make the Alpha channel solid as above. Now in the lower half change the none box to new which brings up a new dialog box.

layerChannelShuffle_14

This New Layer dialog box enables you to create a new ‘Layer’ with up to eight ‘Channels’. First add the name of the new ‘Layer’ you want to create, then add one or more ‘Channel’ names. If you click the Auto: rgba button Nuke will fill in the first four ‘Channels’ automatically for you. Once complete, click OK to return to the Shuffle node.

layerChannelShuffle_15

Left, NEW LAYER DIALOG BOX, Name = oldAlpha, Channels = alphaShape
Middle,SHUFFLE SETTINGS, Note alphaShape is in the first ‘Channel’ of the new ‘Layer’ which is the R ‘Channel’, out of rgba
Right, VIEWER, ‘Layers’ and ‘Channels’ changed to show newly Shuffled Alpha. Note alphaShape is labeled R

Now simply check the box where alphaShape (oldAlpha) and a (rgba) meet, so that your original Alpha channel is shuffled into your new custom ‘Layer’ and ‘Channel’. Please note that, as discussed earlier, when you view this new custom ‘Layer’ and ‘Channel’ in the Viewer it will come up under the ‘R’ label instead of ‘alphaShape’.

Shuffle Copy
Once you’ve got the hang of Shuffle, Shuffle Copy is very straightforward. Let’s assume you have two images, one contains the Beauty pass and the other the zDepth pass. You need both of these passes brought together in the same stream so you can use the ZBlur tool which creates a Depth of Field blur.

The first thing to note with the Shuffle Copy node is that whereas the Merge node’s inputs are labelled A and B, for some reason the Shuffle Copy inputs are labelled 1 and 2. You can assume 1 is A and 2 is B. In Nuke the B input is always the main direction of the stream and the A input adds something into that existing stream. This works the same way for the Shuffle Copy, 2 is the main stream and 1 brings in the extra data.

In the diagram below the Beauty Pass is coming in on the 2 input and the extra zDepth pass is coming in on 1. If you look at the Shuffle Copy properties you’ll see that we have both “1 in” and “2 in” column inputs. Do not panic! It works just the same as the Shuffle node. The red, green, blue and alpha values all need to come from “2 in” which is being fed by the Beauty Pass. We then utilise an existing ‘Layer’ and ‘Channel’ in Nuke called ‘depth’ and ‘Z’ which we open up in the lower section. Into this we feed the red ‘channel’ from the rgba ‘Layer’ of the zDepth Pass (coming in on “1 in”).

layerChannelShuffle_16

More Information on Shuffle and Shuffle Copy
Up until now we have only been using the ‘Layer’ rgba with the Shuffle and Shuffle Copy nodes, but there is no reason why you can’t change the ‘Layer’ and Shuffle other passes around. A common use of the Shuffle node is to split out a multipass openEXR back into individual layers so they can be composited together.

layerChannelShuffle_17

Other CHANNEL Nodes
Some other useful nodes for working with ‘Channels’ are COPY, CHANNEL MERGE, ADD and REMOVE.

COPY
Replaces ‘Channels’ in the B input with ‘Channels’ from the A input.
Note, Over time I have stopped using Shuffle Copy and now use Copy instead

CHANNEL MERGE
Does simple maths between one ‘Channel’ from each input and puts the result in the output ‘Channel’.

ADD
Adds new ‘Channels’ to the image and stream.

REMOVE
Removes ‘Channels’ from the image and stream.


We offer video training in 3D, compositing and visual effects tools and techniques. Choose from a range of software and techniques including Maya, Arnold, Nuke and more. We also offer technique based videos for areas such as Deep Compositing and Core Skills.
Where in the past we offered a range of training from on-site, to video, to consultancy, now spherevfx offers a simple choice of either video training or free written training.
Because all our training is either video based or written we can train anyone, from whole departments, small teams or individual artists/freelancers.
We’ve had the privilege of training artists from some of the top visual effects companies in the world including: Industrial Light and Magic, Framestore, DreamWorks Animation, MPC, Digital Domain, Cinesite, Animal Logic, Ghost VFX, Hybride, Luma Pictures, Pixomondo, Prime Focus, Sony Pictures Imageworks, and Weta Digital.