mattePassMaker.mel

david | dj mod,mel script,mentalray,rendering,tutorials | Monday, August 30th, 2010

You can use this script to create black and white matte renders using mentalray's pass system which means that it takes no longer to render 100 mattes than it does to render one. And since the script makes it so easy to set up these passes you can render a separate matte for every object in your scene, if thats what you want, or for combinations of objects that you select or based on the materials that are assigned.

I'll explain in some detail how the script works, but first I need to provide a bit of history and some credit where credit is due.

In January 2010 I got involved with a discussion on cgTalk where Christopher Emerson shared his ideas for using the mentalray pass system to create matte renders for use in compositing. He pointed out that to create matte passes manually was time consuming and prone to user error and he had written a script called materialMatte.mel to automate the process by creating mattes based on the materials assigned to objects. After trying the materialMatte script I realized that, for my way of working, it would make more sense to create mattes based on selecting objects directly, either individually or in groups. And so I rewrote Christopher's script to incorporate my ideas. The end result is something I have called mattePassMaker.mel which you can download here.

What does it actually do?

Often when doing compositing and special effects in applications like Nuke and Fusion you'll want to isolate parts of your render using a mask (or matte). For example, you may want to color correct just some of the objects in the scene. If you render a black and white matte for those objects - where the selected objects are white and everything else in the scene is black - then you can use the matte sequence to limit your color correction to the selected objects. In the past one way you could have created the matte render was by making a renderLayer and assigning black and white material overrides to the objects on the layer. For every mask you needed you would have to make another renderLayer. And then you would need to render them all, one at a time.

Then a few versions ago maya added the ability to make render passes. These passes are derived from the computations done in the main render (often called the master beauty pass) and they are rendered out at the same time as the beauty pass. Passes are a convenient way to get extra render components without needing to do extra renders.

Unfortunately maya's mentalray pass system has the reputation of being less than perfect, limited to the use of certain materials and often resulting in increased render times.

Fortunately some aspects of the mentalray pass system work really well and mattePassMaker.mel uses the good bits!

mattePassMaker creates a new renderLayer, known as the "Matte"  layer, which will usually contain all the objects in your scene. Everything in the Matte layer gets assigned the default lambert shader as a material override. Lights are left out and finalGather and raytracing are disabled. Matte passes are then created within the Matte layer and each pass has a "pass contribution map" assigned to it. The pass contribution maps are simply lists of the objects that will be rendered as white in each pass. Anything not on the list will be black in that pass. When you render the Matte layer you get a beauty pass which, due to the absence of light, FG, raytracing, textures etc, renders very fast. The Matte layer beauty pass is not very useful since it is totally black (no lights), but it must be rendered to get the matte passes, and these are all rendered simultaneously.

Ok, so how do I use the mattePassMaker script?

First you need to install it, which simply means putting mattePassMaker.mel in your local scripts folder and restarting maya.

Launch the UI using the following command

mattePassMaker;

There are several different ways you might proceed from here, so I'll explain a few things and give some examples.

I have created a scene with several objects and materials and if I render the masterLayer I get the following image.

Before we create any mattes, we should be aware of some preferences that can be set in the edit menu of the mattePassMaker UI.

The meaning of these will become clearer later, but setting them like in the picture above is a good start point.

If there is no Matte layer in the scene, then the UI will look something like this

Source Layer -- The "source layer" defaults to the masterLayer. Only objects in the source layer are used in the creation of the matte layer. Use the drop-down menu to choose a source layer. For this example I will leave it set to masterLayer.

Matte Layer -- It is possible to create more than one Matte layer in your scene and you can add to or replace an existing layer, so there is a drop-down menu for choosing the one you want to edit. In our example we have not yet created a Matte layer so the menu just contains the word <New>.

Action -- You can either "Add To" a new or existing Matte layer or "Replace" an existing one (the deleting the old one). Until we actually create a Matte layer our choice is limited to "Add To"

Layer Objects -- The most common choice here is "All" which means all the objects from the source layer will be added to the Matte layer. That way when mattes are rendered hold-out masks for all other objects in the scene that are in front. Under special circumstances we may wish to limit the objects in the matte layer to be just the objects in the "pass contribution maps" (more about PCMs below). For this example, leave it set to "All".

PCM Objects -- "Pass Contribution Map Objects" are the objects that will be white in a particular matte pass. A Pass Contribution Map (or PCM) is just a list of those objects. So we need to specify how we want the list to be created. There are two choices. "Assigned Materials" means we will select some materials and let the script figure out what objects have those materials. "Selected" means we will select some objects or groups and the script will just use our selection. Set it to "Selected" for the first part of this example.

Mode -- "Individual" means each object in the selection will have its own matte. "Combination" means only one matte will be created for the entire selection. For the example choose "Combination".

Other Options -- Forget about these for now and just set them as shown in the screen snap above. I'll explain later.

Next I'll select some objects in the viewport. I've selected the numbers 1,3,5,7,9 and then I clicked the big green "Create New Matte Layer" button. A pop-up dialog appears so I can enter a PCM Name. If I just click "Ok" it will be called "combination" but since this name will be used for the rendered image fileName I'll change it to something more meaningful, oddNumbers.

Now if we look at the renderLayers window we see a new renderLayer called "Matte" which contains a pass called "oddNumbers".

You will also notice that the big green button has changed to pale blue and says "Add To Matte Layer" since the usual workflow is to keep adding passes to the Matte layer. So I then select the numbers 2,4,6,8,0, click the big button and enter "evenNumbers" as the PCM Name. Now I have created 2 matte passes and the renderLayers looks like this

Next I'm going to use a selection of materials to create some more matte passes, so in the hyperGraph I make my selection.

I reconfigure the choices in the UI as shown below

PCM Objects is set to "Assigned Materials" and Mode is set to "Individual". In this context, "Individual" means I want one matte pass for each material, not for each object. I click the "Add To Matte Layer" button. Observer the renderLayers shown below

Notice how the names for the new matte passes are constructed from the material names. For this reason it would be wise to adopt good naming conventions for your materials (and not just use default material names like in my example).

Next I select the back wall and the floor objects in the outliner.

I want individual mattes for these so in the UI I change PCM Objects to "Selected" and Mode to "Individual". Click "Add To Matte Layer" once again. Two new matte passes are created as shown below

Notice that the matte pass names are constructed from the object names, so good object naming conventions are important.

Now that I've created all the matte passes I'm ready to render. I switch to the Matte layer in the renderLayer window. In the viewport, with the Matte layer active, the objects will most likely be displayed as wireframe even in shaded mode. This is normal and is a side-effect of using a material override on this layer.

At this stage I probably should mention the "Other Options" tab again - the one I skipped over earlier. And I'm only going to cover this briefly now.

Output Channels -- 1 or 3, determines the bit depth for the matte pass image. 3 means it will use the RGB channel.

Output Format -- lets you specify an image output format which will be set as a layer override in render settings. Choosing "None" means you'll use the same format as your master layer, or that you will create the override manually.

Multi Channel -- only applys to EXR format and lets you write all the passes to a single file.

Set File Name Prefix -- creates a layer override on the output file name prefix in render settings, and this will contain a "token" for the pass name in both the folder name and the image name. This is a good way to keep all the matte passes organized. It looks like this

And finally I render a frame...

I used the renderView window and it takes just a few seconds and I'm left looking at a black frame. I can view individual passes in a seperate viewer using the file menu in the renderView window.

If I browse to the output folder with windows explorer I find my passes organized neatly into separate folders

And they look like this

oddNumbers...

evenNumbers...

backWall...

And heres the masterLayer beauty pass again (which was rendered seperately using the masterLayer - remember the Matte layer beauty pass is always black)

Downloads page.

2 Comments »

  1. Thumbs Up!

    Comment by smokedogg — September 13, 2010 @ 3:48 pm

  2. WOW

    David,

    It's been my dream to combine all my mattes into the channels of a single .exr file. This will be so handy. Continued thanks for your contributions and insights.

    Bill...

    Comment by wabash — September 14, 2010 @ 2:50 am

RSS feed for comments on this post.

Leave a comment

You must be logged in to post a comment.

Powered by WordPress | Based on a theme by Roy Tanck