djVRayCreateMatteLayer.mel

david | dj mod,mel script,rendering,tutorials,vray | Wednesday, September 1st, 2010

This mel script creates a renderLayer that can be used to quickly render mattes for selected objects or groups of objects with the VRay for Maya renderer.

As you can see from the screen shot, djVRayCreateMatteLayer.mel looks very similar to the mentalray mattePassMaker.mel script which I wrote about in my previous post. In fact djVRayCreateMatteLayer.mel is an attempt to emulate the mentalray matte pass functionality using "VRay Render Elements".

At this point in time I am fairly new to vray and vray is relatively new to maya so djVRayCreateMatteLayer.mel should be viewed as a work-in-progress, but one you may find useful if you ever need to create mattes using vray.

VRay Render Elements vs. mentalray Passes

The mentalray pass system uses something called a "Pass Contribution Map" to associate renderable objects with passes. PCM's make passes very flexible, since they are not mutually exclusive, which means that one object can be in many PCM's. This makes it possible to create separate mattes for every object in your scene while at the same time create mattes for various groupings of those same objects. These mattes render quickly and simultaneously.

VRay Render Elements are in many ways like mentalray passes and are created at the same time as the main render (sometimes known as the beauty pass) but they do not really share the pass contribution map concept, so in most cases, all the objects in the scene are rendered in all the Render Elements. Another difference is the association with render layers. While mentalray passes are tied to specific render layers, vray render elements are, by default, active on all render layers.

Multi Matte Render Element

The Multi Matte Render Element is one of a handful of render elements that has the ability to be associated with selected objects. It is called Multi Matte because a single image rendered with this element can contain 3 separate mattes - one in each of the red, green and blue color channels. This technique is often used as an efficient way of storing data. In your compositing application you can easily extract the color you need by separating the rgb components.

The multi matte attributes let you specify a filename suffix which is appended to the image file name. And there are object ID attributes which is how an object is associated with one or more of the color channels. To make this work your object needs to have a vray objectID attribute which you have to add to the object since in maya it does not have one by default. Fortunately this is easy and there are several ways to do it. One is through the objects attribute editor as shown below.

This adds the objectID attribute which you'll see in the bottom of the attribute editor in the "Extra VRay Attributes" roll out.

If the number you specify here matches one of the numbers in the Multi Matte render element's color attributes, then that object will be rendered with that color in the image for that element.

Pause for breath...

Ok. It might look a bit long winded written out like this, but once you do it a few times it is quite straight forward. And although I want to resist comparing mentalray to vray subjectively, I think it is worth noting that I find creating vray render elements is less complicated that creating mentalray passes and pass contribution maps. But I also think that both benefit from scripting a UI to manage all the messy stuff, especially when you may wish to create large numbers of mattes.

The Concept

The idea behind my script is that the user selects some objects and clicks a button and a render layer called "Matte" is created with a Multi Matte render element associated with those objects. The script takes care of adding and numbering the objectID attributes, setting the filename suffix and it creates layer overrides so that the Multi Matte elements are only enabled on the Matte layer. It also provides an easy way to create layer overrides and disable other render elements you may have in your scene when using the Matte layer.  The end result is a special Matte render layer that renders all your matte elements quickly and simultaneously.

I decided after some experimentation to use only a single color (red) in the Multi Matte render element. This allows me to be quite specific with the filename suffix and that makes this so much easier to find the matte I'm looking for when I'm compositing.

An Example

To show how the script works, I've built a scene with some colored numbers. If I render the masterLayer I get this

I launch the UI using the following command

djVRayCreateMatteLayer;

Change some preferences

Select some objects. For my example I have selected the odd numbers. And click the "Create New Matte Layer" button.

Enter a meaningful name, which will be used to name the multi matte element and the image file that it renders. Click "OK". A new render layer called "Matte" is created. Click the Matte layer to make it active.

In the render settings window, the Render Elements tab now shows a new element called vrayRE_oddNumbers in the "Added render elements" pane.

If the Matte render layer is active when you click on "vrayRE_oddNumbers" to open the attribute editor you'll see something like this

The "Enabled" flag is orange to indicate a layer override has been created. If you make the masterLayer active and you'll see that the "Enabled" flag is unchecked. The new multi matte element is only enabled on the Matte render layer.

The "Filename suffix" is set to the name I specified, "oddNumbers". The "Red object ID" is set to 1. The script has automatically determined that this is the lowest unused objectID in the scene. If I check the attributes of the objects I selected you can see that they now have an Object ID which has also been set to 1.

Next I select some more objects, the even numbers, and click the "Add to Matte Layer" button, and repeat the steps above to add another Multi Matte render element. These objects get an Object ID of 2, which is the number entered in the Red object ID.

Each time I select more objects the script figures out and assigns the lowest unused object ID.

Here's how the two matte elements look when rendered. I'm viewing them using the drop down menu in the VRay frame buffer.

I've been using the UI in "combination" mode which means the objects I select get the same ID and are associate with the same element. If I switch to "individual" mode, then if I select 10 objects, each will get its own unique ID and each will be associated with a different multi matte element.

Image Sampling Issues

If you look at the RGB color pass (the beauty pass) on the Matte render layer you'll see that it is totally black. This is because lights have been ignored and GI disabled on this layer to make it render as quickly as possible. However, vray's adaptive sampling and hence the anti-aliasing quality depend on the contrast in the beauty layer to figure out how much to sample. A totally black image requires very little sampling so the minimum will be used and that results in badly aliased edges in the multi matte renders.

To anitialias the edges properly djVRayCreateMatteLayer.mel creates an override on the min subdivs in the VRay Image Sampler's Adaptive DMC roll out.

One more thing...

There are a couple of extra functions in the UI's edit menu. These are to help manage render elements in your scene. You can select a number of elements in the list and perform enable and disable operations on them as a group.

Limitations

The main limitation when using object IDs to associate objects with matte elements is that an object can only have one ID, and as such can only be part of the same group as other objects with the same ID. So in my odd & even numbers example, if I was to select all the numbers and add them to the Matte layer, they would all be given a new matching ID, and become associated with the "allNumbers" matte. They would no longer appear in their original mattes. This is where the mentalray pass contribution map system is more powerful. In vray we can use the material ID as an alternative way of associating objects but it does not give us the same freedom of association that we get with PCM's.

Out of breath...

I think this is one of my longest posts. I hope you found some of it useful. I'm interested in hearing how others using vray tackle the creation of mattes so feel free to comment.

As always, you can download the script from my downloads page.

11 Comments »

  1. Hi David,

    This is my first post on your blog, but I'm a long-time follower, actually!
    I have also been using vray for maya at work since about a year ago, I just thought that you might be interested to hear about the technique that I use when setting up my mattes.

    If I'm going to use the Object ID's, I'll use vray's "object properties" node. It's found under "Create> vRay> apply single/multiple object properties node to selection." This allows me to apply object ID's to multiple geometries at once rather than 'adding vray attributes' to geometry one at a time. Also, in case you already have objectID attributes on a 'per-geometry' basis, the objectProperties node will override the existing objectID's.

    Since geometry can (currently, as far as I know) only be part of one "object properties" node at once, you can then use the material ID's to further sort out your geo, say if you want a geometry to be in 'multimatte1', and 'multimatte2'

    In order to have the render element distinguish between using the materialID versus the objectID, just check (or uncheck, accordingly) the box labeled "Use Material IDs" under the "Extra VRay Attributes" rollout in the multimatte attributes.

    I think, if I'm not mistaken, this sort-of addresses the Limitations you wrote about. What do you think? :)

    -pat

    Comment by patrickvidal — September 16, 2010 @ 6:16 am

  2. Thanks for you comments Pat. When I wrote the script, I did consider using the "object properties node", but since the script does all the hard work it was easier to add attributes to each object. And its also easier (for the script) to change things after, without needing to figure out which properties node a particular object is connected to.

    The materialID option is something I may add in the future. Still not a flexible as mentalray's PCMs but better than nothing.

    Comment by david — September 17, 2010 @ 12:04 am

  3. Hi David,

    Thanks for writing this, I am still trying to wrap my head around V-Ray for Maya and have been looking for a quick and easy way to have individual objects render out their own mattes a-la-contribution maps in MR. My problem with the multimatte render element in vray (hooked to object ID) is the amount of time spent configuring each obj ID on the transform nodes, and also the fact that my compositor is getting pissed because I can only fit three objects per image (R, G, B), whereas with mental ray i could just set the contribution map's name to make a matte ... I think I see now why you set them to only use the Red channel, so that the renderelement name could be named after the object, no?

    Anyway, I haven't had much luck getting this to work on Maya 2011 x64 OSX 10.5.8. The menu builds fine, however I get the following erro when trying to create a matte layerr:

    // Error: Cannot find procedure "djVRayCreateMultiMatteElement". //

    Any ideas?

    Thanks again!

    Matthew

    Comment by agenesis — September 29, 2010 @ 4:00 am

  4. Matthew: Yes when I have several mattes, it gets too confusing using the r g b channels, so I prefer to have them named logically and just use red.

    I can't test on OSX. Working ok in windows. Maybe try sourcing the script before you run it.
    For example:

    source djVRayCreateMatteLayer.mel; djVRayCreateMatteLayer;

    good luck

    Comment by david — September 29, 2010 @ 9:23 am

  5. David,

    Thanks for the reply.

    Here's the error I get when sourcing then running:

    source djVRayCreateMatteLayer.mel; djVRayCreateMatteLayer;
    // Error: $helpText = "\n\
    //
    // Error: Unterminated string. //
    // Error: $helpText = "\n\
    //
    // Error: Unterminated string. //

    I'll let you know if I think of anything.

    Best,

    Matthew

    Comment by agenesis — October 1, 2010 @ 3:17 am

  6. Matthew, I suspect this error is being caused by a difference between how macs and pcs interpret end-of-line in text files. I have made a change to the mel script that I think will prevent the error, but I am unable to test this myself. If you download the rar again and give it another try let me know how it goes.

    Comment by david — October 3, 2010 @ 11:29 pm

  7. David,

    Thank you for the updated script, it works well.

    I've noticed, however, that using the multimatte passes created as a pass in a multichannel EXR yields results that after effects CS5 (EXtRactor) is unable to work with. Even just using the Red channel and setting it as the G, B, and A for the pass does not work. Outputting them all as .tga files worked fine, however.

    Secondly, do you know if there is a way to get the exact same sampling results as your beauty pass when using the mutlimatte element? Sounds like a tall order, I know, haha. Oversampling to min 4 max 4 when using DMC seems to produce good results, however still not perfect.

    Thanks again for the scripts!

    Best,

    Matthew

    Comment by agenesis — November 2, 2010 @ 5:06 am

  8. I'm glad it working better now.
    I don't have afx cs5 so I can't check, but last time I looked at extractor in cs4 I had problems with the way it expects the channels to be named - this was prior to me using vray, so I'll take another look and let you know. (I've been using fusion but it also has limited support for multichannel exr's so recently I've moved to nuke - which has excellent support).

    I think the only way to get the same sampling as the beauty pass is to render the multimatte elements on the same render layer as the beauty pass and at the same time. My script creates a separate render layer for the mattes (and disables them on the other layers) so that they can be rendered out quickly, and independently of the beauty pass, but this is simply a workflow choice. If you enable the multimatte elements on your masterlayer (or what ever is your beauty layer) then they will get sampled the same as all the other elements on that layer.

    Comment by david — November 2, 2010 @ 4:58 pm

  9. Thanks again for the update on that script, it's already saved me a bunch of time.

    Using Nuke to comp stuff as we speak ... much better with the EXR.

    One thing, not sure if it's a bug or not - enabling "Prompt for render element name" only seems to take effect when using the "combination" mode ("individual" just takes the name of the DAG path to the object .... which in my case involves a bunch of references/namespaces).

    Have you much experience using NURBs in V-Ray?

    If you get a second, could you take a look at this script I put together? Let me know if you can think of an easier way to manage the NURBs controls in V-Ray for Maya. I'm kind of novice in both MEL and V-Ray, and I'd like to see what you think.

    http://www.creativecrash.com/maya/downloads/scripts-plugins/rendering/other-renderers/c/v-ray-for-maya-nurbs-control--2

    Thanks again for all the help!

    Best,

    Matt

    Comment by agenesis — November 4, 2010 @ 7:18 am

  10. Hi David,
    We've been using the multi-matte render elements with both obj id's and material id's and it works quite well as somebody above suggested. Whenever we have objects that have the same obj id's we'll sometimes put them on render layers and do render layer overrides on the obj id attributes which can be a useful workaround.

    Best,
    Ivan

    Comment by ivanG — August 20, 2011 @ 2:19 am

  11. Layer overrides on the objectIDs. I had not thought of that. Thanks for the tip.

    Comment by david — August 20, 2011 @ 11:24 am

RSS feed for comments on this post.

Leave a comment

You must be logged in to post a comment.

Powered by WordPress | Based on a theme by Roy Tanck