VRay Render Elements & render layers improvement

david | dj mod,mel script,rendering,vray | Sunday, September 5th, 2010

The image above shows the Render Elements tab in the vray render settings window. If you've used it before you may notice some cosmetic differences, and that's because I've modified one of the vray mel scripts to add some functionality that makes render elements easier to use with render layers.

In an unmodified vray installation, when you create a render element it is enabled on all render layers in your scene. You can use a layer override on the element's "enabled" flag. So if, for example, you want the Extra Tex element to be enabled on a single layer only, and you have several render layers in your scene, then you would need to select the masterLayer and disable the element, then select the layer where you want to use Extra Tex and create a layer override on the enabled flag and then enable it. A few clicks here and there but not too difficult.

Having done just that a few times I realized that I did not have an easy way to see which elements were enabled on which render layer. To find out I would have to select the render layer and select the element so I could view the state of the enabled flag in the attribute editor. Now thats just a few more clicks and while still not difficult it can get confusing .

My solution was to modify the mel script that creates the Render Elements Tab. What I've done is change the way that elements are created and how they are displayed in the "Added render elements" list on the right hand side of the window.

If the element is enabled for the currently selected render layer, then [+] is displayed in front of the element name. And if the element is disabled then [_] is displayed. When I switch render layers the list is updated so I have a visual display of enabled and disabled elements.

When I create a render element one of two things happens depending on what render layer is selected. If the masterLayer is selected, then the element is created as it normally would be. But if any other render layer is selected then the element is created and disabled on the masterLayer and then enabled on the selected render layer using a layer override.

I've also changed what happens when you double-click an element in the right hand pane so that instead of removing the element, it toggles the enabled flag of that element. The way this behaves is similar to the creation process. If I toggle an element by double-clicking its name when the masterLayer is selected, it simply changes from enabled to disabled or vice versa. But if I do the same thing when one of the other render layers is selected, the enabled state will be changed, but a layer override will be created first, so that the change only effects the selected render layer.

I posted a copy of my modifications on the wishlist at the Chaos Group VRay for Maya forums. I'm hoping that they'll incorporate my idea in a future update.

If you want to try it out I've put the script on my Downloads page.

Disclaimer: VRay is constantly being improved and with each release the scripts change. My modifications are based on one of their beta releases - 30 August 2010. It may not work with future versions and is untested on earlier versions. To avoid messing up your vray installation I seriously recommend placing this modification in your local scripts folder and not overwriting the original. That way if you get errors you can easily delete it. Furthermore, if you do put it in your local scripts folder, vray will not see it when maya loads, and you'll need to source it once you start maya.

If you are using the vray demo version then you'll need to edit the script and uncomment 4 lines 375, 376, 377, 378.

Update, 14 September 2010: Fixed problem where enabled flag in element list was not being refreshed after scene was saved and reopened.

5 Comments »

  1. First of all, I like your thinking! :D
    that is why i downloaded your script to use it myself...

    something goes wrong however with the size of the scrolllist inside the render elements tab,
    it's so small i can barely read "multimatte" until "refraction"

    if you want i can send you a screenshot of whats happening..

    any ideas?

    cheers.. love your scripts..

    Comment by TinuZ — September 24, 2010 @ 1:38 am

  2. This can happen if the script encounters an error, and that can happen if you do not use the same version of vray as I do (30 Aug 2010). For example if you are using the demo version, see my disclaimer note above.
    All I can suggest is, have a look in the script editor and see if there are any error messages, and I'll try to help you figure it out. I'm use this script in production for the past couple of weeks and it works really well.

    Comment by david — September 24, 2010 @ 9:23 am

  3. hey david,

    thanks for everything on your site. very helpful stuff. this one in particular is a personal sore spot with vray and your solution is very elegant. the only hitch i've hit yet is that imported render elements don't update their visibility toggle. they do update and assign overrides but the [+] or [_] doesn't change. i had a brief look through your script but couldn't see it right away. any thoughts??

    thanks again for all your scripts man,
    -b

    Comment by bob white — December 2, 2010 @ 10:56 am

  4. Glad you like the blog. I enjoy doing it. Only wish I had more time these days.
    I think I know what the problem is. The updates to the interface are triggered by script jobs, which are meant to get created when you load the scene. I think when you open the render settings it scans for elements and creates the script jobs (since I didn't want to save them with the scene). Have you tried closing and reopening the render settings window after you import/open your scene?

    Anyway, I'll have a look at the script and get back to you.

    Comment by david — December 2, 2010 @ 7:38 pm

  5. Ok. I had a look at the script. The script job part is working as it should, so I was wrong about that being the problem.

    But I discovered something that could explain the behavior you experienced. If you took heed of my advice in the disclaimer then you would have put the modified script in your local scripts folder. Normally that would completely override the maya install, but in vray's case its the other way around and the plugin sources all the scripts in the installation folder and ignores the one in your local folder. If you open the render settings window and select the render elements tab it will not be modified. If you then manually source vrayCreateRenderElementsTab.mel from your local scripts folder, the [+] and [_] indicators will appear, but the script job will not get created so the indicators will not update as they should and double clicking the element name will remove it from the scene instead of enable/disabling it.

    What you need to do is source the modified script BEFORE you open the render settings window. Its too late if you already loaded the settings window - you cant just close and reopen the window, you must close and reopen maya and source the modified script. You only need to do this once when you first start maya. Adding a line to userSetup.mel should do the trick. Then all the modifications should apply.

    Alternatively, if you are feeling brave, you might choose to make a backup copy of the original vray script and then copy the modified version into the vray install folder.

    (I hope that solves your problem. If not let me know)

    Comment by david — December 2, 2010 @ 10:52 pm

RSS feed for comments on this post.

Leave a comment

You must be logged in to post a comment.

Powered by WordPress | Based on a theme by Roy Tanck