workaround for AEhardwareTextureTemplate.mel bug

david | dj mod,mel script,mentalray,rendering,vray | Wednesday, January 26th, 2011

Ever since I can remember, maya has had an annoying bug that causes materials to display incorrectly in the viewport in both "Smooth shade all"  and "Textured" modes.

If you've ever encountered something similar to this error

// Error: file: D:/Program Files/Autodesk/Maya2011/scripts/AETemplates/AEhardwareTextureTemplate.mel line 721: Found no attribute match for "mia_material_x1.diffuse" //

or this one

// Error: file: D:/Program Files/Autodesk/Maya2011/scripts/AETemplates/AEhardwareTextureTemplate.mel line 827: Object 'VRayMtl_qualityMenu' not found. //

then you may be interested in a workaround. Its not perfect, but its better than nothing.

The problem comes about when you rename a material. It happens with both vray and mentalray shaders, but differently depending on which you are using.

vray demonstration

Set the default renderer to vray. Create a vray material with a colored diffuse and assign it to a poly object. Switch to "Smooth Shade All" in the viewport. Your object is the color of your shader, as you'd expect. Rename the vrayMtl to something else. Select your object. Open the UV Texture editor window. Your object is now white. Select the material and open the attribute editor. Open the "Hardware Texturing" tab near the bottom. You'll get some errors.

mentalray demonstration

The problem usually happens in "Textured" viewport display mode. It seems to occur when you have renamed your materials and connected some fileTextures, and then later make a change to your shading network. For example you might connect a different mia_material_x to an existing ShadingGroup node. Eventually the texture will nolonger display and your object may look black in the viewport. If you look in the material's attribute editor and open the "Hardware Texturing" tab near the bottom, you'll see "Textured Channel" is set to "None". In theory you should be able to set it to "diffuse" using the drop-down menu, but in practice you can't and get an error similar to the ones printed above.

workaround

The error messages point to a mel script and some line numbers, so I took a look at AEhardwareTextureTemplate.mel to see if I could figure out how to fix it. The problem seems to be that the script is passed the wrong material name. I was not able to determine why, but I was able to remove any parts of the code that referred to the name and bypass the errors.

I have not noticed any negative side-effects, but the reason I'm calling this a workaround and not a fix is that I'm not stopping the initial problem from happening. Materials may still lose their color or texture after a rename and shader network change. But if they do, the "Textured Channel" drop-down menu in the "Hardware Texturing" tab will still be functional, and you can change from "None" to "diffuse" (or whatever else may be appropriate) to restore the correct display. If its a vray material with a color and no fileTexture, then you can still select "diffuse" to get the color back. And with mentalray you may still have to disconnect and reconnect the texture to get it to show.

As you can see - not perfect - but a step in the right direction.

My modified copy of the maya 2011 AEhardwareTextureTemplate.mel is available from my downloads page.

1 Comment »

  1. Thank you very much! Been hassling Autodesk for years about this, cheers for the fix!

    Comment by Kiernan — January 27, 2011 @ 4:03 pm

RSS feed for comments on this post.

Leave a comment

You must be logged in to post a comment.

Powered by WordPress | Based on a theme by Roy Tanck