linear workflow and gamma update

david | mentalray,rendering,tutorials | Monday, October 6th, 2008

Last month I wrote about how I implement a linear workflow in my work. Since then I have noticed that in at least one area I'm doing it differently to many others - and now I know why they are right and I was wrong.

It boils down to this: I've been leaving my frameBuffer gamma at 1 (the maya default) and setting my output gamma to 2.2 via an exposure node, which means I need to degamma my 8 bit fileTextures (but 32 and 16 bit floats are ok). However the other (very popular) approach is to set the frameBuffer gamma to 0.455, and the output gamma to 1, which means generally the fileTexture requires no degamma.

When making my choice I weighed up both methods and I said to myself "The frameBuffer gamma defaults to 1 and the exposure node defaults to 2.2 gamma, so I'll stick with that. After all, it looks the same in the render..." BUT ITS NOT!

I discovered a major difference to the way colors are blended depending on how the framebuffer gamma is set.

I made this image in photoshop.

I loaded it into a fileTexture node in maya which I hooked up to a gammaCorrect node and then to surfaceShader that was assigned to a nurbs plane. I connected the mia_exposure_simple to my camera.

My very simple test setup looked like this in the hypershade

I rendered standard 8 bit images with different gamma combinations to compare the result. I'm showing them here with approx 7x zoom in the renderview window to highlight differences in the multipixel filtering (Please ignore the jpg artifacts)

Image 1:  maya default with gamma=1 for frameBuffer, exposure and gammaCorrect

Image 2: frameBuffer gamma=1, fileTexture gamma=0.455, exposure gamma=2.2

Image 3: frameBuffer gamma=0.455, fileTexture gamma = 1, exposure gamma=1

In my opinion, the way the colors are blended in images 1 and 2 is less pleasing than in image 3 which I think is more natural. But which ever way you look at it the color math is obviously different when you use a different frameBuffer gamma.

So for me, the "nicer color blending" is a good enough reason to alter my workflow and use frameBuffer gamma set to 0.455. Since then I have discovered there is the same difference when you mix colors in photoshop - if you work in 8 bit you get results like the first two images, but if you switch to 32 bit float you get results pretty much the same as image 3.

I also tried other comparisons like rendering to a floating point format and using a floating point (OpenEXR) fileTexture. I won't go into detail here because it gets confusing. If you are interested try it yourself.

29 Comments »

  1. Though I have to dig through my file, find the password, and then log in to post comment.
    I have to say "Thanks a lot", David. Very informative read and clarify my concern on choosing which workflow to adopt for linear workflow. I started as using framebuffer 0.455, then switching to lens shader gamma 2.2 way. Now, you did the great test. Guess maybe the difference you elaborate in the post is why Floze stays with framebuffer workflow.....
    Really interested in what will happen when using float texture and output image. I will find sometime to test it out.
    Also, just read a article here http://www.ypoart.com/tutorials/tone/correcting_textures.php
    mentioning that one should adjust gamma setting in Photoshop to paint textures in linear in Photoshop to fulfill the whole linear workflow in 3D. Just curious if you ever heard of this and/or been using this in production?

    Comment by jasonhuang1115 — October 7, 2008 @ 12:56 am

  2. Interesting indeed! Afaik the adaptive sampling is also affected by the framebuffer gamma.
    I came across some other drawbacks though, check this out:

    http://forum.lamrug.org/viewtopic.php?f=6&t=1144&start=0&st=0&sk=t&sd=a

    It's probably not noticeable in most cases, but one should be aware of it if using the framebuffer gamma I guess..

    Comment by floze — October 7, 2008 @ 4:55 am

  3. Sorry about the password requirement Jason. Its the only way I've been able to stop close to 1000 bot registration attempts each month. If only...
    Thanks for posting that link. Its a great article. I think it is important to prepare file textures for use in rendering anyway, but the linear workflow is an extra consideration.
    So far I have not been doing any degamma in photoshop, I prefer to keep my textures in sRGB. I did consider using a color management system to allow me to keep the images linear and view in sRGB, but I found it to complicated and not easy to work with when collaborating with other artists on other computers. However I have been saving some of my fileTextures in the OpenEXR format (16 bit half float with piz compression). So, prior to saving I duplicate the merged image and set its mode to 32 bit before saving. Photoshop seems to save the data in a linear color space (I guess because its now float) and maya interprets it correctly too(you need to activate the OpenEXR plugin). I'm experimenting with these things a lot lately, so my methods will probably continue to evolve.

    Comment by david — October 7, 2008 @ 9:45 pm

  4. Hi Floze. Thanks for reminding me about that issue. I remember reading your findings back when you started that thread, but that was before I paid too much attention to the linear workflow. I had noticed some differences in the way mr renders the near blacks when working with the opposing gamma setups (frameBuffer v exposureNode), but so far I have not noticed any that would cause problems.
    Looking at your example pictures also reminded me of the importance of good monitor calibration - mine was too dark and I couldn't see those differences.

    Comment by david — October 7, 2008 @ 9:57 pm

  5. Yes its very dark and subtle indeed in the above pictures (perhaps I had my monitor a little too bright at that time :D), but it becomes very obvious in the more exaggerated examples down in that post.. anyhow, I dont suspect that to be a serious and major problem anymore, as it appears only very faint, and in very distinct situations. Still good to know that when dealing with gradients (and thus shading)! Problems would emerge in post though, when underexposed areas need to be seriously lifted for some reason..
    Its good to see all that ideas and experiences about the linear workflow issue coming along however, thanks for the input and tests!

    Comment by floze — October 7, 2008 @ 11:31 pm

  6. Thanks for the info, it's much appreciated! (thanks to floze, also, for his tutes).

    Comment by zeth — October 8, 2008 @ 4:48 am

  7. The image 3 remind some problems we are having on our current production with motion blur. Some of the really blurred edges are getting a pretty strong clear and harsh contour using linear workflow because. The motion blurred edge gradient is so harsh compared to the non linear version that we finally decided to unlinearise the alphas of our linear renders because the compositing is visually more appealing this way.

    Comment by Kel Solaar — October 15, 2008 @ 12:19 am

  8. So am I correct in balking at the required workflow for linear colour correction in Maya ;) :

    Every time we create a lambert, blinn or other material and simply want to make it dark green, light blue or whatever, (not using file textures, just the swatches - which is what we do about 95% of the time), we have to be mindful to make a gamma node with the required colour first, set the gamma to 0.455 and then pipe that into the colour slot of the lambert material?

    Ick.

    Going to be pretty difficult to get the others in the studio to implement that wretched workflow ;) Or have I missed something?

    Comment by geoff3d — November 18, 2008 @ 9:45 am

  9. Ick indeed :)
    It's one of those things that Autodesk could have made so easy, but chose not to.

    In practice it's not as painful as it sounds. I rarely resort to hooking up gamma nodes just to get a color. I start with a 0.455 framebuffer gamma and everything else flows from there. If a color looks to bright I make it darker (no calculator involved either as I dont care about that kind of accuracy, only what it looks like). One thing that I find helps is always setting colors with the color picker set up for HSV (not RGB). That way once I choose my hue and saturation, its usually the value I'm going to be tweaking (mostly).

    Unfortunately I also have to constantly encourage my co-workers to adopt this kind of work flow. Some are very stubborn. Some just don't get it.

    Comment by david — November 18, 2008 @ 11:07 pm

  10. You're right - it's not actually that bad ;)

    Thanks for your reply - and thanks for the awesome blog... I've written a rough little script that'll throw a gammaCorrect with the desired colour into the material if there's nothing already going in, which kind of works, and since it does it en masse it makes things a little less painless.

    Comment by geoff3d — November 19, 2008 @ 11:11 am

  11. Thanx for your great blog.

    I'm sure you're aware of the awesome 300 (or, well, much less but still pretty long) page thread in cgsociety.org forums about this issue. Although a pain in the butt the gamma correct node workflow, imo, is the correct process. Could you post a magnified Photoshop file of the colormap you are displaying in the renders? I'm willing to bet it is more like 1 and 2. Remember, when displaying textures, I want them to look EXACTLY as I pumped them out of Photoshop. I've had Producers come in because we forgot to turn the filter on the texture map to a very low number (even rendering with lanczos) complaining about a blurry cover of a promo magazine that only displays for 5 seconds onscreen. It's an extreme example but one I'm willing to bet is being changed by Maya in option 3 and not in option 1 and 2.

    If a color looks to bright I make it darker (no calculator involved either as I dont care about that kind of accuracy, only what it looks like).
    Aren't mia_materials inherently built in with the right gamma? If so, forget using lambert or blinn and just use a physically correct material.

    You also shouldn't have to go through so much trouble and confusion with the department anyways. I do believe that photoshop natively edits gamma via exposure in Image>adjustments>exposure. I just tested it and it does have a gamma correct feature. So now you can just batch out all of your texture files in a separate folder called (physically correct textures?) via an action in photoshop. Voila, without use of gamma nodes or scripts. ha. If your co-workers get confused then there is no hope lol.

    Comment by spiralof5 — December 7, 2008 @ 2:51 pm

  12. sprialof5: Thanks for you comment. Yes I have been keeping an eye on that thread. Its partly what prompted me to write about my own experiences here.

    The first picture I posted is the photoshop image. It is deliberately made of 4 colors with no filtering between them. Maya adds filtering when this is applied as a texture map and what I was demonstrating is the differences in the filtered colors depending on where gamma is applied. I was not really trying to say one was more correct than the other although I did indicate which I liked the best. Turning off filtering is not an option for me since all my work is animated. Even for stills I believe you would need some filtering - though you could probably get away with less, depending on the image content.

    With regard to mia and gamma, I think they are written in an attempt to be physically accurate, and so it is for that reason that they look their best when used in a linear workflow. Like you, I only care what it looks like and physical accuracy is less of a concern. But what I have found is that if I use a linear workflow them I am able to create better images with less effort. So for me you could say its a win win situation. I do not often use lambert and phong these days, but I have certainly not banned them from my scenes.

    Part of the confusion for groups of people attempting to use a linear workflow is that they suddenly need to be aware of the effect of applying gamma a various stages of the pipeline and this is not helped when maya treats different formats differently. For example, saving OpenExr from photoshop vs using maya IFF as a filetexture. You get different results, depending on how you set you frame buffer gamma and what format you render to. Factor into that artists who would rather not care about any of this and it can get messy. You can force certain rules on how to work, but in my experience confusion has a way of creeping back in.

    Just out of interest, how many people do you work with and what policies, regarding this subject, have you put in place?

    Comment by david — December 7, 2008 @ 5:56 pm

  13. I thought I would add this link to Andrew Weidenhammer's 3dlight for anyone looking to simplify the process of adding gammaCorrect nodes to an existing shader network.

    http://3dlight.blogspot.com/2008/09/gamma-tools-10.html

    Thanks Andrew!

    Comment by david — December 7, 2008 @ 5:58 pm

  14. What are you supposed to do with this .455 Framebuffer method and bump maps, specular maps and displacement maps?

    Are you supposed to gamma correct them to reverse the effect? Or just leave the framebuffer to do it's thing?

    Thanks David.

    Comment by Kiernan — December 15, 2008 @ 10:49 am

  15. In my opinion there is no short answer to your question. Some would say that fileTextures which do not map to color should be left un-gamma'd, but there are too many if's and but's for me to be happy with that as even a general answer.

    For example, with bump it depends on how you created the map, and what format the data is saved in - 8 bit or float. These things affect the way maya treats the data at render time depending on what frame buffer gamma you use and what bit depth you render to.

    These days I still end up doing a render test and adjusting if I think it needs it. Sorry, I cant be more decisive.

    Comment by david — December 15, 2008 @ 6:54 pm

  16. Thanks, I've done a few tests with spec and bump maps, gamma corrected and uncorrected and I've noticed there is a difference between the two methods. However, it wasn't that big of a difference between the two, so I think I'll leave it up to whatever looks the best. Cheers.

    Comment by Kiernan — December 16, 2008 @ 9:14 am

  17. I have 3 questions regarding the Framebuffer 0.455 / float method :

    -how do you preview the results directly in Maya? do you have to batch render and open your comp. software each time you tweak your scene?

    -i heard that this method causes aliasing in areas with high contrast. how do you fix this?

    -i also heard that this method causes inaccurate gradiants. how is this fixed?

    Comment by royter — May 26, 2009 @ 1:52 am

  18. Before I answer each question, I want to say that since I wrote this post I have discovered problems with the 0.455 frame buffer method. Maybe I should write another update, but for now I'll just say that I have gone back to framebuffer gamma=1 and exposure gamma = 2.2.

    Now, your questions:

    Previews in maya are possible in the renderview window, with either gamma method. If you are rendering to a high dynamic range format like EXR then you can add a temporary exposure node to do the preview renders.

    Rendering to high dynamic range, with no exposure node, can lead to high contrast aliassing. My preference is to use an exposure node, even when I render to float, so that the dynamic range of my image is reduced and there is much less aliassing. However some people prefer to fix it in post by adding diffusion and glares like a camera would.

    Gradients depend on you gamma setting, but they also depend on how you make them, so the choices you make with gamma's may affect the end result. How much different will it look though? I have never had a problem them. If a gradient looks wrong I just add a gamma or a degamma to fix it. Most of the time I would not even bother.

    Comment by david — May 26, 2009 @ 9:43 am

  19. David, been awhile since i last spoke. Been doing too much research on all this stuff. Anyway, you mentioned at the end of your blog about rendering out to openexr and was wondering if you could explain further about that.

    So when loading a texture into photoshop, do not do any gamma/color/profiling, etc... except to export out as a 32 bit exr.? You then mentioned on May 26th that you now use a framebuffer gamma of 1 and exposure of 2.2. Meaning you do gamma correct the texture in photoshop to 0.45 and then save as exr to bring into MR?

    Then in maya, for your final render, you then have to remove the exposure node correct? After that you would want to render out to openexr 4x32 float? After that, then bring the image into photoshop/AE and re-apply gamma correction and what ever else?

    I hope in not trying to make this confusing as I seem to be confusing myself, but I have learned so much from you David as well as others. Thanks

    Comment by smokedogg — July 31, 2009 @ 12:43 pm

  20. smokedogg: When I create a texture in photoshop, how I save it depends on what it is being used for. Most of the time I save as a standard 8-bit IFF with no gamma correction, and then I do the degamma in maya. It is not a good idea to degamma and save an 8-bit image. You will reduce the quality of your image by doing that due to the limits of the 8-bit file format.

    Sometimes I do save my textures in the OpenEXR format, using the free after effects plugin from fnord called ProEXR, which enables you to save as 16-bit float with piz compression to keep the file size down. Saving like this stores the image data as linear - meaning it is already degamma'd for you. I would imagine photoshop does the same.

    So, to clarify. No degamma done in photoshop for either 8-bit or float.

    If you render to float you may want to remove the exposure gamma (set it to one) or remove the exposure node completely. This allows you to do your gamma correction and tone mapping in photoshop using the full dynamic range image. This seems to be a popular choice.

    It really depends on how you like to work.

    I prefer to leave the exposure node with gamma=2.2 and get the render looking as close to final as I can. If I render float (half-16 usually) I can still tweak exposure in post without introducing artifacts. This is probably not how I would work if the final output was for film, but for tv broadcast it is fine and saves some time. In fact most of the time I just render to 8-bit IFF, and only go to float renders for the shots where I need the extra control.

    Comment by david — August 2, 2009 @ 6:33 pm

  21. Hello David. Have a question in reguards to converting my image/texture into a linear image. I feel really dumb for asking this, cause im pretty sure i know as i'v been reading a ton on this topic and may have just been overlooking the answer, but i will anyway.
    First, how do i know if i correcrly made my image linear? If using photoshop, as i am, do u go to the histogram or check the pixel with the info pallet?
    In order to make things linear, do i just create a custom ICC profile with gamma of 1.0 and continue on? Or does that just change the appearence on my screen and not the actual image?
    How have you or others made linear images? Any input would be great.

    Comment by smokedogg — August 6, 2009 @ 11:45 am

  22. smokedog: In my last reply I explained that you would only make an image "linear" if you are saving in a floating point format. The 8-bit integer format does not store linear data very well. It is much better to store sRGB data. If your image is 8-bit and it looks ok on your monitor then it probably is in the correct sRGB space.
    Maya will assume your 8-bit image is sRGB.

    If you work in photoshop and change to 32-bit mode then what you see on your monitor depends on the settings in Preferences|Performance|GL Settings|Color Matching. If color matching is ON when you switch to 32-bit mode then the display will look the same, even though photoshop has modified your image data to be linear. Try it with a 50% grey 8-bit image and see what the color values are when you switch to 32-bit mode. (My grey goes from 50% in 8-bit to 22% in 32-bit, but it looks the same on the monitor because color matching is ON). If you turn Color Matching OFF then the 32-bit image will look darker. (On my system the display does not always refresh instantly and I have to minimize/maximize to see the change). It doesnt matter if you work with color matching ON or OFF because in both cases photoshop will store the linear image data when you save the 32-bit image in the EXR format. I leave it on, because then you see the colors as they are intended to look on your sRGB display monitor.
    Maya will assume your 32-bit image is linear.

    Comment by david — August 6, 2009 @ 10:20 pm

  23. Hey David,
    Okay i finally got it with converting my textures into 32 bit exr files. So now when in maya, im ready to render out my final images into 4 x 32 bit float exr. (im not really ready yet, just saying for example purposes) So the question that resides is; Since my textures are now linear, do i still have some kind of gamma encoded 2.2 value where i would have to set the framebuffer to .45? Or is the default of 1.0 ok? I will be compositing in AE CS4 that now has the ability to work in a 32 bit linear space.
    Thank you so much for your time, patience, and help as it is very well appreciated! Its just trying to find answers for certain things can get very cumbersome as so many people have diff. methods, pipelines, and workflows.

    Comment by smokedogg — August 11, 2009 @ 10:40 am

  24. Hi smokedog. The short answer is: If your fileTextures are floating point exr, then you do not need to degamma them in maya. If you render to a float format and plan to do final gamma correction in post, then you can leave both framebuffer gamma and lens shader gamma set to 1.

    The long answer is: As I have said before, there are too many factors involved to be able to say you should just apply the "short answer" and you will always be correct. You need to develop a stategy to be able to test your proposed pipeline and to be able to identify problems. For example create a 50% grey fileTexture and save it as 32-bit exr (actually 16-bit would be enough in most cases). Put it in your scene and render with some lighting that does not over or under expose it. Take the result into after effects. Apply your gamma. The color should still be 50% grey if you got things right. This is just a simple example. You maybe should try it with a color chart so that you see how the colors are working too. The point is you need to understand what you are doing. Knowing how it should work in theory is just the first step.

    Comment by david — August 16, 2009 @ 11:18 pm

  25. David-
    Thank you so much! Thats a great example on how to test the theory of my file textures -rendering - post workflow. I will definitly be trying that out to see the results. Thank you for all your help!

    Comment by smokedogg — August 17, 2009 @ 4:45 pm

  26. Hey David,
    Well doing as you said with the 50% gray example was a great way to test and worked out great. Cleared up some of those questions and got a better understanding of the whole concept. However, I do still have 1 more question. I think this is a dumb question but arent 100% sure of the answer. In my scene i have a couple mia materials using just the diffuse for the color. Do I need a gamma correct node on those? Or are those shaders rendered gamma correctly at there defaults? Thanks!

    Comment by smokedogg — August 27, 2009 @ 11:40 am

  27. Maya's color swatches are designed to look correct on a sRGB monitor, and that means they are like an 8-bit sRGB fileTexture, so they will need to be degamma'd to work in a linear color space. If you want accuracy you could use a mip_gamma_gain connected to your diffuse color. Set your swatch in the mip_gamma_gain and set a 0.455 gamma.
    I rarely do that though. I just have an educated guess and lower the value of the diffuse color and do a test render.

    Comment by david — August 27, 2009 @ 10:18 pm

  28. Well im glad I asked that then because I wasnt sure. I tried following your instructions on how to access that node in the hypershade from your previous blog, but for whatever reason I think im doing something wrong. Wanted to copy and paste or attach my script file onto here but I cant, obviously. Is there a way you can send me the script for the "mentalrayCustomNodeClass"? Here is what I have though:
    // Internal MentalRay Nodes. Not meant to be used with Maya.
    int $enableMIPShaders = (`optionVar -query "MIP_SHD_EXPOSE"`== 1);

    if ((($nodeType == "mip_rayswitch" ||
    $nodeType == "mip_rayswitch_advanced" ||
    $nodeType == "mip_rayswitch_environment" ||
    $nodeType == "mip_card_opacity" ||
    $nodeType == "mip_motionblur" ||
    $nodeType == "mip_matteshadow" ||
    $nodeType == "mip_cameramap" ||
    $nodeType == "mip_mirrorball" ||
    $nodeType == "mip_grayball" ||
    $nodeType == "mip_gamma_gain" ||
    $nodeType == "mip_render_subset" ||
    $nodeType == "mip_matteshadow_mtl" ||
    $nodeType == "mip_motion_vector" ||
    $nodeType == "mip_binaryproxy") &&
    $enableMIPShaders == 0 ) ||
    $nodeType == "misss_physical_phen" ||
    $nodeType == "mi_metallic_paint_output_mixer" ||
    $nodeType == "surfaceSampler" )

    return "rendernode/mentalray/internal";

    return "";

    if ($nodeType == "mip_gamma_gain" )
    return "rendernode/mentalray/material:shader/surface:swatch/mentalRaySwatchGen";
    }

    Is this the inncorrect way of how it is in order or something?
    Thank you very much Dave!

    Comment by smokedogg — September 1, 2009 @ 9:00 am

  29. You were so close. That short line with just return ""; needs to come after your edit, rather than before, otherwise it never gets to your code.

    There is an easier way though. In 2009 you can just enter the following into the maya script editor, to expose all the production shaders:

    optionVar -iv “MIP_SHD_EXPOSE” 1;

    You may have to reopen the hypershade (or even restart maya) to see them though.

    Comment by david — September 1, 2009 @ 8:45 pm

RSS feed for comments on this post.

Leave a comment

You must be logged in to post a comment.

Powered by WordPress | Based on a theme by Roy Tanck