Flat shading in sculpt mode

Spent a little time today looking at improving one small aspect of sculpt drawing: flat-shaded faces in solid view mode with VBO enabled.

In current trunk, flat shading in sculpt mode looks quite different depending on whether VBO is enabled or not. When VBO is not enabled, and immediate mode is used to draw the mesh, each face (triangle or quad) gets drawn separately. Before drawing the face, a single averaged normal and mask value is calculated. The normal and mask values are used for each vertex of the face. In contrast, VBO drawing uses a shared vertex normal and mask value for every face that uses the vertex. This behavior is correct for smooth shading, but looks strange for flat shading.

The benefit to using a shared vertex value is lower storage requirements; the mesh’s faces can simply reference vertices with an index (and the index can usually be a short, so two bytes per index.) When flat shading is enabled, however, OpenGL uses only a single vertex’s value for the entire face. On any particular face this generally looks OK, but the problem is evident when adjacent quads appear to have the same normal despite facing different directions.

Flat shading in sculpt mode

Click to embiggen
Left: current trunk. Middle: averaged normals and interpolated mask. Right: averaged normals and averaged mask.

I’ve implemented a simple fix for this, available from the sculpt-flat-shading branch on Github. If the mesh is flat shaded, the index buffer is scrapped in favor of duplicating vertex values. The GPU storage requirements are in general slightly higher, but the output looks much more correct.

In the image above you can see two variations of the fix. Both have flat-shaded normals, but the middle uses interpolated mask values while the right uses a single average mask value. Using an averaged mask value seems to look a bit better, although it’s technically a bit inaccurate as sculpt brushes operate on vertex mask values, not the average.

One further issue which affects smooth shading is that interpolation over quads doesn’t look very good with OpenGL. Mask values and normals don’t look smooth because OpenGL tessellates quads into two triangles. The vertex values are interpolated within each triangle, without regard to the missing corner. It’s technically possible to solve this with shaders. Instead of using regular vertex attributes, the vertices just get texture coordinates and the attributes are put into a texture. A bilinear (or even bicubic) interpolation can then be coded in the shader. Such fanciness is probably of questionable usefulness though, and at the very least will have to wait on the completion of other improvements to the various drawing systems in Blender.

My current plan is to wait until 2.64 is released, then commit the flat-shading fix (with averaged mask values.)

5 thoughts on “Flat shading in sculpt mode

  1. Hi nicholas, I was trying to think of a way to solve flat shading without raising the storage requiremens for normals by storing the face normal to the vertex that is sampled by gl in flat shaded mode. This requires a way to map vertices to faces in a preprocessing step but should work quite well in general. Unfortunately I haven’t thought of a way to doo this robustly but some quick googling revealed this, it may help you with a few ideas http://www.opengl.org/discussion_boards/showthread.php/172219-VBO-with-flat-shading

    • Another interesting insight is that triangle strips usually have one vertex per face, so essentially stripifying the surface solves the problem Actually the problem is avoiding getting to a situation when matching vertex indices to faces where you have a face whose vertex indices are all taken by adjacent faces for use as “normal” vertices.

      • Interesting stuff. Multires flat-shading does something like that, drawing quad strips and setting one normal per quad. This works OK, although the borders of each multires grid look a bit off. I hadn’t seen before the trick suggested at the beginning of that thread though, adding one extra vertex per triangle. That seems like it could work well.

        Looks like the other ideas will require GLSL, which has to wait for an updated solid view (I guess the GSoC viewport upgrade may help with this?)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>