Sculpt masking, last looks

The sculpt masking code is now finished-ish. I’ve just completed my own review of the code, checking each commit and fixing various bugs.

I’ll be submitting it for code review soon (no rush since it won’t be going into trunk until the start of the 2.64 release cycle), but before I do there’s time for more user testing. No doubt there are still bugs there just waiting for you to find them! Check also the masking documentation on the wiki in case there is anything missing there.

The code is as always available from the masking branch of my github repository. Post builds on GraphicAll if you got ’em.

Builds update:

And just for fun, here’s the commit log as it currently stands, reverse chronological order:

commit a4255631516baaad1e573f1c2f401f83c5d49840
Author: Nicholas Bishop
Date:   Sat Mar 17 08:05:01 2012 -0400

    Add mask-drawing support to GPU_Buffers.

    * For VBO, add color to the VertexBufferFormat structure as three
      unsigned bytes. Since mask elements are scalar the three color
      components are identical to eachother, but the fixed-function OpenGL
      pipeline requires colors to be either three or four components.

    * For the same reason, multires VBO drawing now copies into the
      VertexBufferFormat format as well.

    * Regression: material colors will not show up correctly now, masks
      colors are overriding. Not sure how to fix this nicely (would be
      much easier to fix if drawing with vertex shaders.)

    * Also, masks will only draw PBVH drawing, so only 'solid' drawing
      will work correctly with masks.
commit 466b9027e85080e7e37afef63b409aa6d6f08815
Author: Nicholas Bishop
Date:   Sat Mar 17 07:13:47 2012 -0400

    Use VertexBufferFormat for multires VBO.
commit 578b027bb848f9e93615b5544fb09ad8354c9067
Author: Nicholas Bishop
Date:   Thu Feb 16 06:54:32 2012 -0500

    Add keymap and menu entries for masking.

    * Add CTRL+IKEY to invert the mask.

    * Add ALT+MKEY to clear the mask.

    * Change the 'Hide' menu in sculpt mode to 'Hide/Mask', adds entires
      for clearing, filling, and inverting the mask, as well as hiding
      masked regions.
commit 93d8a02fbbb645ae59c6b6afdb2645ace00da6fa
Author: Nicholas Bishop
Date:   Thu Feb 16 10:20:57 2012 -0500

    Add support for hiding masked regions.

    Add a new mode, PARTIALVIS_MASKED, to the PAINT_OT_hide_show operator.
commit 6f5d05db0ca923e2d1276c014be1b0f14ad6b03a
Author: Nicholas Bishop
Date:   Thu Feb 16 06:53:45 2012 -0500

    Add a paint mask operator to clear, fill, or invert the mask.
commit 4b75f63e427bcfce2dadb0c88bb8028a70ae7523
Author: Nicholas Bishop
Date:   Tue Feb 14 21:37:00 2012 -0500

    Update the keymap for the mask brush.

    * Add MKEY as a toggle for the mask brush. We could use ALT similar to
      SHIFT toggling the smooth brush, but it would conflict with MMB
      emulation (not to mention many window managers.)

    * When the mask brush is active, SHIFT toggles it into smooth mode.
commit 916da0c546600dd4c22a3d9950eb4b76815923ac
Author: Nicholas Bishop
Date:   Thu Mar 8 16:16:28 2012 -0500

    Add mask brush for sculpt mode.

    The mask brush currently has two modes, 'draw' and 'smooth'.
commit 51e4801cf5eb19fa342dbfa4079c9fc5266528ae
Author: Nicholas Bishop
Date:   Tue Mar 20 01:42:44 2012 -0400

    Add new mask-brush icon from Julio Iglesias.
commit 167bd4087efae4fb4e4daa8607f51f24757c6ffb
Author: Nicholas Bishop
Date:   Fri Feb 10 21:39:08 2012 -0500

    Use paint mask when calculating sculpt strength.
commit 427edb1d57bfc3e8b09162e3b84a1c3f78728236
Author: Nicholas Bishop
Date:   Sun Mar 4 23:55:32 2012 -0500

    Add undo/redo support for paint masks.
commit d1883a3d3965d3083bea642f19812b66d216a36a
Author: Nicholas Bishop
Date:   Sat Feb 11 06:24:32 2012 -0500

    Ensure mask layers are always present in sculpt mode.
commit 12c513b025be9544c6bba49925743b466228a249
Author: Nicholas Bishop
Date:   Mon Feb 20 22:50:48 2012 -0500

    Copy GridPaintMask to vertex paint mask when applying multires.

    Adds new subsurf_copy_grid_paint_mask() function similar to
commit 05afcb82ca4c2b81a1eab325a2dcde9cc8320183
Author: Nicholas Bishop
Date:   Mon Feb 27 19:29:08 2012 -0500

    Add mask support to CCGSubSurf and multires.

    * Add new CCG function ccgSubSurf_setAllocMask(). Similar to to
      ccgSubSurf_setCalcVertexNormals(), it sets whether the CCG elements
      have a mask layer and what that layer's offset is. Unlike normals
      however, it doesn't change any behavior during CCG calculation; it's
      there only to give CCGKey information on the mask.

    * Add a new flag to _getSubSurf(), CCG_ALLOC_MASK. If set, space for
      an extra layer is allocated, but the number of CCG layers is not set
      to include it. This is done because GridPaintMasks are absolute,
      rather than being relative to the subdivided output (as MDisp
      displacements are), so we skip subdividing paint masks here.

    * Add a new flag to subsurf_make_derived_from_derived(),
      SUBSURF_ALLOC_PAINT_MASK. This controls whether CCG_ALLOC_MASK is
      set for _getSubSurf(). Related, masks are never loaded in during
      ss_sync_from_derivedmesh(). After subdivision is finished, if the
      alloc mask flag is set, the number of CCG layers is increase to 4
      with ccgSubSurf_setNumLayers().

    * Add a new flag to multires_make_from_derived(),
      MULTIRES_ALLOC_PAINT_MASK. Not all multires functions need paint
      mask data (e.g. multiresModifier_base_apply.) This flag is always
      set in MOD_multires.c so that subdividing a mesh with a mask updates
      properly even when not in sculpt mode.

    * Update multiresModifier_disp_run() to apply, calculate, and add mask
      elements. It's almost the same as the existing operations with xyz
      coordinates, but treats masks as absolute rather than displacements
      relative to subdivided values.

    * Update multires_customdata_delete to free CD_GRID_PAINT_MASK in
      addition to CD_MDISPS.

    * Update multires_del_higher() to call the new function
      multires_grid_paint_mask_downsample(), which allocates a
      lower-resolution paint mask grid and copies values over from the
      high-resolution grid.
commit fe73e0301e9d00400610a11e15756bce9c6fbcb5
Author: Nicholas Bishop
Date:   Sat Mar 17 03:49:31 2012 -0400

    Add GridPaintMask accessor to paint.c.
commit e39465b53eac56ebdfd2602b795fa60536ae9c0d
Author: Nicholas Bishop
Date:   Fri Feb 10 21:27:58 2012 -0500

    Add paint mask access to the PBVH vertex iterator.
commit 4c3639996cc37cff394ea511dce28b7baefcae3b
Author: Nicholas Bishop
Date:   Fri Feb 10 20:58:20 2012 -0500

    Add access to mesh vertex customdata to the PBVH.
commit 9e1f023e477dde407da43bdb95cfddbf13390d3a
Author: Nicholas Bishop
Date:   Fri Feb 10 20:36:17 2012 -0500

    Add DNA and customdata entries for paint masks.

    CD_PAINT_MASK is a layer of per-vertex floats for non-multires
    meshes. Multires meshes use CD_GRID_PAINT_MASK, which is a layer of
    per-loop GridPaintMask structures. GridPaintMask is similar to MDisp,
    but contains an array of scalar floats.

    Note: the GridPaintMask could be folded into MDisp, but this way
    should be easier to add mask layers in the future (if we do fold
    GridPaintMask into MDisp, the mask array should probably be an array
    of arrays with a 'totmask' field so that mask layers can be easily

    Includes blenload read/write support for CD_PAINT_MASK and
commit 158ad101882ad81b4adcb0bda09a8cafc26cc717
Author: Nicholas Bishop
Date:   Sun Feb 26 06:13:53 2012 -0500

    Replace hardcoded DMGridData structure with CCGElem/CCGKey.

    * Changes to DerivedMesh interface: DMGridData has been removed,
      getGridData() now returns an array of CCGElem pointers. Also added
      getGridKey() to initialize a CCGKey (implemented only by

    * PBVH: added BLI_pbvh_get_grid_key().

    * A lot of code is affected, but mainly is just replacing,, and sizeof(DMGridData) with the
      CCG_*_elem functions, removing the reliance on grid elements of
      exactly six floats.
commit 9e55df6fa531f8175107fa9dfdadb5e690d9fa12
Author: Nicholas Bishop
Date:   Thu Feb 23 02:41:04 2012 -0500

    Add CCGKey/CCGElem for accessing CCGSubSurf elements.

    CCGKey caches information about the CCGSubSurf element layout. This
    data, along with the CCG_* inline functions, allows access to
    CCGSubSurf elements with an arbitrary number of layers (as opposed to
    the hardcoded DMGridData structure which assumes xyz coordinates
    followed by three normal components.)

    The CCGElem structure is declared but not defined anywhere, just used
    as a convenient type.
commit a20fc1a4d78acd3fbe5ff9a88a4050e5debc8bda
Author: Nicholas Bishop
Date:   Wed Feb 22 23:29:07 2012 -0500

    Modify CCGSubsurf to subdivide an arbitrary number of (float) layers.

    The layout of vert data in CCGSubSurf is almost the same; previously
    it was three floats (for xyz coordinate) optionally followed by three
    floats for the normal. The only change is that the first three floats
    can now be any number of floats.

    * _getSubSurf takes a numLayers parameter to set the number of layers,
      stored in CCGMeshIFC.numLayers.

    * All calls to _getSubSurf currently have numLayers set to 3, except
      for UV subsurf, where it is reduced to 2 (with a corresponding
      change when reading the results out to use float (*)[2] rather than
      float (*)[3].)

    * The various VertData* macros in CCGSubSurf.c are now functions that
      take a CCGSubSurf pointer, which provides access to CCGMeshIFC,
      which has numLayers.

    * Add ccgSubSurf_setNumLayers() to the API. Only changes the number of
      layers that get subdivided, doesn't change the amount of memory
      allocated. So if space for N layers is allocated, it's safe to set
      the number of layers to less than N, but not more.

    * The rest of the changes are just adding the 'ss' parameter.
commit 8d7b66647b53aa72f961ab0211d5984c13b4f731
Author: Nicholas Bishop
Date:   Tue Feb 14 21:34:50 2012 -0500

    Add new options to PAINT_OT_brush_select, toggle and create_missing.

    The toggle option, if enabled, will toggle back and forth between two
    brushes. (The first brush of the desired tool type will be toggled to,
    running the toggle again switches back to the previously selected

    If no brush of the desired type is found, and the create_missing
    option is enabled, a new brush of that type will be created and set.
commit d11dd91e449740b22996da1045f11715c6649f68
Author: Nicholas Bishop
Date:   Tue Feb 14 21:24:17 2012 -0500

    Add an RNA access function to get an enum item name from its value.

    New function is RNA_enum_name_from_value.
commit a7b510d7b0554284f0f2cc8bd47a535597b14a22
Author: Nicholas Bishop
Date:   Wed Apr 11 16:23:32 2012 -0400

    Code cleanup for multires_dm_create_from_derived().

    Changed name to multires_make_derived_from_derived() and parameter
    order to be more similar to subsurf_make_derived_from_derived().

    Added MultiresFlags enum with flag values to replace the local_mmd and
    useRenderParams parameters.
commit 1b29588e26be79143eeee8d47097e8fe4c924a7c
Author: Nicholas Bishop
Date:   Mon Feb 27 19:20:27 2012 -0500

    Code cleanup for parameters of subsurf_make_derived_from_derived.

    Replaced four boolean parameters with a single flag and a new enum,

8 thoughts on “Sculpt masking, last looks”

    1. Sculpt masking has been in trunk for a bit, so pretty much any recent build (e.g. from the buildbots or from graphicall) should have it.

  1. Really because I’m using 2.63a and it isn’t a masking option, I’ll check buildbots or graphicall. Thanks so much.

      1. Cool, I downloaded ver 2.63.11 it’s ssssssooooooo nice to have masking and I love the skin modifier I got to push them both a little further “BUT BIG UPS” to Nick awesome stuff as usual. Hey do think you’ll be able (I mean have time for I know your able) to make a scale brush so when I mask off areas I scale on all axis? You can check out my website and what I’m working on, my latest post is a super hero sculpt and that’s why I’m so interested in masking.

Leave a Reply

Your email address will not be published. Required fields are marked *