riddled

Articles

Masking Layer Styles

If there’s one reason for doing most of the interface design work in Photoshop, it’s gotta be its “Layer Styles”. For those of you living under a rock, Layer Styles allow to add dynamic—duh—layers like shadows, gradients and borders to any bitmap. Combine them with vector shapes and the result is pretty versatile.

However, there’s a small shortcoming of this great feature: you can’t edit pixels in any dynamic layer directly. So for example, if you want to erase one side of button’s border, you can’t just do it.

It takes more than that, and I’d like to show you how it’s done properly. We’re gonna use another neat and undervalued feature called Layer Mask.

Because mere theory isn’t fun, let’s attempt to make something (kinda) useful – like recreating Mac OS X’s deafult window resizer. Heaps of fun, I know. I’ve decided to build it without much finesse, by gluing vector rectangles one near another. How you do it isn’t so important, but you should end with something like this: (max zoom)

To make it look realistic, we need another three lines of white semitransparent pixels. This is where Layer Styles kick in. Choose Drop Shadow from fx menu and tweak values to resemble this:

These settings will create a 1px drop shadow (without any blur), like this:

As you can see now, the shadow appears underneath every pixel. We don’t want that. Choose menu Layer → Layer Mask → Reveal All (or click Add Mask in the bottom of Layers window). Layer’s icon will change to gray square with dot inside. Then pick Pencil Tool (1px) and draw a straight black line, in this area: (red was used just for presentation)

You’ll notice that a part of the layer has disappeared, but it’s still not perfect. To correct it, choose Layer → Layer Mask → Disable (or choose it from layer’s contextual menu). Layer’s icon will change to gray square crossed with red X and you’ll see the improved look:

Tested in Adobe Photoshop CS4. Hopefully this will increase your real productivity, have fun!