Smudge/Wet Brush interaction with selection

[barel]

Would it be possible to make the smudge/wet brush not sample pixels outside the selection? Right now selection only limits where new pixels can be placed, I think it would be more useful if selection could be used to protect the intended area of effect from adjacent colours.
Here’s an image to illustrate what I mean, left is what happens when the tool is set to sample the current layer only, the tool doesn’t let the white of the background onto the layer, which is what I’d expect when selecting an area that smudge affects.
Right is what happens when with a selection and the smudge sampling all layers, the white of the background leaks in.
Middle has the green on the same layer and smudge sampling the current layer, the green leaks in.
This leads to muddy colours when trying to blend tightly placed tiles and sort of defeats the purpose of the smudge tool for m due to needing to erase and manually correct a lot of areas.

[Lucky Clan]

We will consider that in the future, it is not easy to implement (app must somehow "guess" pixels outside the selection)

[barel]

Thank you for considering, obviously I have no idea what would be involved in this, it’s just something I’ve noticed while using the app.
As an aside, CSP and Paintstorm seem to have this feature from my quick testing, though CSP slips up from time to time.

[mattnava]

We will consider that in the future, it is not easy to implement (app must somehow "guess" pixels outside the selection)

Wouldn’t the solution here be clamp the colors of pixels on the edge of the selection outward beyond it? Essentially, if there is an active selection the smudge/wet effect would sample a temporary version of the image where outside the selection, the colors are clamped to those on the selection border. Probably some sort of jump flood type fill operation could be used to generate that buffer to sample from. Basically repeat edge pixels.


This would be a real nice feature!

[Lucky Clan]

It looks easy for rectangular selection, but selections can be irregular, with holes etc...

We will consider that in the future, it is not easy to implement (app must somehow "guess" pixels outside the selection)

Wouldn’t the solution here be clamp the colors of pixels on the edge of the selection outward beyond it? Essentially, if there is an active selection the smudge/wet effect would sample a temporary version of the image where outside the selection, the colors are clamped to those on the selection border. Probably some sort of jump flood type fill operation could be used to generate that buffer to sample from. Basically repeat edge pixels.


This would be a real nice feature!

[mattnava]

The jump flood thing Im talking about should handle any shape selection/holes/etc. Its possible to visualize using substance designer's distance node, so I thought Id share an an image to clarify what Im talking about. Im not super sure about how the math works, but a distance field from the selection is used to extend edge colors outward from the mask's shape. If something like this was used for sampling colors for smudge/wet brushes when a selection is active, it would ensure colors outside the selection would not have an effect. It might have its own edge cases, for like small holes causing color bleeding from colors across the hole that are also in the selection, but I think thats acceptable and still an overall improvement.

[mattnava]

One more example with a more wild shape!

[mattnava]

Here is an article about how to implement this "dilation" effect I was trying to describe. This is the sort of thing I was imagining might be helpful for smudge/wet effects picking colors outside of selection boundaries.

https://shaderbits.com/blog/uv-dilation

Some images from it: