Same issue here. Sometimes I get delay as big as 4 seconds. Restarting app or system helps somewhat, but still the lowest delay is about 1 second. It's not tied to stylus double tapping, happens with quick menu and keyboard hotkeys as well. Happens when I change color in the color window sometimes too. I use Ipad pro 13 second gen. Swapping tools is one of the most used actions, I hope this gets fixed.