I replace the bit in the hole saw with a solid 1/4" shaft. I used a old screw driver shank since it's hardened.
I had forgotten about that. It's a great idea. It keeps the flutes from the drill bit from hogging out the pilothole. It works especially well if you're not using a drill press with the work clamped down.