Starratt, Milwaukee and Ridgid are good ones.

I think Milwaukee makes the Starratt hole saws, the arbors are identical.

Used to buy Milwaukee's at Orchard Supply Hardware.

Run low speed, use the proper cutting oil WD40 is ideal for aluminum and in fact is preferred by many machinists.

Hotrod46's suggestion to use a blank pilot hole looks like a good one.
Haven't tried it yet, but I will.

One thing that will help - especially in aluminum - is to lay out the cut so you can drill three equally spaced 1/4" or so holes just inside the cut line.
The hole saw should cross the edge of the drilled holes.
The drilled holes give the cuttings a place to escape to and you'll find the hole saw doesn't load up with aluminum so bad.

I've cut a bunch of 18 gage (.063) stainless for dash panels with hole saws.
Tough as the stainless is, the hole saws used are still sharp.

Stay away from Black & Decker hole saws.
They take a different arbor and don't cut as accurate a hole as the abovementioned brands.
They seem to be for wood only . . . although if nowadays the B&D hole saws indicate they are bi-metal they may work ok.

To be fair I have to mention that I cut four 1" hole sawed holes in a .120 thick frame rail with a B&D hole saw using 30 wt for cutting oil and it did ok.
Even so, I find other brands cut more accurate holes in the larger instrument panel sizes.