Are you sure it's oil and not antifreeze?

A couple of thoughts . . .

If it's oil, maybe a leaky intake manifold gasket, letting engine oile migrate up the shaft of the bolt. The fix is a new intake gasket.

If it's antifreeze, pulling the bolts and putting sealant on the threads might stop it.