I would have to agree with the silicon wax and grease remover. You have to use something that will cut wax, which is made to resist water.

A sponge or soft cotton cloth would be the least harmful. If it is cheap paint, or primer, then it may not stand up to serious cleaning.