Use a slide-hammer if you have one or can borrow one. If not, use a pry bar to put tension (outward) on the axle while rapping on the side of the flange with a brass hammer. If it's been in there for a number of years it may be a little stubborn, but it should break loose without too much of a struggle.