How do I get the oil filter bypass valve out?

I am bypassing the original oil cooler that was in the Suburban this engine came out of.

I know that I need to get the bypass valve out from the center hole of the spin on oil filter, but I can't figure it out.

So far all I have been able to do is yank out the spring.

Please help.