If replacing the cam, definitely replace the lifters! If it were mine (on a budget) I'd do the pushrods also and inspect the rockers for wear and damages and then decide based on results.