Having matching numbers helps keep up the value of a collector car, even if some of the other parts have been changed because a good restorer can use NOS miscellaneous parts to "make the car whole" again.

And just because you have matching numbers doesn't mean the value won't be decreased if you start changing all of the other stuff.

If you want to maintain collector value, don't change a thing unless you have to. Then when you have to, use a NOS exact replacement.

If you just want to have fun, change anything you want - but keep the original parts in storage. You might change your mind someday.