I think I would stop there and tear it down before you wipe it all the way out....

I am gonna say too that you either got a lobe or a lifter that is wearing away....This will send tiny metal particles thru the engine and take out the rest with it....Bearings and crank....

As mentioned under "normal" circumstances flat tappet cams do NOT walk....

You got something goin on here that requires attention....save your wallet and yank it and tear it down...

Normally in a car you don't drive they should be started at least once a week especially in a bigblock engine... The camshaft relies solely on splash for lubrication and if you do not start it over long periods of time you are running it after all the oil has dripped off ( running dry) by the time oil gets splashed back on it's usually too late... You can get lucky but sometimes you don't....I would recommned biting the bullit this time and go with a roller cam and lifters.... sure they are expensive and they'll live rebuild after rebuild.....sorry