If the cam and followers are cleaned up and the engine used, the very worst that can happen is the cam will wear, which will stop the valve opening as much as they should, and it will go onto 3 cylinders and a knock may occur.
I would give it a try, other wise replacing the cams and followers is not a 5 min job, plus the cost.
jon