yeah I would leave it on too. I am not sure the rules, but thought the camo blaze orage was not allowed.
One thing I do is to wash the orange in a non-flourescent hunting detergent. It strips off much of the fluorescent shine of the blaze orange. The deer, with full ultraviolet sight, really see the fluorescent shine.
Turkey...I wear the blaze orange in and out, and then take it off when in-position. Just make sure you are NOT wearing other colors, like red,white,etc. There are cases where hunters here were shot at when he pulled out a red handkerchief to blow his nose! Turkey hunting it is allowed to not wear orange....otherwise we would not be able to ever get close to one