During the "regular" gun season - not black powder or archery - any firearm may be used to hunt deer. See: http://www.eregulations.com/vermont/
Page 22.
I used to carry a pistol but the weight added to all the rest of the stuff I carry became too much.