No, it's fine for what it is, which is pretty weak temporary bonding. If you use really strong magnets, there can be concerns with what that's doing to the fire-controls internally, but you're probably not talking about magnets that strong. Even if you are, which you're not, you're probably exerting magnetic attraction laterally, where it's unlikely to disengage any drop safeties, etc.
If you do lightly magnetize anything, it won't do anything but perhaps increase fouling from any magnetic debris in your gun. Not a major concern outside of science fiction movies. And also, shooting the gun should randomize the magnetic dipoles pretty effectively.
Also, neither brass nor lead are magnetic. If you're worried about magnetically-induced failure to feed, because the Terminator zapped your magnets with magic rays, use brass cases with lead bullets.
Science!