Nope, a long-time associate of mine is the senior court officer at Norfolk Superior Court. He brings in prisoners/suspects into their lock-up every day, totally unarmed. I don't think they even carry mace/OC, just cuffs.
It doesn't vary between courthouses wrt white shirts. What does vary is whether or not they allow LEOs in (and which ones) armed or make them lock up the gun at the entrance.
ETA: All this is strictly about MA courthouses. Don't know about Federal courthouses as I've never even stepped foot in one, but I've heard stories that they don't like municipal LEOs to enter armed either.