There's specific case law on this in CT. Police are not allowed to even ask for your permit if you are not committing a crime and the courts have explicitly ruled that open carry does not constitute a crime such as disturbing the peace or the like.
I open carry every day, everywhere. I have never had a police encounter over it even when carrying in front of officers.