If a small thing like that bothers you then you should not carry a gun. If he asked you to see your licence. You say yes, hand him the lic. he gives it back and thank you its over you go on your way.
Well, yes and no. Certainly in this exact scenario, you are required by law to exhibit your license if the officer asks to see it, so you've got quite a battle ahead of yourself if you decide not to comply. Given, that, I agree that just showing the license and moving on is overwhelmingly the best choice.
But to suggest that someone who is bothered by an unwarranted search by a government representative, even if it is not terribly inconvenient, shouldn't carry a gun is patently crazy. People exercising their fourth amendment rights is the only way to preserve those rights.
Now I'm not suggesting hassling any LEO who approaches you. I know the majority of them are decent folks who are just trying to do their jobs. But you can certainly be firm in your communication of your desire to reserve your rights, while still being polite and respectful.