I would have nodded to him and went about my business. I guess I'm weird nowadays, but I don't look at a knife as a weapon without other indications. I see a knife as just part of my everyday pocket load, and (wrongly I know) assume that other men do the same.
I was at a mall, cutting a tag off my kid's toys, and the security guard informed me there was a no weapons policy - I pulled my shirt down to cover my pistol that I thought was printing, and it wasn't. Then I realized the guard was referring to my 1 1/2" mini trapper knife. I asked if he was serious, and was he demanding I leave. He mumbled, "no - I guess you're ok".
While people shouldn't freak out about a pistol, I realize they will. A knife - a non-locking, non tacticool, finger sized knife to boot - should be a revocation of a man card if you get nervous. Sorry OP.
Edit to add:
If he DOES become a threat with his knife, must continue to assess situation:
A) Is he threatening me?
B) Is he threatening my family?
C) Is he a threat only to the liberals around me?
Answer A or B I stop him, permanently if need be.
Answer C - I go about my business, bystanders should arm themselves - not worth my ammo