I particularly like the rapid fire phenomenon you are referring to. That is a handy trade off, but understand accuracy is something we all strive for under as many circumstances as possible be they slow fire, target, defense, rapid fire...we all want the bullet to go where we intend it to.
I've been shooting almost daily for 2.5 years. I started with target then moved on to defensive...but all recreational, nothing competitive except of course trying to out-do myself. But I've come a long way so I don't speak out of my ass, rather from some humble but practical experience.
My first piece of advice would be when you practice take your .22 and your higher calibers. When you do a set shoot 10 rounds of .22, then follow with "X" rounds of a higher caliber. Then go back to the .22 and then the higher caliber, etc, etc, alternating between guns each set. Practicing this technique with the varying calibers during the same range session exercises your skills and will help reveal things like flinches, subtle squeezes, anticipation, etc...and if you can spot them you can fix them.
Nothing is more frustrating than picking up a gun and being spot on, then taking another gun to the range or trying a buddies and being useless. There again, there is nothing more gratifying than being able to pick up just about any gun at any time and be confident you will be within "X-th" of an inch from the intended target.
I find that when I practice with multiple calibers each time I shoot, I am more versatile b/c I am used to the change in applied pressure and firearm feedback it takes to handle varying calibers with accurate results. You won't worry that you "always shoot 9mm so how will I be with a .44", you'll just focus on technique b/c you are used to changing calibers. Every time you change a caliber your whole approach shifts to compensate for the change in force, it's a physical reality...doing that more often = more consistency in your shooting b/c you are used to adjusting your technique on the fly. This not only helps you to be more versatile it helps you to be more CONFIDENT with any given caliber. Confidence plays a strong role.
If you have a severe problem you need to spend a LOT of time and focus on SLOWING DOWN. This problem you are having is a fundamental, basic problem. When you shoot focus on the fundamentals of Stance, Site Alignment, Breath Control, Trigger Squeeze and Follow Through. Address each of the 5 fundamentals individually and do not proceed to the next step of firing until you have completed the prior step as perfectly as you can.
What I mean is don't just say "oh yeah, I got the gun, lift it up then breath out and squeeze." Actually address each step individually. It takes all 5 of them to come together for you to pull off an accurate shot.
Right now it sounds like you are hung up on the trigger squeeze, and probably not paying enough homage to the other 4 steps. Ironically you NEED ALL YOUR FOCUS on the trigger squeeze, but not without ALSO addressing the other 4 steps.
When I have a flinch creep back in what helps me is to address the 5 steps, then when it comes to the trigger squeeze I mentally "freeze" in that moment. All I concentrate on in the world is keeping EVERYTHING ELSE FROZEN and making sure
the only thing that moves is the trigger finger.
You want that trigger finger to bend at the 2nd joint from your finger tip. The first joint remains locked and the trigger is sitting right on the mid point of your finger pad. Sit there now and practice...stick out your hand like you are going to shake hands. Now bend JUST your trigger finger at the second joint until it makes a perfect 90 degree angle...then pull some more...that is YOUR ideal trigger squeeze. Nothing else moves but THAT JOINT. Practice that a bit until you get some muscle memory - this will definitely add some crispness to your trigger squeeze. If you can focus on JUST THAT while doing all other 4 fundamentals you will get your accuracy.
From there you can move on from the flinch and work on other goals like defensive shooting, rapid fire, other calibers, other guns, etc...
Also check out the Todd Jarrett pistol grip lesson on this page. Without a solid grip you are shooting in the dark (hahaha). Before I saw this video I would improve then lose it, then get good again and lose it in a frustratingly random fashion...this grip lesson should help take that much frustration away and then the rest is up to you:
http://www.zombieoutfitters.net/zombiegallery.html
Good luck!