It's all about practice, well to a point, I wouldn't suggest an empty chamber, that usually requires a second hand. And if you're not practicing weak/strong/1-hand/2-hand then you are missing a very real possibility of one hand being unavailable.
As for a manual safety, I can only speak to my experience. I started out with a S&W59, DA/SA with a manual safety, and for CCW a Walther PP, same thing DA/SA manual safter. And both safeties are flip up to fire. For more than a decade I was at the range 2-3 days a week, draw, DA, SA, SA. That's from hammer down safety on with 2 to center mass and one to the head (plus mag changes and practice clearing), for 10s of thousands of rounds.
At this point I don't even notice my flipping the safety, so it's not costing me any time or thought. And I like the added safety.
Of course the other side to all that practice is I haven't completely switched over to the Sig365 yet, you see it's safety is a flip DOWN, and I don't have the kind of spare time I did back them to practice.
Obviously a manual safety offers a little more physical safety, and the counter is that it's not necessary and adds time.
The time point, to me, is about skill and training. I don't draw then flip the safety. The draw and safety occur at the same time, no extra time is required. And frankly I can't remember the last time I missed the safety during the draw.
If these two operations are costing you seconds, then maybe you shouldn't have a manual safety.
Just remember that virtually every ND that happens when putting a gun in a holster, or drawing it out, happens without a manual safety, and it's not always the persons finger that causes it.