For one rod fishing both the Millers and the Swift, you need a 5 weight. The Millers needs a 5 (at minimum) to be able to throw heavier nymphs and medium streamers, and a 5 will still handle the limited dry fly opportunities that river provides.
On the Swift, a 5 is the heaviest I'll fish. Much more of my fishing is done with a 3wt or a 4wt, but a 5 is fine too. The challenge with a 5wt is handling bigger trout on 6X, 7X or even 8X tippet. For me, I can comfortably fish 7X with a 4wt or below, 6X with a 5wt or below. 7X on a 5wt is tougher for me. 8X is almost out of the question. Unfortunately, there's been a few times on the Swift where I've really felt the need for 7X or 8X. A couple of those times, I've been fishing a 5wt. I had more break-offs than I think I would have with a 4 or a 3wt, but I also caught some nice trout. Additionally, a 5wt is a great thing to have on the Swift for the summer, when higher flows and deeper water sometimes demand a heavy nymph rig...
Anyway, if I was in your position, I'd probably scour eBay. There's a ton of more than good enough graphite 5wt rods out there. You can score a great deal on a used rod. Read the 5wt shootout (google it), but don't feel compelled to buy any of those rods. (Although I do have a St Croix Imperial 5wt that I've had for four or five years, and I love the crap out of it.) I have a 3-piece BPS CV 8'6" 5wt, was like 30 bucks off eBay and has been perfectly good for everything I've used it for. 9' would be my second choice; a bit big for the Swift, but a bit nicer to nymph with, and probably better for the Millers.
Anyway. Any name-brand, or any BPS/Cabelas graphite rod made in the last 10 years that's a moderate or moderate-fast action and that's also an 8'6" 5wt will fill this role well enough. From my personal experience, I'd recommend the St. Croix Imperial if you can find one used. I haven't used the Redington Classic Trout in a 5wt, but I've loved both of my 4wt rods.
I usually buy last year's fly lines at closeout, and spend $35-55 for what was a 65-85 dollar line. Premium fly lines are priced high, but they're worth it.
You'll end up tying your own flies. Plan for it now, and start browsing for good deals on a vise.
You don't need to spend more than a $100 trout reel. Right around that price point, the Orvis Battenkill will give you everything you ever need for trout. Stay away from super cheap reels--unlike rods, where graphite and mass production improvements have helped bring the cost down, cheap reels are still pretty shoddy. Poor tolerances, poor anodizing, that kind of thing. I'll continue to by cheap(er) rods for the foreseeable future, but I'm done with crappy reels.
Feel free to PM me if you want any additional info, or if you want to head out some time--I fish those two rivers fairly regularly.