Yep. There are people with legitimate medical (not mental) disorders who, genetically, are on what's called "the intersex spectrum." It's been a thing since the early 70s at least, and Johns Hopkins had a famous surgeon named John Money who believed those people needed to be "gender-assigned" at birth, then raised specifically NOT to be binary. IIRC, most of them are fetuses who were exposed to the wrong amount of testosterone over a critical few days during pregnancy. It's a really fine line, apparently.
He was the most knowledgeable guy about all this, and his theories are probably correct. But alas. He was an unethical bastard who didn't believe in doing proper research. So his findings have been discredited (rightly or wrongly), and they've simply ended up muddying the waters of this issue.
The people legitimately on the intersex spectrum? I feel for them, but they are very few (one live birth out of every 100,000 or so), and most of them keep to themselves because they're just trying to get by. For them, sex and gender are two different things. Unfortunately, a whole mess of people have picked up their torch and deluded themselves into thinking they're on that spectrum as well. Those are generally the people NES gets cranky about, I think.