I agree with pretty much all of that.
Interestingly, I was first introduced to the term in 2008, when I took a class on the inclusion and consideration of folks with disabilities (primarily learning and developmental disabilities, as you might guess). And some of the contexts in which the idea was introduced were certainly rather black-and-white ("reductive" if you will); our society's history of interactions with disabled people includes some rather appalling stories. Of course, most issues on the subject today are likely to be more nuanced (few argue for forced sterilisation of the mentally handicapped any more), and I certainly agree with it being a complex topic. To be perfectly honest I can't remember the last time I used the term myself. But as someone who has seen it used in a proper fashion, I'm going to stand by my resentment of Super's comment.
Snowfire: mmm, there's an obvious difference with men's rights. If someone told me, "I'm in favour of men's rights", I would not dismiss that argument or consider it a toxic term. There are, after all, some very valid discussions worth having on the subject, and "men's rights" remains the best label for that subject I'm aware of. It's the "Men's Rights Movement/Activism" (usually capitalised!) which is more of a banner for toxicity, and yeah if someone told me they were part of that they would immediately get on my "regard with suspicion" list. But I think it's fair to consider an organisation toxic. Terms I'm less sure of.