Universal human nature is a thing, but so is individual and group preference. It's part of why we all group together, but in different freaking groups. Given liberty to do so, groups of people will notice outliers and make them unwelcome. I understand the idea at a very deep level because I'm pretty close to being a psychopathic sociopath in the IQ sweet spot. If I were a little smarter I'd be a CEO, but if I were very much dumber I'd be in prison. I like a fair number of individual people that I meet, but pretty much hate humans and most of their social institutions. If I could find a collection of like-minded people I'd be brutally discriminatory in judging new-comers.
I often wonder how many people are like me in being smart enough to play nice with others and stay out of prison, but not quite smart enough to fuck the world the way I'm inclined to?
But you get to choose and therefore live with the outcomes.
"Don't do the crime if you can't do the time."
Most, many people WOULD commit crimes (as defined by the larger society) if they KNEW they could get away with it. Or if the majority morality accepted it - ie slavery.
That is the social contract Rousseau talked about. It is how we move from small tribes to larger groups. If you don't want to be part of it, you have the liberty to isolate yourself.
I get the tyranny of the majority - is both a desire and a detriment. Choices have consequences however. Such a group as you suggest, based on the criteria you suggest, would lead to it's destruction.