Out-group Homogeneity Bias
The tendency to perceive out-group members as more similar to each other than in-group members.
Out-group homogeneity bias is a well-documented cognitive phenomenon in which people perceive members of groups they do not belong to as more uniform and interchangeable than members of their own group. In human psychology, this manifests as the intuition that "they all look alike" or share the same attitudes, while one's own group is seen as richly diverse. When this bias is embedded in training data — which is collected, labeled, and curated by humans — machine learning models can inherit and amplify it, learning to make coarser, less individualized predictions about people belonging to demographic, cultural, or social groups that are underrepresented or viewed as "other" by the data's creators.
In practice, out-group homogeneity bias in ML systems emerges when training datasets lack sufficient diversity or when labelers apply less granular distinctions to out-group members. A facial recognition system trained predominantly on images of one demographic may learn finer-grained features for that group while collapsing distinctions within others, leading to higher error rates for underrepresented groups. Similarly, natural language models may associate out-group identities with a narrower range of attributes, reinforcing stereotypes in downstream applications like resume screening, content moderation, or risk assessment tools.
Addressing this bias requires both technical and procedural interventions. On the data side, this means actively auditing datasets for representational imbalances and ensuring diverse, high-quality labeling across all groups. Algorithmically, fairness constraints and disaggregated evaluation metrics can surface differential performance before deployment. More broadly, recognizing out-group homogeneity bias underscores why AI fairness cannot be reduced to aggregate accuracy alone — equitable systems must treat individuals within every group with the same degree of nuance and specificity, regardless of their relationship to the majority represented in training data.