What Statistics Can’t Tell Us in the Fight over Affirmative Action at Harvard

Andrew German, Shepard Goel and Daniel Ho:

Yet setting aside legacies and athletes, Asian American applicants are still admitted at lower rates than whites with comparable academic and extracurricular records.

This remaining disparity largely boils down to admissions decisions based on personality, geography, and family. Harvard assesses the “personal” qualities of applicants on a scale from 1 to 6 (“outstanding” to “worrisome”), and on this dimension Asian Americans are rated lower on average than whites. Harvard officials describe “personal quality” as “a subjective determination of a combination of many, many factors,” including “perhaps likability, [and] character traits, such as integrity, helpfulness, courage, kindness.” The university likewise considers where applicants live and their parents’ occupations. Harvard may, for example, favor students from rural communities and disfavor the children of engineers.

section separator

Adjusting and Over-Adjusting for Differences

In many studies of discrimination, race-based or otherwise, an apparent disparity disappears once one accounts for other factors. In Harvard’s case, the gap in admissions rates between Asian Americans and whites largely vanishes after adjusting for differences in legacy status, athleticism, personal ratings, geography, and parental occupation.

Even if a variable helps to explain away a disparity between groups, that variable may itself be the product of discrimination.

In assessing whether Harvard intentionally discriminates against Asian applicants, a key question, then, is whether the factors the university uses to guide admissions decisions are themselves appropriate. If personal ratings were awarded in racially discriminatory ways, it would be inappropriate to appeal to them to explain disparities in admissions. Likewise, if personal ratings bear little relation to legitimate educational goals, then differences in admissions rate should not be justified by differences in the ratings.

This statistical issue—where controlling for illegitimate factors masks evidence of discrimination—is an instance of what is sometimes called “included-variable bias” (as opposed to the inverse problem of “omitted-variable bias,” which entails leaving out variables that ought to be included). In our own research on stop-and-frisk policing, we find that one can underestimate racial bias by improperly controlling for factors such as an officer’s judgment about whether a suspect is behaving “furtively,” since such judgments are often related more to race than risk.