ethics - Is it ethical to apply different criteria for graduate admissions based on country of undergraduate study?

We have undertaken a small statistical study of M.S. students in our department, including their application information and their eventual performance in our program. The goal is to develop criteria for making admissions decisions for new applicants based on their likelihood of success in our program (as predicted by the performance of recent, similar students). I won't get into the details of the methodology.

One outcome of this study was that different attributes predict success for students with undergrad degrees from different countries. For example, for applicants to the program from schools in country X, but not schools in country Y, undergrad GPA is correlated with the students' GPA in our program; for schools in country Y, but not X, GRE scores are correlated with the student's GPA in our program. (I'm simplifying a lot here.) A "toy" example I just completely made up to illustrate is shown below:

(of course, for real applicants the criteria and the relationships are more complicated)

There are many possible reasons for this: for example, we could think that the grading system is more consistent in X so undergrad GPA is a better predictor there, and in X students study "for the exam" so the GRE becomes virtually meaningless as a measure of knowledge. I could speculate, but I don't think it would be helpful. The bottom line is, we find that different factors predict student success among different populations.

Therefore, if we wanted to admit students based on their likelihood of achieving a certain GPA in our program, we would apply an undergrad GPA cutoff for students from X, and a GRE threshold for students from Y. (Again, this is vastly simplified from the criteria our study actually suggested.)

Is it fair to apply different criteria to students with undergrad degrees from different countries in admissions decisions?

Does the answer change if this would significantly skew the admissions decisions in favor of a particular country of undergrad study (because statistically, applicants to our program whose undergrad degrees are from X have done much better than those with undergrad degrees from Y).

My concern is that we're effectively saying, "Students with undergrad degrees from Y with a GRE score < T will be rejected, but students with undergrad degrees from X with GRE < T may still be considered for admission (pending other criteria)."

On the other hand, if we ignore these statistics and reject students with undergrad degrees from X with low GRE scores, we are rejecting applicants even though we have no valid reason to believe that they won't do well in our program.

To those who doubt the results of the study:

• When I say the study is "small" I don't mean it isn't statistically significant - just that it was not designed to be generalizable beyond our applicant pool. (This is the same reason why I won't give too many details about the study - I don't want anybody to read it and try to generalize from our results.)


• As we know, the sample size is not the only factor that determines whether a given effect is significant. We found that the results are significant, given the sample size.



• The results also seem "sane" (which of course is subjective). It's not unexpected that undergraduate grading standards (and the standard-ness of grading standards) differ by country; or that different educational systems and cultures prepare students differently for standardized exams like the GRE and TOEFL; etc. The specifics of the results (i.e., which criteria are good predictors for which undergrad country) are consistent with what students who studied in those countries have told us about grading standards, student culture, and exam prep. So, we really have no reason to doubt them.

Answer

You asked an ethical question and got a lot of scientific, legal, and political answers.

Ethically, you should:

1. 
   

   Do your best to admit people based on merit.

   
   


2. 
   
   

   Be open about your policies.

   
   


3. 
   

   Make a serious and competent effort to ensure that your statistical methods are valid logically and empirically, and that they can withstand scrutiny from people who are experts in the field.

   
   


4. 
   

   Carefully consider the historical legacy of racism, colonialism, and nationalism, and work hard to make sure that you aren't inadvertently reinforcing this legacy.

   
   

Your behavior so far is probably far more ethical than that of most people in your position, since most such people are probably secretive about their practices (you publicly asked for advice) and probably apply various heuristics without carefully considering whether those heuristics could withstand professional scrutiny.

You might want to expand this small, informal, unpublished, nonprofessional study into something more serious and systematic, done by people who have expertise in psychometrics and the (very difficult) methodological issues of the social sciences.

You are unfortunately operating within real-world constraints imposed by (1) the existence of countries where GRE scores are fraudulent, and (2) the existence of countries with such poor undergraduate education that undergraduate GPAs don't mean much.