Randomization in AI Decision-Making Can Enhance Fairness

Integrating structured randomization into AI decision-making processes can improve fairness when allocating scarce resources or opportunities, according to researchers from Northeastern University.

As machine-learning models increasingly help organizations make critical decisions, such as screening resumes or ranking transplant candidates, addressing inherent uncertainties and biases becomes crucial.

Research Insights

The research, presented at the International Conference on Machine Learning, suggests that fairness methods typically used to mitigate biases in AI models are insufficient to address structural injustices and inherent uncertainties. Structured randomization can help balance fairness and efficiency in certain scenarios.

For instance, if several companies use the same AI model to rank job candidates without randomization, a deserving individual could consistently rank low due to how the model weighs specific responses. Introducing randomization prevents such consistent exclusions, ensuring fairer resource distribution.

Randomization Framework

The researchers propose a framework to introduce randomization into AI decisions through a weighted lottery. This method can be tailored to specific situations, enhancing fairness without compromising model efficiency or accuracy.

Shomik Jain, a graduate student at MIT’s Institute for Data, Systems, and Society (IDSS), highlights the importance of addressing uncertainties in AI predictions. As decisions scale, the inherent uncertainties in these scores can amplify. “Fairness may require some sort of randomization,” he explains.

Implications for Fairness

The study builds on previous work that identified harms in deterministic AI systems. The researchers found that deterministic models could reinforce biases and systemic inequalities present in training data. Randomization, as a statistical tool, addresses these fairness demands from both systemic and individual perspectives.

John Broome’s philosophical ideas on using lotteries to award scarce resources underpin the researchers' analysis. Acknowledging different claims to resources, fairness requires respecting all individuals' claims, preventing systemic exclusion or patterned inequality.

Structured Randomization in Practice

The researchers used statistical methods to quantify uncertainty and determine the necessary level of randomization for various situations. In kidney allocation, for example, the uncertainty in projected lifespans warrants higher randomization levels.

While randomization can improve fairness in many contexts, it may not be suitable for all, such as criminal justice. However, it could enhance fairness in areas like college admissions. Future work will explore other use cases and the impact of randomization on competition, prices, and model robustness.

“We are offering randomization as a tool,” says Ashia Wilson, co-author and principal investigator at MIT. “How much you are going to want to do it is up to the stakeholders in the allocation to decide.”