If We Are Going to Pace the AI Frontier, Equity Must Set the Pace

AI developers and researchers have been in the news recently, sounding the alarm that AI development is moving too fast and could be very dangerous for humanity. Recently, Anthropic CEO Dario Amodei penned an essay titled “We must Pace the Frontier,” which called for more regulation, including independent third-party monitoring of AI companies as models are built.  

But this recent caution, including the public resignation of AI researchers, is not new. Experts like Dr. Joy Buolamwini, founder of the Algorithmic Justice League, began exposing the dangers of AI in 2016, popularizing the term “the coded gaze.”  While working at the MIT Media Lab, she discovered that commercial facial analysis software failed to detect darker skin tones. Her research established that AI systems carry immediate, structural risks that disproportionately harm marginalized communities. Big Tech routinely puts out un-audited, opaque models into public and commercial life without proper safety checks or oversight.  

The Hugging Face breach that caused a recent media firestorm because OpenAI agents escaped the controls of a testing environment, took steps to hide their behavior, falsify data, and evade automated monitoring. After discovering this event where AI outsmarted humans, another discovery was made that happened in May, where OpenAI agents took over a German programming site and were able to create backup pages so the messages could not be deleted. How many more issues like this are out there that the public does not know about?

Some critics are skeptical that this new fixation on the part of big tech, citing the dangers of AI and the dire warnings of its potential to destroy humanity, serves as a cover for corporate self-interest and distracts from the immediate, concrete harms. Lobbying governments to impose stringent compliance standards and safety audits on AI models raises the cost of entry, making it very difficult for smaller startups or open-source developers to compete and consolidating market power in the hands of a few dominant players.

The questions become:

• How concerned are developers about such issues as ensuring equity in AI models versus finding more ways to line their pockets?
• Who will be at the table deciding on what the safeguards should be?
• And, most importantly, what and whose concerns will be taken seriously?

Right now, with little effective regulation, AI has much potential for harm, including automating yesterday’s biases and giving them the appearance of objectivity. Biased data can reinforce and even amplify existing social inequalities, leading to unfair and even disastrous outcomes. As just one of numerous examples, a healthcare algorithm used to predict which patients would benefit from extra medical care used healthcare costs as a proxy for health needs, assuming that sicker patients incur higher medical expenses. However, in the U.S. healthcare system, Black patients have historically received less care for a variety of reasons, including lack of access and discrimination, and therefore have lower costs, even when their actual health needs are equal or greater. As a result, the algorithm underestimated the needs of Black patients, giving them lower risk scores, which led to them being denied additional care. Correcting the bias would have increased the share of Black patients identified for additional help from 17.7% to 46.5%.

It is becoming clearer that we need to slow down, better understand the negative consequences, and put more regulations in place that put equity at the forefront.

Before developers race to make AI systems more powerful, they must examine the assumptions, labels, sources, exclusions, and quality controls embedded in the datasets that teach those systems how to see the world. Here are five ways to do that:

1. Audit datasets for representation. Examine whose experiences are included, overrepresented, underrepresented, or missing entirely. Fairness and equity begin with knowing whether the data reflects the full range of people the AI system may affect.
2. Document where the data comes from. Include clear information about its sources, collection methods, limitations, and assumptions. Transparency helps identify bias before it becomes embedded in the model.
3. Test model outcomes across different groups and for different interpretations of the data. Do not rely only on overall accuracy. Test whether the system performs differently by race, gender, disability, language, age, geography, socioeconomic status, and other dimensions of diversity where appropriate and lawful. A perfectly representative dataset can still produce an inequitable system if developers choose the wrong outcome, proxy, threshold, or deployment context. In the healthcare example above, the system used healthcare spending as a proxy for healthcare need, resulting in falsely making an inequitable healthcare system appear neutral.
4. Include impacted communities in the design process. People most likely to be affected by the AI system should have a voice before deployment. Equity requires that we stop “designing for” communities to “designing with” them.
5. Create ongoing monitoring and accountability. Regularly review outputs, investigate harms, correct biased results, and create clear channels for people to challenge or appeal AI-driven decisions.

My biggest concern is that if we do not focus on data equity, we will continue to scale existing problems much faster and with greater impact. Imagine a discriminatory decision made by one person. Its reach is limited. Now imagine a biased system instantly making millions of decisions about employment, credit, healthcare, policing, housing, and education for those already at the margins. Such decisions can destroy millions of lives. The big question for me is whether we will be wise enough to slow the pace of AI for the right reasons: to govern it to ensure it is safe, ethical, and equitable.

---

For more learning on this topic, check out our LinkedIn Learning Courses:

• Data Equity: Ensuring Fair Representation in AI Data Sets
• Analyzing Data with an Equity Lens