AI at a Crossroads: Safety, Ethics, and the Future of Governance

The International Association for Safe & Ethical AI Conference (IASEAI25), held in Paris on February 6-7, assembled a broad spectrum of voices—from government officials and economists to researchers, academics, and technologists—to explore the challenges and opportunities posed by our rapidly evolving AI landscape. As AI reshapes society, discussions ranged from technical risks to the ethical implications of unregulated deployment, calling for regulation that is both robust and nuanced.

1 — Embedding Governance into AI Development

At IASEAI25, Joseph Stiglitz, Nobel Laureate and Professor at Columbia University, challenged the assumption that market forces alone can ensure that AI aligns with the common good, warning that unregulated AI development carries risks akin to those seen in unmonitored financial markets. Later, Gillian Hadfield, Professor of Law at the University of Toronto, introduced the concept of what I would label "adaptive regulatory anchoring" to emphasize that AI regulation must be hooked and integrated into our legal and economic systems rather than applied as a superficial add-on.

Stuart Russell, Professor of Computer Science at UC Berkeley and President of IASEAI25, concluded with a series of calls to action for lawmakers, academics, and the public ahead of the AI Summit in Paris.

With stakes as high as the speakers emphasized, the ground covered at the conference was substantial, spanning key domains of AI governance:

AI Safety & Technical Challenges – Ensuring functionality, robustness, and oversight.
AGI & Existential Risk Mitigation – Addressing long-term control risks and alignment.
Policy & Global Governance – Implementing regulation, compliance, and international cooperation.
Societal, Labor & Knowledge Impacts – Tackling bias, fairness, labor disruptions, and epistemic shifts.
Environmental & Infrastructure Impact – Managing AI's compute costs, sustainability, and supply chain dependencies.

The reason the invisible hand often seems invisible is that it's not there...In a whole set of circumstances, the pursuit of self-interest does not lead to the well-being of society. Once we recognize that, we should recognize that the construction of firms whose interests are not aligned with our society—like the construction of AI—is not likely to be in societal interest. There are challenges in trying to align AI with societal good through regulation and other mechanisms, but it will always be imperfect...Getting alignment is essentially impossible.” — Joseph Stiglitz, Nobel Laureate and Professor at Columbia University

AI safety and governance dominated discussions at IASEAI25. Stuart Russell, Anca Dragan, and Benjamin Hilton emphasized the need for functional robustness and oversight mechanisms like "Safety Cases." The AGI vs. Tool AI debate was a focal point, with Bengio, Tegmark, and Mitchell warning against the unchecked pursuit of AGI and advocating for FDA-like regulatory scrutiny.
Governance was a major theme, with Stiglitz, Hadfield, and Solaiman pushing for AI regulation to be embedded within legal and economic frameworks. Bias, misinformation, and labor concerns surfaced in conversations around data sovereignty (Pistilli, CIVICS dataset), psychosocial impacts (Ressa, Monroe-White), and AI's role in shaping cultural narratives.
Though less prominent, AI's environmental and infrastructure impact appeared in Kate Crawford's talk on sustainability and select poster presentations.
From safety to societal shifts, IASEAI25 reinforced the urgency of aligning AI development with governance, ethics, and long-term human interests. These recommendations, reinforced by Irene Solaiman and Divya Siddarth, set the foundation for a governance framework that prioritizes both technical feasibility and ethical imperatives.

2 — Multi-Agent AI and Autonomous Systems: Navigating Emerging Risks

Although multi-agent systems were not the central point of debate at IASEAI25, 2025 is being hailed as the year of AI Agents—a hot topic that underscored emerging risks in this domain. Several experts delved into the nuances of autonomous systems:

Anca Dragan, Head of AI Safety and Alignment at Google DeepMind and Professor at UC Berkeley, detailed the challenge of ensuring that autonomous agents operate in ways that consistently reflect human values.
Saadia Gabriel, Research Scientist at UCLA, presented her lab's work on detecting false language and auditing models, culminating in a project that utilized multi-agent simulation to model the spread of misinformation on social networks.
Chandler Smith, Research Scientist at Cooperative AI, observed that current evaluation benchmarks fall short in capturing the complex, emergent failure modes inherent in multi-agent systems, calling for more rigorous and transparent methods.
Zhijing Jin, Research Scientist at Stanford University, presented GOVSIM, showing that most AI agents fail at sustainable cooperation due to weak long-term reasoning, while ethical strategies improve outcomes.
Addressing the challenge of moving from risk avoidance to proactive mitigation, Benjamin Hilton, Head of Alignment at the AI Safety Institute, introduced Safety Cases—a formal framework, adapted from high-risk industries, for systematically assessing and managing AI risks.

3 — AGI vs. Tool AI: Setting Clear Boundaries

A central debate at IASEAI25 involved distinguishing between Tool AI—systems designed to augment human capabilities—and the often-hyped pursuit of Artificial General Intelligence (AGI):

Yoshua Bengio, Professor at the University of Montreal and Co-founder of MILA, recounted his radicalizing moment upon first experiencing ChatGPT. That encounter made him realize that without strict limitations, AI could quickly spiral out of control. This realization reinforced his advocacy for a model of tool AI—focused on highly constrained, controlled systems—rather than the unchecked pursuit of AGI.
Max Tegmark, Professor at MIT and President of the Future of Life Institute, argued for a regulatory framework similar to the FDA's oversight, insisting that frontier labs should have their models rigorously inspected before release. He contended that we do not need AGI; instead, our focus should be on developing tool AI within strictly regulated boundaries.
Kate Crawford, Senior Principal Researcher at Microsoft Research, critiqued the negative externalities of AI, noting that efficiency gains can paradoxically lead to increased resource consumption—a modern echo of Jevons' Paradox. She also highlighted the role of models like DeepSeek in democratizing access to advanced technology, thereby intensifying the need for sustainable design.
Margaret Mitchell, Chief Ethics Scientist at Hugging Face, delivered a stark directive:

Fully autonomous AI agents should not be developed.

Her assertion underscored the imperative that AI development remain within clearly defined, interpretable, and ethically aligned boundaries.

4 — Ethical Dimensions: Memory, Media, and Psychosocial Impacts

Ethical considerations were a prominent part of the discussions, addressing how AI reshapes cultural narratives and social well-being:

Maria Ressa, Nobel Peace Prize Laureate and Founder of Rappler, shared a personal narrative about becoming a target due to her data-driven analyses, illustrating how AI can be misused to fragment social communities and undermine media integrity.
After the Q&A, I spoke with Dr. Thema Monroe-White about her paper, "The Psychosocial Impacts of Generative AI Harms". She highlighted concerns about how the rapid spread of generative AI could exacerbate psychosocial challenges for marginalized communities.
I spoke to Giada Pistilli, Principal Ethicist at Hugging Face, about CIVICS, a dataset designed to examine culturally-informed values in LLMs. Instead of relying on automated scraping, they curate prompts through their own team's personal networks, ensuring deep cultural and contextual grounding. The poster raised a key challenge: whose values should guide AI alignment when even human rights frameworks vary across cultures?
In a poster presentation, José Renato Laranjeira de Pereira, Research Fellow at the Yale Digital Ethics Center, warned that without genuine data sovereignty and informed consent, AI diversity initiatives risk reinforcing digital colonialism.
Another poster, "The Virtue Of Artificiality: Islamic Ethics And The Reconceptualization Of AI Caregivers" by Syed AbuMusab, Post-Doctoral Associate at the Yale Digital Ethics Center, offers a compelling counterpoint to Western caregiving models. This work resonates with innovative solutions such as GiveCare—my own AI caregiving product—which is designed to create systems that honor diverse cultural norms.
During lunch and the poster session, I spoke with João Pita Costa about the IRCAI SDG Observatory, an AI-powered platform that tracks AI's impact on SDGs with data-driven insights and bias detection. Backed by UNESCO IRCAI and AI4Gov, it helps governments make informed decisions on sustainability and policy.
Esben Kran, Senior Research Scientist at the Digital Future Institute, introduced DarkBench, a tool that tracks harmful dark patterns by monitoring issues such as brand bias, user retention tactics, sycophancy, anthropomorphism, and problematic content generation.

5 — Agentic AI: Balancing Autonomy and Accountability

A dedicated session on agentic AI brought forward a range of insights into how autonomous systems might self-regulate:

Arthur Allshire, Research Fellow at the Center for Responsible AI introduced the concept of agentic refusal, advocating for AI systems with built-in mechanisms to decline tasks that conflict with ethical guidelines.
Erica Finkel, Director of AI Policy at Meta, discussed how the unpredictable nature of emergent behaviors in autonomous agents necessitates robust self-regulation mechanisms.
Michael P. Wellman, Regents' Professor of Computer Science at the University of Michigan, connected these theoretical discussions to practical concerns by noting that autonomous agents have long been used in finance, suggesting that their presence in high-stakes environments is not entirely new.
I spoke with Rishab Jain, Research Engineer at Google DeepMind about their work on Human-AI Complementarity—the challenge of keeping humans in the loop as AI surpasses individual expertise. Their experiments show that hybrid human-AI oversight improves judgment but risks over-reliance, raising a key question: can human oversight scale as AI systems grow more capable?

6 — Strategic Foresight for Safe and Ethical AI

The final panel, moderated by Dr. Atoosa Kasirzadeh, Director of Strategic Foresight at the Global Institute for Ethical AI, provided long-term perspectives on AI governance:

Gillian Hadfield reiterated the importance of embedding regulatory mechanisms within existing societal structures.
Zico Kolter, Associate Professor of Machine Learning at Carnegie Mellon University, reaffirmed that AI's dual-use nature defies simplistic comparisons to purely destructive technologies.
Toby Ord, Senior Research Fellow at the Future of Humanity Institute, University of Oxford, stressed that even low-probability risks require serious attention given their potential for catastrophic impact.
Nicholas Moës, Research Fellow at the Centre for AI Safety, called for international cooperation and the creation of safety standards that are sensitive to diverse cultural and ethical contexts.

7 — Looking Ahead: Preliminary Insights for the Paris AI Summit

In her upcoming presentation at the Paris AI Summit (February 10–11), Irene Solaiman, Head of Global Policy at Hugging Face, shared preliminary notes outlining key principles to guide future AI governance discussions:

Policy Must Look to and for the Science: Future regulatory frameworks should be grounded in the latest technical research.
Actions Must Be Technically Feasible: Proposals need to be realistic and achievable within current technological capabilities.
Ongoing Evaluation: Continuous monitoring is essential as open models become increasingly widespread.
High Performance at Low Cost: The convergence of high performance and affordability—as seen with models like DeepSeek—reshapes competitive dynamics and regulatory challenges.
Finding Common Ground: Despite divergent values, there remains significant potential for coordinated global governance.

Reflections

The extensive discussions at IASEAI25—spanning robust regulatory strategies, the emergent risks of AI agents, the delineation between Tool AI and AGI, and the profound ethical implications of generative models—offer a roadmap for future governance. As we anticipate the Paris AI Summit, the insights presented here will continue to inform efforts toward a secure, ethical, and culturally attuned approach to AI development.