Who is accountable when AI causes harm? Global leaders debate responsibility amid rising concerns
When AI hurts people who s – On Tuesday, the second day of the inaugural UN summit on AI governance, the question of legal accountability for AI-driven harm emerged as a central topic. Experts from around the world highlighted alarming trends in how the technology is being used to undermine human rights, sparking urgent discussions about the need for global oversight. The debate underscored the growing impact of artificial intelligence on society, with a focus on how to assign blame when its applications lead to tangible suffering.
AI and the rise of digital exploitation
Sonia Livingstone, a member of the Independent International Scientific Panel on AI, presented troubling data on the misuse of AI in the Global South. She noted that in 11 developing countries, one child per classroom has reportedly been subjected to AI-generated sexually explicit deepfakes. These incidents, she explained, are not isolated but part of a broader pattern of digital exploitation. Reports to the US CyberTip line and the National Center for Missing & Exploited Children (NCMEC) are increasing rapidly, with AI now serving as a tool for creating and amplifying harmful content.
“Across 11 Global South countries, up to one child per classroom reported that AI was used to make sexually explicit deepfakes of them. Reports of AI-generated child sexual abuse material to the US CyberTip line, NCMEC, are rising exponentially.”
Livingstone emphasized that the evidence of AI’s role in human rights violations far outweighs its benefits. She argued that the technology’s ability to create and spread disinformation, deception, and distrust has made it a potent force for harm. “Personal data is being taken, manipulated, abused, and exploited by self-learning systems,” she stated, adding that the consequences are increasingly severe for vulnerable populations.
Global collaboration in AI governance
The UN’s stance on AI governance hinges on the technology’s dual potential—both as a tool for good and a source of harm. This duality, according to the organization, necessitates broad participation from all nations in shaping AI’s future. Volker Türk, the UN High Commissioner for Human Rights, echoed this sentiment, stressing that equality, accountability, and human oversight must be embedded in the technology’s design and implementation.
“We don’t consider safety standards for medicines, cars, or aircraft as obstacles to progress. They are the reason why people trust those technologies in the first place.”
Türk’s remarks reflect a call to action for integrating ethical frameworks into AI development. The UN’s approach, he suggested, mirrors how regulatory safeguards have historically supported innovation in other fields. By prioritizing these principles, the organization aims to ensure that AI systems operate transparently and benefit all of humanity.
Environmental and social challenges
Among the most pressing unresolved issues is the environmental footprint of AI. The UN Secretary-General has urged greater transparency regarding the energy-intensive data centers that power current AI platforms. These facilities, while essential for processing vast amounts of data, contribute significantly to global emissions and resource depletion. As AI systems scale, their environmental costs are becoming more pronounced, especially for communities already struggling with energy scarcity.
Sasha Luccioni, co-founder of the Sustainable AI Group, pointed out that the benefits of AI are widely accessible, but its negative effects are concentrated in specific regions. “Anyone with a cell phone, mobile service, or laptop can connect to the cloud and use AI. Yet the impact of its misuse is highly localized,” she said. This disparity raises concerns about the uneven distribution of AI’s consequences.
“Some of the world’s most marginalized are already suffering in terms of water, energy, and emissions, as well as the health impacts that are becoming more severe.”
Jhalak Kakkar, a digital policy expert from the Centre for Communication Governance at Delhi’s National Law University, warned that environmental assessments of AI could exacerbate existing inequalities in the Global South. She questioned how economic value generated by AI technologies is often extracted from regions far removed from those bearing the brunt of its environmental toll.
Gender gaps in AI development
A parallel concern is the lack of diversity in AI creation. The UN Women agency highlighted findings from a survey indicating that 25% of women human rights defenders, activists, and journalists have experienced AI-assisted online violence. Six percent of respondents reported being victims of deepfakes or digitally manipulated images, often without proper documentation or acknowledgment.
“So much of this goes undocumented, goes unnoticed, goes unreported,” said Sima Bahous, Executive Director at UN Women.
Bahous stressed that the underrepresentation of women in AI development is a critical issue. According to the panel’s report, women comprise just 30% of the global AI workforce, with 88% of leading researchers identified as male. This gender imbalance, she argued, perpetuates systemic biases and limits the range of perspectives in AI governance. “When we look at who builds AI, women are largely underrepresented,” she noted, emphasizing the need for inclusive representation to address these challenges.
The path forward: Openness and equity
Experts at the summit urged a shift toward greater transparency, urging the “opening of AI’s black boxes” to better understand its inner workings. Morocco’s Amal El Fallah Seghrouchni, a participant in the discussion, highlighted the complexity of large language models, which operate using around 180 billion parameters daily. “There are very few lawyers or sociologists equipped to fully grasp what these systems are doing,” she said, noting that even AI researchers struggle with the technology’s intricacies.
“Even AI researchers do not always have the tools to comprehend the complexity of these algorithms.”
The first Global AI Dialogue, held in Geneva alongside the AI for Good Summit organized by the UN International Telecommunication Union (ITU), will be followed by a second summit in New York next May. This series of events aims to foster collaborative decision-making, ensuring that AI governance reflects the priorities of all nations. The dialogue also seeks to address the urgent need for integrating equality and accountability into every stage of AI development.
As AI continues to evolve at an accelerated pace, the challenge of aligning legislative frameworks with technological advancements becomes more critical. The summit’s discussions underscore that the path to responsible AI governance requires not only technical solutions but also a commitment to addressing the social, environmental, and ethical dimensions of its impact. With the world watching, the goal is to create a system where AI serves as a force for good, rather than a source of harm for the most vulnerable populations.
