Marcelo Cabrol, advisor on AI to the Vatican: “AI should propose, but a legally accountable human must always dispose.”
One of the most humanist thought leaders on artificial intelligence and a member of the GTF Governance Lab Panel, Marcelo Cabrol, speaks to GTF Insights about the AI black box, the equality paradox, and why small states can only fight Big Tech together.
Government Tomorrow Forum: You wear several hats : Vatican advisor, former IDB Lab CSO, contributor to the GTF Geneva Lab. What brought you to AI ethics specifically? Was there a project, a moment, or a person that made this your life's work?
Marcelo Cabrol: To understand my path into AI ethics, you have to see the tension I have always lived inside: international development work is notoriously slow, and I have always been deeply impatient, especially when it comes to social policy and education. I wanted to accelerate human outcomes.
Thirty years ago I was part of a team at the IDB financing the expansion of Telesecundaria in Mexico. Rural communities between seventh and ninth grade had no qualified teachers, so the state centralised content production and beamed high-quality curriculum out by satellite and videotape. The early evaluations were thrilling: rural students weren't just getting access, they were transitioning to high school at much higher rates than comparison groups. It proved to me that technology could break systemic barriers.
But it also taught me something concerning. When we looked at the small numbers, not the aggregates, kids who had already been behind between grades one and six kept falling behind. And kids who were doing well stopped progressing. Standardised content was equalising the middle, but leaving the tails behind. That was my first hard lesson: technology, unexamined, is not neutral. It can systematically de-equalise.
I moved from satellite television to computer labs, connectivity initiatives, One Laptop Per Child, MOOCs. Each generation of technology arrived with the same tech-utopian promise, and each generation revealed the same underlying issue. Equity in education is not a technical challenge. It is an ethical choice.
About ten years ago, AI became an operational reality and my whole framework had to adapt. Three moments crystallised the transition. First, the black box: I saw that AI was not just automating decisions, it was making uninterpretable ones, deciding, without oversight, whether someone would receive a credit, a housing benefit, an employment opportunity. For an institution like the IDB, whose mission is inclusion, that kind of algorithmic exclusion was anathema.
Second, in 2020 I had the privilege of participating in the launch of Pope Francis's Rome Call for AI Ethics. That completely reframed my perspective, from short-term policy fixes to a civilisational, anthropocentric horizon.
“the rule I would write into the Handbook is direct: no automated termination of social benefits. If the AI flags a family for removal, the system must trigger an intentional speed bump, a forced human verification, a clear notification, a caseworker review, before a single euro is withheld. ”
Third, at the IDB Lab we started applying a Responsible AI matrix to the startups and venture funds we financed, and we proved that founders could integrate deep ethical guardrails into their algorithms and still be profitable and scalable. When people ask why I wear these different hats, Vatican on one hand, venture capital matrices on the other, I tell them it is the same journey I started in rural Mexico. Ensuring that when we use technology to accelerate human progress, we don't accidentally leave humanity behind.
GTF: In your Geneva contribution, you introduced the Friction of Agency as a sector-dependent variable. If a Minister of Social Welfare asked tomorrow where to lower friction and where to raise it, what would you tell them?
Marcelo Cabrol: Look at conditional cash-transfer programmes. I spent years working on Oportunidades in Mexico, Bolsa Família in Brazil, Asignación Universal por Hijo in Argentina, and their equivalents in Nicaragua. CCTs are the perfect testing ground for friction because the whole architecture is about inclusion and exclusion.
Historically, targeting the poorest households relied on proxy-means testing. A social worker would go to a village with a paper clipboard, note whether the house had a dirt floor, running water, a refrigerator, and enter it into a centralised database. It was slow and rigid, but it had inherent, human-scale friction. If a family was wrongly excluded, there was a paper trail and an identifiable social worker you could talk to.
Now introduce AI. A modern welfare ministry can use predictive algorithms, cross-referencing satellite imagery, utility bills, digital transaction footprints. This is where the minister has to understand how to dial the friction up or down. My diagnostic runs on two questions: what is the power asymmetry between the citizen and the model, and how invisible and irreversible is the harm if the model gets it wrong?
On the inclusion side — onboarding — we should lower friction dramatically. If the algorithm detects that a single mother's income has dropped below threshold, the system should automatically enrol her. She should not have to stand in line for hours or prove her poverty repeatedly. Use AI to make access seamless.
But on the exclusion side — offboarding — we must intentionally raise friction. In a traditional CCT, if a family stopped receiving their transfer, they knew why. In an AI-driven system, a model may quietly recalculate weights, detect a slight shift in a proxy variable, and automatically cut off a family's lifeline. That is an invisible, irreversible harm: kids miss meals that week; malnutrition statistics tick up only months later.
So the rule I would write into the Handbook is direct: no automated termination of social benefits. If the AI flags a family for removal, the system must trigger an intentional speed bump, a forced human verification, a clear notification, a caseworker review, before a single euro is withheld. Efficiency for the state must never become cruelty for the citizen.
Marcelo Cabrol at the GTF Governance Lab Geneva session in March 2026
GTF: Your Equality Paradox gets translated to a procurement clause you have called “legible adaptation.” Walk us through what a good and a bad versions look like?
Marcelo Cabrol: The paradox first, because it is the point where mathematical optimisation clashes with social justice. In a deeply unequal society, treating unequal populations with mathematical equality guarantees unequal outcomes. When a government buys a standard AI model from a global vendor, that model optimises for the statistical average. It standardises profiles. But in a country with vast informal economies, marginalised communities, or deep gender divides, “the average” doesn't exist. By forcing a mathematically equal algorithm onto a heterogeneous population, the state accidentally automates and accelerates discrimination. The machine thinks it is being neutral; it is actually reinforcing structural inequality.
Now, the realism. A weak ministry has zero leverage to force a major vendor to rewrite proprietary code. If a small government demands to see the weights and biases, the vendor walks. So we shift the battlefield: from code inspection to validation output.
The procurement clause I would write into the Handbook goes like this: the state does not inspect intellectual property, but the vendor is contractually required to provide a standardised API translation layer that renders the top three decision-drivers into plain, non-technical language for state auditors. Deployment is contingent on passing a Local Stress Test, in which the model is run against a state-provided control dataset representing our most vulnerable demographic subsets. If it fails the stress test, no payment. It is the same principle as testing the brake pads on a fleet of imported police cars, you are not inspecting the manufacturer's design, you are setting a performance standard.
The bad example everyone should read is SyRI in the Netherlands, where the government gave a black-box machine-learning system free rein to cross-reference welfare, tax, and housing data. It optimised for risk, created a massive negative feedback loop targeting poor neighbourhoods, and ruined lives with zero legibility. It was ruled unlawful by the Dutch courts in 2020.
The good example is the UK NHS AI Lab's approach to procuring diagnostic imaging tools. They do not trust the vendor's global validation data. They created a National AI Buyer's Guide and localised sandbox testing environments: your algorithm might work perfectly on a dataset from Boston, but before we deploy it in a diverse borough in London, you must plug it into our secure sandbox and prove it performs equitably across our local demographic mix. That is the gold standard: sandbox validation. Vendor keeps the code, government keeps the citizens safe, and legibility is achieved through proven local outcomes rather than blind trust.
GTF: You have said governments should identify three or four critical leverage points in contracts with foreign-owned AI vendors. The Rwanda-Anthropic MOU is often cited as an example of a small state doing this well. What's your read?
“What actually happened is a classic first-mover trade-off. Rwanda is making a calculated, aggressive bet to attract Big Tech early and establish itself as a regional AI hub. But the operational risk is enormous.”
Marcelo Cabrol: Rwanda-Anthropic is often cited, but it is not, and this is important, a story of a small state successfully dictating terms to Silicon Valley. It is the perfect example of why the Handbook is urgently needed.
Rwanda signed a three-year MOU with Anthropic to integrate Claude across the Ministry of Health, education, and public developer workflows. Structurally, though, it is non-binding. Anthropic gave Rwanda free API credits, Claude Pro licences for teachers, and capacity building. In exchange, Rwanda became Anthropic's flagship “beneficial deployment” proof-of-concept on the African continent.
If you read the terms carefully, Rwanda did not secure hard contractual levers, no data residency, no independent model audit, no protection against model deprecation, no external review mechanism, no parliamentary oversight, no civil-society transparency. What actually happened is a classic first-mover trade-off. Rwanda is making a calculated, aggressive bet to attract Big Tech early and establish itself as a regional AI hub. The operational risk is enormous. Anthropic is quickly becoming load-bearing infrastructure for the Rwandan state's health and education systems before any sovereign governance framework has defined what obligations the vendor faces. If Anthropic changes commercial pricing or deprecates a model in two years, Rwanda faces total vendor lock-in.
This is not state leverage. It is an asymmetric bet on corporate goodwill.
So how do small states solve this? Not alone. A single small country has no bargaining power. But small states already handle other enormous global industries, oncology drugs from Big Pharma, climate financing, through aggregate procurement and multilateral blueprints. That is the model the Handbook should champion. Not “invent your own contract.” Copy templates already pre-vetted by international bodies that vendors have already agreed to sign.
Three real, working precedents. The UNDP and the Digital Public Goods Alliance are drafting Model Contract Clauses for Digital Public Infrastructure, a small state does not write the data-residency clause, it copies one that Big Tech has already accepted in larger deals. The African Union's continental AI strategy is trying to orchestrate the same logic at scale: 55 states demanding unified data-sovereignty and auditing standards under one umbrella. And the G-Cloud blueprints from the UK and Singapore centralised cloud procurement a decade ago — a vendor cannot pitch individual ministries, they must qualify against a central portal on sovereignty and portability. Small states can adapt those existing frameworks to AI today.
The operational recommendation is simple. Collective bargaining and pre-negotiated open-source contracts. Change the narrative from “a weak state fighting Big Tech” to “a state adopting a global standard.” That is how a philosophical idea becomes a hard operational reality.
Marcelo Cabrol at the GTF Governance Lab Geneva session in March 2026
GTF: The sixth pillar you have proposed adding to the Handbook's Five-Point Test is Accountability, with “human fallback” as its operational form. What does a real fallback look like versus a fake one?
Marcelo Cabrol: A human fallback is not a customer-support email at the bottom of a government webpage. That is a passive complaint box. A true governance fallback has three non-negotiable elements.
First, staffing. It requires an interdisciplinary team, not just IT technicians, but data scientists who understand the model, legal experts who understand rights, and social workers who understand the community reality. Call it an Algorithmic Appeals Board.
Second, service-level agreements with real teeth. The speed of human intervention must match the urgency of the human need. If an AI wrongly cuts off a family's cash transfer or denies a healthcare authorisation, the fallback SLA must mandate a binding resolution within 24 to 48 hours. That number is written into the vendor contract.
Third, fire drills. You do not wait for a crisis to test whether the fallback works. Governments must run regular algorithmic fire drills, feeding corrupted data or edge cases into the system to see whether the human team catches it and takes manual control cleanly. Synthetic data works fine. What matters is that the muscle exists.
The negative case is the UK Post Office Horizon scandal. The Post Office deployed a financial software system with bugs that produced unexplained shortfalls in the accounts of local postmasters. Instead of designing a human fallback, a team to independently audit anomalies, the institution suffered absolute automation bias. Over 700 innocent postmasters were criminally prosecuted for theft. Lives were destroyed. That is what happens when a machine's output is treated as indisputable truth with no human bypass.
“The Vatican lens is not new, the Church has centuries of practice thinking about the human person as an end rather than a means. What is new is that the Church is, unusually, ahead of the curve on this particular question.”
The positive case comes from sepsis prediction in hospital AI. Sepsis is a rapid, lethal infection. AI models continuously scan patient vitals to flag early signs hours before a doctor notices. But the architecture is designed entirely around fallback: the AI never prescribes medication or changes treatment on its own. It triggers an alert to a specialised Sepsis Response Team. If the clinician sees that the AI has misread a loose heart-rate monitor wire, they hit an explicit Override button, and that override logs why the human disagreed, feeding data back into the model.
That is the golden rule for the Handbook: AI should propose, but a legally accountable human must always dispose. The machine is the radar; the human is the pilot.
GTF: You work between the Vatican's civilisational horizon and Latin America's operational reality. What do most Western AI-ethics conversations miss, and what makes it harder in practice?
Marcelo Cabrol: The Vatican lens is not new, the Church has centuries of practice thinking about the human person as an end rather than a means. What is new is that the Church is, unusually, ahead of the curve on this particular question. Pope Leo XIV took his name in homage to Leo XIII, the Rerum Novarum pope who wrote the great encyclical protecting labour during the industrial revolution. That was a deliberate signal: the AI moment is being framed as a comparable civilisational transition. Western policy centres, by contrast, are structurally hyper-focused on election cycles and quarterly tech revenues. The Vatican forces a horizon of decades and centuries, and it forces you to look at the human being as the ultimate end, never as a monetisation unit or a data point.
But, and this is where I have to be honest, those principles are much harder to implement in underdeveloped regions. When we talk about technology in the Global South, you already have an asymmetry in access to connectivity, devices, and large models. When we talk about subsidiarity, the Catholic idea that the smallest reasonable level of authority should make the decision, it is easier to invoke in Europe than in Latin America and the Caribbean, where the institutional layer that would exercise that subsidiarity often barely exists.
Which brings me to the hardest paradox in my own work. Friction is good, human agency is good. I believe in both. But my constant struggle is that by raising the ethical barriers to AI deployment, I may be excluding a student in a rural area in Latin America from a better education than she would otherwise have. That is an ethical consideration in itself. The West can afford to be cautious. It can afford to say: slow down, wait until the safeguards are in place. Latin America cannot always afford that patience.
So the honest answer is that the West misses the operational reality of institutions that do not have pristine digital infrastructure or absolute institutional trust. The Vatican makes the horizon visible. Latin America, and every region like it, makes the ground visible. You need both.