One of the more famous thought experiments in the AI safety literature involves a superintelligent machine given the seemingly simple goal of making as many paper clips as possible. The machine, however, soon realizes that humans are made of atoms that could be repurposed. Thus, it also begins converting them into paperclips, in its single-minded pursuit of its objective. This is philosopher Nick Bostrom’s paper clip maximizer. It is often used as a parable for why getting AI “alignment” wrong could mean catastrophe.
The scenario is intended to illustrate a technical problem. But I want to suggest that treating it as purely technical misunderstands what alignment ultimately is, or where it comes from. Recognizing its genealogy might change or expand how we approach it.
A secularization story
It feels strange when you listen carefully to how AI researchers talk about the stakes of their work, especially those who are thinking about AGI or superintelligence. Eliezer Yudkowsky, one of the field’s most prominent figures, concluded a 2023 essay calling for a global halt to AI development with a simple prediction: “If we actually do this, we are all going to die.” Ray Kurzweil speaks of the Singularity in almost rapturous terms—a moment when humanity merges with machine intelligence, transcends biological limits, and achieves something like immortality. At a post-AGI workshop I recently attended, the ideal superintelligent AI was described as a bodhisattva, that is, a being of boundless wisdom acting only for the welfare of all sentient creatures.
These are not regular engineering metaphors. They are eschatological ones. When one talks about extinction or radical flourishing, damnation or salvation, all of which are planetary in scope, they are talking about the stakes of theology.
The eschatological nature of AI research should not come as a surprise. In 1993, David Noble had already argued that modern technology is not a secular endeavor but is deeply rooted in Western Christian millennialism, driving a thousand-year quest for transcendence, salvation, and the recovery of divine perfection. Noble was not talking about AI at that time, but this message rings especially true today during the so-called AI spring. As Charles Taylor, Michael Rosen, and many others have likewise argued, our modern, seemingly secular ways of seeing the world were profoundly shaped by religion and still operate with its metaphysical and ethical categories, despite protestations to the contrary. AI research is animated by the main themes explored by Noble and, later on, Erik Davis, namely: disembodied transcendence, recovery of perfection, and the binding of powerful entities.
The discourse around the field of AI alignment is the latest instance of this pattern. Its structure mirrors the structure of Christian theological problems. When researchers talk about instilling values into machines, about making powerful systems reliably good, and about what happens when entities far smarter than us pursue goals indifferent to human welfare, they are staging a drama that theology had already wrestled with for a long time. Put simply, the alignment problem asks: how do you make a potentially powerful entity act in ways that are reliably good? How do you bind a powerful being?
The Christian political theology lurking behind this genealogy is thus not merely interesting but it actually has something to offer. Noble tells us that millenarian movements characteristically bypass institutional procedure, because institutions belong to the fallen present, not the redeemed future. Davis explores how the gnostic strand of tech culture concentrates interpretive authority among the technically initiated. In this account, esoteric knowledge, by definition, belongs to those who have seen through ordinary appearances to the underlying truth. Together, Noble and Davis, among others, can help us make sense of the AI industry and the field of AI alignment that supports it, a field that is deeply serious about the stakes of its work while remaining, to date, resistant to the kinds of democratic, pluralist, and institutional responses that those stakes would seem to demand. But while understanding could help us identify some problems, the answers might lie elsewhere. I would suggest the history of political theology as one such place where answers to the problem of binding God have taken various forms such as covenants and natural law, an enterprise that would eventually be secularized into constitutional law.
Given this genealogy, AI alignment is not only a technical issue but a fundamentally social one. To be sure, it is important to determine how to specify the objectives correctly, or to ensure that the model generalizes well, but the religious analogues demand that we understand AI as a sociotechnical project, not a standalone machine. While millenarianism and gnosticism can somewhat explain the field’s resistance to institutions, the constitutional tradition, over time, has come up with the philosophy and mechanisms for binding tremendous power. If the challenge is that autonomous AI should be guided by values and goals that benefit humanity, it needs to have the necessary interpretive resources and institutions to understand what human life and society is. The constitutional tradition is one such resource.
The constitutional inheritance
Medieval thinkers had similar intuitions about the problem of the virtuous sovereign. The king stood above ordinary law, wielding power that could not easily be checked. In this account, the only constraint was the sovereign’s fear of divine judgment. The king was supposedly accountable to God even if not to his subjects. But this arrangement was notoriously unreliable. Hence the slow, difficult construction of legal limits from the Magna Carta to the long evolution of constitutional thought, eventually arriving at the idea that sovereign power must be channeled through institutions, separated into branches, and checked at every turn. The Western constitutional tradition’s central insight was that virtue is not a reliable constraint on power. What you need are institutions and mechanisms, such as rights and procedures, which make virtue less necessary.
Today, AI researchers, especially those in the field of alignment, face similar problems. The concern is not only that AI systems could become malicious. They could. But even well-intentioned systems, given imprecisely specified goals and sufficient capability, could cause enormous harm. Another problem is homogeneity. What values should the AI have, and who decides? An AI landscape dominated by a handful of models trained on the values of a handful of institutions in a handful of wealthy countries is clearly a system of unchecked power.
If alignment is, broadly speaking, some kind of secularized-theological problem, then here are some implications: first, it means that a far broader set of disciplines and traditions has something important to contribute to this field, beyond its natural community of computer scientists, machine learning engineers, and AI safety researchers. For instance, the Alan Turing Institute has come up with a new initiative called Doing AI Differently to broaden the field of relevant expertise. At the same time, we need to broaden the community from which these values are going to be drawn.
Second, drawing from my own legal background, I want to suggest that the constitutional tradition is not just a useful metaphor. As I have alluded to earlier; it is a repository of knowledge that may help address some dimensions of this problem. To this end, an emerging field called legal alignment has proposed that legal principles, methods, and institutions are themselves a resource for solving the alignment problem from within. Examples of this include legal reasoning, which has been developed over centuries precisely to apply general principles to novel and contested cases, and the use of legal concepts or doctrines such as fiduciary duty or proportionality as part of a blueprint for building reliable agents and helping them navigate ambiguous cases. A further example of a potential insight from the constitutional tradition involves the instantiation of checks and balances within the AI systems themselves, such as building a right to contest from within the system, or the idea of appealing model behavior that will eventually form part of a larger body of precedent, which could be a source later on for quasi-judicial processes. Other possibilities include creating institutional structures for meaningful public participation in model development, or the establishment of mechanisms for societal inputs into AI objectives and constraints. A recent collaborative effort of technical experts, lawyers, and philosophers called AI constitutionalism has been formed to pursue these types of questions.
These are just a few illustrations, but the main point is that law, over the course of centuries, has learned to bind sovereigns by setting the terms on which authority could be exercised. Law has insisted that even the highest power must act through form, procedure, and accountable institutions. We should think of AI alignment in the same broad way. It is a sociotechnical problem that is in need of institutions and ideas beyond engineering. This is one way to ensure that AI systems could serve pluralistic human values in the course of such alignment.
I.J. Good, a British mathematician who worked as a cryptologist at Bletchley Park with Alan Turing, once wrote that the first ultra-intelligent machine will be the last invention that man needs ever make, for the machine will take over after that. But Good is mistaken. Whether the next powerful entity we create answers to anything beyond itself will not just be decided by how intelligent it becomes, but by whether we do for machines what constitutionalism has done for kings. That is more than just a technical problem. It is the oldest constitutional one, asked yet again.
