Imagine a world in which autonomous vehicles are entrusted with the highest safety responsibilities: transporting passengers, delivering hazardous materials, and even deciding whom to protect in an unavoidable crash. This might seem like the pinnacle of technological progress—untainted by fatigue, distraction, or inconsistency, making split-second decisions with impeccable precision. Yet beneath this vision of a flawless machine lies a fundamental question: can an algorithm understand moral trade-offs, or is it confined to a simulation of ethical reasoning? It might replicate human decisions without improving on them, carrying forward the same biases, blind spots, and cultural distortions embedded in the data it learns from. In trying to emulate us, it might only reproduce our limitations, not transcend them. But there is a deeper concern. Moral judgment draws on intuition, historical awareness, and context—qualities that resist formalization. Ethics may be so embedded in lived experience that any attempt to encode it into formal structures risks flattening its most essential features. If so, an AI would not merely reflect human shortcomings; it would magnify the very rigidity that ethical reflection exists to resist.
Still, many have tried to formalize ethics, by treating certain moral claims not as conclusions, but as starting points. A classic example comes from utilitarianism, which often takes as a foundational axiom the principle that one should act to maximize overall well-being. From this, more specific principles can be derived: for example, that it is right to sacrifice the few to save the many, or that all actions should be judged by their consequences for total happiness. As computational resources increase, AI systems are increasingly well-suited to this kind of calculus—weighing outcomes, estimating probabilities, and reasoning through their implications in complex situations.
But what, exactly, does it mean to formalize something like ethics? The question is easier to grasp by looking at fields in which formal systems have long played a central role. Physics, for instance, has never relied on a single, all-encompassing "theory of everything." Instead, we have many physical theories, each designed to describe specific aspects of the universe: from the behaviour of gases and electrons to the motion of galaxies. These theories often diverge. Aristotelian physics, for example, explained falling objects in terms of natural motion toward earthly centres; Newtonian mechanics replaced this with a universal force of gravity. These explanations are not just different; they are incompatible. Yet both share a common structure: general principles that yield predictions—assumptions about motion, force, or mass—and derive increasingly complex consequences.
Ethical theories have a similar structure. Like physical theories, they attempt to describe a domain—in this case, the moral landscape. They aim to answer questions about which actions are right or wrong, and why. These theories also diverge, and even when they recommend similar actions, such as giving to charity, they justify them in different ways. Ethical theories also often begin with a small set of foundational principles or claims, from which they reason about more complex moral problems.