Artificial intelligence now decides who gets interviewed for a job, what news we see, how police patrol neighbourhoods, and which families get flagged by child-welfare agencies. In response, the world has entered its first serious era of AI lawmaking. The EU AI Act — Regulation (EU) 2024/1689 — entered into force on 1 August 2024 as the world’s first comprehensive, horizontal AI law, with its prohibitions on the most dangerous practices applying from February 2025 and fines reaching as high as €35 million or 7% of global turnover. UNESCO’s Recommendation on the Ethics of AI, adopted by 193 member states in November 2021, set the first global ethical standard. The OECD’s AI Principles (2019, updated in 2024) and the US NIST AI Risk Management Framework (2023) round out an increasingly crowded governance landscape.

Yet drafting rules turns out to be the easy part. The genuinely hard problems in AI regulation are ethical ones — questions about transparency, fairness, and legitimacy that no compliance checklist can resolve. Drawing on work in political philosophy and business ethics, here are the considerations that should sit at the centre of any serious regulatory debate.

1. Legitimacy: who has the right to write the rules?

Legitimacy concerns the acceptable use of power over other people. An exercise of power is legitimate when it can be justified to those subject to it. This matters significantly for AI, because AI dramatically amplifies power — the state’s power to surveil and punish, and platforms’ power to shape what billions of people see and say.

The uncomfortable truth is that many of the most consequential “regulatory” decisions about AI are currently made by private companies with no democratic mandate. When a social media platform’s moderation algorithms decide what counts as hate speech, it is making a quasi-constitutional judgement about free expression — famously stumbling over cases like removing the iconic Vietnam War “Napalm Girl” photograph as child nudity.

Companies have the means to govern speech at scale, but not the standing to do so.

Philosopher Thomas Ferretti argues that public administrations are, in principle, better placed to make such judgements: they contain mechanisms for brokering compromise between citizens who disagree, they satisfy the publicity principle (decisions about basic rights must be open to scrutiny), and they can be held accountable.

The legitimacy problem recurs internationally. The EU AI Act applies extraterritorially, reaching any provider whose system or outputs are used in the EU, and is widely expected to become a global template, much as the GDPR did. That gives one bloc’s values disproportionate power over citizens of countries that had no or limited input in formulating the rules. Meanwhile, much of the Global South lacks the resources to formulate its own AI regulatory priorities, even though its policy needs — agriculture, poverty reduction, digital literacy — differ sharply from the growth-and-competition concerns driving America and Europe.

2. Value pluralism: whose ethics get encoded?

Regulation ultimately encodes values into law, and AI systems encode values into software. But whose values? Google DeepMind ethicist Iason Gabriel, in his influential paper Artificial Intelligence, Values, and Alignment (2020), argues that the deepest challenge is not discovering the “true” moral theory to build into AI, because we live in pluralistic societies where reasonable people permanently disagree. Instead, regulators and designers need principles that people can endorse from many different paradigms — found through devices like an overlapping consensus across philosophical traditions, Rawls’s veil of ignorance, or democratic social choice.

This has a practical implication for lawmaking: procedure matters as much as substance. A technically excellent rule imposed without genuine deliberation will lack the stability and public acceptance that regulation needs to work. It also runs counter to the popular claim that technology is value-neutral. As Gabriel argues, once deployed, a technology tilts the balance of available outcomes in the world — it has a moral valence — so the choice of which values to embed cannot be dodged.

3. Discrimination law wasn’t built for algorithms

Anti-discrimination law is one of our oldest tools for regulating decision-making, and machine learning impacts it negatively. Legal scholars Solon Barocas and Andrew Selbst showed in their landmark article Big Data’s Disparate Impact (2016) that algorithms can discriminate without ever touching a protected attribute, by relying on proxies — postcodes, shopping patterns, speech characteristics — that correlate with race, gender, or disability. Intent-based doctrines of direct discrimination barely apply; and disparate-impact doctrines contain a “business justification” defence that more accurate models will often satisfy, effectively legalising systems that entrench historical disadvantage.

Research by Ifeoma Ajunwa and Daniel Greene on automated hiring platforms documents how bias enters at every stage:

• Targeted job ads that never reach some groups
• Training data built by “cloning your best people” from an unrepresentative workforce
• Video-interview analysis that penalises non-native accents, speech impediments, and darker skin tones — echoing Joy Buolamwini and Timnit Gebru’s Gender Shades findings (2018) on facial analysis error rates

Philosopher Lily Hu further points out that sometimes the algorithm is statistically accurate about an unjust world; in those cases, “collect better data” fixes nothing, because the problem lies in social reality itself.

Regulators are responding in this sphere. New York City’s Local Law 144 (2022) pioneered mandatory bias audits for automated hiring tools, and the EU AI Act classifies employment AI as high-risk. However, audits face real limits: auditors often lack access to code and to the demographic data needed to detect disparities, and technical fairness metrics can miss the social context of deployment.

4. Transparency versus trade secrets — and the limits of explanation

For power to be legitimately exercised, those affected must be able to understand and contest decisions. Virginia Eubanks’s Automating Inequality (2018) recounts caseworkers using a proprietary child-welfare risk model who could predict its outputs but had no idea why it scored families as it did. Intellectual property law here works directly against accountability: it shields the very information that due process requires.

Even full disclosure may not help. According to Burrell, many models are “black boxes” whose outputs cannot be explained in humanly intelligible terms. The field of explainable AI offers feature-importance methods, surrogate models, and counterfactual explanations, while documentation practices such as model cards and datasheets improve baseline transparency. But sceptics such as Babic and Cohen argue that post hoc explanations can be “insincere by design,” while others contend that high-stakes decisions should use only inherently interpretable models. Regulation must therefore decide a genuinely ethical question: is an approximate explanation good enough where liberty, livelihood, or welfare is at stake?

5. Privacy law protects individuals; AI harms groups

The GDPR (2018) set the global benchmark for data protection, yet its architecture — individual consent, individual identifiability — sits awkwardly with how AI actually works. “Anonymised” data, largely outside the law’s protections, can frequently be re-identified. More fundamentally, AI does not need your data to learn things about you: it learns patterns from data disclosed by people like you and applies them back to you.

The canonical example is the Target case reported by Charles Duhigg in 2012 — the retailer inferred a teenager’s pregnancy from purchase patterns learned from other shoppers, and outed her to her father via baby-product coupons. She never consented to anything; there was nothing to consent to. It’s also a textbook violation of Helen Nissenbaum’s theory of contextual integrity: information shared appropriately in one context (a shop till) flowed into another (family life) that no one agreed to.

Technical fixes don’t rescue the individual model either. Differential privacy mathematically guarantees no individual can be identified from a dataset — yet, as Cynthia Dwork and colleagues note, it still permits statistical learning about groups. A model can learn that people with your characteristics probably have a certain disease or vulnerability, and that knowledge can be used to profile and target you even though your record is untouchable. The technical definition of privacy (anonymity) and the ethical concern (protection from inference-based power) have quietly come apart, and regulators need to be explicit about which concept their rules are meant to track.

This is why privacy behaves like a public good, with the same structure as rush-hour traffic: each driver’s choice to take the car is individually rational, but a million rational choices produce a jam nobody wanted. Data works the same way — every disclosure improves the models that make inferences about everyone similar to you. Individually rational choices aggregate into a level of societal privacy nobody would have chosen collectively.

Public-goods problems can’t be solved by asking individuals to choose more carefully, especially when, as Shoshana Zuboff has documented, the digital economy’s core incentive is to maximise data collection within the limits of the law. The remedy must be structural rather than consent-based: purpose limitation, data minimisation, real limits on data brokerage, and scrutiny of what models can infer, not just what records they hold.

6. Responsibility gaps: when harm has no author

Automation can dissolve accountability. When a company deploys a procured hiring model it cannot inspect, and the model discriminates, the vendor blames the deployment context, and the deployer blames the product. The ambiguity here with culpability gives rise to a classic responsibility gap, as framed by Mecacci and Santoni de Sio (2021). Liability doctrines built around identifiable human decisions struggle here. Regulation needs to allocate duties explicitly across the AI supply chain. The EU AI Act attempts to do this by assigning distinct obligations to providers, deployers, importers, and distributors — and firms need internal governance (ethics boards, clear ownership of outcomes, due diligence when sourcing AI) so that someone is always answerable.

7. Markets, monopoly, and the limits of law

Business ethicist Joseph Heath’s “market failures approach” offers a clarifying lens. Heath argues that markets deserve moral respect only insofar as they allocate resources efficiently. Accordingly, firms act unethically when they profit from market failures — externalities, information asymmetry, barriers to entry — even where the law is silent.

AI is a market-failure machine: network effects drive monopoly concentration, opaque algorithms deepen information asymmetry, and energy-hungry data centres externalise environmental costs. Heath’s framework also carries a warning for regulators: competition law corrects inefficiency, not injustice. Using antitrust as an all-purpose morality enforcer against Big Tech confuses two different values and risks achieving neither.

There is a second structural limit: sanctions are just prices. A sufficiently profitable firm can treat fines as a cost of doing business. Furthermore, regulation is inherently reactive — governments, wary of losing the innovation race, typically legislate only after a technology has already established within a market. That is why serious governance thinking pairs hard law with organisational ethics: cultivating corporate and individual virtues (caution, epistemic honesty, the courage to speak up), embedding ethics into performance incentives, and drawing on the hundreds of responsible-AI toolkits now catalogued by the OECD while recognising that no toolkit works off the shelf.

8. Weighing risk under deep uncertainty

Finally, every AI regulation embodies a stance on risk. The EU AI Act’s tiered structure — banning “unacceptable risk” uses outright while permitting minimal-risk systems freely — leans toward the precautionary principle, which involves avoiding actions with a non-negligible chance of severe harm. The US approach has been more reactive and pro-innovation, closer to risk-neutral expected utility reasoning that weighs benefits and harms symmetrically.

Neither rule is obviously correct. Precaution can be paralysingly conservative (Sunstein, 2002) and can cost society enormous foregone benefits — in healthcare, climate, and development — while risk-neutrality can rationalise gambling with low-probability catastrophes. The same technology that acoustically tracks elephant poachers could coordinate targeted violence; dual use is the rule, not the exception. Regulators cannot escape making this value judgement, and they should make it openly.

The thread that ties it together

These eight considerations are not really separate problems. They are all symptoms of a single mismatch: law operates on individuals, intentions, national borders, and deliberate timescales, while AI operates on populations, correlations, global markets, and rapid iteration. Closing that gap will require more than clever statutes. It demands legitimacy in who decides, humility about disagreements over values, updated doctrines of discrimination and liability, structural rather than individualistic privacy protection, and honesty about the risk trade-offs being made on everyone’s behalf.

The frameworks now in place — the EU AI Act, UNESCO’s Recommendation, the OECD Principles, NIST’s RMF — are a genuine start. But they will succeed only if we treat AI regulation as what it fundamentally is: not a technical compliance exercise, but an ongoing exercise in collective self-government.

---

Sources and further reading

Course and academic sources: Gabriel, I. (2020), “Artificial Intelligence, Values, and Alignment,” Minds and Machines 30(3); Heath, J. (2014), Morality, Competition, and the Firm: The Market Failures Approach to Business Ethics (Oxford University Press); Ajunwa, I. & Greene, D., “Algorithms and the Social Organization of Work,” in The Oxford Handbook of Ethics of AI; Barocas, S. & Selbst, A. (2016), “Big Data’s Disparate Impact,” California Law Review 104; Buolamwini, J. & Gebru, T. (2018), “Gender Shades,” PMLR 81; Burrell, J. (2016), “How the Machine ‘Thinks’,” Big Data & Society; Eubanks, V. (2018), Automating Inequality; Hellman, D. (2008), When Is Discrimination Wrong?; Sunstein, C. (2002), “Beyond the Precautionary Principle”; Gorwa, R., Binns, R. & Katzenbach, C. (2020), “Algorithmic Content Moderation,” Big Data & Society; Vredenburgh, K. (2022) on the right to explanation; LSE, Ethics of AI course materials (2024).

Regulatory and policy sources (pre-April 2025): European Union, Regulation (EU) 2024/1689 (the AI Act), in force 1 August 2024, prohibitions applicable 2 February 2025; UNESCO, Recommendation on the Ethics of Artificial Intelligence (November 2021); OECD, AI Principles (2019, updated 2024); NIST, AI Risk Management Framework 1.0 (2023); New York City Local Law 144 (2022) on automated employment decision tools; EU General Data Protection Regulation (2018).