Who Controls the Machine?
AI, power and the uncomfortable question of who remains in control
In Person of Interest, billionaire programmer Harold Finch builds an artificial intelligence capable of watching almost everyone.
The Machine sees through cameras, follows communications, processes enormous amounts of data and predicts acts of violence before they happen.
But Finch is not primarily afraid that his Machine will become evil.
He is afraid of power.
So he restricts it.
He limits what it can communicate. He prevents it from freely identifying people. He builds safeguards. At one point, when another artificial intelligence project becomes too dangerous, the response is wonderfully primitive: sabotage the computer and pull the plug.
It is fiction.
But more than a decade later, Finch's question has become remarkably real:
Who controls the Machine?
And perhaps more importantly:
What happens when pulling the plug is no longer enough?
The Wrong Question
Much of the popular debate about artificial intelligence revolves around one dramatic question:
Could AI turn against humanity?
It makes excellent science fiction.
It may also distract us from a much more immediate problem.
Artificial intelligence does not need to hate us to become dangerous.
It does not need consciousness.
It does not need anger, ambition or a desire to survive.
It merely needs capability, access and authority.
Give a sufficiently capable system access to communications, financial infrastructure, databases, surveillance systems, software environments or critical infrastructure, and the relevant question changes.
It is no longer:
What can the model say?
It becomes:
What can the system do?
That distinction matters enormously.
A chatbot can recommend an action.
An AI agent may be able to execute it.
And somewhere between those two lies one of the most important legal questions of the coming decade.
AI Does Not Need to Become Sauron
There is an older metaphor for this problem.
A ring.
In Tolkien's The Lord of the Rings, the danger of the One Ring is not merely that an evil creature possesses it.
The deeper danger is that extraordinary power changes the position of whoever controls it.
Even those who intend to use the Ring for good are warned against taking it.
Why?
Because the fundamental problem is not simply who holds power today.
It is what becomes possible once that power exists.
Artificial intelligence presents a surprisingly similar governance problem.
AI can concentrate extraordinary capabilities in governments, corporations and increasingly small groups of individuals.
The person controlling such a system does not need supernatural powers.
They may simply need a computer.
And access.
The Machine Behind the Machine
This is where the discussion about "AI taking over" can become misleading.
Perhaps the first question should not be whether machines will control humans.
Perhaps it should be:
Which humans will control the machines?
Who determines their objectives?
Who chooses the data?
Who decides what they may access?
Who determines when an AI system may act autonomously?
Who can inspect those decisions?
Who can challenge them?
And who has the authority to stop the system?
These are not science-fiction questions.
They are questions of law, governance and institutional power.
Europe has already begun constructing a legal response. The EU AI Act introduces obligations based on risk and imposes particular requirements on certain high-risk AI systems.
But regulation faces an uncomfortable structural problem.
Law regulates what exists.
Technology continuously creates what did not exist when the rules were written.
And increasingly, we are not merely building models that produce answers.
We are building systems capable of pursuing tasks.
From AI to Agents
That transition deserves considerably more attention.
Traditional generative AI largely waits.
You ask.
It responds.
Agentic AI changes the relationship.
An agent can potentially receive an objective, break it into steps, use external tools, obtain information, interact with other systems and continue working toward the objective with less human intervention.
None of that requires consciousness.
That is precisely why the fixation on whether AI can "wake up" may be the wrong debate.
A system does not have to be alive to exercise power.
An algorithm denying someone credit is not alive.
A surveillance system identifying someone in a crowd is not alive.
An automated trading system moving millions within seconds is not alive.
Yet each can produce very real consequences.
Power does not require consciousness.
That may be the sentence we should remember.
The Administrator Problem
Person of Interest gives its Machine an administrator: Harold Finch.
He tries to teach it boundaries.
He attempts to encode morality into architecture.
And he deliberately restricts the Machine because he understands something deeply important:
You do not design safeguards only for what a system can do today. You design them for what it may become capable of doing tomorrow.
That principle should sound familiar to lawyers.
We do it constantly.
Constitutions do not assume governments will always be benevolent.
Contract law does not assume parties will always behave honourably.
Corporate governance does not assume directors will never face conflicts of interest.
Data protection law does not assume organisations will always use personal information responsibly.
Law exists partly because good intentions are not a governance mechanism.
Why should artificial intelligence be different?
The Kill Switch Is Not Enough
There is something reassuring about the idea of a red button.
If AI behaves badly, switch it off.
Problem solved.
Except modern digital infrastructure does not live inside Harold Finch's laptop.
Cloud computing is distributed. Systems are replicated. Services cross borders. Infrastructure may involve numerous providers, jurisdictions and dependencies.
That does not mean AI has developed an instinct for self-preservation.
It means humans have deliberately built resilient infrastructure.
And that creates a fascinating paradox.
We spend enormous resources making digital systems difficult to interrupt.
Then we reassure ourselves that, if necessary, we can interrupt them.
So perhaps every discussion about powerful autonomous AI systems should include questions that sound almost embarrassingly simple:
Who has the off-switch?
Does the off-switch actually work?
And:
Can the person who needs it legally and technically reach it?
Who Watches the Watcher?
There is another danger.
AI may become so useful that humans voluntarily surrender oversight.
Not because a machine conquers us.
Because it is faster.
Cheaper.
More accurate.
More convenient.
First it recommends.
Then it decides.
Eventually, perhaps, humans merely approve.
And at some point the human-in-the-loop risks becoming something rather different:
the human-who-clicks-OK.
That is not meaningful human control.
It is administrative theatre.
If human oversight is to remain a genuine safeguard, the human must have enough information, competence, authority and time to disagree with the machine.
Otherwise "human oversight" becomes a comforting phrase attached to an automated decision.
The Real Alignment Problem
AI researchers often speak about alignment: ensuring that artificial intelligence acts consistently with human values and intentions.
But there may be another alignment problem.
With whose values?
Humanity does not have one administrator.
Governments disagree.
Cultures disagree.
Corporations have commercial interests.
Citizens have competing rights.
Security agencies prioritise security.
Businesses prioritise efficiency.
Individuals prioritise autonomy.
The hardest problem may therefore not be teaching AI what humans want.
It may be deciding which humans get to decide what AI should want on our behalf.
That is not merely a technical problem.
It is constitutional.
Who Controls the Machine?
Perhaps artificial intelligence will never become conscious.
Perhaps there will never be a dramatic moment when a computer announces that it has awakened.
Perhaps nobody will ever need to fight an all-knowing Machine.
But none of those things need to happen for AI to transform the distribution of power.
The more capable these systems become, the more important some very old legal principles become with them:
Accountability.
Transparency.
Due process.
Limits on authority.
The ability to challenge decisions.
The separation of powers.
And ultimately, the possibility of saying:
No.
The greatest danger may not be that artificial intelligence develops a desire to rule humanity.
It may be that humans acquire unprecedented technological power over other humans — and discover that the systems exercising that power have become too complex, too autonomous or too indispensable to meaningfully challenge.
Harold Finch understood the problem before most of us had ever spoken to an AI.
Building the Machine was only the beginning.
The real question was always:
Who controls it?