The Question Before the Answer

AI, the illusion of control, and the systems we're trying to preserve

We are spending an extraordinary amount of time asking how to govern artificial intelligence. Perhaps we should spend a little more asking what we’re governing towards.

There is something almost theatrical about the current conversation surrounding artificial intelligence.

Everywhere, people are calling for control. Regulation. Alignment. Oversight. Guardrails. Kill switches.

Technology companies promise transformation. Governments promise protection. Institutions promise responsible innovation.

And somewhere in the middle of this increasingly elaborate production, a rather inconvenient question remains largely unanswered.

What, exactly, are we trying to optimise for?

It’s a deceptively simple question.

We might say human flourishing. Safety. Prosperity. The greatest good. Reasonable aspirations, all of them.

But whose prosperity? Whose safety? What constitutes flourishing? And who gets to decide when the pursuit of one comes at the expense of another?

These are not new questions. Philosophers, economists and political theorists have wrestled with them for centuries.

What is new is the possibility that we are developing systems capable of pursuing objectives at a scale and speed that may expose just how poorly we’ve defined those objectives ourselves.

And perhaps that’s where the conversation becomes uncomfortable.

The Architecture of Wonderland: Who Makes the Rules?

Much of our thinking about AI begins with an assumption that rarely receives the scrutiny it deserves: that our existing systems provide the appropriate reference point.

We ask how AI can be aligned with our laws, institutions, economic structures and social expectations.

But what if those structures are themselves imperfect expressions of the values they claim to uphold?

Consider accountability.

When something goes wrong, our instinct is frequently to identify responsibility, assign blame and demonstrate that action has been taken.

Sometimes that is entirely appropriate. Deliberate wrongdoing demands accountability.

But what happens when the failure is systemic? When responsibility is distributed across institutions, technologies, processes and decisions made over years?

Does finding someone to blame necessarily bring us closer to understanding what failed?

Or have we simply become rather adept at shouting “Off with their heads!” while the machinery of Wonderland continues operating exactly as before?

The pursuit of blame can become another mechanism through which the underlying architecture escapes examination.

We see variations of this across cybersecurity, public administration, healthcare, financial systems and technology governance.

An incident occurs. An inquiry follows. Recommendations are made. Assurances are given.

And sometimes the conditions that produced the failure remain largely intact.

Are we optimising for better outcomes, or for the appearance of having responded?

That distinction matters enormously when we begin introducing increasingly capable AI into these environments.

Because automating an inadequate system does not necessarily improve it. It may simply make its inadequacies faster, cheaper and considerably more difficult to challenge.

The Curious Business of Control

There is another assumption worth examining. That intelligence must be controlled.

Of course, powerful technologies require safeguards. Systems capable of causing significant harm demand rigorous oversight, meaningful accountability and mechanisms for intervention.

But is control the same thing as governance?

And is governance necessarily synonymous with maintaining existing distributions of authority?

Much of human history has been organised around hierarchies of knowledge and power. Those who possess information, resources or institutional authority have traditionally exercised considerable influence over those who do not.

Artificial intelligence complicates that arrangement.

Not simply because it may perform tasks previously reserved for particular professions or institutions, but because it challenges some of the assumptions upon which those arrangements were constructed.

• What happens when expertise becomes more widely accessible?

• When decisions can be interrogated at unprecedented scale?

• When the mechanisms through which authority has historically been exercised become increasingly visible?

  
  

There are legitimate reasons to be concerned about the concentration of AI capabilities, the misuse of autonomous systems and the potential erosion of human agency.

But there is another question alongside those concerns.

How much of our response is motivated by the desire to protect humanity, and how much by the desire to preserve familiar arrangements of power?

The two are not necessarily mutually exclusive. And that is precisely why the distinction deserves examination.

And what if the disruption we're witnessing isn't simply the emergence of a new form of intelligence, but the exposure of contradictions that have existed within our systems all along?

Through the Looking Glass: The Question of Alignment

We talk constantly about aligning AI with human values. It’s an important undertaking. But humanity is not a single coherent set of values.

We disagree about justice, freedom, fairness, equality, responsibility and even what constitutes a good life.

Our institutions reflect those disagreements. Sometimes through careful compromise. Sometimes through historical accident. Sometimes through the exercise of power.

So when we say AI should be aligned with humanity, what are we actually asking?

• Aligned with our aspirations?

• Our laws?

• Our prevailing social norms?

• Our economic incentives?

• Or our demonstrated behaviour?

Because those things are not always consistent.

And perhaps the most interesting question is not simply whether we can align artificial intelligence with humanity.

It’s whether the process of attempting to do so might force us to examine the contradictions within humanity itself.

Diversity of thought is essential, I’m not suggesting we should or even could, eliminate disagreement. There is utility in productive disagreement to challenge and expand.

But we must consider that the objectives we encode into our technologies are inseparable from the assumptions we make about ourselves.

Curiouser and Curiouser: Questioning the Familiar

Lewis Carroll created a world in which elaborate rules governed almost everything, yet those rules frequently appeared absurd to the person encountering them.

Alice kept asking questions.

Not because she possessed all the answers, but because she hadn’t yet accepted that the way things were was necessarily the way they ought to be. There is something useful in that disposition.

We are entering a period in which many familiar assumptions about intelligence, authority, knowledge and human capability will be tested.

Some will endure. Others may prove remarkably fragile.

The temptation will be to respond by strengthening the familiar structures, tightening the rules and insisting that the future conform to the architecture of the past.

The Opportunity: Rethinking What's Possible

Perhaps the more productive response is to examine the architecture itself.

What if coexistence with increasingly capable intelligence requires more than establishing boundaries around what AI may do? What if it also requires reconsidering what we, individually and collectively, are trying to achieve?

This is not an argument for abandoning governance, diminishing human responsibility or surrendering decisions to machines.

Quite the opposite.

It’s an argument for taking those responsibilities seriously enough to question the assumptions beneath them.

For recognising that meaningful accountability requires more than identifying someone to punish.

That genuine alignment requires more than declaring a set of values.

And that progress requires more than making existing systems operate more efficiently.

We need to become a little more comfortable asking questions that don’t have immediately convenient answers.

After all, we’ve become remarkably proficient at building machines that produce answers.

It would be rather unfortunate if we neglected to improve the questions.