The Right to Understand the Systems That Govern You

Somewhere in your city, there is a piece of software making decisions about people’s lives. It decides who qualifies for housing assistance. It determines which students get placed in which schools. It calculates whether a family is eligible for food support. It routes emergency services. It allocates public resources.

Nobody elected this software. Nobody voted on its rules. In most cases, nobody outside the development team can even read how it makes decisions. The rules are hidden in code, written by engineers who may have never met the people affected by their work, maintained by companies whose primary obligation is to their shareholders.

And yet these systems govern. They are, in every meaningful sense, the invisible bureaucracy of modern life.

I’ve built systems like these. I’ve written the code that encodes policy into logic, that turns human judgment into automated decisions. I did it for over two decades, across multiple countries, at scales ranging from small businesses to national-level platforms. I’ve come to believe something: the way we build these systems is fundamentally broken. Not because the technology is bad, but because it makes knowledge invisible.

Here’s what I mean.

When a government agency decides that families with children under five get priority for housing assistance, that’s a rule. In the old world, that rule lived in a policy document. Anyone could read it. Anyone could challenge it. Anyone could propose changing it. The rule was visible.

In the new world, that rule lives in a conditional statement inside a codebase. It might be in a function called calculateEligibility, or it might be distributed across three microservices and a machine learning model. To understand the rule, you need to read the code. To change the rule, you need a developer. To verify the rule is being applied correctly, you need an audit that most organisations can’t afford.

The rule hasn’t changed. But it’s become invisible to the people it affects. And invisible rules are dangerous, not because they’re malicious, but because they can’t be questioned.

This gets worse with AI. A traditional rule-based system is at least theoretically auditable. You can trace the logic. With machine learning, the decisions emerge from patterns in training data that no human fully understands. The model might be biased. It might encode assumptions from a different context. It might make exceptions that nobody intended. And the people affected by these decisions have no mechanism to understand why, let alone push back.

We’ve created a world where the systems that govern people’s lives are increasingly opaque to the people they govern. And we’ve normalised this. We call it “digital transformation.” We call it “efficiency.” We call it “innovation.” We rarely call it what it is: a transfer of power from communities to platforms.

I think there’s a different way. Several different ways, actually.

The most obvious is the transparency regulation. The EU’s AI Act is a start, requiring that high-risk AI systems provide explanations for their decisions. But legislation alone won’t solve a structural problem. You can require a company to explain its algorithm, but if the explanation requires a PhD in machine learning to understand, it’s not really transparency. Real transparency means the people affected by a decision can understand the rule that produced it, in language they speak, at a level of detail that lets them meaningfully challenge it.

Another path is what you might call structured readability. Instead of writing rules in code and then trying to explain them after the fact, you write rules in a format that’s inherently readable: formal enough for a machine to execute, clear enough for a domain expert to verify. This isn’t a new idea. Legal systems have been doing something similar for centuries: statutes are written in precise language that both courts and citizens can interpret. Software has no equivalent. It could.

Imagine if every decision-making system were required to publish its rules in a structured, human-readable format. Not the source code, which is too technical. Not a marketing summary, which is too vague. The actual decision logic is expressed in a specification that a policy expert, a community advocate, or a journalist could read and evaluate. When a food bank’s distribution logic is expressed in readable rules, the community can verify that the prioritisation is fair. When a school’s enrollment system declares its logic openly, parents can understand why their child was placed where they were.

Open-source software points in this direction but doesn’t quite get there. The code is technically available, but readability and availability are different things. A Python function is “readable” only if you read Python. What we need is a layer above code: a representation of logic that’s accessible to the people the logic affects, regardless of their technical background.

There’s also the question of portability. When a community’s rules are locked inside a vendor’s system, the community is captive. If they want to switch providers, they lose the institutional knowledge encoded in the old system. Data portability laws help with the information, but they don’t address the logic. You can export your data from a platform. You can rarely export the rules that govern how that data is used.

I think about employment in this context, too. The conversation about AI and jobs usually focuses on automation: which tasks will machines do instead of humans? But there’s a deeper displacement happening. When community knowledge gets absorbed into opaque systems, the people who held that knowledge lose their role. Not because a robot took their job, but because their expertise became invisible. The system works, nobody knows why, and the humans who used to understand the domain become operators instead of experts.

The fix isn’t to slow down technology. The fix is to build technology that keeps human knowledge visible and human expertise relevant. Technology where the person who understands how something really works can see their understanding reflected in the system that implements it. Where expertise is encoded, not erased.

I know the counterargument. Transparency is expensive. Readable rules are slower to implement than just writing code. Explaining yourself is harder than operating in silence. All true. But the cost of opacity is higher. The cost of opacity is communities that can’t understand the systems they depend on. Citizens who can’t challenge decisions made about them. Organisations that can’t adapt their tools when reality changes because the logic is locked inside a vendor’s proprietary codebase.

The cost of opacity is a world where power flows toward those who can read code and away from everyone else.

The right to understand the systems that govern your life isn’t a feature request. It’s a fundamental condition of a fair society. And technology should expand that right, not eliminate it.

Osamah Alghanmi is the co-founder of Almadar, a software studio based in Ljubljana, Slovenia. He spent over two decades building enterprise systems, including national-scale platforms in Saudi Arabia, before deciding that the most important problem in software isn’t how to build it faster, but how to make it accountable.