Responsibility, ethics, and governance

Target audience: non-technical leadership + technical | Prerequisites: 5.6 Continuous improvement: from measurement to decisions

What you'll learn

After this document, you'll be able to:

• explain what governance is and why, in a growing system, individual people's diligence is no longer enough;
• compile a decisions table — which decision, who makes it, who approves it, where it's written down;
• ask the four ethics questions for every automation decision;
• write down a simple risk register and tie it to the quarterly review;
• act correctly when something goes wrong — learn instead of hunting for culprits.

In plain terms

In a small system, people can settle everything important by talking. As the system grows — more workflows, more people, more customers — every decision starts to depend on who happens to be around. That's where rules come in: who may make which decisions, who approves, where it's written down. Writing down the rules isn't about control — it's protection and clarity for everyone: the person knows what they may decide themselves, the manager knows what needs their signature, and when something goes wrong, it's visible where the decision was made.

Governance: who may make which decisions

Governance (the body of steering rules: who may make which decisions, how risks are assessed, who is responsible). 5.2 Team workflows and standards wrote down how work gets changed; governance looks one rung higher — not the work, but the decisions.

What happens without it. A builder launches a new system because it seemed like a good idea — half a year later, someone asks who decided that. A colleague sends customer data to a new provider because the service is free — and now the question lands with the manager. Without rules, every decision is an exception and every exception is a risk.

Governance's simplest tool is the decisions table (decision | who makes it | who approves | where it's written). Five rows to make a start:

In plain terms: the decisions table works like signature rights at a bank — nobody debates whether the accountant “may” transfer a large sum; it's written down. When it's written that sending data to a new provider needs the manager's and the data protection expert's consent, nobody has to guess or wait for a meeting: the path is known.

The table isn't a gag. The vast majority of day-to-day decisions — reversible ones and those contained within one system — stay outside the table: the system owner (the owner — the person responsible for a system/prompt, a single name; the 5.2 principle) makes those alone. Only decisions that are hard to reverse, that affect customers, or that carry responsibility are in the table.

Responsibility: “the model erred” is not an excuse

Responsibility for an AI mistake always stays with people and the company (1.6 Roles and responsibility in a project) — the customer, the bank, and the supervisory authority turn to the company, not the model. “The model erred” isn't an excuse anyone accepts: a model has no contract, signature, or registration — so it has no responsibility either.

In practice, responsibility rests on two things:

1. Every system has an owner. The question “who is responsible for this?” is answered by a name, not “the team” — because everyone's responsibility quickly becomes nobody's responsibility.
2. Every significant decision has a written decision protocol: what was decided, why, who approved, when. Three lines, two minutes — and years later nobody is forced to reconstruct from memories why it was decided exactly that way.

The audit trail (a record of every automated action, 3.5) underpins all of it: when asked “why did the system act exactly this way?”, the record answers, not anyone's memory.

In plain terms: written-down decisions are the company's insurance. When someone asks “who decided that the chat logs are kept for 90 days?”, the answer is in the document, with the reasoning — and that protects both the company and the person who made the decision. Better three lines in writing today than an investigation over memories tomorrow.

The four ethics questions

Ethics here isn't about feelings but four concrete, answerable questions to ask with every automation decision:

1. Transparency (the person knows they're talking to a system). Question: does the customer know the chat partner isn't a human? Honesty is also the simplest path: add to the start of the chat “I answer you through a system — if you need a human, here you'll reach one.” Leaving the customer believing they're talking to a human is short-term convenience and long-term loss of trust.

2. Fairness (similar cases — similar answers). Question: does the system behave differently toward different groups — by language, name, place of residence? A simple test: compare answers to similar cases that differ in only one attribute — the same question in Estonian and in English, the same case with two different names.

3. Harm prevention. Question: what is the worst thing the system can do — and does the design prevent it? This is 3.5's safety test: if the answer is “we hope the model behaves,” the system isn't ready.

4. Human dignity. Question: are we automating something that must stay human? Receiving complaints, delivering bad news — these are the places where a person expects a person. Speed isn't a value here if the price is talking to a machine at a hard moment.

The risk register

Governance can't manage risks that nobody has named. That's what the risk register is for (a table of significant risks written down: risk | probability | impact | who watches it).

Four rules that keep the register alive:

• 5–10 rows are enough — better five real rows than fifty generic ones.
• Every row has a name — “the team watches it” means nobody watches it; the same principle as the owner (5.2).
• Probability and impact on three levels — low, medium, high; the assessment must stay at minutes, not days.
• Updated once a quarter — the natural place is the quarterly review (5.4 Cost strategy at scale), where the leadership goes over the numbers anyway: which risk materialized, which faded, which appeared.

Step-by-step example: Remedy House writes the rules down

The fictional “Remedy House” — the e-pharmacy chat system that 3.5 and 5.3 already know: the chat answers about stock and prices and routes health questions to a pharmacist. Over a year the system grew: considerably more customers, an English-language chat was added, and two new workflows. The pharmacy's manager, Liis Telline, noticed that decisions had started to depend on who happened to be in the building — and wrote the rules down.

1. The decisions table:

The health row is the table's most important wording: not “forbidden, except,” but forbidden — because health advice is a sensitive domain where human-in-the-loop (a workflow where a human approves the result before use) is always present.

2. The four ethics questions at Remedy House. Transparency: a sentence was added to the chat's introduction — “You're chatting with the Remedy House system; for medication questions you'll reach a pharmacist.” Fairness: the language equality test — the same question in Estonian and in English; stock, price, and routing were equivalent; the test went on the calendar once a quarter. Harm prevention: the worst scenario is misleading medication information — the protection isn't hope but design; such questions are always routed to a human. Dignity: complaints stay with a human.

3. The risk register — the first five rows:

4. The case: when something went wrong. A customer asked how to store a medication that has to be kept in the refrigerator — the system answered confidently “at room temperature.” The error was noticed in the weekly summary: instead of routing, the system had answered itself.

What followed was learning, not culprit-hunting:

• Facts from the log (3.4 Errors and error handling): when, which question, where the system got the answer from.
• No culprits were sought — the fault wasn't in a person but in a rule: the system was still allowed to answer outside its own source.
• The fix: the source was made a mandatory part of the answer (4.2 RAG) — answers only from the pharmacy's trade-information data and with a source; a result without a source routes to a human.
• Approval and the lesson: the owner approved the instruction change, the case went into the risk register as a new row, and the next quarterly review checked whether the fix held (5.3's “learn” step).

In plain terms: the case didn't hunt for culprits: the system erred, the rule was fixed, and everyone saw who and why. A company where every mistake is a shame hides its mistakes; a company where a mistake is information fixes its rules. And written-down decisions are what protect the company — before both customers and supervisors.

Summary

• Governance is the body of steering rules — and its simplest form is the decisions table: decision, who makes it, who approves, where it's written.
• Responsibility always stays with people and the company — “the model erred” isn't an excuse; an owner for every system and a written protocol for every significant decision.
• The four ethics questions: transparency, fairness, harm prevention, human dignity — each answerable with one test.
• The risk register: 5–10 rows, a name next to every row, updated at the quarterly review.
• When something went wrong: facts written down, fix the rule, don't hunt for culprits — written-down rules are protection and clarity for everyone.

What's next?

• previous → 5.6 Continuous improvement: from measurement to decisions
• next → 5.8 Template library: checklists and samples
• back → handbook index