n.V6-4.08 | WHEN THE AGENT IS WRONG
01| It will be wrong. Not occasionally and visibly, which is manageable, but sometimes systematically and invisibly, which is not. Designing for that is the whole of governance.
02| The failure modes we plan against. Capability shortfall — agents absorb far less than designed, the curve flattens, the funding gap opens. Quality decay — output that looks fine in aggregate and contains a consistent error nobody catches without a targeted audit: a compliance scan that misses one category, a translation that insults people in one language, matching that quietly disadvantages one kind of project. Over-reliance — the platform integrates agents so deeply that when they go down, a CITI stops working. Replacement creep — cost pressure walks AI into roles we said were human, one reasonable exception at a time, until the platform scales and is no longer IkoCiti. Bias — trained on one barrio's data, deployed in another, flagging normal local practice as a violation. Gaming — people learn the patterns and exploit them; fake projects pass the scan, reputations get manufactured. Regulation — legal today, illegal next year, in one jurisdiction of many.
03| What we build against them. A standing oversight function with real authority to pause, adjust or override an agent when its behaviour departs from the mission — authority that does not require permission from the people who built it. Audits run independently of the builders, on a schedule and after incidents. An explicit failure budget: thresholds at which an agent is rolled back and humans take the work again, decided in advance and in writing, because in the moment there is always a reason to wait one more week.
04| Three commitments to the user. You always know whether you are dealing with an agent or a person — no hidden AI in anything user-facing, ever. Every consequential agent decision is logged, retrievable, and explainable to the person it affected, on request. And every deployment carries the conditions under which it gets retired; no agent is assumed permanent.
05| What we sacrifice: audit trails, rollback paths, oversight and explainability all cost compute, engineering time and speed. They make us slower than a competitor who skips them. They are the price of being trusted by people who have been let down by institutions before, and that trust is the asset we cannot rebuild once spent.
06| One question I have not resolved and will not paper over. When an assistant misleads a Maveriq and real damage follows, who is accountable — the division that runs the agents, the division that deployed it into their workflow, or the person who acted on the advice? Every answer has an ugly side. Blaming the user makes the assistant a liability nobody should touch; absolving the user entirely makes the platform responsible for every judgment made in a barrio. It is open, and it needs to be closed before scale, not after the first incident.
