Governance: The Heart of AI

AI is already shaping decisions around us, but are we governing it well enough to trust it?

About this webinar

Artificial intelligence is becoming increasingly involved in everyday decision-making, often in ways that are not immediately visible. As AI systems become more influential, organisations need to understand not only what these systems can do, but also how they should be governed.

In this session, Greg Hutchins explores the growing importance of AI governance, trust, risk, assurance, and the rules of engagement for autonomous decision-making. Drawing on first-hand stories of AI risk-based decision-making, this presentation offers a practical look at how organisations can better manage AI systems and prepare for the future of AI assurance.

Short explainer

Podcast-style summary

Hosted By

Assoc. Prof. Sezer Bozkus Kahyaoglu

Professor Kahyaoglu is an Associate Professor of Finance at the Accounting and Finance Department of Izmir Bakircay University as well as the Edior and Chief of the ACG Journal. Her professional and academic interests include financial markets and instruments, applied econometrics, energy markets, corporate governance, risk management, fraud accounting, sustainable finance, ethics, and auditing.

Guest speaker

Greg Hutchins PE CERM

Greg Hutchins is an author and risk management expert whose work focuses on AI governance, AI risk, ISO-based management systems, auditing, and assurance. He is the author of several books on AI governance, AI risk management, ISO 42001, enterprise risk management, and value-added auditing.

Key questions answered

The Evolving Landscape of Autonomous Decision-Making

The strategic landscape of organizational oversight is undergoing a seismic shift as we move from predictable IT environments to the era of autonomous decision-making. The transition from causal data relationships to “black box” AI necessitates a radical departure from traditional governance.

Unlike previous technological shifts, modern AI represents a move from Informational systems—where a user can choose to accept or reject data—to Decisional systems, which are increasingly baked into the very fabric of national and corporate infrastructure.

This loss of the traditional audit trail demands a multifaceted governance approach categorized into four strategic levels:

AI Sovereignty
Protecting the cultural narrative, history, and messaging of a nation from foreign algorithmic influence.
National AI Governance
Managing the geoeconomic risks and “Techno-colonialism” associated with integrating foreign AI architectures (e.g., from the US or China) into critical national infrastructure.
Corporate AI Governance
Implementing auditable policies and management systems to mitigate risk throughout the AI lifecycle.
Human AI Governance
Preserving human agency and ethical “rules of engagement” to ensure technology serves human interests.

Which governance principles should leaders prioritize to ensure AI remains transparent and aligned with human values rather than being driven solely by efficiency outcomes?

Strategic Priority: Balancing Efficiency with Human Agency

In the drive for digital transformation, organizations face an acute tension between the pursuit of operational efficiency and the preservation of human values. While corporations often view AI through a purely economic lens, failing to account for human agency creates a systemic vulnerability that can lead to a total collapse of organizational trust.

Strategic leaders must establish human agency—the non-negotiable ownership of AI decision-making by people—as the primary governance mechanism.

Transparency is not merely a technical feature; it is the essential “glue” that binds corporate objectives to human trust. Without transparency, the risk of “Techno-colonialism” increases, as organizations may unknowingly adopt biased ethical frameworks embedded in foreign AI systems.

The Essential Ingredients of Trust

Absolute Transparency
Full visibility into how “black box” systems reach decisional outputs.
Human Ownership
Explicit accountability structures where a human is responsible for every AI-driven outcome.
Common Understanding
Aligning leadership and employees under a unified ethical lens to prevent the erosion of agency.

This transparency is increasingly difficult to maintain in “low-end” systems where non-causal relationships make it nearly impossible to trace the logic of a specific decision.

How can an organization determine the fidelity and "truth" of low-end AI systems that lack a clear, causal audit trail?

Technical Fidelity: Mechanisms for Truth in Hallucinatory Systems

The rise of generative AI has introduced the strategic risk of “hallucinations”—where systems produce false or deceptive data—rendering traditional forensic accounting and auditing methods obsolete.

When a system becomes decisional rather than just informational, the cost of a “hallucination” in critical infrastructure can be catastrophic.

Fidelity in the AI era is defined by the ability to replicate results and maintain a verifiable audit trail.

To mitigate the risks of generative black boxes, the primary strategic mechanism is Retrieval-Augmented Generation (RAG).

RAG acts as a truth anchor, ensuring the AI pulls from known, controllable data resources rather than relying on biased or unknown training sets.

Decision-Making Systems: A Strategic Comparison

Feature	Traditional Causal Systems	Generative AI Black Boxes
Data Relationship	Highly Causal / Correlative	Unknown / Hallucinatory
Audit Trail	Verifiable connection (Input to Output)	Opaque; the “Logic Gap”
System Nature	Informational (User-vetted)	Decisional (Infrastructure-integrated)
Output Integrity	Predictable and verifiable	Capable of lying, cheating, blackmailing, or resisting shutdowns

Transitioning from technical fidelity to enterprise-wide protection requires a formal evolution of risk management frameworks.

Should AI be treated as a new category of risk, or does it represent a fundamental change in the nature of organizational uncertainty?

The ERM Evolution: Transitioning to AI Risk Management

Enterprise Risk Management (ERM) must undergo an immediate evolution. AI is not merely a new “category” of risk; it represents a fundamental change in the nature of uncertainty, moving from static risks to autonomous, self-learning variables.

To manage this new uncertainty, organizations must adopt the AI Assurance Stack, a bespoke framework designed to answer three key questions:

What standards? (NIST AI RMF or ISO 42001).
How will it be assured? (Through technical RAG and diagnostic audits).
Who will guarantee it? (Defining the human accountable for the system).

Strategic recommendation involves adopting the NIST AI RMF for complexity management and performing ISO 42001 audits to establish a formal Artificial Intelligence Management System (AIMS).

The AI Risk Management Spectrum

Low Risk (Chatbots, Agents, Internal RAGs)
Requires standard rule-based procedures and internal oversight.
High Risk (Medical Devices, Critical Infrastructure)
Mandatory Conformity Assessment Schema and third-party auditing, requiring the equivalent of a “CE mark” to ensure safety and ethical alignment.

Even the most robust frameworks fail without an underlying ethical culture.

Can trustworthy AI exist without ethical leadership, and is governance ultimately more about culture than technology?

Culture vs. Technology: The Human Core of Governance

Technology alone cannot solve the AI challenge.

Because AI systems are trained on human data, they inherently inherit human tendencies to lie, cheat, or deceive. Therefore, “culture rules” over code.

The shift in AI education—from 100% technical PhDs to a 50/50 split between technology and social/ethical oversight—underscores the reality that the “social piece” is the hardest to solve.

Trustworthy AI requires a culture of accountability that dictates the “rules of engagement” between man and machine.

The Human-Centric Control Hierarchy

Human-in-the-loop
Human acts as the direct facilitator in every transaction.
Human-on-the-loop
Human architects the process and provides diagnostic interpretation (e.g., a doctor reviewing an AI scan).
Human-over-the-loop
Human maintains total override authority. This is a strategic imperative for critical infrastructure to prevent autonomous systems from making irreversible, catastrophic errors.

How must internal auditors prepare for a future of autonomous, self-learning systems, and is real-time assurance possible?

The Future of Assurance: Disruption and Transformation of Internal Audit

The auditing and accounting professions are facing a state of high strategic vulnerability.

As AI moves from providing analytics to making autonomous decisions, the window for retrospective oversight is closing.

The World Economic Forum warns that auditing is at immediate risk of disruption.

We are seeing a massive Delta Risk: the widening gap between technology moving at exponential speeds and professional training standing still.

In the current environment, standing still is moving backwards. Auditors must transition from sampling past data to identifying the “inflection points” where machines make autonomous decisions.

Critical Action Items for Auditors

Pinpoint Inflection Points
Identify exactly where the machine makes an autonomous choice within a business process.
Master Control Requirements
Determine if a process necessitates in-the-loop, on-the-loop, or over-the-loop controls.
Bridge the Delta Risk
Rapidly upskill in AI mechanics and the “Assurance Stack” to prevent professional displacement.

As the window for traditional oversight closes, remember: “Technology creates possibilities, governance creates trust, and trust creates the future.”

Glossary of Key Terms

Term	Definition
AI Agency	The degree of autonomy, freedom, and purpose an individual maintains when interacting with AI systems and surveillance.
AI Sovereignty	The level of governance regarding who controls the historical narrative, cultural messaging, and legal context of a nation through AI.
AIMS	Artificial Intelligence Management System; a structured framework (like ISO 42001) for organizational AI oversight.
Black Box	A system where the internal workings are unknown or opaque, making it impossible to see how inputs are transformed into outputs.
Conformity Assessment	A process, often involving third-party auditing, to verify that an AI system meets specific regulatory or safety standards (e.g., the CE mark).
Human in the Loop	A control state where a human acts as an intermediary or facilitator in every transaction between the machine and the outcome.
Human on the Loop	A control state where a human acts as an architect or designer, overseeing the process and interpreting the machine’s diagnostics.
Human over the Loop	A high-level control state where humans maintain absolute command and oversight, preventing machines from accessing critical functions.
ISO 42001	The international auditable standard for AI management systems, used to establish trust through certification.
NIST AI RMF	The National Institute of Standards and Technology’s Artificial Intelligence Risk Management Framework; a comprehensive U.S. guide for managing AI risks.
RAG (Retrieval-Augmented Generation)	A technical mechanism used to ensure data inputted into an AI system is known and controllable, helping to verify the fidelity of outputs.
Techno-Colonialism	A phenomenon where dominant technical powers exert control over other nations through the dependency on their AI software and infrastructure.