AI for Human-AI Teaming — Calibrating Trust in Autonomous Systems

June 25, 2026 — Human-AI Teaming — Trust Calibration — Adaptive Automation — Explainable AI — Defence AI — IDEaS

Human-AI Trust Calibration — Bidirectional feedback loop between operator state estimation, adaptive automation, and explainable AI

Introduction

The most dangerous moment in human-autonomous teaming isn't when the AI fails — it's when it fails silently and the human doesn't notice. Over-trust in automation leads to complacency; under-trust leads to disuse. Calibrating operator trust to match actual AI capability is one of the hardest problems in defence AI, and it's where NovaFuse's approach to explainable AI and uncertainty quantification becomes a force multiplier.

Canada's defence modernization increasingly depends on autonomous and semi-autonomous systems — from unmanned maritime patrol to AI-assisted air defence. But inserting AI into kill chains and decision loops demands more than accuracy scores. It demands that operators understand what the AI knows, what it doesn't know, and when to intervene. This is the human-AI teaming challenge, and solving it is critical for both operational effectiveness and compliance with emerging AI governance frameworks.

The Trust Calibration Problem

Human factors research in military AI identifies three failure modes in human-automation interaction:

Failure Mode	Cause	Operational Risk
Automation complacency	Over-trust leads to monitoring cessation	Miss critical AI failures
Algorithmic aversion	Distrust after rare failures	Manual overrides degrade outcomes
Mode confusion	Misunderstanding AI state	Inappropriate interventions

These aren't theoretical risks. The USS Vincennes incident (1988), automation-related aviation accidents, and recent autonomous vehicle near-misses all demonstrate the same pattern: the technology worked as designed, but the human-AI interface failed.

For defence, where decisions carry lethal consequences, the stakes are even higher. A commander who doesn't understand when to override an AI targeting recommendation — or when to trust it over their own assumptions — creates operational risk that no amount of model accuracy can mitigate.

What Effective Human-AI Teaming Requires

Real-Time Trust Calibration

Effective teaming requires the AI to communicate its confidence and reasoning in real time. Not just a confidence score — that leads to probability gaming — but contextual explanations that help operators build accurate mental models of AI capability.

Approaches include:

Uncertainty visualization: Showing not just the answer but the distribution of possible answers, with explicit “I don’t know” states
Decision traces: Explaining what evidence led to a recommendation and what evidence contradicts it
Failure mode disclosure: Proactively communicating what the AI cannot detect or predict

Adaptive Automation Levels

Not every decision should be fully automated. The appropriate level of autonomy depends on task criticality, information completeness, time pressure, and operator workload. NovaFuse's framework models these as dynamic automation levels that shift in real time — increasing autonomy when the operator is overloaded and the AI is confident, requesting human input when uncertainty is high or lethality is involved.

Understanding Operator State

Emerging research on human-AI teaming measures operator cognitive state through secondary task performance, physiological indicators, or behavioral proxies. A system that detects operator overload can increase automation; one that detects complacency can inject uncertainty cues to re-engage conscious monitoring.

For CAF applications — particularly air defence, maritime patrol, and C2 — operator attention management is a critical bottleneck. AI teammates don't just need to be smart; they need to be good teammates.

IDEaS Connection: Cognition and Trust

The June 7 IDEaS challenge call for “Cognition and trust: Real-time dynamic calibration for human-autonomy teams” targets exactly this problem space. It seeks solutions for:

Measuring and modeling human trust in autonomous systems in real time
Dynamically calibrating automation levels based on human cognitive state
Improving team performance through adaptive human-machine interfaces

NovaFuse's work in explainable AI, uncertainty quantification, and human factors-aware decision support positions us to compete here. The key differentiator: our approach integrates machine uncertainty with human state estimation, creating a two-sided trust calibration framework.

Technical Approach

Building effective human-AI teaming for defence requires several integrated capabilities:

Multi-modal operator state estimation. Fusing behavioral, physiological, and task-performance signals to estimate cognitive load, attention, and trust level. This doesn't require invasive sensors — behavioral proxies like response time patterns, decision consistency, and information-seeking behavior provide rich signals.

Contextual explanation generation. Producing explanations tailored to the operator's current task, expertise level, and information needs. A junior operator may need different explanation granularity than a senior commander facing the same AI output.

Graceful capability degradation. When the AI's capability degrades (sensor failure, novel threat, adversarial attack), the system must communicate this clearly and transfer appropriate control to the human — not silently produce degraded outputs.

Training integration. Human-AI teams need to train together. Simulation environments that expose operators to AI failures in controlled settings build appropriate trust calibration over time. This connects directly to our simulation & wargaming capabilities.

The Canadian Advantage

Canada's defence ecosystem is uniquely positioned for human-AI teaming:

DRDC research pedigree: Decades of human factors research through DRDC's Human Systems program
Academic strength: Leading programs at Carleton, uOttawa, and Toronto in human-autonomy interaction
Operational experience: CAF operators working with increasingly autonomous systems across domains
Allied alignment: Five Eyes intelligence sharing creates opportunities for allied human-AI teaming standards

IDEaS Fit and Timeline

The IDEaS challenge on human-autonomy teams remains one of the strongest fits for NovaFuse's capabilities. Our existing work in uncertainty quantification (Blog #12), explainable AI (Blog #6), and military simulation (Blog #24) provides the technical foundation, while our human factors approach addresses the evaluation criteria that IDEaS assessors prioritize.

Component 1a (for TRL advancement) would fund prototype development and evaluation with DND operational stakeholders. NovaFuse is well-positioned to be a prime applicant or subcontractor to a larger consortia led by a prime integrator.

Key Takeaways

Challenge	NovaFuse Approach
Automation complacency	Real-time uncertainty visualization + adaptive automation levels
Algorithmic aversion	Contextual explanations + decision traces
Mode confusion	Transparent AI state communication + failure mode disclosure
Operator overload	Multi-modal state estimation + dynamic task allocation
Trust calibration	Integrated human-AI uncertainty framework

Conclusion

The future of defence AI isn't autonomous systems replacing humans — it's humans and AI working together, each compensating for the other's limitations. The challenge isn't building smarter AI; it's building AI that makes humans smarter too.

NovaFuse's approach to explainable AI, uncertainty quantification, and adaptive human-machine interfaces addresses the core of the human-AI teaming problem. As IDEaS and DND increasingly recognize that AI effectiveness depends on human factors, this capability becomes a competitive advantage — and an operational necessity.