What does fairness look like in a hospital roster? Before we piloted an AI self-rostering system in a New Zealand medical imaging department, the answer seemed to be equal distribution of night shifts, weekends, and holidays. But AI-powered self-rostering added another layer to the discussion of fairness, by incorporating staff preferences, raising autonomy and choice. So why did half the staff love it while the other half remained skeptical?
Over the last decade, regulatory approvals of AI health technologies have grown nearly exponentially, from image analysis tools, pharmaco alerts, transcription, summarising tools, and intelligent automated rostering systems. However, evidence suggests that AI rostering in healthcare is still failing to deliver on its promise. In healthcare settings, relational elements are often central to non-adoption, including a lack of training, context fit, usability, transparency, and accountability.
Given the immense investment in resources to achieve technical validation, it is no surprise that implementation is often an afterthought. As one commentator puts it, “we slap new technology onto completely broken processes and then act shocked when magic doesn’t happen” (Blackford Middleton, 2025). This is especially true in complex, dynamic health settings, where change is constant, and where expectations about how work should happen often differ wildly from what actually happens on the floor.
What we learned from our experience with AI self rostering
In 2023, our research team ran a simulated pilot of RosterLab’s AI self-rostering system in a medical imaging department in New Zealand. The aim was to explore whether the system could help manage complex shift rosters and support better work–life balance for staff. In other words, could AI scheduling in healthcare support genuinely fair staff rostering, rather than simply redistributing unpopular shifts more visibly?
As practising radiographers, my research partner and I assumed we had a good grasp of how radiology rosters worked. However, we quickly realised our assumptions of rostering practices were rose-tinted.
When looking at the existing rostering system, we could see that staff shift-swapped their way to better rosters. With the introduction of AI, we saw something more subtle: staff who officially worked the same shifts often ended up with quite different roster patterns. This, we learned, highlighted shift patterning that sat over and above the baseline expectations that the system operated under. This common example among others, showed us how nuanced aspects of work culture influenced how the AI was received.
Why fairness didn’t feel fair
Notably, although the AI aimed to balance shift distribution, staff did not consistently perceive the outcome as “fairer.” The group was split - not because of the outputs alone, but because of how those outputs were interpreted. Some valued consistency and transparency, while others placed greater trust in human judgment and predictability. Although perceived to be odds, these elements were in fact complementary within the AI system and much time was spent talking about the human processes that sat behind the AI roster; the one-to-one ‘check-ins’, queries and adjustments that would still occur before roster publication.
As we spoke with participants, it became clear that these views were shaped in-part by the transition itself and how people understood the technology rather than by the presented rosters. Misconceptions about what the system was doing – how it balanced rules, preferences, and fairness - also influenced trust This experience highlighted that when AI is introduced into everyday clinical work, staff perceptions and work culture are central to adoption.
Why staff don't trust AI, even when it works
Looking at the bigger picture of trust in healthcare, it is essential to consider how high-stakes decision making plays into trust. Every health professional using AI technology is making micro-decisions about when and how to use the tool - often deciding on the spot whether to rely on the AI or their usual practice. Health Professionals know they are ultimately responsible for the outcomes for patients or staff. So, when users aren’t confident that the technology makes the best decisions, or when it appears unreliable, inaccurate, opaque, or poorly suited to their context, they are unlikely to use it fully. This leaves the technology vulnerable to low adoption, unintended use, and underperformance.
In the mix of misconceptions, responsibilities, and expectations, one thing is clear: successful implementation depends on developing accurate, balanced trust. Trust must align with both the abilities and limits of the technology and the professional judgment of the user, so health professionals know when to follow AI’s recommendation, and when to override it, and how to integrate it into safe sustainable workflows.
So, how do we foster a culture of trust?
Central to this is the promotion of Human-AI collaboration.
Educate, re-educate, and re-educate again
Healthcare staff are busy and tired. When users don’t understand how a technology operates - what data it uses, what constraints it optimises for, and where it may fail - it limits their ability to integrate it into their professional toolkit. This slows learning and, under time pressure, can lead some to abandon the technology or others to trust it blindly. Effective education is reciprocal: it should create efficient pathways for feedback to move between user to developer, building confidence over time, and demonstrating that the system can adapt to real-world needs. As we progressed to the piloting stage, increasing the amount of education and feedback sessions helped us to address misunderstandings, knowledge gaps and concerns earlier in implementation, and enabled us to promote use through individualised support.
Find champions within your context
Evidence shows that relationships with staff who are engaged with the technology are key to adoption. Champions should understand the ‘why’—the core problem the AI is solving. They are also able to support the technology, testing its boundaries and identifying gaps in their practice, helping ensure it remains responsive and flexible. In our self‑rostering pilot, trusted champions acted as intermediaries, helping colleagues interpret unfamiliar AI-generated rosters and helping us better understand the nuances of the contexts that we were working in This collaboration was key to adaptive integration and ultimately, to improving technological fit.
Make the reasoning visible
Explainable AI (XAI) is essential for accountability and transparency.AI tools used in healthcare workflows should guide users, helping them understand outputs as they use them, rather than functioning as black boxes. This does not necessarily mean a chatbot. It can be a toolkit that explains why a particular roster, warning, or recommendation was produced, provides confidence indicators, visualises trade-offs, and allows users to ask questions and challenge results in a structured way.
By making the decision logic more visible, XAI helps clinicians and managers calibrate their trust appropriately.Based on our research, XAI could be used to support staff to understand reasoning behind the allocation of requested shifts, why the received or missed a requested shift. For example, highlighting when requests clashed with union rules.
So, where did we land?
Ultimately, without engaging users meaningfully, the “magic” will not happen. The lesson is clear: successful AI implementation in healthcare depends not just on technical validation, but on fostering trust through transparency, education, and human-centred design, and real collaboration between staff and technology.
When AI systems, whether for diagnosis, documentation, or self-rostering, are implemented with these principles in mind, they are far more likely to be adopted, used appropriately, and deliver genuine improvements for both patients and healthcare workers.
And as AI implementation practices continue to evolve, a deeper question arises: can explainable AI do more than educate and inform users? Can it surface the contextual nuances that currently only emerge in real-world practice – providing the next level of human-centered integration through increased responsivity.
