Key findings
- Healthcare’s AI challenge is no longer adoption alone. It is visibility, control and accountability
- Because AI is increasingly embedded in SaaS applications, AI governance and SaaS governance must operate together
- Healthcare leaders need to connect application discovery, data access, utilization, cost and risk before they can govern AI or prove its value
As artificial intelligence becomes embedded across software as a service (SaaS) applications, healthcare organizations face a more immediate question than whether to adopt it: Do leaders know where AI is already operating, what data it can access, what it costs and whether the right governance is in place?
That governance gap is widening as clinical, operational and administrative teams gain access to AI through software they already use. A new capability can appear through a vendor update, a premium license or an employee-led purchase without entering the organization through a dedicated AI program. The result is a fragmented environment in which policy may exist on paper while actual AI use remains difficult to see.
For healthcare CIOs, CISOs, CFOs and digital leaders, this is both a technology and leadership problem. AI can support faster analysis and better-informed decisions, but organizations can’t manage risk, spending or value when adoption outpaces visibility.
Five questions turn AI policy into an operating model
Effective AI governance starts by answering five practical questions:
- What AI-enabled software do we have?
- Who is using it?
- What data can it access?
- What does it cost?
- Who owns the risk, governance and resulting value?
These questions translate an abstract policy discussion into an operational healthcare leadership agenda. They connect the people creating AI policy with the teams responsible for cybersecurity, privacy, procurement, software asset management, finance and clinical or business outcomes. They also reveal why AI governance can’t remain separate from SaaS governance.
Most healthcare organizations aren’t building every AI tool themselves. They’re consuming AI through SaaS applications, cloud platforms and vendor products that span patient engagement, research, workforce management, finance, security and other functions. Each application can introduce different data permissions, contractual terms, model behavior, security controls and costs. Without a common view of that ecosystem, leaders may apply policy inconsistently or discover risk only after an incident, audit or budget review.
Independent guidance reinforces the need for human accountability
The World Health Organization (WHO) discussion paper on artificial intelligence and evidence-informed policy describes how AI can support research synthesis, data integration, scenario modeling and monitoring across the health policy lifecycle. Used well, those capabilities can help policymakers identify emerging risks, compare interventions and incorporate new evidence more quickly.
WHO also emphasizes that AI should augment human expertise rather than replace it. Transparency, explainability, accountability and human oversight remain essential, particularly when AI affects sensitive information or high-impact decisions. Risks involving bias, privacy, cybersecurity, regulatory gaps and vendor concentration make governance a prerequisite for responsible use, not a brake on innovation.

That principle is visible in WHO’s 2026 program on responsible AI use for health systems. The program addressed practical use cases such as disease surveillance, clinical decision support, diagnostics and health information management while also emphasizing regulation, workforce readiness, ethical safeguards, reliable data, interoperability, validation and ongoing monitoring.
The implication for healthcare leaders is straightforward: Governance should make responsible adoption possible. It should define where AI may be used, which controls apply, who remains accountable and how the organization will monitor outcomes over time.
The visibility gap creates a healthcare governance problem
Industry research shows why policy alone is not enough. The Flexera 2026 AI Pulse Report found that 85% of organizations believe gaps in IT visibility pose a significant risk. Nearly half don’t always know how or when AI tools are being used, even as AI investment continues to grow.

The Flexera 2026 State of ITAM Report adds another layer: 84% of organizations identify tracking and managing AI applications as a top challenge, only 31% report visibility into AI software in use and 59% saw wasted AI spend increase year over year.
For healthcare leaders, these figures point to a fundamental governance problem: You can’t apply AI policy to technology you don’t know exists. You also can’t demonstrate the value of AI investment if you can’t connect adoption, utilization, cost and risk.
The consequences extend beyond the IT budget. An unknown AI feature may access protected or commercially sensitive information. Duplicate tools may spread across departments. Employees may rely on outputs without understanding limitations. Contract terms may change as vendors add AI capabilities. A promising pilot may continue to consume licenses after usage declines. Visibility gives leaders the evidence to address these scenarios before they become persistent risk or waste.
What healthcare leaders should do now
Drawing on these governance principles and Flexera’s research into AI visibility, adoption and spending, healthcare organizations can build a practical operating model across five areas. They don’t need to wait for a perfect enterprise-wide AI framework to begin.
1. Discover AI across the software estate
Create an inventory that includes sanctioned applications, embedded AI features, trials and employee-led tools. Update it continuously because vendor capabilities and licensing terms change quickly. Discovery should cover clinical, administrative and operational environments rather than stopping at a central AI project list.
2. Map identity, use and data access
Connect each application to users, departments, permissions and data categories. This allows security, privacy and compliance teams to focus reviews where exposure and potential impact are highest. It also helps leaders distinguish experimentation from tools influencing recurring workflows or decisions.
3. Assign accountable owners
Define who owns the business outcome, technical control, data decision, vendor relationship and financial result for each material use case. Human accountability should remain explicit even when an AI capability contributes analysis, recommendations or automation.
4. Connect governance to procurement and change management
Review AI capabilities before purchase and again when vendors add or materially change them. Contract, privacy, security and clinical or operational reviews should reflect the application’s intended use and risk. Governance should also include offboarding, license reclamation and data-handling requirements when a tool is retired.
5. Measure value alongside risk and spend
Track adoption, utilization, cost, control status and intended outcomes together. A low-risk tool with no sustained use may still represent waste. A widely used tool that supports an important outcome may warrant additional investment, stronger controls or both. This shared evidence helps finance, IT and business leaders make decisions from the same view.
Responsible AI adoption starts with visibility
You can’t govern what you can’t see. Healthcare leaders need a clear understanding of the SaaS applications and AI capabilities operating across their environment, who is using them, what they cost and what information they can access.
AI governance shouldn’t begin after adoption. Visibility, governance and financial accountability need to be built in from the start. When organizations connect these disciplines, they can move beyond broad principles and make informed decisions about where AI belongs, which risks require action and whether investment is delivering value.
Learn more about AI Cost Management
What is AI governance in healthcare?
AI governance in healthcare is the framework of policies, responsibilities, controls and oversight used to manage how AI is selected, adopted and used. It addresses transparency, accountability, privacy, cybersecurity, financial stewardship and human oversight so organizations can pursue value while managing risk.
Why should healthcare organizations connect AI governance with SaaS governance?
AI capabilities increasingly arrive through SaaS applications that healthcare organizations already use. Governing the AI therefore requires understanding the application, its users, data access, vendor terms, cost and ownership. Connecting the two disciplines closes gaps that appear when policy and software operations are managed separately.
Why is software visibility important for AI governance?
Visibility shows which AI-enabled applications are in use, who is using them, how they interact with data and what they cost. That evidence allows leaders to apply policy, prioritize reviews, monitor adoption and assess value where governance requirements actually need to operate.
What role does human oversight play in healthcare governance?
Human oversight keeps healthcare professionals, policymakers and organizational leaders accountable for decisions supported by AI. It includes reviewing outputs, understanding limitations, validating appropriate use and monitoring whether a tool continues to perform as intended.
How can healthcare leaders reduce wasted AI spend?
Leaders can connect application discovery with license, usage, contract and outcome data. That makes it easier to identify duplicate tools, unused licenses, overlapping capabilities and pilots that are not producing measurable value while protecting investments that are delivering useful results.