Skip to content
Expert Perspective

AI Governance ACROsS THE MEDICAL DEVICE LIFECYCLE

AI governance in medical devices must preserve qualified review, traceable evidence, and accountability as AI-generated work moves across regulated processes.

Shane Dultz
Medical device professional holding a tablet while reviewing an ultrasound system

The Real Takeaways

AI can generate the work, but accountability remains with the organization. That means governance in medical devices cannot stop at tool approval. It must ensure qualified review, traceable evidence, and ownership as AI-generated work moves across teams. This connected approach helps organizations scale AI while maintaining regulatory readiness.

Where AI Governance Breaks Down

Medical device teams are using AI to draft software requirements, generate test cases, analyze code, and support documentation. That work rarely stays within one team.

A generated requirement can influence architecture, testing, and risk analysis as it moves through the development lifecycle. Each handoff raises practical questions: What changed? What evidence is required? Who is qualified to approve it?

Many organizations have yet to answer those questions consistently. In a survey of 50 director-level and above leaders for Perficient’s State of the Healthcare & Life Sciences Industry Report, 78% say accountability for AI-influenced decisions is not clearly defined and documented, and 68% say governance and operational readiness are a barrier to scaling AI. The survey covered the full industry, but the report is direct about medical devices: balancing governance and AI performance is the difference between an AI implementation that survives an FDA audit and one that creates regulatory liability.

Approving an AI tool determines where it can be used. It does not determine the level of scrutiny its outputs require.

 

Oversight Should Reflect the Work & Its Risk

The appropriate level of oversight depends on what AI produces and the consequences of an undetected error. Under IEC 62304, software is classified as A, B, or C based on the harm a failure could cause, and that classification determines the rigor required across development, testing, verification, and documentation. AI can carry more of the load for Class A software, where a failure cannot injure anyone. On Class C software, where a failure could cause death or serious injury, the review bar must rise, not stay the same. Put simply, a meeting summary does not carry the same risk as a safety-related requirement, hazard analysis, postmarket report, or submission document.

Governance should be calibrated to the level of risk, considering:

  • The work AI generated or materially influenced
  • Its role in the quality system
  • The consequences of an undetected error
  • The expertise needed to evaluate it
  • What the reviewer must independently verify
  • How AI involvement, review, and approval are documented

A human signature alone does not constitute meaningful oversight. Without clear accountability and objective evidence of review, “human-in-the-loop” remains a label rather than a defensible control.

Higher-risk work demands deeper scrutiny. Reviewers need the expertise to challenge AI-generated output, not simply approve it. Because AI errors often appear polished, well-formed, and plausible, review processes designed to catch human mistakes may fail to detect them.

 

“As AI accuracy improves, reviewer vigilance can erode. In medical device development, a plausible wrong answer that gets approved creates more risk than an obvious error that gets caught, and that gap widens as AI gets more convincing.”

 

 

What Connected Governance Looks Like

Consider a software requirement drafted with an AI tool. A qualified reviewer evaluates it against the system architecture, risk analysis, and original source material. Its relationships to related test cases, hazard analyses, and submission documents remain traceable. That traceability preserves the evidence behind downstream decisions.

If the requirement, its source information, or the AI model changes, the organization can identify which related work requires reassessment and who is responsible for reviewing it. The control follows the requirement across the product lifecycle rather than ending when the AI tool is approved.

The right controls depend on whether AI supports product development, operates within the device, or does both. Relevant mechanisms include:

Connected governance brings these mechanisms together, keeping responsibility clear and decisions and evidence aligned across the product lifecycle.

 

When Governance Breaks Down, Accountability Stays With the Organization

An April 2026 FDA warning letter shows what can happen when AI-generated work moves forward without adequate review and accountability.

The action involved a Michigan manufacturer of homeopathic drug products that used AI to create product specifications, procedures, and production records. The FDA found, among other violations, that the company had not adequately reviewed those materials for accuracy and compliance. Company personnel also indicated that required process validation had not been performed because the AI system had not identified the requirement.

Although the case involved drug manufacturing rather than medical devices, it reinforces a principle that applies across regulated industries: organizations remain responsible for the AI-generated work they use.


AI can produce the work. It cannot accept accountability for it.

 

Connected Controls Support Responsible Scale

Preventing similar failures requires software, risk, quality, and regulatory controls to work as one decision-making system. When these controls remain aligned, medical device leaders can make more deliberate decisions about how AI is used.

They should be able to answer three questions:

  1. Where can AI carry more of the workload without creating unacceptable risk?
  2. Where must qualified human review increase?
  3. Can a change be traced across every affected process and work product?

Clear answers help organizations scale AI with greater confidence and control.

Learn how your peers across healthcare and life sciences are strengthening governance, clarifying accountability, preparing their workforce, and building the operating models needed to scale AI in Perficient’s State of the Healthcare & Life Sciences Industry Report.

 

Let's work together

Ready to put AI to work for your business?

Build solutions and optimize technologies so you can move faster, work smarter, get ahead – and stay there.