Practical considerations for using AI in analyzing clinical data and preparing FDA submissions

Begin the analysis with intended use

One of the clearest themes in FDA's draft AI guidance is that the intended use of an AI system is at the center of regulatory expectations. The agency is less concerned about whether a tool is labeled “AI” and more focused on what the tool is being asked to do, whether the tool is fit-for-purpose for that task, and how its outputs will be relied upon.

In practice, companies should avoid adopting broad rules about how to use AI, and instead define and evaluate specific narrow use cases. Several broad categories of use cases tend to emerge, including those discussed below.

Example of higher risk AI use: Interpreting clinical study data

If AI serves as a primary tool to evaluate patient-level study data and formulate conclusions about that data, FDA will likely expect a very high level of rigor.

Unlike traditional statistical tools (e.g., SAS or R), where key features are fully reviewable (e.g., assumptions built into the programming and statistical coding) and the outputs may be independently reproduced by the reviewing statistician for verification, many commercial or custom-built AI tools have challenges meeting FDA expectations, including:

• Limited transparency into underlying algorithms, training, tuning and testing data, and change controls;
• Difficulty locking models or establishing stable, auditable versions; and
• Gaps in audit trails and documentation sufficient to support regulatory decision-making.

Absent these controls, FDA may view AI generated analyses as less reliable than conventional statistical analyses, particularly where AI is proposed as a substitute for, rather than a supplement to, established analytical approaches. As a result, AI is generally more defensible when used as a drafting, exploratory, or hypothesis generating tool, rather than as a replacement for validated statistical analysis workflows accepted by FDA reviewers.

Where AI will be used for these complex analytical uses, such as patient stratification (e.g., risk factors for serious adverse events), endpoint assessment, or interpretation of clinical outcomes, meaningful human oversight is not optional. FDA is likely to expect that final analytical judgments remain grounded in validated statistical methods, with AI outputs serving a clearly defined and reviewable purpose within a broader evidentiary framework.

For these higher risk use cases, companies should expect FDA to scrutinize not only the model itself, but also how, by whom, and in what context AI outputs are generated, reviewed, and relied upon. FDA's focus is likely to extend well beyond model performance and governance to issues of transparency, validation, auditability, and meaningful human oversight, including an understanding of the training, tuning, and testing data used to develop the model, as well as how the model has been modified or updated over time.

Companies should expect to:

• Clearly define the context of use, including what regulatory decisions the AI output will ultimately support;
• Conduct and document a risk-based credibility assessment aligned with FDA's seven step framework, discussed below;
• Demonstrate that the model is fit-for-purpose, including transparency around data used to train and test the AI tool, performance limitations, and potential bias;
• Leverage robust human-in-the-loop approaches supported by adequate AI governance and training to ensure AI outputs are appropriately reviewed, contextualized, and validated by qualified subject matter experts prior to use in regulatory decision making; and
• Implement ongoing monitoring and lifecycle management.

In our experience, this is where companies most often underestimate the effort required. FDA is likely to closely scrutinize each specific use of an AI model that could directly influence regulatory decisions, even if the AI platform has broad acceptance for general use.

If sponsors intend to rely on AI for higher risk statistical analysis functions, early engagement with FDA is strongly advised. Early discussions can help clarify acceptable contexts of use, expectations around validation and transparency, and whether proposed AI applications meaningfully exceed what could be achieved through traditional statistical programming alone.

In addition, using AI tools to process and analyze clinical trial participant data raises a number of potential privacy and security issues, even if the data is coded or deidentified. Companies should make sure they have conducted sufficient diligence of the AI tool, including assessing the safeguards and protections needed to mitigate risk to data and to minimize risk of hallucinations and confabulations; determining how data will be used, stored, and processed by the tool; and evaluating whether data that will be used to retrain the AI model or algorithm may result in additional risks and obligations.

Example of lower risk use: AI as a drafting assistant

By contrast, AI tools are gaining broad adoption for initial drafting support, such as:

• Background sections of regulatory submissions;
• Executive summaries of clinical study reports that were prepared by traditional methods;
• Cover letters or other narrative, non analytical content; and
• Preliminary reviews of draft regulatory filings for completeness or consistency.

When AI is used solely for these purposes — and qualified humans critically review, edit, and take ownership of the final content — the risk profile is materially different.

In these scenarios, the AI tool functions more like an advanced word processing aid than a system interpreting clinical data. These uses often fall within the realm of operational efficiencies, which are excluded from the scope of FDA's draft guidance.

Sponsors should be aware that it can be difficult to draw the distinction between (1) AI used as a drafting assistant, and (2) AI that plays a central role in analyzing data to make substantive regulatory representations. This underscores the importance of clearly defining the AI tool's intended use. Where a human-in-the-loop approach is not rigorously applied, AI-generated drafts can potentially include biased results, unverified generated conclusions, or inaccurate representations.

Notably, FDA recently issued a CGMP warning letter where it emphasized that when AI is used as an aid in document creation, sponsors must carefully review AI outputs to “ensure they were accurate and actually compliant with” FDA regulations.

Part 11 compliance: An important and often overlooked question

A recurring question is whether commercial AI tools (such as general purpose large language models) must themselves be validated and compliant with 21 CFR Part 11 when used in regulatory workflows.

To briefly summarize, Part 11 requirements apply to any records in electronic form that are created, modified, maintained, archived, retrieved, or transmitted, under any records requirements set forth in agency regulations, such as under 21 CFR Part 314 (NDAs) or 601 (BLAs). FDA's own guidance on electronic systems used in clinical investigations, however, reinforces that Part 11 compliance is assessed through a justified, documented risk assessment, not a categorical rule applied to all software. The key considerations underpinning this type of risk assessment include the following:

• The importance of the records generated;
• The role those records play in regulatory decision making; and
• The system's potential impact on patient safety or trial reliability.

Where AI is used to generate electronic content that directly becomes part of a regulatory record, Part 11 considerations are likely to be front and center. This includes scenarios where AI is used in conjunction with conventional electronic systems validated under Part 11 (e.g., electronic clinical outcomes assessment tools).

Why traditional validation is challenging for commercial LLMs

In our experience, fully validating a general purpose commercial AI tool under traditional Part 11 paradigms is extraordinarily difficult if not impossible in practice.

Common challenges include:

• Lack of transparency into model logic and reasoning;
• Continuous vendor driven updates outside the company's control; and
• Non deterministic outputs that undermine reproducibility assumptions.

FDA's draft AI guidance implicitly acknowledges these realities by shifting the focus away from system level validation of AI models and toward use case definition, documentation, and governance.

A practical compliance framework for lower risk uses of AI

A defensible approach to using AI for lower risk drafting support could include:

• A documented risk assessment, tailored to the specific AI use case and aligned with FDA's credibility assessment framework;
• Clear internal policies defining what the AI tool may and may not do, including restrictions on inputs and required human review;
• Human-in-the-loop controls, with subject matter experts carefully reviewing, verifying, and approving any AI-generated content before it enters regulated systems; and
• Clear separation of systems, ensuring the AI tool is not treated as a GxP system of record, with validated document management systems remaining authoritative.

This approach allows companies to benefit from AI as a drafting tool while remaining aligned with FDA's expectations.

Vendor considerations for AI use

Importantly, although vendors may build, host, or maintain AI technology, FDA will continue to hold the sponsor accountable for how the tool is used and how its outputs are verified and relied upon. Meeting FDA regulatory expectations for AI use may be more challenging when AI tools used for clinical data analysis or FDA submissions are developed or operated by third‑party vendors. As a result, robust vendor contracting and active sponsor oversight are essential to ensuring that AI — particularly higher‑risk uses — can appropriately support regulatory decision‑making.

Within this context, sponsors should ensure that vendor agreements require their vendors to:

• Support FDA aligned use and compliance, including compliance with FDA Quality System Regulations, 21 CFR Part 11 (where applicable), and relevant state AI and data privacy laws, with clear obligations around data integrity, cybersecurity safeguards, and timely incident reporting.
• Enable transparency and sponsor understanding by providing sufficient access to relevant model documentation, audit trails, performance metrics, training data descriptions, and change history necessary to support regulatory inspections and submissions.
• Integrate into the sponsor's risk-based AI framework, including vendor participation in AI risk assessments, defined change control processes for model updates, and advance notice of changes that could affect performance or regulatory use.
• Maintain adequate AI governance through documented procedures and policies covering the AI lifecycle, including accountability, human oversight, transparency, traceability, continuous monitoring, maintenance, cybersecurity, audits, and escalation of issues affecting compliance or data integrity.

Embedding these requirements into vendor agreements allows sponsors to align vendor-provided AI tools with their intended use analysis and credibility assessments. When paired with internal policies and human-in-the-loop controls, thoughtful contracting helps ensure that AI — whether developed internally or provided by a third party — can be used appropriately to support clinical data analysis and FDA submissions.

Companies should also seek contractual assurances around configuration of the AI tool, privacy and security controls and safeguards, and data storage and use by the vendor (including ideally preventing the use of company data to retrain the AI model or algorithm). In addition, companies should limit use to enterprise versions of the AI tool where the tool may be used for sensitive, proprietary, or confidential participant information.

Key takeaways

• FDA's approach to AI is pragmatic but disciplined — the agency's expectations are calibrated to the regulatory risk of the use case.
• Companies should invest time upfront in defining intended use and conducting appropriate risk assessments.
• AI tools used to analyze clinical data could potentially attract significantly greater scrutiny than those used purely for drafting support.
• Thoughtful case-by-case governance — not blanket prohibition or unchecked adoption — is emerging as the most sustainable path forward.
• Where sponsors outsource the development or deployment of AI tools to vendors, robust contracting and active sponsor oversight are essential to fulfilling the sponsor's obligation to be ultimately responsible and to ensure that AI can be used to support regulatory decision-making.

As FDA continues to encourage sponsors to embrace AI through initiatives like the forthcoming real-time clinical trials (RTCT) pilot program, sponsors should ensure AI is deployed with the appropriate rigor, transparency, and human oversight that ensures compliance with the sponsor's traditional regulatory obligations. The agency is currently conducting a request for information (RFI) on the proposed RTCT pilot program to assess how AI-enabled technologies can improve efficiency, speed, and quality of decision-making in early phase clinical trials. This RFI presents a unique opportunity for sponsors to shape the agency's approach to use of AI in clinical trials and regulatory decision-making more broadly. The RFI solicits industry feedback on important topics like system performance, trustworthiness, comparative evaluation, decision quality, data integrity and patient safety.

If you would like help navigating FDA's draft guidance on the use of AI in regulatory decision-making, such as regulatory submissions, clinical trials, and manufacturing, or assistance with developing a comment for the RFI, please reach out to the authors of this alert or the Hogan Lovells attorneys with whom you usually work.

Authored by Robert Church, Blake Wilson, Melissa Levine, Elizabeth Jungman, Heidi Gertner, Kelliann Payne, Ashley Grey, Will Tenbarge.