Pharma races ahead on AI, but governance struggles to keep pace

The use of AI in pharmaceutical development is becoming widespread, but the rulebook is still being written.

McKinsey reports that 71% of businesses across various industries now use generative AI, with life sciences sitting among the fastest adopters. However, only 53% actively mitigate AI risks. This gap now leads to unease, as pharmaceutical companies deploy autonomous systems across R&D and manufacturing while simultaneously building their governance frameworks.

The conversation has moved on from whether AI belongs in drug discovery or production lines. The big question is how those businesses secure innovation without restricting the impact that makes AI so valuable.

Operational vulnerabilities

AI systems introduce risks that legacy pharmaceutical quality frameworks were not designed to address. Model drift often occurs when AI projections become inaccurate over time, as training data deviates from the primary production conditions. For drug discovery, this can mean compounds flagged as promising based on outdated datasets. In manufacturing, this can result in quality control algorithms that fail to detect process abnormalities.

Data leakage between contract research companies and manufacturing partners creates another layer of exposure. Proprietary models trained on clinical trial data may inadvertently expose patient information or intellectual property when shared across structural boundaries. Without strict provenance controls, pharmaceutical companies lose visibility into how their data is used and whether models remain compliant with privacy regulations.

The added risk of bias continues to distort results. Regulatory reviews conducted by the FDA and EMA between 2024 and 2025 identified cases where AI-driven algorithms systematically excluded certain demographic groups from clinical trials, thereby compromising both regulatory compliance and therapeutic efficiency. FDA’s 2023 Cybersecurity in Medical Devices: Quality System Considerations and Content of Premarket Submissions guidance requires medical device manufacturers to demonstrate that their systems, including those using AI or machine learning, are designed with resilience, integrity, and recovery capabilities throughout the product lifecycle. These expectations directly influence clinical trial design, regulatory submissions, and most importantly, patient safety.

Compliance in action

Regulatory frameworks have a profound effect on how pharmaceutical companies approach AI governance. The ICH E6(R3) update, finalised in early 2025, strengthens requirements for all computerised systems used in clinical trials, including AI. Section 4.2.2 mandates that metadata relevant to the trial, including audit trails, shall be retained to allow for reconstructing the course of events. Section 3.16.1 requires that all alterations to data must be attributable, recorded and reconstructable. These are requirements that affect every AI-assisted trial submitted for regulatory approval.

The EMA’s Reflection Paper on the Use of Artificial Intelligence in the Medicinal Product Lifecycle again reinforces these expectations for machine learning systems. It emphasises that applicants and marketing authorisation holders must ensure that algorithms, models, datasets and data processing pipelines are transparently documented and auditable. The paper further states that, particularly in clinical trials, model architecture, logs, training data and processing pipelines should be available for regulatory review, supported by appropriate explainability and governance measures.

This stance change has accelerated structural change within organisations. A reported 13% of businesses overall have created dedicated AI governance or risk roles, recognising that regulatory expertise must sit alongside technical capabilities. AI centres of excellence are emerging as operational hubs where data scientists, regulatory affairs professionals, and quality assurance teams work together to validate models.

These centres improve accountability by establishing clear ownership. They improve reproducibility by standardising documentation and increase speed by preventing compliance failures that delay submissions. Our own AI Adoption Lab provides a secure environment for prototyping and validating models before deployment. The real advantage will go to companies that treat compliance as an operational discipline, rather than an administrative burden, gaining approval faster and scaling with confidence.

Future readiness

The next wave of operational risk is clear. This September 2025 analysis emphasises that as agentic AI becomes embedded in workflows, companies must establish strong governance frameworks with clear accountability, guardrails to prevent unintended consequences and regular audits to ensure compliance. Agentic AI systems can execute multi-step workflows across R&D and manufacturing without human oversight. This autonomy accelerates discovery and optimises production, but it also creates vulnerabilities.

Observable systems are therefore essential. Each agent decision must be logged, every data source validated and every action auditable. Continuous monitoring must become the operational standard, not an afterthought. Pharmaceutical companies that build these capabilities now will be prepared for the next decade. Those who defer will face costly remediation when regulatory scrutiny intensifies.

Leadership accountability remains the differentiator. AI security is not a problem that can be solved solely by engineering teams. It requires executives who understand that trust, validation and transparency are the foundations of operational speed.

Protecting patients, data and innovation are no longer separate objectives. They are interdependent priorities that define how pharmaceutical companies operate in the intelligence economy.