Compliance Architects

Consulting Technology Outsourcing

John (Jack) Garvey Has Been Selected by Our Partners QbDVision to Join Their Newly Formed Board of Advisors

FDA Data Integrity

John (Jack) Garvey Has Been Selected by Our Partners QbDVision to Join Their Newly Formed Board of Advisors

Taken from LinkedIn…

“At QbDVision, we firmly believe that finding the right guides is just as important as discovering the right path. We’re incredibly fortunate to have found both!

As we map the path to #DigitalCMC, we’re excited to announce that we’ll be guided by an exceptional new Board of Advisors: Joschka Buyel, Gloria Gadea-Lopez, John (Jack) Garvey and Rebecca Waterbury. They bring our team an incredible depth of insight on a wide range of critical aspects of the CMC process and have already made an invaluable impact on multiple areas of our product and business.

Big thanks to these remarkable leaders for the guidance and expertise they’ve already shared with our team. We couldn’t ask for a more exceptional group of advisors!”

Learn more about this visionary board and how they round out our roster

A bit more from Managing Partner Jack Garvey –

“I’m honored to have been asked to be an Advisory Board member at QbDVision! I’ve always been ‘software-neutral,’ but QbDVision has changed the game with a process-driven, data-centric approach to regulated product commercialization that is currently unmatched. What excites me though — a quality and compliance guy — #quality #compliance #fda #software #data is how QbDVision creates a knowledge map that will greatly enhance the ability to succeed in FDA and other regulatory inspections.”
|||

Data Integrity: The Foundation of Medical Product Manufacturing Compliance

The FDA and other global regulators rely on reviews of data during inspections to determine whether a pharmaceutical or medical device manufacturer is operating within regulatory requirements and expectations. The credibility of that data is a crucial parameter in the regulator’s assessment of whether operations are compliant—or not. Data integrity refers to the credibility—meaning reliability, validity, authenticity and trustworthiness—of various data presented for regulator review. Questions about the integrity of data presented in regulator-reviewed documentation always raise red flags about the compliance state of your operation and the levels of control within the medical product manufacturing system.

Since data collection and reporting are foundational for virtually all cGMP and QSR requirements, the notion of data integrity is foundational to all aspects of regulatory compliance. And today, all regulators, FDA and other international bodies, take the concept very seriously. Questions about data integrity are among those most frequently appearing as FDA Form 483 observations and warning letter citations. Out of 15 cGMP-related warning letters issued to drug firms and four QSR-related letters to device companies in 2019 and posted to the FDA website, approximately nine included at least one citation related to data integrity, according to a review by Compliance Architects®.

The issue was highlighted by a Bloomberg article last year, which examined ongoing issues at Mylan manufacturing sites, many involving data integrity.

And this is most definitely not a new trend. In a January 2017 guest column for Pharmaceutical Online, Barbara Unger of Unger Consulting highlighted the emphasis on data integrity in FDA enforcement activity during the 2013-2016 period. Figure 1, below, illustrates this growing emphasis going back years now.

Figure 1: FDA Data Integrity Enforcement Trends

It’s important to note that the words “data integrity” may not appear in such citations. Rather, companies might see such statements as the following:

  • Raw materials, intermediates, and finished API analytical results found to be failing specifications or otherwise suspect (e.g. OOT) are retested until acceptable results are obtained. These failing or otherwise suspect results are not reported.
  • Failure to document production and analytical testing activities at the time they are performed.
  • Failure to prevent unauthorized access or changes to data and to provide adequate controls to prevent omission of data.
  • Appropriate controls are not exercised over computers or related systems to assure that changes in master production and control records or other records are instituted only by authorized personnel.
  • Failure to maintain complete data derived from all testing, and to ensure compliance with established specifications and standards.
  • During our inspection, we observed multiple examples of incomplete, inaccurate, or falsified laboratory records.

And although the wording may vary, all such citations relate to the faith—or lack thereof—that agency investigators have in the information provided by drug and device companies about their manufacturing operations. Characteristics that the FDA considers when determining the credibility of data include attributability, legibility, contemporaneous recording, originality and accuracy, often abbreviated as ALCOA.

Companies can avoid regulatory and other fallout due to poor data integrity practices by comprehensively implementing a seven-program-element approach within their overall quality systems (see box above).

Filling the data integrity gaps

To fully understand the concept of data integrity, medical product manufacturers need a clear picture of what comprises data from the FDA and other regulators perspective. Examples of data include charts that record information, paper and electronic lab notebooks, product release and approval information, batch release documentation, certificates of analysis, raw data, instrument printouts and computer-based data, among others.

Companies can face numerous obstacles to having reliable data and information to present for inspections. Many challenges come down to unintentional errors or negligent conduct due to:

  • Lack of awareness;
  • Lack of defined expectations from management;
  • Lack of procedural or positive controls;
  • Lack of adequate supervision and oversight;
  • Failure to prioritize importance of data integrity;
  • Tolerance for sloppy or unprofessional work;
  • Lack of periodic “checks” on performance;
  • Lack of technology controls;
  • Pressure on personnel to achieve outcomes; and
  • A “whatever it takes” culture.

And in some instances, data integrity gaps may be due to purposeful, deliberate conduct aimed at gaining monetary, professional or personal gain. This can occur when the risk from non-compliance is greater than that from wrongful conduct. If management creates a culture that presents data manipulation as a victimless crime, or directs staff to fudge numbers to meet customer service requirements or for financial gain, employees may fail to ensure the integrity of data presented in FDA-facing records that are reviewed during inspections.

But the FDA and other regulators don’t care whether data integrity gaps are due to deliberate or unintentional acts. As far as FDA is concerned, if it cannot rely on the data, then it cannot be assured that manufacturing operations are compliant. And the ramifications for noncompliance can be serious—both in enforcement and customer perception terms—for pharma and medical device manufacturers.

As with other cGMP and QSR requirements, the FDA expects medical product manufacturers to apply a risk-based approach to data integrity. The overall data integrity compliance risk profile is the sum of all data integrity gaps, which can be grouped into three categories: known and documented gaps, known and undocumented gaps, and unknown, undocumented gaps.

Pharma and medical device manufacturers are responsible for identifying, documenting and addressing all gaps. Thus, the first step toward ensuring data integrity across all quality-related systems and activities is a comprehensive assessment of all potential gaps, so that all areas of concern are identified (known) and documented.

Using the comprehensive seven-program-element approach (listed in the box above), companies can be confident that this structural approach will maximize the prospect of ensuring data integrity throughout all operations. Figure 2, below, illustrates how several of the seven program elements, which Compliance Architects® will explore in greater depth in future articles, can work together to create a comprehensive data integrity program that produces data in which the FDA and other regulators will have faith.

Figure 2: How It Fits Together – A Structural Approach

The Secrets We Keep — FDA Data Integrity, Elizabeth Holmes, Theranos, Bad Blood and HBO’s “The Inventor?”

By now, most everyone in FDA regulated industry is aware of Theranos, the Wall Street Journal’s John Carryrou, and his award-winning book Bad Blood: Secrets and Lies Inside a Silicon Valley Startup. But have you seen HBO’s new documentary, “The Inventor: Out for Blood in Silicon Valley?” I was lucky enough to come across the documentary while doing my usual evening channel surfing, and I decided to tune in. While most of the information covered in the documentary has already been told, being able to observe the tone, mannerisms, and delivery of everyone’s story was truly fascinating. To see the people involved discuss a story with such scope, scale and unrealized promise was both sad and riveting.

Some still find it baffling that Theranos was able to raise upwards of $900 million without any tangible evidence that their marquee product, the Edison, worked. Even more puzzling, at least to me, was how many high-profile investors were eager to support the company. These were well-respected, high-net-worth individuals and venture capital (VC) funds with hundreds of years of collective investing expertise. How could they be SO FAR OFF?

The Motive? Ego and Money

Take a look at the Theranos pitch, it’s easy to see the allure:

“The Pitch: It was a marvelous pitch. A black box, capable of running upward of 240 blood tests — diabetes, HIV, cholesterol, cocaine — a la carte, from just a single drop of blood. Think about the level of industry disruption. Labs, doctors, Big Pharma, all brought to their knees and consolidated into a Macintosh-like box in the back of your Walgreens. Patient-consumer data at the prick of a fingertip, with far less blood and funding needed to make things happen. We’d beat a gatekeeping, profiteering healthcare system overnight. This could save untold millions of lives with early detection. And who wouldn’t want to see that in their lifetime? Developments in nanotechnology, modern medical innovation, and Silicon Valley gumption seem like perfect excuses for an idea like this to be within reach.”[1]

Who wouldn’t want to ride this wave? “Disruption”, “Macintosh-like”, “save untold millions of lives”, “nanotechnology,” the Theranos pitch had it all. If there’s one thing that makes investors drool its buzzwords. Especially when delivered with a slick presentation backed up by an all-star board of directors. Despite some of the brightest minds in science stating the idea was technologically unfeasible, investors continued to throw money at the company. In due time, everyone learned exactly what the problem was…the box (and the company’s culture) didn’t work.

Why VCs Should Care About FDA Data Integrity

Data integrity continues to be a hot topic in our industry. Despite all that’s been written and spoken about it, bad data (or downright fraudulent data) is still being produced daily. For life sciences companies to accurately represent their products, procedures and processes, data needs to be accuracy, reliable and reproducible. For the investors, the quality of such data is foundational to the value placed on a funded company. When journalists, industry experts and, most recently documentarians, dug into the Theranos story — fabricated data and data manipulation was a central tenet. Incredibly, the high-profile VCs who continued to pour money into the company didn’t even want to do a casual assessment regarding the integrity of the data.

One scene in the documentary describes a particularly egregious example of data manipulation. Theranos would bring investors into a conference room and ask someone to volunteer a blood sample. Theranos executives would insert the sample into the Edison machine, and while they were “waiting for the results,” take the VCs out of the conference room for a tour of the facility. While investors were away, other Theranos employees took the blood sample elsewhere for diagnostics and then manually input data into the Edison machine.

While this sounds crazy, it was relatively common practice according to Business Insider:

“The first incident…involved a blood test ordered in 2013 for a patient using the company’s proprietary Edison lab machine. The lab worker assigned to the test reportedly found problems that indicted accuracy problems with the device. When the lab worker told superiors, the report said, an employee in research and development came to the lab and deleted the data. In the second incident…an employee sent an email to CEO Elizabeth Holmes in 2014 suggesting the company ‘cherry-picked’ data when comparing the Edison machines to traditional blood-testing machines to make the machines look more accurate.”

The article goes on,

“…for one test, the device’s accuracy rate increased sharply after some information was deleted and manipulated, the employee wrote. Edison machines also allegedly failed daily quality-control checks often.”[2]

A Fundamental Cause of Data Integrity Issues? Poor Quality (and Compliance) Culture

FDA Quality Culture starts at the top. As we learned from the above examples, regardless of how great the potential of a patent, technology, or scientific innovation, it is impossible to assign value to a company’s potential when Senior Management willingly participates in generation of fraudulent data – or – turns a blind eye to making sure the data “works” regardless of its validity. This failure is fundamentally the result of a company’s poor Quality Culture.

When Theranos was questioned about the above occurrences, “Theranos representative Brooke Buchanan told the Wall Street Journal she didn’t believe the first incident ever happened. In the second incident, Buchanan put the blame on the employee, who she said was too inexperienced to ‘make these types of comments.’” [3]

Compliance Architects®’ CEO, Jack Garvey, has been on the cutting edge of FDA Data Integrity program development and FDA Quality Culture assessment. At a recent conference, Jack was quoted:

“Understanding the drivers of behavior behind quality and compliance outcomes can be frustrating and challenging. Stated corporate intent and policy can be disconnected from actual employee conduct. Pockets of information can lead to lack of transparency, and to personnel not working together as a team towards common goals, and with common understandings.”

Jack often states that “quality culture trumps quality systems when you need to improve quality and compliance performance and outcomes.”

A Comprehensive Approach to Solving FDA Data Integrity (and FDA Quality Culture) Problems

To address problems with FDA Quality Culture leading to non-compliant and possibly fraudulent Data Integrity outcomes, Compliance Architects®  has developed a 7-element program that considers culture and data integrity together to ensure an effective, lasting outcome for a data integrity improvement or remediation program.

Although comprehensive in nature, the program can be summarized as follows:

  1. Program Element 1- Regulations, Enforcement, Guidances, Industry Standards: Review and develop a consolidated requirements document from the various guidances and model approaches that reflect external stakeholder expectations. Consolidated requirements must reflect local market expectations and should be consistent across the enterprise.
  2. Program Element 2 – Policy, Expectation, Culture, Incentives & Punishment: Ensure all personnel understand the Corporate intent, requirements and expectations; to create positive culture and incent proper conduct; and to establish the significant ramifications from failure to adhere to Corporate principles.
  3. Program Element 3 – Quality System / Positive Compliance Controls: Establish a rigorous quality system and related control framework that incorporates data integrity considerations throughout all operational processes. Positive controls are an expansive term encompassing the quality system control framework and discrete, individual controls over focus activities and functions.
  4. Program Element 4 – Data Integrity Focus Areas: Identify those operational, quality and support activities that have particular risk for data integrity concerns and ensure both activity execution controls and proactive challenges are implemented rigorously.
  5. Program Element 5 – Data Lifecycle Management Controls: Establish comprehensive data lifecycle management controls that create clear, unequivocal requirements for how to record, migrate, report, repurpose and forward-migrate data to ensure integrity of data and information.
  6. Program Element 6 – Proactive Challenges: Establish a program that regularly challenges, probes and identifies occurrences of data mishandling and that will seek out and identify, to the extent possible, possibly fraudulent activity.
  7. Program Element 7 – Governance: Develop and implement an appropriate governance approach that ensures regular reporting of data integrity assurance practices performance, and that serves as an escalation point for deviations from directed practices and is responsible for maintaining the contemporary value of the overall program.

If Theranos had embraced such a comprehensive program, and if the Theranos technology actually worked, the company might have achieved transformational medical innovation. We can only hope the next Theranos considers this cautionary tale and embraces these principles before patients are put at risk, and investors lose their shirts on “snake oil” investments.

More information on Compliance Architects®’ 7-element data integrity program can be found here.

Bad data and quality culture are like the plague — they can spread and quickly overwhelm your organization. If you are concerned about your data integrity practices, call us before the FDA shows up at your door and identifies the negative outcomes that can occur.

For further information or for help on how to improve your organization’s quality and compliance operations, please contact Jeff Grizzel at jeff.grizzel@compliancearchitects.com or at 703-587-8990.

 

[1] https://consequenceofsound.net/2019/03/film-review-the-inventor-hbo/

[2] https://www.businessinsider.com/bombshell-report-alleges-theranos-deleted-data-to-make-technology-look-more-accurate-2015-12

[3] https://www.wsj.com/video/elizabeth-holmes-discusses-theranos-at-wsjdlive-2015/20CE68A0-CAEE-48E0-BAB4-FD6C47D283BE.html?mod=e2tw

Contact Us Today