Compliance Architects

Consulting Technology Outsourcing

How Quality Records and Documents Create Your Biggest FDA Inspection Risk

Writing for Compliance

How Quality Records and Documents Create Your Biggest FDA Inspection Risk

Risk management is an essential activity for regulatory compliance within pharmaceutical and medical device companies.  While it is often challenging for medical product manufacturers to incorporate risk-based principles into their quality and compliance approaches, it is essential that compliance-implicated documents are drafted in a way that considers risk principles to facilitate FDA reviews during an inspection.  Risk based principles are important because poorly written documents covering critical compliance topics can create one of the biggest risks a company might face during an FDA inspection.

Every development, commercialization and manufacturing process has risks – which FDA expects companies to mitigate. While the reduction of all risk to zero is unrealistic, the FDA expects that manufacturers bring risks down to acceptable levels. Similarly, the development of quality records presents its own set of risks. Every compliance document presented to investigators must be created with risk mitigation principles in mind.

Quality Document Risk 

While most quality organizations and professionals understand product and process risk considerations, what are the risks associated with documents themselves?  Simply put, poorly written documents often fail to create an easy-to-understand story of how a company identifies, considers and manages risk. FDA investigators are not obligated to go out of their way to obtain additional information to clarify information or concepts they don’t understand in a document. Instead, they often will simply issue a 483 observation and be on their way.  Enough bad documents, and one observation can lead to multiple observations, which in time, could lead to a warning letter if initial concerns are not comprehensively responded to and addressed.

What are Good Documents? 

What constitutes a good document for the FDA’s inspection purposes? First and foremost, it must contain the information necessary to demonstrate compliance with regulatory requirements. Information conveyed must be based on sound technological and scientific principles. But it’s not enough for a company to believe — even to know — that its operations are scientifically sound and compliant with FDA requirements and expectations. If that information is not written clearly and in a way that is easy for investigators to understand, it may not withstand scrutiny during an inspection.

For FDA inspection success, it is critically important to understand that if a document does not clearly communicate compliant activities and adherence to SOPs, investigators are likely to have significant questions or concerns, and those issues will likely be reflected in 483 observations.

Obstacles to Developing Inspection-Ready Documents

There are many reasons why a company might find it difficult to ensure that its documents are well-written. Employees tasked with producing documents might lack skills and training to write well, or, may not have enough time to craft good documents. Training in writing principles and techniques are generally not included in training curricula for employees or may not be among the skills sought during the hiring process.

In many cases, companies may just not emphasize – or even understand – how critical good writing really is. There can be a tendency within the drug and device industries to consider scientific and technical competence  as the most important contributing factor for successful outcomes of quality operations. This can lead to a lack of focus on the importance of writing for clear communication. Similarly, companies may fail to understand and convey the regulatory basis for written documents, not considering their purpose(s), structure and usability for both internal and external readers.

Identifying Problem Documents and Problem Writing 

When determining which documents pose the greatest inspection risk, companies must first look at which documents are most likely to be reviewed by the FDA during an inspection. Focus documents include quality, technical and problem-solving records.  These are also, not coincidentally, the documents most likely to result in 483 observations. Further, technical records, such as validation protocols and reports, design records, and documents detailing complaint handling, deviations, non-conformances and CAPA, are among the highest-risk documents. The FDA may also ask to review SOPs, device history records, device master records and master batch records to check for consistency between these and the quality records memorializing the compliance activities.

The key to identifying — and then correcting — problem documents is to establish a system of ongoing document reviews where an internal person or consultant examines each compliance-implicated document in the same way the FDA would. If a given report does not meet expectations, the reviewer must identify exactly what is wrong — possibly listing questions that an investigator might have about the document — and make recommendations for improvement.

Document Organization and Structure — Essential to a Good Story

Document organization and structure — the third step in the Writing for Compliance® structured writing process (see sidebar above) — is critical to ensuring that investigators and others can easily follow the story being told in each document. An outline of required document sections, including one that lists names of all people involved in activities related to the record, is critical. The organization of content in the document should mirror the chronology of the activity, moving carefully from the beginning, through the middle, to the end of the story. An executive summary — written last — can give readers the gist of the document’s purpose and conclusions. A table of contents or content map can also be useful. Other key organizational points:

  • Naming conventions need to be established;
  • Acronyms and terms should be defined to ensure proper understanding;
  • Determine where in the document peer and/or technical reviews are needed;
  • Establish accurate, helpful and easy to follow references to all attachments, making sure attachments are easy to find and review; and
  • Where electronic systems are used, companies need to ensure that these can produce readable, user-friendly printed documents.

The way in which a company organizes content within the structure of a regulatory document can also play a critical role in how easy is to follow and understand the story being told. Bold paragraph headings and frequent paragraph breaks, along with use of bullets to isolate important information can support this goal. Writing should be clear as to the relationships between cause-and-effect and problems and solutions. Finally, readability is improved when authors write in the third person and use an active voice.

Context: Key to Understanding and Comprehension 

Underpinning easy to digest content within a regulatory document are the situational and technical contexts that establish foundational information for the entire document.  This step of the Writing for Compliance® structured writing process provides the background necessary to understand the compliance story being told by the report in question.

To establish situational context, the company needs to describe the genesis of the event, activity or problem and, explain its significance.  This is where the involved parties should be identified and the preliminary who, what, when, where and why are established.  The document must explain what requirements and expectations were applicable and state the desired outcome. Technical context will explain the underlying engineering and scientific concepts and lay the foundation for considerations such as risk to the product or patient. Technical context helps to define significance in a compliance document, while linking that information to performance requirements and clinical or functional considerations. By thoroughly documenting situational and technical context, companies can apply risk-based principles to their writing process.

Other common document problems to watch for include:

  • Failure to define the overall document purpose (discussed in Article 2 of this series);
  • Failure to support the document purpose with appropriate content and information;
  • Writing for science, for management, or for some reason other than compliance;
  • Failure to adequately support content and information with facts/data;
  • Too much or not enough detail;
  • Including irrelevant information;
  • Assuming too much audience knowledge;
  • Over-use of jargon and/or acronyms; and
  • Poorly structured sentences and paragraphs.

Well-structured documents incorporating risk-based principles help to reduce inspection risk by ensuring that a manufacturer’s compliance story is clearly and thoroughly told. This allows the company to control the dialogue with the FDA. When an investigator begins asking questions about operations due to deficient writing that doesn’t effectively relate a company’s story, the FDA controls and directs the dialogue.

To summarize; well-structured, well-written documents lead investigators to the conclusions your company desires and makes them difficult to challenge. Conversely, poorly written documents can create incorrect conclusions that can lead to 483 observations — or worse — that will often require time-consuming (and possibly unnecessary) remediation actions to address.

In our final article in the series on Writing for Compliance®, we will discuss the importance of investigation and analysis; what is different between the two; and how to ensure you are developing well-reasoned conclusions from these critical activities.  Stay tuned.

 

FDA “Live Review” Inspection of Company Documentation: Are You Ready?

The following was included in our July email newsletter.

There are many training sessions conducted each year by GMP Training Managers and company quality and compliance staff on “How to Manage an FDA Inspection”. Regularly, FDA inspection management is a “headliner” on the agendas of industry trade association meetings. And of course, there are numerous industry experts who consult and advise companies on how to manage FDA inspections as one of their key offerings.

Times are changing though. The FDA is adding a new, modern approach to its arsenal of approaches for conducting site inspections. This change is not a subtle change – it is a significant change.

Traditionally, FDA has conducted detailed, paper-based reviews. Requested documentation was presented to FDA by the company in hard-copy format. Timely retrieval and presentation of documentation and data by the company to the Investigator has been, and continues to be, a critical-to-quality attribute of a successful inspection.

To ensure timely and accurate presentation of documentation and data, many companies have developed an extensive “back room” process to retrieve and review documents requested by FDA prior to presentation in the front room. This process often includes a “pre-review” and “dress rehearsal” with the individuals presenting the documentation. A recent addition to back-room / front-room communications includes the use of computer-based tools (IM / chat / screen presentations) to enhance communications and document availability responsiveness.

The Live Review: New, Modern FDA Inspection Technique

Recently, I had the opportunity to participate first-hand in a different type of FDA inspection. Having directly managed (and saved) many, many critical FDA inspections, I know what to expect, and how to lead and manage the inspection process. However, when the detailed document reviews began, the traditional process of hard-copy record review was circumvented when the FDA Investigator requested a “live review” of the documents contained within the IT system deployed by the company to manage documentation for one of its core quality sub-systems. In this case, the Investigator was reviewing the complaint process. As part of the “Live Review”, the Investigator:

  • Requested an electronic copy (on a memory stick) of the summary tables for the past x years of product quality complaints for “y” products; and
  • Requested the ability to randomly select specific complaint documentation to review “live” on the company’s computer system.

What You Can Do

This new approach to FDA inspections is likely to occur with increasing frequency. Because of this, now more than ever, exceptional writing skills are essential for all levels of the Quality organization to ensure quality records and other compliance-implicated documents stand on their own – successfully – during a Live Review inspection. Because the Live Review inspection approach bypasses the traditional “managed approach” to FDA inspections, direct actions during the inspection to improve inspection outcomes are limited. There is little to no opportunity to “verbally augment” the written record during a Live Review inspection. Your compliance documents need to be written, reviewed and approved with this new, potential inspection approach top of mind.

During numerous client engagements, Compliance Architects® has identified that writing skills for compliance implicated documents are deficient. This lack of skills leads directly to poor inspection outcomes, since quality records are the source of FDA’s determination of a company’s compliance status. However, every company can take action to ensure your quality records meet FDA expectations, and provide a complete and thorough story of compliance for Investigators to review. To find out more about the problem of compliance writing, and how you can improve your FDA inspection outcomes, please visit our Writing for Compliance® website.

FDA’s new approach to inspecting quality system documents and quality records will, over time, substantially change the traditional company inspection management process, reducing the importance of the “back room” and creating more uncertainty during inspections. For additional insight into FDA’s Live Review inspection approach, download our white paper. To find out more about our experiences with this new approach, please contact me, Teresa Gorecki, at teresa.gorecki@compliancearchitects.com, or call me direct at 310-465-7285.

Teresa Gorecki
Practice Director
Compliance Architects LLC

FDA’s Upcoming Quality Metrics Submission Requirement: Data Driven Decision-Making or Letting the Students Issue Their Own Report Cards?

Click Here to See a Case Study on Sharepoint Metrics Capability for FDA Quality Metrics

Although Corporate America has embraced the concept of metrics and dashboards for close to 20 years now, the US Food and Drug Administration (FDA) has only recently come to the conclusion that incorporating a review of company-provided product quality metrics into its inspection and enforcement prioritization model will provide the Agency with an increased understanding of product quality performance at pharmaceutical manufacturers.   Significant activity has been occurring in this space of late, and given the heightened interest, on December 9-10, the Parenteral Drug Association (PDA) is hosting a conference dedicated to the topic of pharmaceutical quality metrics.    Notable, Dr. Janet Woodcock, Director of the Center for Drug Evaluation and Research (CDER) will be present at this conference.   The lineup of industry experts shows that this topic is ready-for-prime-time and is most certainly one that will get a high-level of attention from the quality and compliance community over the next couple of years.   Given all this interest, and the long-standing use of metrics by Corporate America, it would seem that it should be full-steam aheadSklamberg report for both manufacturers and the Agency in collecting, aggregating, reporting, and most-importantly relying on such data.    However, based on our experience from a recent, substantial consulting engagement with a leading multi-national pharmaceutical company, where we drove the development and implementation of a SharePoint-based metrics reporting portal, there a number of things to consider before “jumping into the pool” and faithfully relying on corporate quality metrics.

1.  Metrics are the Illegitimate Siblings of Statistics.

Most everyone has heard the saying “Lies, Damn Lies and Statistics!”    What is interesting about this phrase, and about the use of statistics generally is that, despite the completely scientific and mathematical basis for probability and statistics, they can be easily misused to not only obfuscate, but often to completely misrepresent reality.   In my course on Writing for Compliance™ I cover multiple ways to characterize information that can lead you from the legitimate “representation” of information or data, across the line into the murky realm of “misrepresentation.”   Crossing that line into misrepresentation is easy enough with statistics;  however, it’s even easier with metrics, as metrics, per se, have no scientific or mathematical basis.  By definition, they are in fact, made up—to meet the needs of their creator.   Metrics can be defined to represent information in whatever manner you want it to be considered.    If you want metrics to look good, it’s easy to do so.   Just measure the things that are consistently positive, and characterize them as more significant than they are.  And it has been my distinct experience that very few people have the insight, capability, and/or desire to challenge a corporate metric, once published.

2.  Metrics, and Specifically Quality Metrics, are Often Based on Inaccurate, Misleading or Poorly-Defined Source Data.

Even if an organization or company is well-intentioned (and as the saying goes, you know where that leads you), metrics are, by definition, measurements.   However, in the context of quality metrics, the items (or better said, information objectives) being measured are often not easy to obtain; and even if they are, they may not be based on consistent definitions.   Due in substantial part to industry’s fear of 21 CFR Part 11 and the cost and risk of implementing computer-based systems for quality management because of the requirements for Computer Systems Validation (CSV), far too many companies are still using either paper-based information systems, or computer-facilitated, paper-based systems.  This is in contrast to data-driven approaches to information management that have been embraced mostly by the very large FDA-regulated organizations, and even there, has resulted in expensive systems that companies are reticent to change or modify.

The downside of the foregoing is that it is a rare company that is lucky enough to have it’s operational and quality information readily-available in the right formats for easy analysis, aggregation, and/or reporting, since each of these steps creates an additional “functionality” over basic record-keeping (the 40-year outdated objective of most of FDA’s quality system regulations).    Securing the source information for the calculation of quality metrics is often an entirely manual exercise, done mostly with spreadsheets from proprietary system exports, if at all.   It is also often a laborious, paper-intensive exercise, relying on manual review of paper-based records to write down previously “recorded” information for input to spreadsheets to enable basic aggregation, analysis and reporting.

Finally, regardless of whether the information (data) is readily-available and easy to aggregate, there is a real danger in “definitional disconnect”.    In other words, unless a significant effort is made during the source data recording activity to ensure that people are recording information that is “apples to apples”, or, that within the data collection and aggregation phase, that the information is normalized for field names, content scope, date range, procedural differences, requirements differences (e.g., from division to division), etc., then the resulting metrics may very well mean different things, regardless of the intent to provide coherent, meaningful measures.    Thinking of how hard it is to be consistent with information within an organization, and thinking of how these concerns apply to the new FDA intended-approach to using metrics for quality assessment, and it’s challenging to think of how the FDA is going to be able to meaningfully rely on metrics to assess differences in quality performance from company to company.    It’s a great intention; but, remember what I said above — that good intentions often lead us into places we don’t really want to be.

 3.  Too Much Focus on Metrics Leads to “Metrics Mania”.

Having spent a large part of my professional career inside large corporations, and now often delivering consulting services to these same large companies, I have seen firsthand how the well-intended use of metrics for performance management, performance improvement, and legitimate measurement turns quickly into what I not-so-affectionately term “metrics mania.”    This affliction is based on the dangerous assumption that once you eliminate judgment and the corresponding subjectivity from decision-making, that the “pure” nature of mathematics — numbers — will lead you to the best outcomes.   Hence, incredible amounts of time are spent on capturing, analyzing and reporting metrics, which then turns into the inevitable need for “correction” when the metrics show poor performance.   Most often, though, in situations where the metrics are red, the objective is not to find the underlying structural contributors to poor performance — it’s not to find that the organization is improperly-resourced — that the physical plant is inadequate — that the process or product is poorly designed — that the specifications are not aligned to product performance — no, the corrective measures are usually focused on simply figuring out what to do to change the metric from red to green.   In this approach to characterizing and representing corporate achievement, once the metrics turn green, everyone is happy.

4.  Outcome Metrics are Often Not Linked to Operational “Levers”

The original intent of most companies in their zeal to embrace quality and operational metrics is to provide objective information that will lead to both increasing performance relative to outcomes that drive business success, and to identify activities and outcomes that require remediation.   As noted above though, the exercise often turns into a “do whatever it takes” effort to turn the metric from “red” to “green.”    The reason this occurs is that companies often don’t have the first idea as to what granular activities and efforts influence the outcome methods, nor how much they influence, or in what proportion to other related activities and efforts.

Good outcome metrics need to be well-characterized and tied to pre-defined
Light bulboperational “levers.”   To use an analogy, if we have a light that’s attached to a wall switch, and the light is off (outcome), there is in almost all cases, only one of two reasons (levers) for it:  1) the switch is off, or 2)  the bulb is not functioning.  Pull the right lever — turn the switch back on or fix the bulb, and the light goes back on.  Similarly, metrics (although they’re generally a bit more complicated than a light bulb) should be sufficiently characterized such that we understand what the pre-defined levers are.   You could also call these contributing factors or any one of a number of terms that tie effect and cause together.   Unfortunately though, whatever you call them, defining these levers well is rarely done in the metrics programs I’ve seen.

So, in light of the foregoing is the FDA’s plan to use submitted quality metrics in a corporation-to-corporation risk comparison something we should be concerned about?   And, give the challenges that individual companies have with the establishment of reliable data and metrics, should companies be comfortable “submitting” such information to the FDA “metrics reviewers” without a substantial focus on data integrity, reliability and validity?   If nothing else, the FDA’s plans should be highly-scrutinized by both FDA’s policy wonks, and by industry critics.

Of course, in spite all of the foregoing, the establishment of meaningful, well-characterized metrics aligned to an organization’s quality and operational objectives can be accomplished.    Such metrics can be extremely beneficial to consistently achieving high-quality operational outcomes.    Done well (and it can be done well) metrics serve as a (but not the only) important tool in effective quality and operational management.   Understanding where a metrics program can go wrong is a first step in ensuring the most value and in ensuring that you don’t rely on the wrong information in development of your metrics program.

For help in creating a quality metrics program or adapting an existing program to the upcoming submission requirements of the FDA; and / or to move towards a fully-data centric, web-based metrics solution, please contact Jack Garvey of Compliance Architects® at 888-734-9778 to review your organization’s needs and objectives.

Contact Us Today