Statistical Approach to Aseptic Process Simulation
Introduction
Aseptic Process Simulation (APS) is a critical component in sterile pharmaceutical manufacturing. For some, it’s a challenge to ensure that the conditions simulated in APS are truly representative of the actual routine aseptic manufacturing process. Regulatory and industry guidelines1-4 provide a framework for designing APS, but they don’t always offer sufficient guidance on how to demonstrate that specific conditions - like the duration or number of corrective Aseptic Interventions (AI) - are representative of routine operations.
Moreover, periodic APS is sometimes mistakenly viewed as a retrospective analysis designed to cover past manufacturing conditions. Trend analysis used to evaluate the number and duration of AIs simulated during APS should not only assess past performance but also set alert limits for future aseptic manufacturing. This approach helps ensure that the APS condition covers each manufactured batch; otherwise, sound justification should be required before batch release.
Regulatory and industry guidelines require that manufacturers design their APS to mimic routine aseptic conditions as closely as possible1-4. This includes simulating both inherent (also called routine) and corrective (also called non-routine) aseptic interventions. Inherent interventions (e.g., set-up, change of settle plates) occur regularly, while corrective interventions (e.g., needle replacements, opening doors in Restricted Access Barrier Systems) happen when there is a need to fix or adjust something during aseptic production. In some instances, unplanned interventions that are not classified as qualified aseptic interventions and that may occur during routine batch manufacturing should typically be simulated unless a sound justification is provided4 . The assessment of unplanned intervention would define the risk level (risk of contamination) of the intervention taking into consideration several elements such as AI (a) duration, (b) complexity, (c) proximity to product or break of first air using a sterile or non-sterile item, (d) Human exposure (e.g. open or closed door)5 .
EU GMP Annex 1 (9.34) emphasizes that the APS should account for a variety of aseptic manipulations and interventions observed during routine manufacturing, including worst-case scenarios. It specifically requires:
- Both inherent and corrective interventions are to be performed in a manner and frequency similar to routine aseptic operations.
- The inclusion and frequency of interventions should be based on assessed risks to product sterility.
The US FDA Aseptic Processing Guideline2 emphasizes that media fill studies should closely simulate aseptic manufacturing operations, incorporating worst-case scenarios and challenging conditions to test aseptic conditions. The FDA recommends that media fill programs address key factors, including:
- A representative number, type, and complexity of routine interventions performed in each run, as well as non-routine interventions and events (e.g., maintenance activities, equipment stoppages, or adjustments).
- A representative number of aseptic interventions, such as charging containers, closures, and sterile ingredients, or transfers.
One question often arises: How can we confirm that the number and duration of AIs simulated in the next APS are truly representative of routine conditions over a recent manufacturing period, say the last six months? The answer lies in statistics6-8, rather than relying solely on empirical or extreme approaches.
An empirical approach would average (also called mean) the number of AIs performed over the last six months, while an extreme approach would consider the maximum number observed during that period. Both methods have limitations, as they fail to fully capture the variability or trend in the data. This is where statistical analysis becomes essential.
Recommendation
Wiesbaden, Germany8/9 April 2025
Single-Use Systems in Sterile & Biomanufacturing
To maintain this article's length, we will focus solely on the number of corrective interventions. However, the same approach can be applied to the duration of aseptic interventions. The AI duration is an important factor in media fill design4, ensuring that operator variability is accounted for and that the time taken for each intervention is neither shorter than in the smoke study nor longer than necessary. However, this duration is often overlooked and not consistently tracked or monitored during batch reviews, even when the type of corrective AI as well as the start and end times are documented in the batch record.
Discussion
The mean of a population itself represents the central tendency of the data, but it doesn't directly specify how much of the population it covers. However, if you're referring to the coverage of the population in a normal distribution, the mean alone divides the population into two equal halves, with 50% of the data points below the mean and 50% above it.
If you want to measure how much of the population is covered within a range around the mean, you need to include standard deviations (sigma or SD):
- Within 1 standard deviation (±1σ): About 68.26% of the population is covered (34.13% on each side of the mean).
- Within 2 standard deviations (±2σ): About 95.44% of the population is covered.
- Within 3 standard deviations (±3σ): About 99.72% of the population is covered.
Instead of relying on averages or extremes, we recommend a simple statistical approach. Consider that the mean (average) number of corrective AIs plus one standard deviation (SD) represents roughly 68.26% of the population. For those wanting a more conservative approach, using the mean plus two or three standard deviations will cover more of the population. By using this method, sites can justify that the number and duration of corrective AIs simulated in APS are representative of routine operations.
Let’s walk through an example of an aseptic facility manufacturing between 20-40 batches every six months. The first objective is to set the number and duration of corrective AIs to simulate in the next APS, ensuring they represent routine aseptic conditions. The number and duration of AI should be determined as Mean + x.SD, where "x" can range from 1 to 3, depending on the level of precaution and justification used. The second objective is to treat the Mean + x.SD as an alert limit for the number or duration of corrective AI during routine aseptic manufacturing. If any batch exceeds the alert limit, an investigation or justification is required before batch release9. The third goal is to use statistical analysis to justify atypical results, such as unusually long aseptic AI durations or high AI numbers, by demonstrating that they still fall within a representative range (Mean + 3 SD). If a batch falls outside this range and no valid justification is provided, an extreme (worstcase) approach should be applied for that specific AI type in the next APS.
Figure 1: Levey Jennings Chart of AI-C2 over 2 periods of batch manufacturing. The dotted red line is the mean+SD (Alert limit as per the APS performed after Period 1), the red line is Mean+3SD and the green line is the mean.
Table 1 (at the and of the article) provides information on the type of corrective aseptic interventions (AI) and the number performed in each batch. Each period in the table represents a six-month timeframe (e.g., Jan. to Jun. or Jul. to Dec.). The first period (Period 1) establishes the baseline for the number of interventions to be simulated in the next periodic APS. This baseline, based on Period 1, is defined in our example as the mean + standard deviation (Mean + SD).
For those who prefer a more conservative approach, the following rule can be applied: if the calculated Mean + SD is less than 3, the intervention should be simulated 3 times during the APS. The number simulated during the APS would be considered the alert limit (represented by a dotted red line in Figure 1), while the Mean + 3 SD is the maximum number of interventions (e.g., AI C-2) that can still be considered typical for a specific batch and representative of the aseptic conditions covered in the previous APS.
The next step is to answer the question, How do we detect if there’s an upward trend in the number or duration of aseptic interventions? A simple answer would be, by performing a trend analysis that compares data across periods (e.g., January to June vs. July to December). Any rightward shift in the mean on the x-axis signals an increase in AIs, while a leftward shift indicates a reduction.
However, an upward trend is more than just a small change in the mean; it reflects a significant increase in the number of AIs that requires investigation. Our proposed approach involves comparing the percentage change in mean and standard deviation (SD) between periods, referred to as "%PoP" (percentage period over period), to detect statistically significant upward trends.
Formula 1: %PoP Mean = ((New Mean / Previous Mean) 1) x 100
Recommendation
Barcelona, Spain23-25 September 2025
Modern Microbiology Laboratory
Focusing on the mean alone might not be sufficient, as a shift to the right could occur without being statistically significant. Therefore, it's crucial to also consider the variation in standard deviation (SD) between periods. The SD provides insight into the spread of data within and between batches. A statistically significant change in SD should prompt an investigation to determine whether an upward trend is emerging.
Formula 2: %PoP SD = ((New SD / Previous SD) 1) x 100
To ensure the observed differences are not due to chance, the optimal %PoP limit should be set. A p-value below a significance threshold (e.g., 0.05) indicates a statistically significant difference. For aseptic interventions, a p-value of 0.1 or lower is typically considered sufficient (based on the author’s experience), providing 90% confidence that any observed difference in the mean is meaningful for identifying upward trends.
Based on experience and statistical analysis from multiple sites manufacturing 20 to 40 batches, a 50% increase in %PoP mean is usually enough to trigger an early alert of an upward trend, while an increase greater than 50% ensures a statistically significant difference between two or more periods (as shown in Figure 3). The threshold limit for increase in %PoP can be adjusted and justified using statistical tests like a t-test or ANOVA, depending on the sample size (N) and the number of periods, with a p-value of less than 0.1. This ensures a 90% chance of identifying an outlier or upward trend.
Figure 2: Levey Jennings Chart of AI-C2 over 3 periods of batch manufacturing. Only 1 batch is outside the Mean+3SD over 3 periods which would require a justification of being covered by APS conditions.
When there is unequal variance, the probability of detecting the same %PoP change increases compared to conditions with equal variance. As a result, trend analysis should investigate significant changes in both the mean and SD, especially when increase in %PoP is more than 50%.
It's important to note that different statistical tools may be required to compare variations when the means are similar. For example, Welch’s ANOVA is particularly effective for comparing periods with similar means but unequal variances, allowing for a more accurate assessment of differences in variance between periods.
Finally, setting an optimal increase in %PoP limit helps ensure early detection of potential upward trends, enabling a timely CAPA plan. However, this is only possible if data is logged and analysed after each batch, rather than waiting until the end of a period.
Let’s get back to our example (Table 1). Based on the retrospective statistical analysis comparing Period 1 and Period 2, AIs C-2, C-5, C-6, and C-7 have a %POP mean exceeding 50% (highlighted in red in table 1), suggesting an upward trend that warrants further investigation. Additionally, AIs C-2, C-6, C-7, C-8, C-12, and C-14 show an increase of more than 50% in %POP SD, which also requires investigation to justify the upward trend.
The alert limit can be adjusted after each APS to ensure that all future manufactured batches align with the established APS conditions, allowing for early detection of potential atypical batches. It's important to note that not all investigations require a full 5M (Man, Machine, Material, Method, Mother Nature) analysis. In many cases, a simple review of the batch record documentation can quickly clarify the reasons behind an increased number of aseptic interventions. The goal is to be proactive, not reactive.
In the example provided (Figure 1), batch AB20 and AB25 exhibit atypical AI-C2 numbers, as they exceed the Mean + 3 SD for Period 1. Similarly, batch AB40 is atypical for Period 2. If a retrospective analysis is performed at the end of Period 2, it can be concluded that batches AB20 and AB26 are covered by the APS conducted after Period 2, using data from both periods. However, for batch AB40, if no valid justification is provided, worst-case conditions should be simulated during the APS to ensure the aseptic conditions for this, and other batches are covered.
It’s crucial to note that batches AB20 and AB26 would only be considered as covered if the data from AI-C2 for batch AB40 is retained and not excluded from the statistical analysis. This ensures that the analysis reflects the true variability and helps in maintaining representative process conditions.
Focusing on each period in isolation is not sufficient, as APS is con ducted between periods. To support batch release and demonstrate that manufactured batches truly reflect the APS conditions, the data must be analyzed holistically (see Figure 2). This approach includes providing robust justifications for any outliers, such as issues during a batch that resulted in an increased number of aseptic interventions (AIs).
Therefore, analysis should be conducted not only between individual periods but also across multiple periods. This ensures comprehensive oversight of aseptic conditions compared to the "normal" baseline, offering a clearer understanding of trends and potential deviations.
Figure 3: using Jmp 18 application to analyze the AI C-2 over 3 periods using ANOVA test, and Each pair Student's t-Test.
Setting the optimal %PoP increase limit:
Using an ANOVA test, we can show that a %PoP mean greater than 50% yields a Prob > F value of less than 0.0001 (p-value < 0.001), indicating an upward trend that is statistically significant and warrants further investigation (Figure 3). The analysis confirms that the threshold limit of an increase in %PoP mean greater than 50% is relevant for early upward trend identification. Additionally, by utilizing the JMP 18 application, we can compare the means of each period (indicated by circles on the right side of Figure 3). It reveals that Period 2 is statistically different from both Period 1 and Period 3 in regard to the number of AI C-2, suggesting that the corrective actions implemented during or after Period 2 effectively addressed the issues observed during that period.
Conclusion
The purpose of the statistical approach in defining the representative number and duration of corrective aseptic interventions (AIs) is important for ensuring that the aseptic conditions simulated in the Aseptic Process Simulation (APS) accurately reflect routine aseptic conditions. This statistical approach not only provides a solid justification for the representativeness of the number and duration of each corrective AI simulated during the APS, but it also facilitates trend analysis, which can trigger early investigations into any upward or atypical results. To be effective, data should be logged immediately after batch completion or in real-time, e.g. when using electronic batch records.
Furthermore, this trending analysis enables users to demonstrate, based on data, that the APS effectively covers routine aseptic conditions. It also offers a framework to support investigations into discrepancies related to the number or duration of AIs performed on specific batches. The statistical approach allows for retrospective analysis, which can help define prospective alert limits and even simulate potential routes of events.
Table 1: Number of AI per batch for Period 1 (e.g. from Jan to Jun) and for Period 2 (e.g. Jul to Dec) and so on. The table provides information on the number of interventions (e.g. AI C-2) performed per batch (e.g. AB01) and across batches (E.g. From AB01 to AB02), the mean and the Standard Deviation (SD). Users can add the minimum and maximum number of each intervention performed during a specific period for a total number of batches (e.g. 40 batches).
Note: this is an excerpt of the full table, which can be viewed in full here.
About the Author
Walid El Azab ... is Co-founder and Managing Director of QP Pro Services. He provides consultancy support through QPM Consulting and QP Pro
Services, acting as a strategic partner for pharmaceutical industry business continuity.
Noten:
1 EudraLex The Rules Governing Medicinal Products in the European Union, Volume 4: EU Guidelines to Good Manufacturing Practice Medicinal Products for Human and Veterinary Use, Annex 1 Manufacture of Sterile Medicinal Products, (2008).
2 U.S. Department of Health and Human Services Food and Drug Administration, Guidance for Industry Sterile Drug Products Produced by Aseptic Processing — Current Good Manufacturing Practice, (2004)
3 Swiss Medicine Inspectorate, Interpretation of GMP Annex 1 2022 (Rev. 1), I-SMI.TI.25e, (2023)
4 Parenteral Drug Association, Technical Report No. 22 (Revised 2011), Process Simulation for Aseptically Filled Products, (2011)
5 Hal Baseman, Subrata Chakraborty and Michael A. Long, Interventions Risk Evaluation and Management in Aseptic Manufacturing, PDA J Pharm Sci and Tech 2022, 76 485-496.
6 Zulfiqar Ali and S Bala Bhaskar, Basic statistical tools in research and data analysis, Indian J Anaesth. 2016 Sep; 60(9): 662 669.
7 Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, An Introduction to Statistical Learning with Applications in R, Second Edition, 2013.
8 Larry Wasserman, All of Statistics A Concise Course in Statistical Inference, 2004
9 EudraLex The Rules Governing Medicinal Products in the European Union, Volume 4: EU Guidelines to Good Manufacturing Practice Medicinal Products for Human and Veterinary Use, Annex 16
Certification by a Qualified Person and Batch Release, (2016).
