Monitoring Assay Performance

This application is a continuation of International Patent Application No. PCT/US2023/084655, filed on Dec. 18, 2023, which claims priority to U.S. Provisional Patent Application No. 63/433,598, entitled “Monitoring Assay Performance,” filed on Dec. 19, 2022, the disclosure to which is incorporated herein by reference in its entirety.

The present disclosure relates generally to techniques for monitoring and evaluating the performance of analytic biological procedures. More specifically, the present disclosure relates to methods for identifying and evaluating the significance of changes in data collected during an analytic biological procedure.

Taking repeated measurements of a process, or one or more aspects thereof, over time has the potential to provide valuable insights into changes occurring within the process. However, techniques for the analysis of such data to confidentially identify process change is currently lacking.

Sources of process data for analyzing process changes are numerous. For example, in the production of a biologic therapeutic, such as an antibody, immunoassays are useful for tracking the concentration of the biologic between different production runs. The production of a biologic therapeutic is very complex and involves numerous reagents and components (such as living cells), instruments, manufacturing and testing environments, and human operators, all of which are susceptible to change over time. Careful monitoring of biologic therapeutic manufacturing processes, including monitoring of assays using controls, is essential to the production quality control of such therapeutics. An additional example can be found in the analysis of digital biomarkers, such as from measurements of the human body performed over time from medical and/or consumer smart devices, e.g., a smart watch. The analysis of these biomarkers produce complex multi-parameter data, including linked information regarding heart rate, glucose level, blood-oxygen content, GPS coordinates, gyroscope data, and environmental conditions.

Current techniques for evaluating such data, which is often irregularly spaced, noisy, multi-dimensional, and contains limited ground truths, involve tedious manual processes with a lack of an established metric for identifying process outliers. Moreover, current techniques only enable the identification of process deviations well after the process is performed, thus making the rectification of an issue undesirably slow.

As described above, process data often includes a time series of repeated measurements of a process. Provided are methods for determining the causes of statistical changes in process data by identifying locations in process data where significant changes in the data have occurred. The methods may allow laboratory analysts to efficiently identify periods of time when important shifts in the data have occurred. Once the relevant periods of time have been identified, the analysts can investigate potential causes of the shifts in the data by evaluating parameters associated with the measurements taken during the identified periods of time.

In some embodiments, the methods described can both identify locations in process data where changes in the data have occurred and validate the significance of said changes with respect to the data set as a whole. Once the potential locations are pinpointed algorithmically, the significance of each potential location may be validated by quantifying statistical changes in the data surrounding the location. This quantification of the statistical changes can be used determine whether the changes that occur at that location are significant—and, therefore, potentially indicative of measurement error or other causes worth evaluating—or insignificant (e.g., the result of random statistical fluctuations). By validating the significance of the identified changes in the process data, the methods provided herein can, in some embodiments, help analysts spend time investigating important changes that have a high probability of impacting the results of the process being measured.

An example of a method for determining a cause of a significant statistical change in a plurality of concentration data points obtained from an assay may comprise performing a plurality of instances of the assay to obtain the plurality of concentration data points, receiving assay information comprising the plurality of concentration data points and a plurality of assay parameters, wherein each assay parameter of the plurality of assay parameters is associated with an instance of the plurality of instances of performing the assay, identifying a significant change point that corresponds to a location in the plurality of concentration data points at which one or more statistical characteristics of the plurality of concentration data points change by more than a threshold value, correlating one or more assay parameters of the plurality of assay parameters with the identified significant change point by identifying an instance of the plurality of instances of performing the assay that corresponds to the location of the significant change point in the plurality of concentration data points, and determining the cause for the change in the one or more statistical characteristics of the plurality of concentration data points at the identified significant change point based on the correlation between the one or more assay parameters and the significant change point.

In some embodiments of the method, the assay is configured to measure a concentration of an analyte in a sample.

In some embodiments of the method, the analyte is a therapeutic analyte.

In some embodiments of the method, the analyte is a therapeutic polypeptide.

In some embodiments of the method, the analyte is an antibody or a fragment thereof.

In some embodiments of the method, the sample is a cell culture sample or a derivative thereof.

In some embodiments of the method, the assay is an immunoassay.

In some embodiments of the method, the assay is a competitive assay.

In some embodiments of the method, the assay is a non-competitive assay.

In some embodiments of the method, the assay is a non-homogenous assay.

In some embodiments of the method, the assay is a homogenous assay.

In some embodiments of the method, the assay is an ELISA assay.

In some embodiments of the method, the ELISA assay is a direct ELISA assay.

In some embodiments of the method, the ELISA assay is a sandwich ELISA assay.

In some embodiments of the method, the ELISA assay is a competitive ELISA

In some embodiments of the method, performing the plurality of instances of the assay comprises performing two or more of the plurality of instances at two or more times.

In some embodiments of the method, the two or more times constitute a time course of at least about one week.

In some embodiments of the method, performing the plurality of instances of the assay comprises performing two or more of the plurality of instances simultaneously.

In some embodiments of the method, the plurality of concentration data points comprises data points pertaining to a concentration a target analyte.

In some embodiments of the method, the plurality of concentration data points comprises data points pertaining to a concentration of a control.

In some embodiments of the method, the control is a negative control, non-specific binding control, blank control, detection antibody control, negative matrix control, or positive control.

In some embodiments of the method, the plurality of concentration data points comprises data points pertaining to a solution concentration.

In some embodiments of the method, the plurality of concentration data points comprises data points pertaining to an absolute amount of a target analyte.

In some embodiments of the method, the plurality of concentration data points comprises data points pertaining to a measurement associated with concentration.

In some embodiments of the method, the measurement associated with concentration is an optical density (OD) measurement.

In some embodiments of the method, the plurality of concentration data points comprises data points pertaining to a mean, lowest standard deviation mean, highest standard deviation mean, or middle control concentration.

In some embodiments of the method, the significant change point reflects inter-assay variability.

In some embodiments of the method, the significant change point reflects intra-assay variability.

In some embodiments of the method, two or more concentration data points of the plurality of concentration data points are in the same format.

In some embodiments of the method, the one or more assay parameters correlated with the significant change point comprise an environmental factor.

In some embodiments of the method, the environmental factor comprises a temperature, humidity, light, or contaminant.

In some embodiments of the method, the one or more assay parameters correlated with significant change points comprise an equipment factor.

In some embodiments of the method, the equipment factor comprises a reagent change, reagent aging, reagent expiration, reagent contamination, reagent failure, hardware change, hardware aging, hardware contamination, hardware failure, instrument change, instrument failure, or instrument calibration.

In some embodiments of the method, the one or more assay parameters correlated with the significant change point comprise a human factor associated with one or more humans who performed or assisted in performing the assay.

In some embodiments of the method, the human factor comprises performance variability, performance error, or operator change.

In some embodiments of the method, identifying the significant change point comprises determining an expected change point population in the plurality of concentration data points.

In some embodiments of the method, identifying the significant change point comprises selecting a first segment of concentration data points of the plurality of concentration data points, determining a first median value associated with the first segment of concentration data points, selecting a second segment of concentration data points of the plurality of concentration data points, wherein the concentration data points in the second segment are consecutive to the concentration data points in the first segment, determining a second median value associated with the second segment of concentration data points, comparing the first median value to the second median value, and determining, based on the comparison between the first median value and the second median value, whether a candidate change point is located between the first segment and the second segment.

In some embodiments of the method, the first segment and the second segment comprise at least a threshold number of concentration data points.

In some embodiments, the method comprises receiving the threshold number of concentration data points from a user.

In some embodiments of the method, the threshold number of concentration data points is determined based on the assay.

In some embodiments, the method comprises, for the first segment and the second segment, generating one or more mean values, generating one or more data point clusters, wherein each data point cluster is associated with a mean value of the one or more mean values and comprises concentration data points in the segment that are closest to the associated mean value, updating the mean value for each data point cluster of the one or more data point clusters, wherein updating the mean value for a data point cluster comprises identifying a centroid of the data point cluster, and iteratively repeating the steps of generating one or more data point clusters and updating the mean values for each data point cluster until the mean values for each data point cluster no longer change.

In some embodiments, the method comprises generating one or more data point clusters within each of the first segment and the second segment, wherein the concentration data points in each data point cluster are normally distributed and have a unique mean value and a unique standard deviation value.

In some embodiments, the method comprises identifying, for the first segment and the second segment, a principal data point cluster of the one or more data point clusters, wherein the principal data point cluster comprises at least a threshold percentage of a total number of concentration data points in the segment.