Evaluation of Risk and Uncertainty for Model-Predicted NOAELs of Engineered Nanomaterials Based on Dose-Response-Recovery Clusters

Inhalation exposure to engineered nanomaterials presents the potential for an emerging occupational hazard. The unique combinations of properties of these small particulates may produce unique effects in individuals who are exposed to them in significant quantities. However, engineers, chemists, and materials scientists have an opportunity to guide in the future directions of this technology toward less harmful paths, if we can discern those safer choices before we are faced with a growing set of occupational illnesses that could arise from this industry.

Experimental toxicology studies, the best way we have to anticipate the effects of human exposure, are often time consuming and require considerable investment. A viable solution to the problem is testing the NMs on a priority basis—targeting the more toxic variants first. The development of computational approaches targeted at predicting the toxicity of the NMs based on their attributes can be used as a means to isolate potent nanomaterial variants and guide the future synthesis of less hazardous variants. Precaution against hazardous nanomaterials can be taken by the imposition of recommended exposure limits (RELs) and permissible exposure limits (PELs), The National Institue of Occupational Safety and Health (NIOSH) and the Occupational Safety and Health Administration (OSHA) are responsible for their development, respectively. NIOSH has currently proposed REL of 1 μg/m³ for carbon nanotubes (CNTs) and RELs of 50 μg/m³ for respirable Crystalline Silica, these limits are derived from the extrapolation of pulmonary exposure in rats to humans [1–3]. NIOSH treats metal oxide nanoparticles in a similar fashion, the current proposed REL for Titanium dioxide (TiO₂) currently stands at 2.4 mg/m³ and 0.3 mg/m³ for fine and ultrafine (includes nanoscale) species of TiO₂ [1]. It remains uncertain as to whether these recommendations will prove feasible as occupational exposure limits due to the difficulty of detecting nanoparticle concentrations at low levels in the context of realistic occupational environments.

The establishment of a recommended exposure limit is typically accomplished through identification of the no observed adverse effect level (NOAEL) in a toxicological study. The NOAEL is the highest dose for which no discernable adverse or negative change of health in the test subject could be detected. Since test subjects are typically animals or sometimes just cells, a series of adjustment factors or uncertainty factors are used to reduce the experimentally derived NOAEL to an established recommended exposure level or reference dose. Depending on the quantity and quality of the data, these adjustment factors can range from 3 to 1000 or more.

Though not yet used for the establishment of government recommendations in the U.S., several researchers have documented NOAELs [4] and corresponding adjustment factors for other nanomaterial variants that could presumably be used in such a way. The NOAELs and adjustment factors in these studies have varied between 0.1 and 0.37 mg/m³ and 2–50, respectively [2,5]. The NOAEL has traditionally played a key role in the determination of RELs. Given the diversity in responses to different varieties of nanoparticles, further subdivision of the exposure limits may be useful by incentivizing the production and use of less toxic varieties of nanoparticles in the future. Experimental evaluation of toxicity and exposure limits are subject to a certain degree of error or bias and in some cases, the results may be difficult to replicate. To overcome some of these inefficiencies, forecasting methods have been proposed to predict the potential toxicity and environmental impact of new NMs [6–11].

Attempts at using computational modeling to predict the effects on toxicity of changes in particle properties (quantitative structure activity relationships (QSARS), meta-analysis, etc.) have been limited in their scope to date. The rapid increase in the variety of engineered nanoparticles synthesized reveals a need to enhance the currently available state of knowledge and investigate exposure patterns [12]. A 10-year study in 2011 estimated that roughly six million people could be at risk of exposure to NMs by 2020 [13]. Categorizing NMs displaying similar effects as a means of grouping the NMs based on their toxicity is a potential solution to the existing problem. To address the question of how different nanomaterials can be and still be considered the same substance an algorithmic approach applied to meta-analysis of extant in vivo nanomaterial toxicity data was accomplished [14]. Progress in this direction could be significant in the analysis of new and emerging NMs which can be prescreened and prioritized for more careful evaluation if predicted to be more hazardous. In-vivo screening of nanoparticles has previously been attempted on isopods [15]. The screening concluded that duration of exposure had greater effect than the quantity consumed. The financial factor impacting the potential regulation of NMs cannot be ignored with cost estimations for hazard testing of nanoparticles projected to range between $250 million to $1.2 billion [16]. The evaluation of updated exposure limits for the new NMs is also necessary to assess their agreement with the existing recommendations for workplace exposure. The challenges associated with the assessment of the risk posed by nanoparticles have been documented extensively [17,18]. An alternative to the traditionally determined NOAEL is the “benchmark dose” (BMD) which in similar fashion to the NOAEL is a single number that is statistically derived and can be used to quantify the exposure risk to substances [19–22]. A detailed and critical comparison between the BMD and NOAEL has been conducted previously which suggests that NOAEL derivation methodology is antiquated and dependent on several experimental factors such as dose selection, dose spacing, and sample size [23]. The advantages associated with NOAELs lie in their ease of derivation and ability to work with all types of exposure data. The BMD process takes into account multiple factors of which the shape of the dose-response curve is an important factor, there are multiple available BMD models to choose from depending on the variation between the dose and response [23]. This paper illustrates the derivation of a model predicted NOAEL (MP-NOAEL) which in similar fashion to the BMD utilizes a fitted dose-response model as a critical component of the methodology.

The data utilized for the meta-analysis of the in vivo pulmonary toxicity of the engineered nanoparticles is curated from peer-reviewed journal articles. The publications are all independent studies, a study can be defined as a complete experiment that consists of several levels of exposure and recovery times to one or more different nanoparticle variants (a batch of nanoparticles defined by a set of chemical and physical properties). Studies are comprised of multiple exposure groups, each group is a set of animals characterized by their exposure to a specific dosage, method of exposure (inhalation, instillation, aspiration, etc.), postexposure recovery length, and their resultant toxicological measurements. There are five quantitative response measures from the bronchoalveolar lavage (BAL) fluid that are the focus of our meta-analysis: Total cell count (TCC), macrophage count (MAC), total protein concentration (TP), polymorphonuclear neutrophil count (PMN) and lactate dehydrogenase concentration (LDH). These five measures were reported in a sufficient number of publications to be included in this analysis. The measured response reported in each publication is expressed as a fold (multiple) of its respective control group's measure, this has been done in accordance with previous work which suggests that to compare different exposure modes (instillation, aspiration, and inhalation) a normalized response is needed. For the purposes of this study, in vivo pulmonary toxicity data was chosen from two major divisions of nanomaterials, namely, CNTs (either single-walled carbon nanotubes (SWCNTs) or multi-walled carbon nanotubes (MWCNTs)) and metal oxide nanoparticles (MONPs). Table 1 below is a comprehensive summary of the data sources utilized for the analysis presented in this publication. All the showcased exposure groups and reported endpoints were used in the analysis presented by this publication.

The four parameters, a (signifies the response at dose = 0), b (the toxic potency of the nanoparticles), c (the maximum relative shift in response), and d (slope of the response decay) are the key parameters that can be used to quantify and reflect the potential of the particle to be a hazard.

Lower values of AIC are preferred when comparing the clusters as they indicate the model is closer to the data in those cases. AIC needs to be adjusted when dealing with small datasets, this corrected form is denoted as AICc

$AICc=AIC+2k(k+1)n−k−1$⁠, where n is the number of observations and k is the number of model parameters.

The AICc weighs both the complexity and performance factors to provide the most balance, it helps in filtering about overfitted solutions that provide marginal improvement while increasing the complexity of the model. A detailed discussion behind the process and approach can be found in previous publications by the same author's [14,85].

This publication presents a methodology to derive model-predicted NOAEL (MP-NOAEL) values for clusters of both the CNTs and MONPs. The process utilizes the measured control group response for each particle tested and the standard deviation associated with the control group response to derive the MP-NOAEL. For the purpose of our analysis, we assume our MP-NOAEL response to be the dose that corresponds to the response measured at two standard deviations above the control group response. The dose-response curve (Eq. (1)) for each cluster can be used to estimate the dose that corresponds to any given value of the measured response, the recovery term from Eq. (1) is set to 0 as there is no observed adverse effect onset at t = 0. Since the dose-response-recovery curve for each cluster is different we are able to obtain model predicted NOAELs which are unique to each cluster. The NOAEL values predicted by this methodology might not necessarily agree with what eventual experiments predict. Some of the factors responsible for the differences can be the inherent variance arising from the multiple studies included in our analysis, the NOAELs being a function of the dose levels tested (variation in dose levels tested between publications leads to diverse NOAELs), diversity in the animal species used in the exposure studies, and multiple nanoparticle sources. These factors are just a few factors that could be contributing to the difference between model-predicted MP-NOAELs and experimentally determined NOAELs.

The type of model used to model the dose-response variation directly impacts the MP-NOAEL calculation since the MP-NOAEL dose value is back-calculated using the MP-NOAEL response, altering the type of dose-response model used results in directly modification of the MP-NOAEL values. The default option for all clusters is the model represented by Eq. (1), in some special cases, it would be favorable to substitute the traditionally exponential dose-response model for a simple linear dose-response relationship (y = mx + c + ε) or a simple exponential model (⁠$y=a*e−bx$⁠), where y is the measured response and x is the corresponding delivered dose. The substitution is necessary due to compatibility issues between the original dose-response model and the data encapsulated by the particular cluster. Common examples where model substitutions become necessary to include situations where there is a lack of data, such cases might be better modeled using a linear model than an exponential model. Similarly, when the empirical evidence suggests nonconformity and poor model fit across all model choices, we can choose the simplest model. The 95% confidence interval of the parameter estimates for the dose-response model from Eq. (1) is used to derive the corresponding confidence limits of the MP-NOAELs; the parameter estimates themselves are based on how well the model conforms to the available data.

The novelty behind the approach described by this publication lies in the data collection process and model-driven approach adopted to derive NOAEL limits. The addition of a temporal component to the dose-response relationship to model response decay, choosing to cluster the nanosubstances together based on their dose-response-recovery profiles rather than physicochemical characteristics, and working on an assembled in vivo dataset are just some prominent examples where the adopted methodology for deriving NOAELs differs from the traditional. This methodology has the potential to provide a diverse range of NOAELs for similar substances categorized by their relative toxicity rather than a single all-encompassing limit. Stratification of the NOAEL limits would be highly effective to isolate toxic variants and prioritize their testing. While there are differences between the exposure limits determined using the method highlighted in this paper and the BMD methodology, the NOAELs suggested by the authors are meant to supplement the existing information in the domain rather than supplant the existing benchmarks.

The clustering analysis was conducted on two datasets of engineered nanoparticles, CNTs, and MONPs. The dataset for the MONPs is comprised of multiple studies where more than one unique set of nanoparticles is analyzed, such studies were identified and separated to ensure each study reflects a single particle. Both sets of data were analyzed for the five endpoints discussed earlier and their ideal cluster combinations were derived. An ideal cluster size of four was consistently achieved across the five responses for the CNTs, there is no consensus cluster size that can be agreed upon across the five responses for the MONPs due to the varying size of the number of particles analyzed for each response. A detailed reasoning behind the ideal choice of clusters being four has been addressed previously by the authors [14,85].

The existence of these clusters is not based solely on the similarity of their in vivo toxic effects. Detailed examination of the particles within the clusters is necessary to study the characteristics of the constituent particles and generate hypotheses to ascertain why one cluster or class of nanoparticles may be more or less toxic than another. The particles were not separated into groups based on their physical/chemical characteristics before the clustering process. The reasoning is that physically and/or chemically similar nanomaterials need not necessarily be similar in their dose-response-recovery relationships as well, depending on which characteristics are used to define similarity. Differences in the relative toxicity of the clusters could be correlated to significant variation in the particle attributes between them, these attributes are also hypothesized as key contributors to the separation of the clusters at each stage.

Examination of the properties of all the particles in the CNT and MONP clusters revealed that the CNT particles can be identified by their physical characteristics. Table 2 is an illustration of the distribution of physical properties (median length and median diameter) and impurity content of 4 CNT clusters using PMN as a response. We can observe upon inspection of the physical properties the potential to define provisional thresholds which can be used to assign the clusters to different categories. By virtue of these thresholds, we can designate rudimentary labels to the clusters for ease of reference. Setting thresholds of 15 nm for the diameter leads to clusters 2 and 3 being designated as “thin” clusters whereas clusters 1 and 4 can be regarded as “thick.” Using 2200 nm as a threshold for length we can label the four clusters as “long” and “short” based on their median length. A similar approach adopted yielded consistent results across the other four responses analyzed although with varying threshold limits for the length and diameter. A single uniform range of values for length and diameter for all the response variables could not be established due to the shift in the constituent members of a cluster across the responses. It should be noted that these labels are provisional and cannot be verified as statistically meaningful across the five responses analyzed due to insufficient characterization data amongst the tested CNTs. The labels have been highlighted here to point to the presence of a pattern which could possibly be examined in thoroughly in the future. The presence of negative values within the model predicted NOAELs is symptomatic of the lack of data at lower dose levels, this results in negative intercepts when performing low dose extrapolation. These values while appearing to be irrational are statistically relevant since they are evidence to the need for increased testing at lower dose ranges.

Similarly, we analyzed the MONP clusters for the PMN response in the same fashion as the CNTs to identify and highlight any trends between the clusters and their particle properties. The ideal number of clusters for the PMN response were found to be eight based on the AICc values. Table 3 showcases the particle properties and cluster composition of the MONP clusters for the PMN response. The “particle typical size” measurement is the mean diameter for the spherical MONPs that were analyzed. We can observe that there is no perceivable trend between the particle size and the cluster composition for this particular response. Analysis of the four other responses for the MONPs revealed the same. This could imply that unlike the CNT particles, categorizing the metal oxide nanoparticles purely using their physical properties might not be possible and hence a more detailed approach might be required which considers both the physical and chemical characteristics of the particles.

No-observed-adverse-effect-levels can be predicted for each of the clusters generated using the algorithm. The NOAELs can be defined as the highest experimental dose that does not produce an adverse effect . Tables 2 and 3 are the measures of the predicted NOAELs for the CNTs and MONPs respectively using PMN as a response. Increase in the PMN response is an indicator of inflammation, they are also preferred due to their relative sensitivity during inhalation-based experiments [86,87]. Large values for the NOAEL of a substance imply that higher dosage is required before adverse effects are observed in an organism and hence lower the potency of the substance.

The initial clustering process treated the MONPs analyzed from each study as a unique particle. This leads to different instances of the same substance appearing in multiple clusters as evidenced in Table 3. The information provided in Table 3 for the MONP clusters do not seem to provide evidence of any discernable trend between the particle properties, cluster composition and the predicted NOAELs for the respective clusters. A repeat of the clustering process was performed by isolating the various particles on a chemical basis. Table 4 contains the measures of particle size and predicted NOAELs for the new set of clusters derived along with their respective compositions.

We can observe from Table 4 that not only are the optimum number of clusters reduced from 8 to 4 for the MONP dataset compared to the earlier iteration but also that two of the clusters contain isolated chemicals (TiO₂ and iron oxides). A key observation between the two clustering variants is the presence of irrational MP-NOAEL values for the second clustering iteration, the cause of these values and their possible treatment has been addressed in the following section.

The model-predicted NOAEL estimates discussed earlier for the CNTs and MONPs are point estimates that are determined using the dose-response model. The dose-response model plays a crucial role in both the clustering process and the determination of MP-NOAELs. The confidence interval for the MP-NOAEL values can be visualized using a “confidence spread.” Evaluating the confidence spread of the dose-response curve would aid in visual assessment of the fit of the model to the experimental data, a tight spread can be viewed as representative of low error and a good fit to the data. Conversely, larger spreads imply larger error in the parameter estimates. The confidence spreads for the MP-NOAEL are determined using the parameter estimates for each cluster, we used the “nlnfit” and “nlparci” functions made available through the “statistics and machine learning toolbox” on matlab (2017b) to obtain the estimate intervals for each parameter and proceeded to plot the spread [88]. Figures 1 and 2 are helpful in visually symbolizing the confidence spread for the dose-response models associated with each cluster across all five responses. Based, on the evidence presented in Fig. 1 we can see that only a handful of CNT clusters (5 out of 20) display the signs associated with good fit and low error.

The cause greater uncertainty associated with the estimated dose-response model parameters could be attributed to several reasons. We can see from Fig. 1, there are three clusters (A, “C,” and “D”) for the PMN response which show and conform to the typical dose-response model relationship. Amongst the other clusters only cluster A for the “MAC” and “TCC” responses are consistent with good fit, low error representations. The majority of the remaining clusters show larger spreads which could imply that the fitted model is not the right choice for the data evaluated. Examining the models for the larger spread cases we observed the regression model parameters were all deemed significant at a 0.05 level of significance. The possibility of the model despite passing the test of significant not being the right choice for the data was considered and alternatives were tested, the alternatives did not show significant improvements. The lack of insignificant parameters for the dose-response model could also indicate that the data itself could be a source of error. Detailed exploration of the available data revealed that there is indeed a lack of data in those cases showing large spread, the quality of data could also be a factor leading to larger error. The data collection process is also a factor especially in the case of cluster “B” under the TCC response. The source publication only reported the results of evaluation at only two dose levels which were well spaced from one another leading to large error [29].

The evidence presented in Fig. 2 shows much of the same issues plaguing the clustering process for the MONPs that did in the case of CNTs. The increase in the number of sources for the MONPs reflects a greater number of particles analyzed and therefore an overall increase in the number of data points available for analysis for each cluster. Figure 2 shows that half of the clusters (10 out of 20) overall are consistent with expectations. This could directly be a result of an increase in available data for the overall process. Detailed inspection of the data available to the clusters with large uncertainty spreads revealed that the underlying issue could be due to the data collection process, there was a large difference in the range of dose-levels tested. For example, cluster “I” under the “PMN” response for the MONP particles showed that the lowest dose tested for was at 800 μg/kg while the highest was 20,000 μg/kg with no recorded results at dose levels in between. The lack of available data between the two dose levels tested denies a detailed look at the complete progression of the response across the dose range. This disparity in dose-levels is even more apparent in cluster I under the MAC response, the large spacing between the doses forced a change in the type of dose-response model used, a linear dose-response relationship was substituted for that particular cluster. Cluster I under the TCC response and cluster “III” under the “TP” response, in both cases an exponential model was used. Additional reasons for the large spreads at higher doses could be attributed to variability in the biological response of the animals sourced and tested, and the lack of data covering a wide range of dose levels. The uncertainty could also be a property of the underlying data used for the analysis, this hypothesis could be verified in the future using newer data sources and checking if the same level of error is observed.

Tables 5 and 6 below are the model predicted NOAEL value ranges for the 20 clusters generated by the clustering process for the CNTs and MONPs, respectively. The NOAEL calculation is not impacted by the dose-response modeling process directly as it corresponds to the highest dose delivered that does not induce any response in the subject. Errors present in the dose-response model similarly do not affect the experimentally derived NOAEL. The process described earlier for the determination of the model predicted NOAELs is highly sensitive to the parameters for the dose-response model whereas the parameters themselves are estimated based on the data used.

We can observe from Tables 5 and 6 that the clusters with viable estimations of the MP-NOAEL are most of the same clusters that showed a tight confidence spread. Most of the clusters are missing their designated MP-NOAEL because the calculated range contained negative values and was deemed undefined, since the minimum possible NOAEL value is 0 which corresponds to a case of no exposure. NOAELs of zero (0) do exist for substances that have no threshold of safe exposure. However, a range of entirely negative predicted MP-NOAELs would indicate that some adverse response is present at a dose of zero. Such an occurrence must be either an artifact of limited data to reliably establish model parameters or an artifact of the experimental design. The common link between the clusters with “U” designations is that they displayed large spreads of uncertainty in the model parameters.

There are certain MP-NOAEL measures that have been highlighted in both the tables that are derived from clusters with a tight spread. The main reasoning behind the occurrence of negative values in the range for the NOAELs is the position of the MP-NOAEL response relative to the spacing of the confidence spread. For example, cluster D under the PMN response for the CNTs displays a tight confidence spread consistent with a good fit and low error yet its estimated MP-NOAEL is negative, this is because the MP-NOAEL response falls below the confidence spread spacing and hence their intersection can only possibly occur at negative dose values. The relative shift is also observed in cluster B for the PMN response and clusters A and B for the MAC response in the CNTs, cluster “II” for the PMN response and cluster III for the “LDH” response in the MONPs. This shift in the confidence spread occurs mainly in response to the fitted model parameters which are highly dependent on the data analyzed and specifically the dose spacing. Clusters with viable MP-NOAEL values were comprised of source publications that reported the BAL response at multiple low dose (<200 μg/kg) levels and where the spacing of the dose values tested was more even. A potential solution to the “relative shift” issue is to set bounds of the estimated parameters to ensure that the MP-NOAEL response is always within range, this methodology was successful and was used to derive the estimates for the MP-NOAEL for the PMN response for the CNTs and MONPs (Data in Tables 3–5), extension of the same technique to the other responses resulted in model parameters which failed the test of significance at the 0.05 level.

The utility of these predicted NOAELs and associated confidence ranges for these nanomaterials is twofold. First, these values provide a simple means for evaluating our current understanding of nanomaterial safety from the perspective of inhalation exposures, which is of interest for nanomaterial manufacturers as well as health and safety research organizations. For materials or clusters of materials where the data is current insufficient to predict a NOAEL, there is a need to prioritize the continued collection of in vivo toxicological data especially for those clusters anticipated to be more toxicologically potent based on the point estimates. Second, these values permit anticipation of future exposure limits. While recommendations have only been published for CNTs and nano-TiO₂, these values allow nanomaterial development scientists and toxicologists and policy analysts to anticipate at what levels such future recommendations and potentially future regulatory limits will be set. For instance, since the MP-NOAEL for the silica-containing cluster is approximately half of the titania-containing cluster, one should currently expect the future recommended exposure limit for nanosilica to be about half of the NIOSH REL for nano-TiO₂. Organizations working with such materials could take this provisional guidance into account to help ensure worker safety while the available dataset continues to evolve.

The development and application of a hierarchical clustering algorithm to perform a meta-analysis of engineered nanomaterials in vivo pulmonary toxicity in rodents has been documented previously [14]. NOAELs were predicted for the clusters using their respective dose- response curves and correlated to which cluster of particles can be categorized as relatively more or less potent. The model predicted NOAELs were derived for the clusters belonging to a single response as point estimates. These estimates reflect the available data and the dose-response model used. A detailed sensitivity analysis of the clusters and their predicted NOAELs suggests that the model predicted NOAELs are dependent on the dose range evaluated for a response, the error in the model parameter estimates and the form of the model chosen to represent the dose-response relationship. Reducing the overall observed error between the fitted dose-response model and the data would be beneficial to the derivation of model predicted NOAELs. The MP-NOAEL predictions could be further enhanced and developed by incorporating multiple functional model forms to create a database of models capable of explaining wide ranging dose-response behavior. Multiple clusters showed substantial uncertainty which results from limitations in data availability across the five responses analyzed. The uncertainty can be mitigated by constantly expanding the available data to be analyzed and ensuring that BAL responses to particles are recorded at multiple dosage levels. Similarly, researchers could be encouraged to test particles at multiple dose levels further expanded from their initial designs, the additional time and financial investment required might preclude that possibility.

• NIOSH (Grant No. 1 R03 OH010956-01; Funder ID: 10.13039/100000125).

• Center for the Environmental Implications of Nano-Technology (CEINT), National Science Foundation (NSF) and the Environmental Protection Agency (EPA) (NSF Cooperative Agreement EF0830093; Funder ID: 10.13039/100000001).

BAL =

bronchoalveolar lavage

CNT =

carbon nanotubes

LDH =

lactose dehydrogenase

MAC =

macrophage count in BAL fluid

MONP =

metal oxide nanoparticles

MP-NOAEL =

model-predicted no observed adverse effect level

NOAEL =

no observed adverse effect level

PMN =

polymorphonuclear neutrophil

TCC =

total cell count

TP =

total protein concentration in BAL fluid