Development and deployment of a supply-chain digital tool to predict fluid milk spoilage due to psychrotolerant sporeformers

Psychrotolerant sporeformers pose a challenge to maintaining fluid milk quality. Dynamic temperature changes along the supply chain can favor the germination and growth of these bacteria and lead to fluid milk spoilage. In this study, we aim to expand on our previous work on predicting milk spoilage due to psychrotolerant sporeformers. The key model innovations include (i) the ability to account for changing temperatures along the supply chain, and (ii) a deployed user-friendly interface to allow easy access to the model. Using the frequencies and concentrations of 8 Bacillales subtypes specific to fluid milk collected in New York, the model simulated sporeformer growth in half-gallons of high-temperature short-time (HTST) pasteurized fluid milk transported from processing facility to retail store and then to consumer. The Monte Carlo simulations predicted that 44.3% of half-gallons of milk were spoiled (defined as having a bacterial concentration > 20,000 cfu/mL, a conservative estimate that represents the Pasteurized Milk Ordinance [PMO] regulatory limit) after 21 d of refrigerated storage at consumer’s home. Model validations showed that the model was the most accurate in predicting the mean sporeformer concentration at low temperatures (i.e., at 3 and 4°C; compared with higher temperatures at 6 and 10°C) within the first 21 d of consumer storage, with a root mean square error of 0.29 and 0.34 log 10 cfu/mL, respectively. Global sensitivity analyses indicated that home storage temperature, facility-to-retail transportation temperature, and initial spore concentration were the 3 most influential factors for predicting milk spoilage on d 21 of shelf-life. What-if scenarios indicated that microfiltration was predicted to be the most effective strategy to reduce spoilage. The implementation of this strategy (assumed to reduce initial spore concentration by 2.2 log 10 cfu/ mL) was predicted to reduce the percentage of spoiled


INTRODUCTION
Bacterial spoilage due to psychrotolerant sporeformers is a major hurdle to maintaining fluid milk quality.The genera Bacillus and Paenibacillus are commonly isolated from fluid milk (Fromm and Boor, 2004;Huck et al., 2007).These microorganisms can contaminate raw milk from various on-farm sources such as soil (Christiansson et al., 1999), feed (Vaerewijck et al., 2001), and bedding materials (Magnusson et al., 2007).The processing plant may provide an additional pathway to contamination of raw milk as Bacillus and Paenibacillus spp.can survive and persist in processing equipment (Doll et al., 2017).Once present in the milk, these sporeformers can survive pasteurization and subsequently germinate.Following germination, some Bacillus and Paenibacillus spp.are capable of growing to high enough levels where enzymatic activity (Trmčić et al., 2015) causes defects that are unacceptable to consumers, including off-flavors and coagulation.As milk travels along the various stages of the supply chain, dynamic temperature changes can provide opportunities for germination and subsequent growth of psychrotolerant sporeformers.
The occurrence of spoilage in fluid milk is influenced by multiple factors, including the initial raw milk spore concentration, sporeformer subtypes (and associated growth characteristics), storage and transportation durations, as well as temperature conditions along the supply chain.Therefore, it is difficult to accurately predict spoilage based on any single factor alone, or human intuition which was argued to be unreliable in estimating probability (Tversky and Kahneman, 1983).Thus, an assessment tool that can systematically estimate psychrotolerant sporeforming bacterial spoilage is needed for stakeholders to make decisions regarding supply chain management to reduce spoilage and extend the shelf-life of fluid milk.The model developed here was built based on a previously reported initial model (Buehler et al., 2018), which also simulates the growth of psychrotolerant sporeformers in fluid milk, but requires a major assumption of a single static temperature (i.e., 6°C) for the entire supply chain continuum.Thus, the objective of this study was to develop a digital tool that can accurately predict psychrotolerant sporeformer growth as well as the percentage of spoiled half-gallon milk containers at different time points in a complex yet realistic supply chain that consists of 5 stages that can each represent a different temperature.

Experimental Data
Sample collection.To create a validation data set, pasteurized milk samples were collected from 2 conveniently selected New York dairy processing facilities, both with annual production capacity between 10 million and 100 million liters.For each facility, one set of samples representing 4 milk types (skim, 1%, 2%, and whole milk) was collected on a given day; samples representing the same milk type from a given facility (e.g., skim milk from the facility A) were commingled according to the SMEDP (Robertson and Black, 1949) and then aliquoted into 200-mL portions, each dispensed into a separate sterile 250-mL, screw-capped Pyrex bottle (Corning Inc.), yielding a total of 8 different sample categories (2 facilities x 4 milk types).Each of the 8 samples categories was incubated for 7, 14, 21, 28, and 35 d at each of 3 different storage temperatures, including (i) 4°C or 6°C held constant over the whole incubation time, and (ii) simulated supply chain temperatures ("SC temperatures"), which represented (a) 4°C for 36 h to simulate the storage at the processing facility, (b) 4°C for 5 h to simulate transportation from facility to retail, (c) 2°C for 24 h to simulate storage at retail, (d) 10°C for 26 min to simulate consumer transportation to home, and (e) 4°C up until the designated experimental storage time to simulate consumer storage at home.These specific temperatures and times (e.g., 4°C to simulate facility storage temperature) were obtained from rounded median values of temperature and time distributions in the corresponding supply chain stages (e.g., a Uniform [3.5,4.5]distribution that represents the temperature variability at facility) (Table 1).

Microbiological testing.
For each incubation temperature, samples were incubated over 35 d and tested every 7 d for total aerobic plate count (APC) and total Gram-negative count.APC testing was performed in duplicates by using a spiral plater (Neu-tec Eddy Jet 2, Farmingdale, NY) to plate 50 μL of appropriate dilution of milk samples onto standard methods agar (EMD Millipore Corporation, Billerica, MA), followed by 48 h incubation at 32°C and subsequent enumeration.Total Gram-negative counts were determined by spiral plating 50 μL of sample onto crystal violet tetrazolium agar, a selective media for Gram-negative bacteria, in duplicate, followed by incubation at 21°C for 48 h and subsequent enumeration of only red colonies according to Standard Methods for the Examination of Dairy Products (SMEDP) (Frank and Wehr, 2004) using an automated colony counter (Q-count, Advanced Instruments, Norwood, MA).Total Gram-negative counts were used to identify samples that were contaminated by Gram-negative bacteria (presumably due to postpasteurization contamination); samples that showed evidence for the presence of Gram-negative bacteria were not included in the validation data set (as our model only assessed spoilage due to growth of sporeforming Gram-positive bacteria).For the samples with APC count 2 log greater than total Gram-negative counts and total Gram-negative counts less than 2 log, we consider psychrotolerant sporeformers to have outgrown putative Gram-negative bacteria and therefore those samples were not assigned post-pasteurization contamination.In addition, if a test unit (i.e., a sample with the unique combination of replicate, facility, milk type, and storage temperature) showed post-pasteurization contamination in 2 consecutive testing days, samples representing subsequent time points for this test unit were not tested and hence not included in our data set.For example, for a skim milk sample from facility A that was stored at 6°C, if post-pasteurization contamination was observed for bottles that have been stored for 14 and 21 d, we discontinued storage and testing for the bottle that should be stored for 35 d.

Model Overview
The model reported here was built based on a previous preliminary model (Buehler et al., 2018), which was restricted to a single static temperature (i.e., 6°C) for the overall period between manufacturing and the end of shelf life.The model developed here simulates the growth of psychrotolerant sporeformers in 1,000 (hypothetical) lots of 10 halfgallon milk containers from manufacturing to consumer storage for up to 35 d.The simulated concentrations were used to calculate the percentage of half-gallon  milk containers from these 1,000 lots that exceeded the 20,000 cfu/mL, which is the regulatory limit for pasteurized fluid milk in the US according to the FDA Pasteurized Milk Ordinance (PMO).Based on these data, the end of fluid milk shelf life was arbitrarily defined as the day in consumer storage when at least 50% of total milk containers exceed the regulatory limit of 20,000 cfu/ml; this definition is used consistently throughout the manuscript.Key modifications from the previous model described by Buehler et al. (2018) include (i) the ability to include up to 5 different stages in a supply chain, which each can have a separate time and temperature distribution, (ii) containers that are assigned an initial load of <1 spore per container are assigned a value of "0," thus allowing no growth, and (iii) a modified psychrotolerant sporeformers subtype (i.e., allelic type [AT]) table (as input parameter for the model) that excludes no-growth ATs.

Model Parameters
The "baseline model" (i.e., a model assessing a baseline fluid milk spoilage scenario) was initiated to simulate the transportation and storage of 1,000 lots (n lot ), each consisting of 10 units representing half-gallon milk containers (n unit ).The parameters used in this model include (i) AT frequency (f AT ) (referred to as "NY AT frequency" as it represents the frequencies of 8 different subtypes of psychrotolerant sporeformers isolated from NY raw milk), (ii) initial spore concentration (n i ) and (iii) growth characteristics of these 8 subtypes, as previously reported (Buehler et al., 2018).Growth characteristics include the maximum growth rate (μ max ), lag phase duration (lag), and maximum microbial population (N max ).The AT frequency table was modified so that 2 isolated ATs that showed no growth in skim milk broth were excluded.The purpose of this step is to prevent the underestimation of microbial growth and subsequent spoilage in our model.
In addition to the subtype-specific growth parameters, specific temperature and time distributions were obtained for 5 stages of the milk supply chain (Table 1), which include (i) storage at facility, (ii) transportation from facility to retail store, (iii) storage at retail store, (iv) transportation from retail store to consumers' home, and (v) storage at home.For these parameters, the units for duration and temperature are consistently days and °C, respectively.Duration of storage at a processing facility (t F ) and temperature at the same stage (T F ) were modeled using a Uniform (1,2) distribution and a Uniform (3.5, 4.5) distribution, respectively (Table 1), based on the range of time and temperature collected in 2021 from the same 2 commercial milk processing milk facilities where validation milk samples were collected from (T. T. Lott, unpublished data).For transportation from facility to retail store, the duration (t T ) was modeled using a Triangular (1, 10, 5) distribution, and temperature (T T ) was modeled using a Triangular (1.7, 10.0, 4.4) distribution (Table 1), both based on the data from expert elicitation in a risk assessment of listeriosis from soft-ripened cheese consumption (FDA and Health Canada, 2015).The assumption was made because both pasteurized fluid milk and soft cheese need to be stored and transported at refrigeration temperature for a relatively short period, and therefore these 2 dairy products share similar transportation temperature and time variability.When milk is stored at retail, the duration of storage (t S ) was modeled using a Normal (1.821, 3.3) distribution truncated at 0.042 and 10 d and the temperature at retail storage (T S ) was modeled using a Normal (2.3, 1.8) distribution truncated at −1.4 and 5.4°C (Table 1).The normal distribution was selected based on the shape of histogram and these distribution parameters are based on summary statistics (i.e., mean, standard deviation, and range) generated by a previous study conducted by the Cornell Milk Quality Improvement Program (MQIP) in 2013 measuring storage temperature and time for fluid milk packages in 26 different retail locations (Supplementary Table 2).While the temperature measurement was taken at the exterior of packaging using an infrared thermometer, and therefore some readings were below 0°C, we believe overall this is a reasonable approximation of actual product temperature.For the transportation from retail store to consumer's home, the specific temperature and time distributions were obtained from consultation with an expert in the Cornell Dairy Extension team.The duration (t T2 ) and temperature (T T2 ) were modeled using a Normal (0.04, 0.02) distribution truncated at 0.01 and 0.24 d, and a Normal (8.5, 1.0) truncated at 0 and 10.0°C, respectively (Table 1).Lastly, during consumer storage, the duration (t H ) was assumed to at most be 35 d, and temperature (T H ) was modeled using a Laplace (4.06, 2.31) distribution truncated at −1 and 15°C (Table 1).This temperature distribution was obtained from a study (fitness not provided from literature) that investigated the temperature of ready-to-eat foods in the US (Pouillot et al., 2010), and we, therefore, assumed the same temperature profile can be used to represent the refrigeration conditions in US households.
Implementing multiple stages that each have their temperature distributions required the implementation of procedures that account for changing maximum growth rate and lag-phase duration as microorganisms adapt to new environments.The following rules were modified from Zwietering et al. (1994) and applied to address the shift in lag phase duration as the result of temperature shift when a container was transferred from one stage to another.The model first calculates adjusted lag phase duration (lag adj ) based on the new temperature (T new ) in the current stage, old temperature (T old ) in the previous stage, minimum growth temperature (T 0 ), and theoretical lag phase duration at a given temperature T old (lag old ).(Equation 1).
Then the model checks whether the bacteria are still in the lag phase in the previous supply chain stage by calculating the proportion of elapsed lag phase duration (lag elap ) using Equation 2, where i is stage number (e.g., facility storage = 1, facility-to-retail transportation = 2, etc.,) and s is the number of stages that milk traveled including the current stage.
If the bacteria from the previous stage were no longer in the lag phase, we assumed that in the new stage, the lag phase duration is 0 (i.e., bacteria are in the exponential growth phase).Otherwise, the new lag phase duration (lag new ) was calculated as the product of remaining proportion of lag phase duration (1-lag elap ) and lag adj .
The maximum growth rate at each stage was also adjusted depending on the shift in temperatures, calculated as follows: If the new temperature (T new ) is within 25% of the old temperature (T old ), then we assume that the new maximum growth rate (μ new ) equals to the old maximum growth rate (μ old ); otherwise, the μ new was determined as described in Pradhan et al. (2009).
These calculated growth rate (μ new ) and lag phase duration (lag) were then used as model inputs for the Buchanan growth model (Buchanan et al., 1997) to calculate the concentration of sporeformers (N) at the end of each supply chain stage.Additional inputs for these calculations were (i) sporeformer concentration from the previous stage as the initial concentration (N 0 ), (ii) maximum microbial population (N max ), and (iii) duration of the current stage.In the equation below, t max is defined as the time when the sporeformer concentration reaches N max .

Model Validation
The developed model was validated using 3 data sets with different sets of input parameters.First, our model was validated using the same validation data set (n = 30) as described by Buehler et al. (2018) ("validation 1").To evaluate model performance under different static temperature profiles and to assess model generalizability with a different source of raw milk, we used a previously described data set (n = 268) (Lott et al., 2023) for the second validation ("validation 2").Lastly, to ensure our approach to model microbial growth upon temperature shift and to evaluate model performance under dynamic temperature profiles, we used the experimental data (n = 68) generated in this study to conduct the third validation ("validation 3").The details of the validation procedures are provided below.
Validation 1 A historical data set described by Buehler et al. (2018) was used to perform validation 1.This historical data set represents shelf-life data for fluid milk with spoilage due to sporeformer growth and was obtained through shelf-life testing performed on commercial pasteurized milk collected through Cornell's Voluntary Shelf Program (VSL) between October 2016 and June 2017 (referred to as "VSL data").The same input parameters described by Buehler et al. (2018) were used, including NY AT frequency (f AT_ Val ), initial spore concentration (N 0_ Val ), storage temperature (T Val ) set at 6°C (with no temperature variation, to mimic the incubation temperatures that were used to generate the validation data) and storage time (t Val ) set at 14 d.The simulated sporeformer distribution was compared with the actual APC distribution from VSL data (Table 2) using the Kolmogorov-Smirnov (K-S) test from R package 'stat' (R Core Team, 2022), which evaluated the null hypothesis that 2 populations (i.e., predicted and actual microbial concentration distribution) come from the same underlying distribution with a significance level of 0.05 (Wilcox, 2005).

Qian et al.: Predictive digital tool for milk spoilage
Validation 2 We subsequently proceeded to validation of the model described here with a new set of input data (Table 2), using previously described validation data (Lott et al., 2023).Briefly, in this study, 4 different sets of raw milk samples were obtained from a processing facility in Texas and heat treated using high-temperature short-time (HTST) pasteurization; pasteurized milk samples were then incubated at one of 3 temperatures (3, 6.5, and 10°C) for up to 42, 42,  and 17 d, respectively.Pasteurized milk samples held at both 3 and 6.5°C had been evaluated throughout shelflife on d 0, 7, 14, 21, 28, 35, and 42 whereas samples held at 10°C had been evaluated on d 0, 3, 5, 7, 9, 13, and 15 for APC as well as the presence of Gramnegative bacteria, using crystal violet tetrazolium agar.Different inputs used in validation 2 included (i) new AT frequency data (f AT_Val; See Supplementary Table 1) which were assembled based on ATs representing 121 sporeformer isolates that can grow in skim milk broth at 6°C and are associated with raw milk collected in TX, including (a) 91 isolates from Lott et al. ( 2023) and (b) 30 isolates obtained from raw milk collected on 22 Texas farms (Kent, unpublished), to represent a different diversity of psychrotolerant sporeformers in Texas milk and, and (ii) storage temperature (T Val ) and time (t Val ) (Table 2).However, the same raw milk initial spore concentration (N 0_Val ) was used because K-S test that compared the psychrotolerant spore count between previously collected raw milk samples from Texas (data not shown) and New York showed no significant difference (p-value = 0.17), suggesting the initial spore concentrations in raw milk samples from these 2 states have similar distributions.The model was used to simulate spore outgrowth for these replicated samples at 3, 6.5, and 10°C for various storage times to obtain the distribution of predicted sporeformer concentrations at the end of storage.Because APC in the validation data set also included background microflora assumed to show no growth at cold storage, we adjusted the predicted psychrotolerant sporeformer concentration by adding the concentration of background microflora, which was represented by the mean APC of all pasteurized milk samples on d 0 of storage, based on the assumption that APC on d 0 predominantly consisted of microflora other than psychrotolerant sporeformers; this assumption is justified by a previous study in NY as d 0 APC was 2.00 ± 1.00 log 10 cfu/mL, while psychrotolerant spore count of raw milk were usually below 0.01 MPN/ml (Masiello et al., 2017).The adjusted concentration distribution for each time and temperature combination was compared with the actual distribution from validation data collected under the same condition using boxplots.The root mean square error (RMSE) was selected as the performance metric with the error term quantified as the difference in mean concentration between the actual distribution and adjusted distribution at different storage temperatures for each storage duration.
Validation 3 The experimental data collected in this study were used to conduct validation 3. Input parameters for this validation (Table 2) included (i) the same set of raw milk initial spore concentrations (N 0_Val ) used in validation 1 and 2, (ii) the NY AT frequency (f AT_ Val ) (i.e., the same frequencies used for validation 1), and (iii) the set of storage temperature (T Val ) and time (t Val ) used in the experiment for generating this validation data set.The same sets of raw milk spore concentrations and AT frequency from validation 1 were used because milk samples used in both validation data sets were sourced from NY and thus were expected to have similar microbial quality and diversity.Using input parameters summarized above for model simulation, the predicted psychrotolerant sporeformer concentrations were then adjusted and validated following the same procedure as in validation 2.

Sensitivity Analysis
Two types of global sensitivity analysis, partial rank correlation coefficient (PRCC) (Marino et al., 2008) and conditional random forest (Antoniadis et al., 2021), were performed to rank the relative importance of input parameters (e.g., facility storage temperatures) on influencing whether or not spoilage (i.e., predicted psychrotolerant sporeformer concentration >20,000 cfu/mL) occurred at d 21.PRCC measured the monotonic relationship between each model parameter and the outcome (i.e., spoilage at d 21) after removing the effects from other model parameters.Conditional random forest captured the nonlinear relationship between input and output parameters and ranked the input parameters in terms of predictive power.The PRCC was conducted using the R package 'epiR' (Stevenson et al., 2022), while the conditional random forest was performed using the R package 'caret' (Kuhn et al., 2020).10-fold cross-validation was repeated 3 times for training the conditional random forest model.

What-If Scenarios
Two categories of what-if scenarios were assessed in this study.The first category assessed the common strategies to reduce spore concentration in the milk including microfiltration, singleand double-pass bactofugation.The second category included methods to control transportation or storage temperatures including setting up an alarm system, manually reducing the temperature, and improving the cooling system (to reduce temperature fluctuation) at different stages (see Table 3 for specific implementation of these scenarios).

Model Deployment and Data Availability
The model was developed in the R programming language (R Core Team, 2020) with the R package "shinydashboard" (Chang and Ribeiro, 2021).The local seed was set to 1 so that the same set of random numbers are generated from each distribution every time the code was executed to allow the consistent analysis of model results.The source codes as well as raw data used to construct the model are available at https: / / github .com/FSL -MQIP/ MC _MilkSporeModel.The developed model was deployed on an online server (https: / / www .shinyapps.io/ ) with a user-friendly interface (Figure 1) designed in R Shiny (http: / / shiny .rsturuo.com/ ) and is free to access at https: / / mqip .shinyapps.io/MilkSporeModel/ .This model application was designed to be fully reactive, customizable, and accessible (e.g., through different platforms including mobile) to reduce the technical difficulty in assessing milk spoilage.

Fluid Milk Shelf-Life Validation Data
In addition to 2 previously reported validation data sets used here (i.e., validation data sets 1 and 2), we also created a new set of validation data (validation data set 3) using pasteurized milk samples collected from 2 commercial fluid milk processing facilities.The milk samples obtained were used to prepare 48 unique test units (2 plants x 4 milk types x 3 storage temperatures x 2 replicates).From a total of 240 samples, 62 samples were removed and not tested for APC and total Gram-negative counts due to 2 consecutive testing days with evidence of post-pasteurization contamination on the same test unit (e.g., a 2% milk sample from facility A stored at 6°C was not tested on d 28 because samples of the same test unit showed presence of Gram-negative bacteria on d 14 and 21).For the remaining samples, the average APC on day initial was approximately 1,200 cfu/mL (representing 8 sample categories; 2 facilities x 4 milk types); average APC on day initial for facilities A and B were approximately 1,700 and 700 cfu/ml, respectively.Plating on crystal violet tetrazolium agar showed that 61.8% of samples (i.e., 110 out of 178 aliquots) had growth of Gram-negative organisms, which indicates post-pasteurization contamination.For samples stored at 4°C, 6°C and simulated supply chain temperatures, 72.4% (42/58), 63.3% (38/60), and 50.0%(30/60) showed growth of Gram-negative bacteria, respectively; these samples were excluded from the validation data set as they are not representative for growth of Gram-positive sporeformers.Hence 38.2% of samples (68/178) were retained to form the validation data set (see https: / / github .com/FSL -MQIP/ MC _MilkSporeModel/ blob/ main/ InputFiles/ ValidationData3 .csvfor a complete set of validation data).

Model Validations
After completion of the sporeformer spoilage model, we performed an initial validation (validation 1; see Figure 2) to compare the model reported here to a previous model (Buehler et al., 2018), which had also been validated with validation data set 1.For this validation, we compared the APC (which was assume to be dominated by psychrotolerant sporeforming bacteria) distribution after 14 d of storage at 6°C in commercial milk samples that are included in validation data set 1 with the model-simulated distribution obtained when the temperature was fixed at 6°C.The K-S test indicated no significant difference between the actual APC distribution and predicted sporeformer concentrations (P = 0.17), similar to the validation performed for the model reported by Buehler et al. (2018).Summary statistics confirmed these results as median observed and simulated concentrations at d 14 were very similar (2.65 log 10 cfu/mL and 2.73 log 10 cfu/mL, respectively), although the actual concentrations had a smaller interquartile range (0.71 log 10 cfu/mL) compared with the simulated concentrations (1.49 log 10 cfu/mL).
Validation 2 (Figure 3) was subsequently performed to evaluate the model performance with milk from a different source and incubated at different temperatures (i.e., storage temperatures of 3, 6.5, and 10°C).As milk samples for this validation were sourced from Texas (as opposed to NY for validation 1), validation 2 used, as one of the inputs, Texas-specific frequencies for different sporeformer subtypes (i.e., ATs) present in raw milk (Supplementary Table 1).The overall model performance was evaluated by RMSE, in which the error term was defined as the difference between the mean of predicted and actual concentration at different storage times.The calculated overall RMSE was 1.13 log 10 cfu/mL.Temperature-specific RMSEs were 0.54, 1.35, and 1.32 log 10 cfu/mL for 3, 6.5, and 10°C, respectively.When the model performance was evaluated for validation milk samples stored no more than 21 d, the RMSE results showed that the model performed substantially better at 3 and 6.5°C with RMSEs of 0.29 and 0.20 log 10 cfu/mL compared with 10°C with an RMSE of 1.32 log 10 cfu/mL (note that due to rapid spoilage at 10°C no samples stored at this temperature were tested past d 14).
Finally, we performed an additional validation to assess the ability of the model reported here to predict the outgrowth of sporeformers in both milk samples stored under static conditions (i.e., 4 and 6°C over the Qian et al.: Predictive digital tool for milk spoilage entire shelf life) and dynamic temperatures simulating temperature changes that could occur during storage and transportation of fluid milk from a processing facility to a consumer refrigerator (Figure 4).The overall RMSE across all 3 storage conditions was 1.22 log 10 cfu/ mL.For prediction of sporeformer concentrations up to d 35, temperature specific RMSEs were 0.34, 1.96, 0.71 log 10 cfu/mL for 4, 6°C and the simulated supply chain temperatures, respectively (note that at 4°C all milk samples stored later than 21 d were removed from validation data set due to post-pasteurization contamination).Considering that predictions of sporeformer concentrations might be less relevant beyond 21 d, we further evaluated the model performance for predictions within 21-d storage.The RMSEs for the sporeformer concentration distributions predicted at 4°C and simulated supply chain temperatures were 0.34 and 0.59 log 10 cfu/mL, respectively, compared with 1.43 log 10 cfu/mL at 6°C.

Model Prediction for Baseline Scenario
The baseline Monte Carlo simulation model, which simulated sporeformer outgrowth for HTST milk produced from raw milk typically for NY predicted that 28.6% and 44.3% of half-gallons were spoiled after 14 d and 21 d of storage, respectively (see Supplementary Figure 1).The model was simulated for 10,000 iterations using the initial spore concentration distribution and AT frequencies based on data collected in NY (the same distribution used for validation 1 and 3).The mean predicted psychrotolerant sporeformer concentration in the simulated half-gallons of milk was 2.33 log 10 cfu/mL on d 14 and 3.20 log 10 cfu/mL on d 21.The shelf life of milk, which was defined as the longest consumer storage time at which less than 50% of milk containers exceed 20,000 cfu/mL bacterial concentration, was predicted to be 25 d for the baseline model.

Sensitivity Analysis
The 2 sensitivity analyses (i.e., partial rank correlation coefficient and conditional random forest, see Figure 5) performed with the baseline model detailed above agreed that the top 4 most influential model parameters, which in order of influence, were consumer storage temperature (T H ), facility-to-retail transportation temperature (T T ), initial spore concentration in milk (N 0 ), and facility-to-retail transportation time (t T ).It is expected that T H is the most influential parameter as milk is simulated to be stored for substantially longer at the consumer's home (i.e., up to 35 d) as compared with other stages (e.g., facility-to-retail transport).Overall, the trained random forest model showed an accuracy of 0.82 and a kappa score of 0.64 when the hyperparameter "mtry" (i.e., the number of variables used in each split) was optimized at 6.The performance of the random forest model suggested that the importance rank of parameters was reliable.

What-If Scenarios
Results from what-if scenarios (Table 3) showed that, in general, methods to reduce spore levels in milk were predicted to be substantially more effective in reducing spoilage as compared with temperature control methods.In particular, microfiltration, which was assumed to achieve 2.2 log 10 cfu/mL reduction of spores was found to be the most effective intervention, as it was predicted to reduce spoilage on d 14 and 21 by 15.1 and 17.0 percentage points, respectively.However, at low efficacy (assumed to achieve 0.6 log reduction), microfiltration only provides marginal benefits as it was predicted to reduce spoilage on d 14 and 21 by 4.3 and 4.7 percentage points, respectively.Double-pass bactofugation was found to be the second most effective intervention strategy, as it was predicted to reduce spoilage on d 14 and 21 by 14.1 and 15.8 percent points, respectively.Microfiltration and double-pass bactofugation were predicted to extend the shelf life of milk from 25 d to 35 and 34 d, respectively (Table 3).In contrast, temperature control strategies were found to be less effective in reducing spoilage.The temperature control strategy predicted to be the most effective was to prevent the temperature from rising above 4°C during facility-toretail transportation, which could be achieved through the use of a temperature alarm system; 100% efficient truncation of facility-to-retail temperature to <4°C was predicted to reduce spoilage at d 14 and 21 by 6.5 and 6.6 percentage points, respectively.For temperature control strategies applied at facility and retail storage, strategies to reduce the mean temperature tend to reduce spoilage more effectively as compared with strategies that reduce temperature variability.For example, reducing the mean temperature at facility by 1°C was predicted to reduce spoilage on d 14 and 21 by 0.7 and 0.9 percentage points.By comparison, strategies that reduce temperature variation (simulated by reducing the standard deviation of the storage temperature by 50%) were predicted to reduce spoilage at d 14 and 21 by 0.0 and 0.1 percentage points, respectively; shelf-life extension by reducing the mean storage temperature could utilize an approach whereby improved cooling systems are installed at the processing and retail stages.

DISCUSSION
We substantially modified a previously developed Monte Carlo simulation model (Buehler et al., 2018) that can predict milk spoilage due to the growth of psychrotolerant sporeformers in the pasteurized milk to account for temperature variations in a supply chain system.Our model was initially validated by a previously reported data set from the NY fluid milk voluntary shelf-life program and further validated with 2 data sets from additional storage studies.Overall, our data show that (i) 28.6% and 44.3% containers of half-gallon milk were simulated to be spoiled after 14 d and 21 d of consumer storage, (ii) consumer storage temperature (T H ), facility-to-retail transportation temperature (T T ), initial spore concentration in milk (N 0 ), and facility-to-retail transportation time (t T ) are top 4 model parameters identified by sensitivity analyses, and (iii) microfiltration was predicted to be the most effective strategy to reduce spoilage.

Consumer Storage Temperature Is Key to Reducing Spoilage
Both sensitivity analyses conducted in this study ranked the temperature at consumer storage as the most important model parameter.This is unsurprising as the 21-d duration of consumer storage (as sensitivity analysis used milk's spoilage status at d 21 as the outcome variable) is longer than the average durations of other storage stages during the supply chain, including facility storage (1.5 d) and retail storage (1.82 d).Our finding is consistent with several previous studies which suggested home refrigeration temperature has an impact on the shelf life of food products, including fruits and cooked meat products (Jung et al., 1996;Brown et al., 2014;Jofré et al., 2019).Specifically in the context of fluid milk, lowering temperature has been demonstrated to be able to extend the shelf life by minimizing the growth of psychrotolerant Gram-negative bacteria (Chandler and Mcmeekin, 1985;Schaffner et al., 2003).Home storage temperature has also been identified as an influential parameter in several quantitative microbial risk assessments (QMRA) conducted for dairy products.For example, a risk assessment conducted for Staphylococcus aureus in natural and processed cheese in South Korea (Lee et al., 2015) showed a strong relationship between home storage temperature and risk of illness, which is directly related to microbial growth.Our previous model (Buehler et al., 2018) also predicted that by reducing the storage temperature from 6 to 4°C, the percentage of spoiled milk at d 21 can be reduced from 66% to 9%, suggesting a substantial impact of lowering storage temperature.The shelf life is defined as the last date on which at least 50% of containers of milk have sporeformer concentrations less than Pasteurized Milk Ordinance (PMO) limit (20,000 cfu/mL).This limit as well as percentage value are subject to change based on users' own management preference. 2 The baseline scenario does not include any intervention strategies and assumes no changes to the temperature distribution.
3 N/A = Not applicable.
Many factors can impact refrigeration temperature.For example, a previous study (Evans and Redmond, 2016) that investigated time and temperature profiles indicated that room temperature and reported door opening frequency are both significantly correlated (pvalue <0.05) with the refrigeration temperature.On top of that, refrigeration temperature can be influenced by temperature fluctuations due to cooling circles, the position of food storage within the refrigerator, and seasons (Jofré et al., 2019).All these factors can increase the variability of refrigeration temperature, which suggests that modeling microbial growth during consumer storage using a single static temperature might not be sufficient.To enhance the model utility in the real world, temperature fluctuations in time series should be integrated into the model to have a more precise and real-time estimate of milk's shelf life.The realization of such a model needs technological advancement in refrigerator design, such as real-time temperature recording.The model developed here can be potentially embedded in the advanced refrigerator coupled with a temperature logger to optimize the shelf life of milk in real time.For example, a model developed by Fu et al. (1991) was shown to have the potential to predict the growth of Pseudomonas fragi in milk stored in flasks with data from time-temperature integrators.Other technological improvements that enable more precise temperature control in different spaces within a refrigerator can also be promising for the shelf-life extension.For example, one study showed that using a 5-compartment fridge can optimize the storage temperature of different food products, including dairy, and therefore extend the overall shelf life and prevent food waste (Holsteijn and Kemna, 2018).
In addition to technological solutions to prevent milk spoilage, consumer education to drive behavioral change should also be emphasized as consumer's knowledge and attitude associated with recommended A dashboard web design that consists of 3 tabs, which are "Introduction," "Model" and "Output."The homepage (i.e., the "Introduction" tab) provides general information about the model.The left panel under the "Model" tab is the main layout of the model application interface, featuring 3 sections: menu, user input, and result visualization.The user input section allows the user to submit key model parameters, including mean and standard deviation of spore concentration in milk, spoilage threshold, and the number of days to simulate during the consumer storage.Furthermore, intervention strategies described in this study can be selected in the check box for users to simulate their effects on reducing the percent spoilage.The simulated proportion of spoiled half-gallon milk containers will be displayed on the right panel, and associated spoilage data are downloadable following the instructions under the "Output" tab.refrigeration temperature can directly influence their behaviors toward managing actual refrigeration temperature.A review paper (Redmond and Griffith, 2003) indicated that only 40-56% of US consumers claimed they are aware of proper refrigeration temperature.In other countries and regions, data from early 2000 to 2015 showed that 15-88% of respondents did not know the recommended temperature for refrigeration while 48-87.9% of respondents did not know the actual temperature in their fridges (James et al., 2017).Data from the same country (i.e., the United Kingdom) showed no improvement in knowledge over the years (Prior et al., 2011(Prior et al., , 2013)).lack of knowledge could potentially lead to negligence of elevated temperatures that shorten the shelf life of perishable foods and HTST milk.In addition, consumers need to be mindful of where to store perishable food products as the temperature within the refrigerator depends on the location of temperature measurement.A study (Godwin et al., 2007) that collected the temperature profiles in 200 US homes showed that for 66% of refrigerators, the temperature at the door was over the recommended temperature (4.4°C) for longer than 8 h per day, suggesting that storing milk in the refrigerator door can increase the likelihood of spoilage.
Following the consumer storage temperature, facilityto-retail transportation temperature and time were identified as the 2nd and 4th most important model parameters.The relationship between these 2 variables and spoilage (i.e., microbial growth) was stronger in our study than some other risk assessments for dairy products, including a QMRA for Listeria monocytogenes in Canadian soft cheese in which Spearman's rank correlation coefficients for transportation temperature and time were 0.06 and 0.05 (FDA and Health Canada, 2015).This might be because (i) a different methodology was used, and (ii) the variable significance was diluted by more variables included in that QMRA study, such as more variability in serving size and time of aging.In addition, while most risk assessments included transportation temperature and time from facility to retail as model parameters, some did not include them in sensitivity analysis, and therefore the relative importance of these 2 parameters was not evaluated (Lee et al., 2015;Ding et al., 2016).However, our finding is consistent with the observation that the distribution of food products is less controlled by processors and thus more likely to fall outside the optimal condition (Koutsoumanis and Gougouli, 2015).To further improve the control of transportation conditions, precise tempera- ture measurements via temperature data loggers can be implemented during facility-to-retail transportation in combination with our model to enable a real-time estimation of milk quality.This is particularly important during extreme weather conditions (e.g., hot days in summer) so that products that undergo temperature abuse can be re-located and properly managed.
In addition to the 3 important parameters mentioned above, the initial spore concentration was also found to be among the top 4 influential factors.The importance of the initial bacterial load has also been demonstrated Qian et al.: Predictive digital tool for milk spoilage Figure 3. Results from validation 2, comparing the actual APC (white boxes; containing 268 data points) and simulated psychrotolerant sporeformer concentrations (gray boxes; containing 268,000 data points) for HTST pasteurized milk (produced from raw milk procured in Texas) stored at 3, 6.5, and 10°C.The regulatory limit designated by Pasteurized Milk Ordinance (PMO) is used as the spoilage threshold and indicated by the dotted line.Actual concentrations below the detection limit (i.e., < 10 cfu/ml) were assigned as 2.5 cfu/ml (i.e., 25% of the detection limit) before log 10 transformation.In each boxplot, the middle line is the median value.The bottom and top box boundaries represent the first and third quantiles, respectively, while the end of the whiskers represents 1.5x the interquartile range beyond the quantiles.Data points presented as dots outside the whiskers are outliers.
in multiple microbial spoilage assessments conducted on dairy foods (Buehler et al., 2018;Lau et al., 2022;Qian et al., 2022).The sensitivity analysis performed by Buehler et al. (2018) showed that reducing the initial spore concentration by 2 log can lower the mean sporeformer concentration at day-21 storage from 4.54 ± 1.71 to 3.08 ± 1.83 log 10 cfu/mL.In a Monte Carlo simulation model that predicts milk spoilage due to post-pasteurization contamination, Lau et al. (2022) reported that an increase in initial Gram-negative bacteria level by 60% can result in a percentage point increase of spoiled milk by 8.9 and 7.2 on d 7 and 10 respectively.Note that a 60% increase is a small amount of change considering the original mean initial concentration is 0.38 log 10 cfu/mL.Overall, the importance of initial spore concentration suggests we need to better characterize its variability in the raw milk supply between facilities and from lot to lot to improve model predictions.

Spore Reduction Strategies Are More Effective at Reducing Fluid Milk Spoilage Than Temperature Control Strategies Before Consumer Storage
Our what-if scenario results showed that 3 scenarios associated with reducing initial spore concentration were the top 3 most effective strategies for reducing fluid milk spoilage.The effectiveness of spore reduction techniques in reducing spoilage and extending the shelf life of dairy products has been supported by both previous experiments (Hoffmann et al., 2006;Schmidt et al., 2012;Doll et al., 2017) and models (Buehler et al., 2018;Griep-Moyer et al., 2022).For example, Buehler et al. (2018) reported that a Monte Carlo simulation model for milk stored at a single temperature predicted that the percentage of milk spoilage was reduced from 66% to 13% when microfiltration was implemented, a magnitude of reduction (53 percentage points) that is higher than our predictions (17 percentage points).One explanation could be that in a more complex supply chain, such as the one presented in our study, the effectiveness of microfiltration could be potentially re- duced due to the impact of other abnormal conditions along the supply chain, such as storage temperatures higher than recommended temperature.In another study (Griep-Moyer et al., 2022), the same Monte Carlo simulation model reported by Buehler et al. (2018) predicted that implementing one-phase bactofugation prolonged the shelf life of pasteurized skim milk stored at 6°C by approximately 3 d; in this study, the shelf life was defined as the time by which 50% containers showed psychrotolerant sporeformer concentrations >20,000 cfu/mL.In comparison, our model predicted that single-pass bactofugation can extend shelf life by 6 d.The discrepancy between shelf-life extension can be attributed to the difference in the effectiveness of bactofugation to reduce psychrotolerant spores.In the Griep et al. (2022) study, psychrotolerant spore count was only reduced from 0.88 ± 0.49 to 0.63 ± 0.47 log 10 MPN/mL as compared with 1.4 log reduction efficiency (Gésan-Guiziou, 2010) used in our study.This suggests that the reduction efficiency of bactofugation might be highly variable at different initial spore levels.In addition, the variability can also source from different equipment and specifications used, and therefore implementation of bactofugation in the model scenarios should be refined in future studies to accommodate this complexity.Similarly, the efficacy of microfiltration can also vary from batch to batch, potentially ranging from 0.6 to 3.1 log reduction with the average efficacy of 2.1 log reduction, as shown by a previous study (Doll et al., 2017).This variability can have a substantial impact on shelf-life extension as model predictions in what-if scenarios showed that low-efficacy microfiltration (assumed to achieve 0.6 log reduction) can only extend the shelf-life of fluid milk by 2 d, which are 8 d less than the microfiltration with an expected 2.2 log reduction efficacy.Thus, it is advised that a processing facility should evaluate the variability in the efficacy of spore reduction intervention as part of decision-making process.In addition to the variation in reduction efficiency, factors such as financial budget, operation size, and targeted shelf life also need to be taken into account when deciding intervention strategies.To address this multifaceted issue, an optimization model has been developed which can help processors optimize interventions depending on the specific situations and consumer preferences for length of shelf life (Enayaty-Ahangar et al., 2021).
Although what-if scenarios suggested spore removal strategies were generally more effective than temperature control strategies, we found that in sensitivity analyses, consumer storage temperature and facilityto-retail transportation temperature were ranked more important than initial spore concentration.These 2 findings might seem counterintuitive but are in fact not contradictory.First, we did not evaluate the consumer storage temperature in our what-if scenarios because it is impractical for dairy processors to implement control strategies that impact consumer storage temperature.Second, our model implementation of temperature control strategies at transportation only removed milk transported at temperatures higher than 6°C while the spore removal strategy was applied to all milk, regardless of their initial spore concentrations.Thus, despite being ranked higher in sensitivity analysis in the model, facility-to-retail transportation temperature was only controlled in the case of extreme temperature conditions (simulated by the model), which are infrequent, and thus the corresponding control strategy was less effective compared with spore reduction strategies.This is consistent with a spoilage assessment for late-blowing defects in Gouda cheese, in which spore reduction was shown more effective in minimizing bacterial growth (and thus defects) as compared with lowering the ripening temperature (Qian et al., 2022).Nonetheless, our data still indicate that controlling temperature during facility-to-retail transportation was more effective as compared with temperature control at other stages of the supply chain, which is consistent with our sensitivity analysis.

Deployable Digital Tools are Needed for Dairy Product Spoilage Management
Currently, while there are multiple digital decisionsupport tools available on the farm level for cow health monitoring, economic management, and animal welfare (Cabrera, 2018;Cabrera et al., 2020;Ferris et al., 2020) and on the processor level for quality control (e.g., compositional analysis) (Pu et al., 2020), the dairy industry still lags in digital technologies that can predict the microbial spoilage and facilitate risk management.The reasons for the lack of digital tools in this area could be multifaceted, including but not limited to (i) sensitivity of microbial data, (ii) technological difficulty to rapidly detect and quantify spoilage microorganisms, (iii) high variability and uncertainty of microbiological data that can potentially limit the application performance.To address this lack of digital tools, in addition to the functional model, we further developed a customizable and user-friendly model interface based on the practical situations (e.g., different sources and microbial quality of milk) for different dairy processors.We believe this could be a starting point to drive more innovations in digital tools and to integrate databases and data sharing systems that would allow for continuous improvement of the model.

Challenges and Limitations
One of the challenges during model development is modeling microbial adaptation to a new environment (i.e., a new temperature condition).Based on multiple previous studies that applied different mathematical expressions to predict microbial growth in a dynamic environment (Fu et al., 1991;Baranyi and Roberts, 1994;Zwietering et al., 1994;Baranyi et al., 1995;Van Impe et al., 1995;Koutsoumanis et al., 2006), we could generally divide modeling microbial growth into explicit and dynamic approaches.This is based on whether, in the primary growth model, the lag phase duration is explicitly defined and used as a model parameter.In general, an explicit growth model (e.g., a 3-phase linear model [Zwietering et al., 1994]) would require adding more rules to adjust lag time duration during temperature shifts.Different from the explicit approach, a dynamic approach, which was initially proposed by Baranyi and Roberts (1994), does not explicitly model the lag phase duration; the lag phase duration instead is only defined as inversely proportional to the maximum growth rate.This theory for Baranyi's approach was later evaluated in a dynamic model for predicting the growth of the spoilage microorganism Brochothrix thermosphacta on agar (Baranyi et al., 1995).Adopting the same approach, one study (Koutsoumanis et al., 2006) developed a growth model from growth data under different temperatures and pH and predicted Pseudomonas growth in ground meat under fluctuating temperatures between 0 and 24°C, suggesting the potential for real-world applications to predict meat shelf-life.In both approaches, however, the assumption was made that the specific growth rate changes instantaneously in a new environment.In our study, we applied a modified approach from Zwietering et al. (1994) as our primary growth model was developed based on an explicit primary growth model.Applying Baranyi's approach in our study would have required re-fitting a dynamic growth model to previously collected growth data.To sufficiently validate a dynamic model would require continuously collecting temperature data via temperature loggers, which could be a future area of improvement.
Another challenge in predicting the spoilage in milk is the lack of a clear definition for spoilage.While in this model we defined "spoilage" as exceeding regulatory limit (i.e., 20,000 cfu/mL), previous literature (Carey et al., 2005;Martin et al., 2012) suggested that sensory defects are associated with a higher bacterial concentration (i.e., 1,000,000 cfu/mL).We addressed this issue by including both threshold levels in the user interface so that model users can make their own decisions.In the meantime, we acknowledge that it is important to develop a dose-response relationship between the total bacteria count and the likelihood of spoilage to develop a 2-dimensional Monte-Carlo simulation model that can include both variability and uncertainty in the prediction.Due to the lack of a dose-response curve, the current model outcome only includes variability in the predicted microbial concentrations, and therefore the model cannot estimate the uncertainty around the percentage of milk containers exceeding the regulatory limit.
One limitation of our model is the inability to predict milk spoilage due to post-pasteurization contamination; due to this a large proportion of milk samples that showed presence of total Gram-negative bacteria results had to be removed from our validation data set 2 (Lott et al., 2023) and 3 (see Materials and Methods above).A more comprehensive model that can simultaneously predict spoilage due to both post-pasteurization contamination and psychrotolerant sporeformers should be developed to address this limitation.Nonetheless, the current model still provides value in evaluating strategies to minimize the spoilage due to psychrotolerant sporeformers.

CONCLUSION
The model developed in this study can be valuable for stakeholders along the supply chain of fluid milk to make decisions with respect to control strategies to minimize spoilage due to psychrotolerant sporeformers.This model is flexible for customization depending on the specific user's temperature and time distributions at any stage of the supply chain as well as the initial spore level of HTST pasteurized fluid milk.To extend the utility of the developed model, we deployed this model through R shiny (accessible at https: / / mqip .shinyapps.io/MilkSporeModel/ ) to make it publicly available to dairy processors.Overall, this work provides a promising digital tool that can facilitate rational management decision-making.

ACKNOWLEDGMENTS
This work was funded by Allied Milk Producers' Cooperative, Inc through project #86959.This project was also in part supported by a grant from the Foundation for Food & Agriculture Research (FFAR, Washington, DC; award no.CA18-SS-0000000206) and by a grant to Rochester Institute of Technology's NYS Pollution Prevention Institute from the Environmental Protection Fund as administered by the NYS Department of Environmental Conservation (Subcontract: A5183-02).Any opinions, findings, conclusions, or recommendations expressed are those of the author(s) and the model is represented by a crate of 10 milk containers. 2 Milk containers within the same lot were assigned with the same AT. 3 based on the range of time and temperature from a commercial milk processing milk facility (T.T. Lott, unpublished data).4 In a TruncNormal (a, b, c, d) distribution, a = min, b = max, c = mean and d = standard deviation.5 In a TruncLaplace (a, b, c, d) distribution, a = min, b = max, c = location parameter and d = dispersion.
Qian et al.: Predictive digital tool for milk spoilage Qian et al.:  Predictive digital tool for milk spoilage Table3.What-if scenarios and their simulated effects on reducing spoilage and extending the shelf life of pasteurized milk Figure1.A dashboard web design that consists of 3 tabs, which are "Introduction," "Model" and "Output."The homepage (i.e., the "Introduction" tab) provides general information about the model.The left panel under the "Model" tab is the main layout of the model application interface, featuring 3 sections: menu, user input, and result visualization.The user input section allows the user to submit key model parameters, including mean and standard deviation of spore concentration in milk, spoilage threshold, and the number of days to simulate during the consumer storage.Furthermore, intervention strategies described in this study can be selected in the check box for users to simulate their effects on reducing the percent spoilage.The simulated proportion of spoiled half-gallon milk containers will be displayed on the right panel, and associated spoilage data are downloadable following the instructions under the "Output" tab.

Figure 2 .
Figure 2. Results from validation 1 with the boxplot (top) and empirical cumulative density plot (bottom) representing simulated psychrotolerant sporeformer concentrations (n = 1,000) and actual APC that was obtained from New York milk (n = 30) collected from multiple commercial fluid milk processing facilities plants via Cornell Voluntary Shelf-life Program (VSL) and stored at single static temperature of 6°C for 14 d.In each boxplot, the middle line is the median value.The left and right box boundaries represent the first and third quantiles, respectively, while the end of the whiskers represents 1.5x the interquartile range beyond the quantiles.Data points presented as dots outside the whiskers are outliers.

Figure 4 .
Figure 4. Results from validation 3, comparing the actual APC (white boxes; containing 68 data points) and simulated psychrotolerant sporeformer concentrations (gray boxes; containing 68,000 data points) for milk (procured from 2 New York fluid milk processing facilities) stored at 4°C, 6°C and simulated supply chain temperature (i.e., 4°C for 36 h, 4°C for 5 h, 2°C for 24 h, 10°C for 26 min, and 4°C until the end of storage).The regulatory limit designated by Pasteurized Milk Ordinance (PMO) is used as the spoilage threshold and indicated by the dotted line.In each boxplot, the middle line is the median value.The bottom and top box boundaries represent the first and third quantiles, respectively, while the end of the whiskers represents 1.5x the interquartile range beyond the quantiles.Data points presented as dots outside the whiskers are outliers.

Figure 5 .
Figure 5. Sensitivity analysis for evaluating the impact of key model parameters on milk spoilage status (i.e., with bacterial concentration >20,000 cfu/mL) on day-21 of consumer storage.(Top) The partial rank correlation coefficient.(Bottom) variable importance plot of conditional random forest.
Qian et al.: Predictive digital tool for milk spoilage Qian et al.: Predictive digital tool for milk spoilage do not necessarily reflect the views of the Allied Milk Producers' Cooperative, the Foundation for Food & Agriculture Research, the Rochester Institute of Technology, and its NYS Pollution Prevention Institute, or the NYS Department of Environmental Conservation.

Table 1 .
Qian et al.:Predictive digital tool for milk spoilage Summary of variables used in the model with the baseline scenario

Table 2 .
Qian et al.: Predictive digital tool for milk spoilage Summary of model inputs used in model validations Step 1Cornell voluntary shelf life (VSL) program that collects commercial milk samples for quality assessment.2Simulated supply chain temperatures: 4°C for 36 h, 4°C for 5 h, 2°C for 24 h, 10°C for 26 min, and 4°C until the end of storage.