ABSTRACT
Study objectives were to (1) describe the diagnostic test characteristics of an automated milk leukocyte differential (MLD) test and the California Mastitis Test (CMT) to identify intramammary infection (IMI) in early- (EL) and late-lactation (LL) quarters and cows when using 3 different approaches to define IMI from milk culture, and (2) describe the repeatability of MLD test results at both the quarter and cow level. Eighty-six EL and 90 LL Holstein cows were sampled from 3 Midwest herds. Quarter milk samples were collected for a cow-side CMT test, milk culture, and MLD testing. Quarter IMI status was defined by 3 methods: culture of a single milk sample, culture of duplicate samples with parallel interpretation, and culture of duplicate samples with serial interpretation. The MLD testing was completed in duplicate within 8 h of sample collection; MLD results (positive/negative) were reported at each possible threshold setting (1–18 for EL; 1–12 for LL) and CMT results (positive/negative) were reported at each possible cut-points (trace, ≥1, ≥2, or 3). We created 2 × 2 tables to compare MLD and CMT results to milk culture, at both the quarter and cow level, when using each of 3 different definitions of IMI as the referent test. Paired MLD test results were compared with evaluate repeatability. The MLD test showed excellent repeatability. The choice of definition of IMI from milk culture had minor effects on estimates of MLD and CMT test characteristics. For EL samples, when interpreting MLD and CMT results at the quarter level, and regardless of the referent test used, both tests had low sensitivity (MLD = 11.7–39.1%; CMT = 0–52.2%) but good to very good specificity (MLD = 82.1–95.2%; CMT = 68.1–100%), depending on the cut-point used. Sensitivity improved slightly if diagnosis was interpreted at the cow level (MLD = 25.6–56.4%; CMT = 0–72.2%), though specificity generally declined (MLD = 61.8–100%; CMT = 25.0–100%) depending on the cut-point used. For LL samples, when interpreted at the quarter level, both tests had variable sensitivity (MLD = 46.6–84.8%; CMT = 9.6–72.7%) and variable specificity (MLD = 59.2–79.8%; CMT = 52.5–97.3%), depending on the cut-point used. Test sensitivity improved if interpreted at the cow level (MLD = 59.6–86.4%; CMT = 19.1–86.4%), though specificity declined (MLD = 32.4–56.8%; CMT = 14.3–92.3%). Producers considering adopting either test for LL or EL screening programs will need to carefully consider the goals and priorities of the program (e.g., whether to prioritize test sensitivity or specificity) when deciding on the level of interpretation (quarter or cow) and when selecting the optimal cut-point for interpreting test results. Additional validation studies and large randomized field studies will be needed to evaluate the effect of adopting either test in selective dry cow therapy or fresh cow screening programs on udder health, antibiotic use, and economics.
Key words
INTRODUCTION
With increased pressure to reduce antibiotic (Ab) use in food animals, and because the majority of Ab use on dairy farms is for the prevention or treatment of mastitis, the dairy industry must evaluate how we use Ab and, where possible, develop and adopt new mastitis control practices that can reduce Ab use while still maintaining or improving udder health, productivity, and animal wellbeing. One opportunity to reduce Ab use on dairy farms is the adoption of selective dry cow therapy (SDCT) programs, whereby Ab treatment of either the cow or individual quarters at the end of lactation is based on assessment of the IMI status of the cow or quarter, respectively. As the presence of IMI at calving is associated with increased risk of clinical mastitis in that lactation (
Green et al., 2002
), then another potential opportunity to enhance udder health could be to identify and apply selective treatment to individual quarters or cows with subclinical IMI very soon after calving, although it should be noted that the practicality and cost-benefit of implementing such programs requires investigation. However, to be successful, SDCT or fresh cow screening and treatment programs will require use of diagnostic tests with sufficient accuracy and utility (e.g., speed, cost, ease of use) to identify infected quarters or cows. Whereas both high diagnostic sensitivity (Se) and specificity (Sp) would be desirable, we can argue that SDCT programs will need to prioritize high Se to identify infected quarters or cows that will benefit from Ab treatment; conversely, fresh cow screening programs may want to prioritize high Sp to avoid false positives that would result in unnecessary use of Ab and associated costs such as increased discarded milk.Although milk culture is recognized as the gold standard to determine IMI status, the cost and timeliness of submitting samples to reference laboratories for milk culture may be a deterrent for some farms to adopt this approach for routine dry or fresh cow screening programs. Options for on-farm diagnostic tests to diagnose IMI at either the quarter or cow level could include direct tests, such as rapid on-farm culture (OFC) of milk, or indirect tests, which typically measure inflammatory cells, enzymes, or other products of inflammation in milk that are often associated with the presence of bacterial infection. Two recent studies of SDCT programs that used rapid OFC systems to guide either cow- or quarter-level treatment decisions were successful in that authors reported equal udder health in cows assigned to the SDCT and BDCT groups, and reduced Ab use by 21% (cow-level program) or 48% (quarter-level program), respectively (
Cameron et al., 2014
; Godden et al., 2016
). For the former study, the reported Se and Sp of the Petrifilm-based OFC system (3M, Canada, London, ON, Canada) when culturing cow-level composite milk samples were 85.2 and 73.2%, respectively (Cameron et al., 2014
). For the latter study, the Se and Sp of the MN Easy 4Cast (University of Minnesota, St. Paul, MN) when culturing quarter milk samples were 82.4 and 73.2%, respectively (Godden et al., 2016
). However, and despite the success of using OFC in these 2 SDCT programs, it is acknowledged that adopting OFC systems may not be convenient or easy to implement for some farms. As such, interest is growing in evaluating the performance of indirect tests, such as SCC or the California Mastitis Test (CMT), for use in SDCT or fresh cow screening programs.One recent study, using the composite SCC results from the last 3 mo with a cut-off value of 200,000 cells/mL and clinical mastitis history, yielded a Se of 69.7% and a Sp of 62.4% to identify infected cows at dry off, as compared with microbial culture of milk on day of dry off (
Torres et al., 2008
). This lower Se could result in many infected quarters going undetected and untreated in a SDCT program. Another option, the CMT, is a simple, rapid, and inexpensive cow-side test that indirectly indicates the level of SCC in milk (Schalm and Noorlander, 1957
). The CMT can be performed quickly at cow-side and costs approximately $0.15/cow (considers CMT solution and paddle) plus labor. It works by using a reagent that disrupts the cell membrane of somatic cells present in a milk sample; the DNA in those cells then reacts with the test agent to form a gel. The degree of gelling is scored as negative, trace, 1, 2 or 3, with the score being positively associated with SCC in the sample. Studies have reported that the Se of the CMT test to identify infected quarters or infected cows is less than 57 and 70%, respectively (Sargeant et al., 2001
; Middleton et al., 2004
; - Middleton J.R.
- Hardin D.
- Steevens B.
- Randle R.
- Tyler J.
Use of somatic cell counts and California Mastitis Test results from individual quarter milk samples to detect subclinical intramammary infection in dairy cattle from a herd with a high bulk tank somatic cell count.
J. Am. Vet. Med. Assoc. 2004; 224: 419-423
Sanford et al., 2006
). Similar to the SCC test, a concern with the CMT is that the lack of Se could result in many infected quarters going undetected and untreated in a SDCT program.A new testing option, a commercial automated milk leukocyte differential (MLD) test (QScout MLD. AAD, Durham, NC) has recently been introduced and is being promoted for use in SDCT or early lactation mastitis-screening programs. This test works on the principle that measures of different leukocytes in milk, especially the phagocytes (e.g., neutrophils and macrophages), can be incorporated with total SCC to indicate the presence and stage of disease (
Dohoo et al., 1981
; Leitner et al., 2003
). For the automated MLD test, fresh milk from 4 quarters is applied to each of 4 quadrants of a single-use cassette. The cassette is then loaded into an automated reading device that uses a fluorescence-imaging system to take images to identify and count lymphocytes, neutrophils, and macrophages, and then assesses the information based on an index of infection and provide a test result (positive or negative) for each individual quarter. After approximately 3 min (smart mode) or 15 min (research mode), the system reports an answer in the form of positive, borderline positive, borderline negative, or negative for each quarter. The MLD test allows the producer to select which class of animal to program the test for, including early-lactation (EL) or late-lactation (LL) cows. Within each class of animal, it is possible for the user to select different threshold settings to declare a quarter positive or negative, with settings options on a scale of 1 to 18 for the EL index or on a scale of 1 to 12 for the LL index. Results may be interpreted at the quarter or cow level. The manufacturer currently recommends using a setting of 12 and 6 for the EL and LL index, respectively, based on internal calculations suggesting that these thresholds will optimize return on investment if that cow is treated. Costs for the MLD test are approximately $18,000 for the reader, $5.00 per cow for the cassette, plus the cost of labor for sampling and running the test. Also, samples would need to be collected at least 1 milking before dry off to allow time for testing. Although the MLD test has potential as a rapid on-farm diagnostic test, studies are needed to describe the MLD diagnostic test characteristics to identifying infected quarters or cows.The process of evaluating a new diagnostic test is typically done by comparing results of the new test against those of a gold standard or accepted reference test. For clinical or subclinical mastitis, microbial culture of milk is the accepted gold standard for determining IMI status; however, researchers may use different approaches to sampling or interpreting milk culture results when defining IMI status. For example,
Andersen et al., 2010
suggest that culture results from duplicate milk samples should undergo series interpretation, whereby both samples must show concordant results (i.e., growth of the same pathogen in both samples) to declare IMI present. Other researchers have used an approach similar to parallel interpretation of culture results from duplicate milk samples, whereby IMI is diagnosed if a pathogen is present in 1 or both milk samples, provided that pathogen is present at high enough concentrations (Pinzón-Sánchez and Ruegg, 2011
). A third approach is to use culture of a single milk sample; Dohoo et al., 2011a
reported that the relative Se of using culture results from a single milk sample to detect IMI (as compared with 3 consecutive samples) varied by pathogen, but was less than 90% for all species except Staphylococcus aureus. With the exception of CNS, Sp exceeded 94% for most species. Based on their results, Dohoo et al., 2011a
recommended that when using culture of a 10-μL plated sample volume from a single milk sample to determine IMI status, 1 or more colonies could be considered significant for any organism with the exception of CNS, for which 2 or more colonies were recommended to declare IMI present. In a separate study comparing the relative test characteristics when using different definitions of IMI in quarters, Dohoo et al., 2011b
reported that series interpretation of duplicate or consecutive samples yielded the highest Sp but lowest Se, whereas parallel interpretation of duplicate samples collected at the same time, or consecutive samples collected at different milkings resulted in the highest Se but lowest Sp. Overall, triplicate samples provided the best combination of Se and Sp but, compared with a single sample, provided only a modest gain in Sp and little or no gain in Se. Clearly, an investigator's choice of how to define quarter IMI status from milk culture has the potential to affect the estimated test characteristics of a new diagnostic test being compared against milk culture results. However, studies investigating this question are lacking. Objectives of the current study were to (1) describe the diagnostic test characteristics of an automated MLD test (QScout MLD, AAD) and the CMT to identify IMI in EL and LL quarters and cows when using 3 different approaches to define IMI status from milk culture, and (2) describe the repeatability of MLD test results at the quarter and cow level.MATERIALS AND METHODS
Study Herds
Three commercial Holstein herds from western Wisconsin participated in the study. Early-lactation cows were enrolled from a transition cow facility that housed approximately 300 late-gestation (dry) cows and heifers and approximately 145 early-lactating cows between 1 and 21 DIM (herd A). Cows in herd A were housed in a freestall barn using deep-bedded recycled digested manure solids. Cows were milked 2× in a single 12 BouMatic parallel parlor (BouMatic, Madison, WI). Average DIM, milk yield, and bulk tank SCC for this herd was 11 d, 29.5 kg/cow per day, and 390,000 cells/mL, respectively. Late-lactation cows were enrolled from 2 commercial dairy herds milking 1,304 (herd B) and 1,208 (herd C) cows. Cows in herd B were housed in a freestall barn using mattresses bedded with recycled digested manure solids, and were milked 3× in a double 17 Boumatic parallel parlor. Cows in herd C were housed in a freestall barn using deep-bedded recycled digested manure solids, and were milked 3× in a double 20 Boumatic parallel parlor. Average DIM, milk yield, and bulk tank SCC were 251 d, 38.2 kg/cow per day, and 333,000 cells/mL for herd B, and 234 d, 38.6 kg/cow per day, and 343,000 cells/mL for herd C, respectively. Routine bulk tank culture analysis conducted using samples collected from each herd on 3 consecutive days during the study period indicated that herd A (EL cows) was negative for Staphylococcus aureus and Mycoplasma spp. However, herd B (LL cows) was positive for both Staphylococcus aureus and Mycoplasma spp., and herd C (LL cows) was negative for Staphylococcus aureus but positive for Mycoplasma spp.
Cow Enrollment and Milk Sample Collection
Animal sampling activities were approved by the Institutional Animal Care and Use Committee, University of Minnesota. All milk sampling activities were conducted by University of Minnesota technicians and the principal investigator between January 4 and 12, 2016. For EL cows to be eligible for inclusion, cows had to be between 7 and 21 DIM and must not have been treated with systemic or intramammary drugs during the previous 14 d. For LL cows to be eligible for inclusion, cows had to be at least 275 DIM and must not have been treated with systemic or intramammary drugs during the previous 14 d. Quarters with active clinical mastitis (visibly abnormal milk) and blind quarters were noted but not sampled. Milk sample collection procedures began as cows entered the parlor for regular morning milking and underwent the regular udder prep routine by parlor staff (i.e., forestrip onto the floor, predip with a premilking teat disinfectant and leave on for at least 30 s, wipe dry). Following udder preparation, but before milking unit attachment, study technicians wearing clean, disposable gloves collected milk samples using the following sequence of events for each cow.
(A) Clinical Mastitis Screening and CMT Testing
First, a stripping sample of milk from each quarter was collected into the 4 wells of the CMT paddle If the milk in the paddle was visibly abnormal before adding the CMT reagent, then that quarter was identified as having clinical mastitis and was not sampled further. One of 2 trained technicians performed the CMT test and recorded results as negative (no thickening of mixture), trace (slight thickening), 1 (distinct thickening but no tendency to form a gel), 2 (immediate thickening, with a slight gel formation), or 3 (gel is formed and surface of the mixture becomes elevated;
Ruegg, 2005
). Finally, the CMT paddle was rinsed off with water between each test.(B) Milk Sample Collection for MLD Testing
A single stripping sample of milk from each eligible quarter was collected into each of the 4 wells of the Q4 QScout milk collection device, filling it to the fill line indicated on the device. The Q4 milk collection device was then capped, labeled (cow identification), and placed in an unchilled container at room temperature
(C) Milk Sample Collection for Bacterial Culture
The technician thoroughly scrubbed the teat end and teat orifice using a gauze square soaked in 70% isopropyl alcohol. When standing behind the cow, the left front (LF) and right front (RF) teats were disinfected first, followed by the left rear (LR) and right rear (RR) teats. Milk samples were then collected from rear teats first, followed by the front teats. The initial 3 strippings of milk (foremilk) from the quarter were discarded onto the floor. Approximately 5 to 10 mL of milk was collected into a sterile 20-mL flip-top sample vial, ensuring that the teat did not touch the open vial end and that no extraneous material contaminated the sample. Duplicate milk samples were collected from each quarter at the same time. Vials were labeled [cow identification, quarter, date, sample number (S1 or S2)] and immediately placed in a chilled closed cooler for transport to the University of Minnesota
Immediately after completion of on-farm sampling, chilled vials of individual quarter milk samples for bacterial culture and unchilled samples in the QScout Q4 collection devices were transported directly to the Veterinary Diagnostic Laboratory, an accredited veterinary diagnostic laboratory at the College of Veterinary Medicine, University of Minnesota (St. Paul, MN).
Milk Sample Testing
Microbial Culture of Quarter Milk Samples
Upon arrival to the Veterinary Diagnostic Laboratory, duplicate milk samples (S1 and S2) from all quarters were submitted immediately for bacterial culture as fresh samples. Ten microliters of each quarter milk sample was pipetted onto a blood agar plate (BD and Co., Franklin Lakes, NJ) using a sterile calibrated pipette, and the milk was distributed across the surface of the agar with a sterile spreader. Plates were then incubated at 37°C for 48 h. Plates were examined by an experienced laboratory technician for bacterial growth at both 24 and 48 h. The final number of bacterial colonies visible after incubation were counted and recorded for each colony-type of bacteria. If 3 or more colony-types of bacterial growth were discovered, the sample was recorded as contaminated and no further identification was performed. Identification of bacterial isolates was performed using MALDI-TOF, using a Bruker MALDI-TOF Biotyper version 3.1.66 (Bruker Corp., Billerica, MA). A confidence level for each bacterial identification was assigned by the MALDI software. Confidence levels greater than 2.0 were accepted to the species level, whereas a level between 1.8 and 2.0 was acceptable to the genus level. Identification of samples below 1.8 was repeated or a determination was made by the technician as to the level of identification that was acceptable (e.g., gram-positive rod, gram-negative organism).
Immediately after initial plating, the S1 and S2 milk samples were frozen at −20°C. Because bulk tank culture of samples collected during the same time period indicated that herds B and C were positive for Mycoplasma spp., the S2 frozen quarter milk samples from all 3 herds were thawed within 7 to 14 d of freezing and underwent culture for Mycoplasma spp. For this process, quarter milk samples from each cow were pooled together by aseptically pipetting 1 mL of milk from each quarter into a sterile vial. The pooled sample was then swabbed on the surface of a Mycoplasma agar plate and incubated for 7 d at 37°C with increased CO2. If the pooled result for the cow was positive, then the quarter samples were rethawed and cultured individually for Mycoplasma spp. to establish which individual quarters were infected.
Results recorded for microbial culture included cow identification, quarter (LF, RF, LR, RR), farm, parity, lactation phase (EL or LL), DIM, sample number (S1 or S2), organism(s) present (if any), and pathogen colony count (cfu/mL). Technicians conducting and recording the bacterial culture results from milk samples were blinded to results from the MLD and the CMT tests and vice versa.
QScout MLD Testing of Milk Samples
Fresh milk samples collected into the Q4 QScout milk collection device underwent MLD testing within 3 to 8 h of milk sample collection according to manufacturer guidelines (M. Hockett, Advanced Animal Diagnostics Inc., Morrisville, NC, personal communication). This was completed by the same 2 trained technicians on each sampling day, and using 6 QScout Farm Lab readers made available for the study by AAD. Testing was done with the MLD test set in the research mode and according to the manufacturer's directions and settings. Paired (duplicate) MLD testing was conducted for each single milk sample collected. Specifically, the Q4 milk sample collection device from each cow was manually agitated for 5 s, then loaded onto each of 2 MLD cassettes. After programing the readers to the correct class of animal (EL or LL), the 2 cassettes were then inserted into each of 2 different QScout Farm Lab readers and processed in the research mode (approximately 10–15 min per sample). The choice of which 2 of 6 readers to use for any 1 paired set of samples was rotated randomly between samples. The MLD results were automatically transferred onto a hard drive and later retrieved electronically by AAD for interpretation.
Definitions of Infection
Milk Culture Results
Quarter IMI status was determined using 3 different methods of interpretation of milk culture results (Table 1):
- •a single milk sample (SS;Dohoo et al., 2011a),
- •duplicate milk samples collected at the same time with parallel interpretation (DP;Pinzón-Sánchez and Ruegg, 2011), and
- •duplicate milk samples collected at the same time with series interpretation (DS:Andersen et al., 2010).
Table 1Criteria used to determine quarter infection status when using 3 different approaches to interpretation of milk culture results
| Method of defining true IMI status from quarter milk culture results | Microbiological diagnosis of sample 1 (S1) | Microbiological diagnosis of sample 2 (S2) | Diagnosis of quarter infection status |
|---|---|---|---|
| Single milk sample | Pathogen | NA | Infected-pathogen |
| No growth | NA | Not infected | |
| Contaminated | As identified | As identified | |
| Duplicates, parallel interpretation | Identical to S2 | Identical to S1 | As identified |
| Pathogen | No growth | Infected-pathogen | |
| Pathogen | Contaminated | Infected-pathogen | |
| Pathogen | Missing | Infected-pathogen | |
| No growth | Contaminated | Not infected | |
| Duplicates, series interpretation | Identical to S2 | Identical to S1 | As identified |
| Not identical to S2 | Not identical to S1 | Omitted |
1 Isolation of 1 or more colonies of bacteria (required ≥2 colonies for CNS and ≥5 colonies for Bacillus spp.).
2 NA = not applicable (S2 culture result not considered).
3 Isolation of 3 or more different colony types.
4 As identified = could be infected, not infected, or contaminated, depending on the results from the S1 and S2 samples.
5 Pathogen, no growth, or contamination.
6 Isolation of ≥3 colonies of the same type of bacteria (required ≥5 colonies for Bacillus spp.).
7 Omitted = if either of the S1 or S2 samples were contaminated or if discordant results were attained between the S1 and S2 samples, then the quarter was omitted from analysis.
One deviation applied to the 3 aforementioned definitions of IMI was that, for Bacillus spp., an IMI was defined as ≥5 colonies isolated from a 0.01-mL milk sample. As no peer-reviewed studies have determined a cut-off point for the latter organism, the definition for IMI for Bacillus spp. used was that established during an informal discussion among mastitis experts conducted during the 2011 Mastitis Research Workers' Conference (Nov. 1, 2011, Chicago, IL). For all 3 definitions of IMI status, a single IMI was defined as the presence of only 1 type of pathogen in the sample, whereas mixed infections corresponded to the presence of 2 different bacterial species. If ≥3 different colony types were present, the quarter sample was considered contaminated. Another deviation from
Dohoo et al., 2011a
, when interpreting the results of only a single milk sample (SS), if S1 was contaminated then the result of S2 was used. If both S1 and S2 were contaminated, then the quarter was omitted from further analysis. This approach to interpreting single milk sample culture results has been used in previous field studies (Lago et al., 2011
; Arruda et al., 2013
).Cow-level infection status was determined by evaluating milk culture results of the 4 quarters. If 1 or more quarters had an IMI, the cow was classified as infected. All 4 quarters had to be uninfected for the cow to be classified as uninfected. However, if 1, 2 or 3 individual quarters were uninfected, but any remaining quarters were contaminated or, as was prevalent for the DS method, had discordant results between S1 and S2, then the infection status of the cow could not be determined with certainty and the animal was omitted from the final cow-level analysis.
MLD Test Results
Results from the MLD test were automatically transferred onto a hard drive, and later retrieved electronically by AAD, which interpreted the results as though the machine had been set in the 3-min smart mode (as would be the method of interpretation used on a farm). Results included cow identification, quarter (LF, LR, RF, RR), test number (1 or 2), total leukocyte count (TLC), total neutrophil count (N), percentage neutrophils (N%), total lymphocyte count (L), percentage lymphocytes (L%), total macrophage count (M), percentage macrophages (M%), and QSCORE result, a final score derived using a proprietary AAD algorithm. Additionally, for each sample and test (1 or 2), and for each possible threshold setting (1–18 for EL; 1–12 for LL), a result of positive, borderline positive, borderline negative, or negative was determined. All MLD test results were transferred electronically back to the principal investigator at the University of Minnesota for the paired MLD tests for each quarter sampled. Technicians at AAD were blinded to CMT and milk culture results. For the purpose of statistical analysis conducted by the principal investigator, quarters assigned a result of borderline positive or borderline negative were assigned a final classification of positive or negative, respectively. Cow level MLD test status (positive/negative) was determined by examining the test results from the individual 4 quarters. If 1 or more quarters were test-positive using the MLD test, the cow was classified as MLD-positive.
CMT Test Results
Quarter test results were classified as positive or negative using each of 4 different cut-points for defining a positive test: trace, ≥1, ≥2, or 3. To determine cow-level CMT test status (positive/negative), if 1 or more quarters had a positive CMT test result at the defined cut-point the cow was classified as being CMT-positive at that cut-point.
Data Analysis
Target sample sizes were derived to calculate MLD test Se (%) while achieving a desired level of confidence and precision for the estimate. A priori assumptions used for this calculation included (1) predicted MLD test Se of 80%; (2) desired confidence level for Se estimate at 95%; (3) desired precision of Se estimate width of 10% (±5%); (4) herd size has 100 eligible cows (400 quarters); and (5) 20% of all quarters will truly have IMI ( = population sample size). Using a sample size calculator (http://epitools.ausvet.com.au/content.php?page=1Proportion), it was estimated that we would need a sample of 61 infected quarters (304 total quarters, or 76 cows) for each lactation phase. This was inflated to 90 cows per lactation phase—90 EL and 90 LL cows—to allow for some anticipated losses due to blind quarters or contaminated samples.
Statistical Analysis
A quarter-level database was assembled which included the following information for each quarter: cow identification, quarter (LF, RF, LH, RH), herd, sample date, parity, lactation phase (EL or LL), DIM, and all test results (positive/negative), including the quarter-level milk culture results when using each of the 3 different definitions of IMI (SS, DP or DS), the corresponding QScout MLD duplicate test results (test 1 and test 2) for each possible threshold setting, and CMT test results for each possible cut-point. A cow-level database was also assembled that described cow identification, herd, sample date, parity, lactation phase, DIM, and all test results (positive/negative), including the cow-level milk culture results when using each of the 3 different definitions of IMI, the corresponding QScout MLD duplicate test results (test 1 and test 2) for each setting, and CMT test results at each possible cut-point.
All analyses were performed using SAS version 9.3 (SAS Institute Inc., Cary, NC), and were completed separately for EL and LL quarters and cows. Descriptive statistics were first generated to describe basic cow characteristics for both EL and LL cows (e.g., parity, DIM). Results of milk culture and interpretation using each of the 3 methods of interpretation were reported as the proportion of quarters and cows with infection present, and the quarter-level bacterial etiology of infection was described. For the MLD test results (tests 1 and 2), the number and proportion of quarters with a positive MLD test result was calculated for each MLD setting (1–18 for EL; 1–12 for LL). Descriptive statistics were also generated to describe the mean, median, standard deviation, and range values for the various quantitative outcomes of MLD testing, including the TLC, N, N%, L, L%, M, M%, and QSCORE result. For the CMT, descriptive statistics were generated describing the number and proportion of CMT-positive quarters and cows using the different cut-points for interpretation (trace, ≥1, ≥2, or 3).
Objective 1
The first objective was to describe the diagnostic test characteristics of an automated MLD test and the CMT to identify IMI in EL and LL quarters and cows when using 3 different approaches to define IMI status from milk culture.
For quarter-level MLD test results, 2 × 2 tables were created comparing laboratory culture results when using each of the 3 different definitions of IMI (SS, DP, DS) against the first MLD test result (test 1, positive/negative). These tables were then used to calculate the diagnostic test characteristics (including 95% confidence limits) for the MLD test including true prevalence of IMI (TP), apparent prevalence of IMI (AP), Se, Sp, predictive value of a positive test (PPV), and predictive value of a negative test (NPV), with calculations performed as defined by
Dohoo et al., 2009
. True prevalence was defined as the proportion of quarters tested that were diagnosed as infected using the referent test. Apparent prevalence was defined as the proportion of quarters tested which received a positive MLD test result. Sensitivity was the proportion of infected quarters (according to culture) that received a positive MLD test result. Specificity was the proportion of uninfected quarters (according to culture) that received a negative MLD test result. Predictive value of a positive test was the proportion of MLD-positive quarters that were diagnosed as infected based on culture, and NPV was the proportion of MLD-negative quarters that were diagnosed as uninfected based on culture. Overall accuracy (the proportion of all quarters tested which were correctly classified by the MLD test) and a Kappa statistic (agreement beyond chance between milk culture and MLD test results) were also calculated but, in the interest of brevity, these will not be reported in the results. These test characteristics were calculated for each possible threshold setting (1–18 for EL; 1–12 for LL). A similar approach was used to calculate diagnostic test characteristics for the MLD test results when interpreted at the cow level (positive/negative), as compared with laboratory culture results when interpreted at the cow level (infected/uninfected).A similar approach to that described in the previous paragraph was used to calculate the CMT test characteristics for both EL and LL quarters when using each of the 3 different definitions of IMI by milk culture (SS, DP, DS) to compare against CMT results and when using each possible cut-point for interpreting CMT results (trace, ≥1, ≥2, or 3). The same process, as described in the previous paragraph, was also used to calculate diagnostic test characteristics for the CMT when results were interpreted at the cow level (positive/negative).
Objective 2
Our second objective was to describe the repeatability of MLD test results at both the quarter and cow level. The Pearson correlation coefficient was calculated to compare the various continuous outcome measures between the paired quarter-level MLD test results (test 1 vs. test 2), including TLC, N, N%, L, L%, M, M%, and QSCORE result. Also, paired quarter-level MLD test results (test 1 and test 2) were compared against each other to calculate agreement (Kappa) for a positive test result, for both EL and LL quarter samples, and at each possible MLD threshold setting (1–18 for EL; 1–12 for LL). The same process was repeated at the cow level to calculate agreement between the MLD cow-level test results (test 1 and test 2) for a positive test result.
RESULTS
Characteristics of the MLD and CMT to Identify IMI in EL Quarters and Cows
A total of 336 functional quarters were sampled from 86 EL cows (8 quarters were blind). The mean (SD; range) parity and DIM for these EL cows was 2.3 (1.2; 1 to 6) and 11.0 (2.2; 7 to 15) days, respectively. Of the 336 EL quarters sampled, 30 (8.9%) were omitted from further analysis due to contamination in both samples, leaving 306 EL quarters for the final analysis of MLD and CMT test characteristics when using SS or DP definitions of IMI from milk culture. However, when applying the DS definition of IMI to these 306 EL quarters, a further 146 (47.7%) samples were omitted from the final DS analysis because one or the other of the paired S1 and S2 results were contaminated (n = 90) or S1 and S2 had discordant pathogen results (n = 56), leaving 160 samples for the final DS analysis. The TP of IMI in EL quarters using the SS, DP, and DS definitions was 25.2 (77 of 306), 18.0 (55 of 306), and 14.4% (23 of 160), respectively (Table 2). Gram-positive infections were the predominant type of IMI detected in EL quarters, with a remarkably similar proportion for all 3 definitions of IMI (SS = 89.2%, DP = 91.7%, DS = 91.3%; Table 2).
Table 2Prevalence and etiology of infection in early and late lactation quarters as determined by culture of quarter milk samples (reference test) when using 3 different definitions of IMI (percentages in parentheses)
| Pathogen group | Pathogen | Single milk sample | Duplicate samples, parallel | Duplicate samples, series | |||
|---|---|---|---|---|---|---|---|
| Early lactation | Late lactation | Early lactation | Late lactation | Early lactation | Late lactation | ||
| Total quarters (N) | 306 | 283 | 306 | 283 | 160 | 152 | |
| No growth (n) | 229 (74.8) | 210 (74.2) | 251 (82.0) | 223 (78.8) | 137 (85.6) | 119 (78.3) | |
| Infected (n) | 77 (25.2) | 73 (25.8) | 55 (18.0) | 60 (21.2) | 23 (14.4) | 33 (21.7) | |
| Gram positive | |||||||
| CNS | Staphylococcus chromogenes | 26 (31.3) | 19 (24.1) | 20 (33.3) | 19 (29.2) | 12 (52) | 14 (36.8) |
| Staphylococcus epidermidis | 0 | 1 (1.3) | 0 | 1 (1.5) | 0 | 0 | |
| Staphylococcus haemolyticus | 3 (3.6) | 1 (1.3) | 2 (3.3) | 2 (3.1) | 1 (0.6) | 1 (2.6) | |
| Other Staphylococcus spp. | 3 (3.6) | 1 (1.3) | 5 (8.3) | 0 | 0 | 0 | |
| Environmental streptococci group | Enterococcus saccharolyticus | 0 | 9 (11.4) | 0 | 10 (15.4) | 0 | 8 (21.1) |
| Lactococcus garvieae | 1 (1.2) | 0 | 1 (1.7) | 0 | 1 (0.6) | 0 | |
| Streptococcus dysgalactiae | 5 (6.0) | 1 (1.3) | 4 (6.7) | 1 (1.5) | 0 | 1 (2.6) | |
| Streptococcus uberis | 8 (9.6) | 3 (3.8) | 9 (15.0) | 3 (4.6) | 4 (17.4) | 3 (7.9) | |
| Other Streptococcus spp. | 4 (4.8) | 3 (3.8) | 1 (1.7) | 1 (1.5) | 0 | 1 (2.6) | |
| Other gram positive | Bacillus spp. | 3 (3.6) | 6 (7.6) | 6 (10.0) | 8 (12.3) | 0 | 0 |
| Corynebacterium spp. | 12 (14.5) | 23 (29.1) | 4 (6.7) | 12 (18.5) | 2 (1.3) | 4 (10.5) | |
| Gram-positive cocci | 0 | 1 (1.3) | 0 | 1 (1.5) | 0 | 0 | |
| Gram-positive rod | 9 (10.8) | 4 (5.1) | 3 (5.0) | 2 (3.1) | 1 (0.6) | 0 | |
| Total gram positive | 74 (89.2) | 72 (91.1) | 55 (91.7) | 60 (92.3) | 21 (91.3) | 32 (84.2) | |
| Gram negative | |||||||
| Coliform | Acinetobacter spp. | 4 (4.8) | 1 (1.3) | 1 (1.7) | 0 | 0 | 0 |
| Enterobacter spp. | 1 (1.2) | 0 | 0 | 0 | 0 | 0 | |
| Klebsiella spp. | 1 (1.2) | 5 (6.3) | 1 (1.7) | 4 (6.2) | 1 (0.6) | 3 (7.9) | |
| Other gram negative | Serratia spp. | 0 | 1 (1.3) | 0 | 1 (1.5) | 0 | 0 |
| Other gram negative | 1 (1.2) | 0 | 0 | 0 | 0 | 0 | |
| Total gram negative | 7 (8.4) | 7 (8.9) | 2 (3.3) | 5 (7.7) | 1 (0.6) | 3 (7.9) | |
| Other organisms | Yeast | 2 (2.4) | 0 | 3 (5.0) | 0 | 1 (0.6) | 0 |
| Total organisms | 83 (100) | 79 (100) | 60 (100) | 65 (100) | 23 (100) | 38 (100) | |
Seventy-three cows were included in the EL cow-level analysis when using SS or DP definitions of IMI from milk culture. However, when applying the DS definition of IMI, 51 (70%) cows had to be omitted from the analysis because 1 or more quarters for a cow had a contaminated (n = 42) or discordant (n = 9) pathogen result, thereby leaving only 22 EL cows for the final DS analysis. The TP of infection in EL cows using the SS, DP, and DS definitions was 71.2 (52 of 73), 53.4 (39 of 73), and 81.8% (18 of 22), respectively.
MLD Performance in EL Quarters
Though some variation existed, the MLD test characteristics to identify IMI in EL quarters were very similar, whether using SS, DP, or DS definitions of IMI from milk culture as the referent test (Table 3). For example, when using the SS, DP, and DS methods to define IMI, the Se estimates for the MLD test at threshold setting 1 were 32.5, 30.9, and 39.1%, respectively, and the Sp estimates at setting 1 were 83.8, 82.1, and 83.9%, respectively (Table 3). Although MLD test characteristics were calculated at all possible settings (1–18 for EL; 1–12 for LL), we have elected to only report the MLD test characteristics for the following 4 settings of interest: (1) setting yielding maximum Se (usually setting 1), (2) setting yielding maximum Sp (usually the highest setting), (3) setting yielding maximum accuracy, and (4) the factory setting currently recommended by the manufacturer for use on farms (12 for EL; 6 for LL; Table 3). In reviewing these results (Table 3), it can be seen that Se, Sp, PPV, and NPV estimates for the MLD test varied somewhat among these 4 settings. For example, when considering the MLD test characteristics when compared with DP interpretation of milk samples as the referent test, in general Se was low, ranging between 12.7 (setting 18) to 30.9% (setting 1), and Sp was good to very good, ranging between 82.1 (setting 1) and 95.2% (setting 18). Predictive value of a positive test was low, ranging between 27.4 (setting 1) and 43.5% (setting 16), whereas NPV was good, ranging between 83.3 (setting 18) and 84.4% (setting 1; Table 3).
Table 3Test characteristics of the milk leukocyte differential (MLD) test to diagnose IMI in early- and late-lactation quarters as compared with laboratory culture of milk samples when using 3 definitions of IMI [estimate (95% confidence limit)]
| MLD setting | No. of positive quarters | Apparent prevalence (%) | Sensitivity (%) | Specificity (%) | Positive predictive value (%) | Negative predictive value (%) |
|---|---|---|---|---|---|---|
| Early-lactation quarters | ||||||
| Single sample (true prevalence = 25.2%) | ||||||
| MLD #1 | 62 of 306 | 20.3 (15.8, 24.8) | 32.5 (22.0, 42.9) | 83.8 (79.1, 88.6) | 40.3 (28.1, 52.5) | 78.7 (73.6, 83.8) |
| MLD #18 | 19 of 306 | 6.1 (3.5, 8.9) | 11.7 (4.5, 18.9) | 95.6 (93.0, 98.3) | 47.4 (24.9, 69.8) | 76.3 (71.4, 81.2) |
| MLD #16 | 23 of 306 | 7.5 (4.6, 10.5) | 15.6 (7.5, 23.7) | 95.2 (92.4, 98.0) | 52.2 (31.8, 72.6) | 77.0 (72.1, 81.9) |
| MLD #12 | 32 of 306 | 10.5 (7.0, 13.9) | 20.8 (11.7, 29.8) | 93.0 (89.7, 96.3) | 50.0 (32.7, 67.3) | 77.7 (72.8, 82.7) |
| Duplicate samples, parallel (true prevalence = 18.0%) | ||||||
| MLD #1 | 62 of 306 | 20.3 (15.8, 24.8) | 30.9 (18.7, 43.1) | 82.1 (77.3, 86.8) | 27.4 (16.3, 38.5) | 84.4 (79.9, 89.0) |
| MLD #18 | 19 of 306 | 6.2 (3.5, 8.9) | 12.7 (3.9, 21.5) | 95.2 (92.6, 97.9) | 36.8 (15.2, 58.5) | 83.3 (79.0, 87.6) |
| MLD #16 | 23 of 306 | 7.5 (4.6, 10.5) | 18.2 (8.0, 28.4) | 94.8 (92.1, 97.6) | 43.5 (23.2, 63.7) | 84.1 (79.8, 88.4) |
| MLD #12 | 32 of 306 | 10.5 (7.0, 13.9) | 20.0 (9.4, 30.6) | 91.6 (88.2, 95.1) | 34.4 (17.9, 50.8) | 83.9 (79.6, 88.3) |
| Duplicate samples, series (true prevalence = 14.4%) | ||||||
| MLD #1 | 23 of 160 | 19.4 (13.3, 25.5) | 39.1 (19.2, 59.1) | 83.9 (77.8, 90.1) | 29.0 (13.1, 45.0) | 89.1 (83.8, 94.5) |
| MLD #18 | 11 of 160 | 6.9 (3.0, 10.8) | 13.0 (−0.7, 26.8) | 94.2 (90.2, 98.1) | 27.3 (1.0, 53.6) | 86.6 (81.1, 92.1) |
| MLD #12, | 14 of 160 | 8.8 (4.4, 13.1) | 21.7 (4.9, 38.6) | 93.4 (89.3, 97.6) | 35.7 (10.6, 60.8) | 87.7 (82.3, 93.0) |
| Late-lactation quarters | ||||||
| Single sample (true prevalence = 25.8%) | ||||||
| MLD #1 | 135 of 283 | 47.7 (41.9, 53.5) | 69.9 (59.3, 80.4) | 60.0 (53.4, 66.6) | 37.8 (29.6, 46.0) | 85.1 (79.4, 90.9) |
| MLD #12, | 77 of 283 | 27.2 (22.0, 32.4) | 46.6 (35.1, 58.0) | 79.5 (74.1, 85.0) | 44.2 (33.1, 55.2) | 81.1 (75.7, 86.4) |
| MLD #6 | 103 of 283 | 36.4 (30.8, 42.0) | 56.2 (44.8, 67.5) | 70.5 (64.3, 76.6) | 39.8 (30.4, 49.3) | 82.2 (76.6, 87.8) |
| Duplicate samples, parallel (true prevalence = 21.2%) | ||||||
| MLD #1 | 135 of 283 | 47.7 (41.9, 53.5) | 73.3 (62.1, 84.5) | 59.2 (52.7, 65.6) | 32.6 (24.7, 40.5) | 89.2 (84.2, 94.2) |
| MLD #12, | 77 of 283 | 27.2 (22.0, 32.4) | 53.3 (40.7, 66.0) | 79.8 (74.6, 85.1) | 41.6 (30.6, 52.6) | 86.4 (81.7, 91.1) |
| MLD #6 | 103 of 283 | 36.4 (30.8, 42.0) | 61.7 (49.4, 74.0) | 70.4 (64.4, 76.4) | 35.9 (26.7, 45.2) | 87.2 (82.3, 92.1) |
| Duplicate samples, series (true prevalence = 21.7%) | ||||||
| MLD #1 | 74 of 152 | 48.7 (40.7, 56.5) | 84.8 (72.6, 97.1) | 61.3 (52.6, 70.1) | 37.8 (26.8, 48.9) | 93.6 (88.2, 99.0) |
| MLD #12, | 46 of 152 | 30.3 (23.0, 37.6) | 66.7 (50.6, 82.8) | 79.8 (72.6, 87.0) | 47.8 (33.4, 62.3) | 89.6 (83.8, 95.4) |
| MLD #6 | 60 of 152 | 39.5 (31.7, 47.2) | 78.8 (64.8, 92.7) | 71.4 (63.3, 79.5) | 43.3 (30.8, 55.9) | 92.4 (87.0, 97.8) |
1 Setting with maximum sensitivity.
2 Setting with maximum specificity.
3 Setting with maximum accuracy.
4 Factory setting of MLD test recommended by manufacturer for use on farms.
CMT Performance in EL Quarters
When examining the CMT test characteristics for EL quarters, again the test characteristics did not vary much whether using SS, DP, or DS definitions of IMI from milk culture as the referent test (Table 4). For example, when using the SS, DP, and DS methods for defining IMI, the Se estimates for the CMT test at the cut-point trace were 51.9, 49.1, and 52.2%, respectively, and the Sp estimates at this same cut-point were 70.7, 68.1, and 68.6%, respectively (Table 4). In reviewing the CMT test characteristics for EL quarters it can be seen that Se, Sp, PPV, and NPV estimates varied moderately depending on which CMT cut-point was used to declare a positive test. For example, if we consider CMT test characteristics when compared with DP interpretation of milk culture results, Se was low, ranging between 3.6 (cut-point = 3) to 49.1% (cut-point trace), and Sp was fair to excellent, ranging between 68.1 (cut-point trace) and 99.6% (cut-point = 3). Predictive value of a positive test was low to fair, ranging between 25.2 (cut-point trace) and 66.7% (cut-point = 3), whereas NPV was good, ranging between 82.5 (cut-point = 3) and 85.9% (cut-point trace; Table 4).
Table 4Test characteristics of the California Mastitis Test (CMT) to diagnose IMI in early- and late-lactation quarters as compared with laboratory culture of milk samples when using 3 definitions of IMI [table reports estimate (95% confidence limit)]
| CMT setting | No. of positive quarters | Apparent prevalence (%) | Sensitivity (%) | Specificity (%) | Positive predictive value (%) | Negative predictive value (%) |
|---|---|---|---|---|---|---|
| Early-lactation quarters | ||||||
| Single sample (true prevalence = 25.2%) | ||||||
| Trace+ | 107 of 306 | 35.0 (29.6, 40.3) | 51.9 (40.8, 63.1) | 70.7 (64.8, 76.6) | 37.4 (28.2, 46.6) | 81.4 (76.0, 86.8) |
| 1+ | 38 of 306 | 12.4 (8.7, 16.1) | 20.8 (11.7, 29.8) | 90.4 (86.6, 94.2) | 42.1 (26.4, 57.8) | 77.2 (72.2, 82.3) |
| 2+ | 15 of 306 | 4.9 (2.5, 7.3) | 11.7 (4.5, 18.9) | 97.4 (95.3, 99.4) | 60.0 (35.2, 84.8) | 76.6 (71.8, 81.5) |
| 3 | 3 of 306 | 1.0 (0, 2.1) | 3.9 (0, 8.2) | 100 (100, 100) | 100 (100, 100) | 75.6 (70.7, 80.4) |
| Duplicate samples, parallel (true prevalence = 18.0%) | ||||||
| Trace+ | 107 of 306 | 35.0 (29.6, 40.3) | 49.1 (35.9, 62.3) | 68.1 (62.4, 73.9) | 25.2 (17.0, 33.5) | 85.9 (81.1, 90.8) |
| 1+ | 38 of 306 | 12.4 (8.7, 16.1) | 20.0 (9.4, 30.6) | 89.2 (85.4, 93.1) | 28.9 (14.5, 43.4) | 83.6 (79.1, 88.0) |
| 2+ | 15 of 306 | 4.9 (2.5, 7.3) | 12.7 (3.9, 21.5) | 96.8 (94.6, 99.0) | 46.7 (21.4, 71.9) | 83.5 (79.2, 87.8) |
| 3 | 3 of 306 | 1.0 (0, 2.1) | 3.6 (0, 8.6) | 99.6 (99.8, 100) | 66.7 (13.3, 100) | 82.5 (78.2, 86.8) |
| Duplicate samples, series (true prevalence = 14.4%) | ||||||
| Trace+ | 55 of 160 | 34.4 (27.0, 41.7) | 52.2 (31.8, 72.6) | 68.6 (60.8, 76.4) | 21.8 (10.9, 32.7) | 89.5 (83.7, 95.4) |
| 1+ | 18 of 160 | 11.3 (6.4, 16.1) | 17.4 (1.9, 32.9) | 89.8 (84.7, 94.9) | 22.2 (3.0, 41.4) | 86.6 (81.0, 92.2) |
| 2+ | 6 of 160 | 3.8 (0.8, 6.7) | 8.7 (0, 20.2) | 97.1 (94.3, 99.9) | 33.3 (0, 71.1) | 86.4 (80.9, 91.8) |
| 3 | 0 of 160 | 0 (0, 0) | 0 (0, 0) | 100 (100, 100) | NA | 85.6 (80.2, 91.1) |
| Late-lactation quarters | ||||||
| Single sample (true prevalence = 25.8%) | ||||||
| Trace+ | 145 of 283 | 51.2 (45.4, 57.1) | 63.0 (51.9, 74.1) | 52.9 (46.1, 59.6) | 31.7 (24.1, 39.3) | 80.4 (73.8, 87.1) |
| 1+ | 76 of 283 | 26.9 (21.7, 32.0) | 41.1 (29.8, 52.4) | 78.1 (72.5, 83.7) | 39.5 (28.5, 50.5) | 79.2 (73.7, 84.8) |
| 2+ | 46 of 283 | 16.3 (12.0, 20.6) | 27.4 (17.2, 37.6) | 87.6 (83.2, 92.1) | 43.5 (29.2, 57.8) | 77.6 (72.3, 82.9) |
| 3 | 13 of 283 | 4.6 (2.2, 7.0) | 9.6 (2.8, 16.3) | 97.1 (94.9, 99.4) | 53.8 (26.7, 80.9) | 75.6 (70.4, 80.7) |
| Duplicate samples, parallel (true prevalence = 21.2%) | ||||||
| Trace+ | 145 of 283 | 51.2 (45.4, 57.1) | 65.0 (52.9, 77.1) | 52.5 (45.9, 59.0) | 26.9 (19.7, 34.1) | 84.8 (78.8, 90.8) |
| 1+ | 76 of 283 | 26.9 (21.7, 32.0) | 41.7 (29.2, 54.1) | 77.1 (71.6, 82.6) | 32.9 (22.3, 43.5) | 83.1 (78.0, 88.2) |
| 2+ | 46 of 283 | 16.3 (12.0, 20.6) | 28.3 (16.9, 39.7) | 87.0 (82.6, 91.4) | 37.0 (23.0, 50.9) | 81.9 (77.0, 86.8) |
| 3 | 13 of 283 | 4.6 (2.2, 7.0) | 11.7 (3.5, 19.8) | 97.3 (95.2, 99.4) | 53.8 (26.7, 80.9) | 80.4 (75.6, 85.1) |
| Duplicate samples, series (true prevalence = 2.17%) | ||||||
| Trace+ | 80 of 152 | 52.6 (44.7, 60.6) | 72.7 (57.5, 87.9) | 52.9 (44.0, 61.9) | 30.0 (20.0, 40.0) | 87.5 (79.9, 95.1) |
| 1+ | 45 of 152 | 29.6 (22.3, 36.9) | 51.5 (34.5, 68.6) | 76.5 (68.8, 84.1) | 37.8 (23.6, 51.9) | 85.0 (78.3, 91.8) |
| 2+ | 27 of 152 | 17.8 (11.7, 23.8) | 36.4 (20.0, 52.8) | 87.4 (81.4, 93.4) | 44.4 (25.7, 63.2) | 83.2 (76.6, 89.8) |
| 3 | 9 of 152 | 5.9 (2.2, 9.7) | 15.2 (2.9, 27.4) | 96.6 (93.4, 99.9) | 55.6 (23.1, 88.0) | 80.4 (73.9, 86.9) |
1 NA = not calculable.
MLD Performance in EL Cows
Though some exceptions were present, the MLD test characteristics to identify IMI in EL cows were very similar, whether using SS, DP, or DS definitions of IMI from milk culture as the referent test (Table 3). For example, when using the SS, DP, and DS methods to define IMI, the Se estimates for the MLD test at threshold setting 1 were 53.8, 56.4, and 55.6%, respectively, and accuracy estimates at setting 1 were 57.5, 58.9, and 63.6%, respectively (Table 5). In reviewing the MLD cow-level test results (Table 5) it can be seen that Se, Sp, PPV, and NPV estimates varied somewhat among the 4 threshold settings reported. For example, when considering MLD test characteristics when compared with DP interpretation of milk culture results, in general Se was low, ranging between 25.6 (setting 18) to 56.4% (setting 1), and Sp was fair to very good, ranging between 61.8 (setting 1) and 88.2% (setting 18). Predictive value of a positive test was fair, ranging between 62.9 (setting 1) and 71.4% (setting 18), and NPV was also fair, ranging between 50.8 (setting 18) and 55.3% (setting 1). It should be noted that the Sp and PPV estimates for all MLD settings were 100% when using DS interpretation of milk culture as the reference method, though these calculations were derived using a rather small number of samples.
Table 5Test characteristics of the milk leukocyte differential (MLD) Test to diagnose IMI in early- and late-lactation cows as compared with laboratory culture of milk samples when using 3 definitions of IMI [table reports estimate (95% confidence limit)]
| MLD setting | No. of positive cows | Apparent prevalence (%) | Sensitivity (%) | Specificity (%) | Positive predictive value (%) | Negative predictive value (%) |
|---|---|---|---|---|---|---|
| Early-lactation cows | ||||||
| Single sample (true prevalence = 71.2%) | ||||||
| MLD #1 | 35 of 73 | 47.9 (36.5, 59.4) | 53.8 (40.3, 67.4) | 66.7 (46.5, 86.8) | 80.0 (66.7, 93.3) | 36.8 (21.5, 52.2) |
| MLD #18 | 14 of 73 | 19.2 (10.1, 28.2) | 25.0 (13.2, 36.8) | 95.2 (86.1, 1.0) | 92.9 (79.4, 1.0) | 33.9 (21.8, 46.0) |
| MLD #2 | 32 of 73 | 43.8 (32.5, 55.2) | 51.9 (38.3, 65.5) | 76.2 (58.0, 94.4) | 84.4 (71.8, 97.0) | 39.0 (24.1, 54.0) |
| MLD #12 | 24 of 73 | 32.9 (22.1, 43.7) | 42.3 (28.9, 55.7) | 90.5 (77.9, 100) | 91.7 (80.6, 100) | 38.8 (25.1, 52.4) |
| Duplicate samples, parallel (true prevalence = 53.4%) | ||||||
| MLD #1 | 35 of 73 | 47.9 (36.5, 59.4) | 56.4 (40.8, 72.0) | 61.8 (45.4, 78.1) | 62.9 (46.8, 78.9) | 55.3 (39.5, 71.1) |
| MLD #18 | 14 of 73 | 19.2 (10.1, 28.2) | 25.6 (11.9, 39.3) | 88.2 (77.4, 99.1) | 71.4 (47.8, 95.1) | 50.8 (38.1, 63.6) |
| MLD #12, | 24 of 73 | 32.9 (22.1, 43.7) | 43.6 (28.0, 59.2) | 79.4 (65.8, 93.0) | 70.8 (52.6, 89.0) | 55.1 (41.2, 69.0) |
| Duplicate samples, series (true prevalence = 81.8%) | ||||||
| MLD #1–7, | 10 of 22 | 45.5 (24.6, 66.3) | 55.6 (32.6, 78.5) | 100 (100, 100) | 100 (100, 100) | 33.3 (6.7, 60.0) |
| MLD #17,18 | 5 of 22 | 22.7 (5.2, 40.2) | 27.8 (7.1, 48.5) | 100 (100, 100) | 100 (100, 100) | 23.5 (3.4, 43.7) |
| MLD #12 | 9 of 22 | 40.9 (20.4, 61.5) | 50.0 (26.9, 73.1) | 100 (100, 100) | 100 (100, 100) | 30.8 (5.7, 55.9) |
| Late-lactation cows | ||||||
| Single sample (true prevalence = 64.4%) | ||||||
| MLD #1 | 55 of 73 | 75.3 (65.5, 85.2) | 80.9 (69.6, 92.1) | 34.6 (16.3, 52.9) | 69.1 (56.9, 81.3) | 50.0 (26.9, 73.1) |
| MLD #12 | 41 of 73 | 56.2 (44.8, 67.5) | 59.6 (45.5, 73.6) | 50.0 (30.8, 69.2) | 68.3 (54.0, 82.5) | 40.6 (23.6, 57.6) |
| MLD #3 | 52 of 73 | 71.2 (60.8, 81.6) | 78.7 (67.0, 90.4) | 42.3 (23.3, 61.3) | 71.2 (58.8, 83.5) | 52.4 (31.0, 73.7) |
| MLD #6 | 50 of 73 | 68.5 (57.8, 79.1) | 74.5 (62.0, 86.9) | 42.3 (23.3, 61.3) | 70.0 (57.3, 82.7) | 47.8 (27.4, 68.2) |
| Duplicate samples, parallel (true prevalence = 49.3%) | ||||||
| MLD #1 | 55 of 73 | 75.3 (65.5, 85.2) | 83.3 (71.2, 95.5) | 32.4 (17.3, 47.5) | 54.5 (41.4, 67.7) | 66.7 (44.9, 88.4) |
| MLD #12 | 41 of 73 | 56.2 (44.8, 67.5) | 69.4 (54.4, 84.5) | 56.8 (40.8, 72.7) | 61.0 (46.0, 75.9) | 65.6 (49.2, 82.1) |
| MLD #10 | 43 of 73 | 58.9 (47.6, 70.2) | 72.2 (57.6, 86.9) | 54.1 (38.0, 70.1) | 60.5 (45.9, 75.1) | 66.7 (49.8, 83.5) |
| MLD #6 | 50 of 73 | 68.5 (57.8, 79.1) | 75.0 (60.9, 89.1) | 37.8 (22.2, 53.5) | 54.0 (40.2, 67.8) | 60.9 (40.9, 80.8) |
| Duplicate samples, series (true prevalence = 75.9%) | ||||||
| MLD #1, | 23 of 29 | 79.3 (64.6, 94.1) | 86.4 (72.0, 100) | 42.9 (6.2, 79.5) | 82.6 (67.1, 98.1) | 50.0 (10.0, 90.0) |
| MLD #7–12 | 20 of 29 | 69.0 (52.1, 85.8) | 72.7 (54.1, 91.3) | 42.9 (6.2, 79.5) | 80.0 (62.5, 97.5) | 33.3 (2.5, 64.1) |
| MLD #6 | 21 of 29 | 72.4 (56.1, 88.7) | 77.3 (59.8, 94.8) | 42.9 (6.2, 79.5) | 81.0 (64.2, 97.7) | 37.5 (4.0, 71.0) |
1 Setting with maximum sensitivity.
2 Setting with maximum specificity.
3 Setting with maximum accuracy.
4 Factory setting of MLD test recommended by manufacturer for use on farms.
CMT Performance in EL Cows
When examining the CMT test characteristics for EL cows, again the test characteristics generally did not vary much whether using SS, DP, or DS definitions of IMI from milk culture as the referent test (Table 6). For example, when using the SS, DP, and DS methods for defining IMI, the Se estimates for the CMT test at the cut-point trace were 69.2, 61.5, and 72.2%, respectively, and the Sp estimates at this same cut-point were 47.6, 32.4, and 25.0%%, respectively (Table 6). In reviewing the CMT test characteristics for EL cows, we found mild to moderate variation in Se, Sp, PPV, and NPV estimates depending on which CMT cut-point was used to declare a positive test (Table 6). For example, when considering CMT test characteristics when compared with DP interpretation of milk culture results, in general Se was low to fair, ranging between 5.1 (cut-point = 3) and 61.5% (cut-point trace), and Sp was low to excellent, ranging between 32.4 (cut-point trace) and 100% (cut-point = 3). Predictive value of a positive test was fair to excellent, ranging between 51.1 (cut-point trace) and 100% (cut-point = 3), whereas NPV was low, ranging between 42.3 (cut-point trace) and 48.4% (cut-point = ≥2; Table 6).
Table 6Test characteristics of the California Mastitis Test (CMT) to diagnose IMI in early- and late-lactation cows as compared with laboratory culture of milk samples when using 3 definitions of IMI [table reports estimate (95% confidence limit)]
| CMT cut-point | No. positive cows | Apparent prevalence (%) | Sensitivity (%) | Specificity (%) | Positive predictive value (%) | Negative predictive value (%) |
|---|---|---|---|---|---|---|
| Early-lactation cows | ||||||
| Single sample (true prevalence = 71.2%) | ||||||
| Trace+ | 47 of 73 | 64.4 (53.4, 75.4) | 69.2 (56.7, 81.8) | 47.6 (26.3, 69.0) | 76.6 (64.5, 88.7) | 38.5 (19.8, 57.2) |
| 1+ | 23 of 73 | 31.5 (20.9, 42.2) | 34.6 (21.7, 47.5) | 76.2 (58.0, 94.4) | 78.3 (61.4, 95.1) | 32.0 (19.1, 44.9) |
| 2+ | 11 of 73 | 15.1 (6.9, 23.3) | 19.2 (8.5, 29.9) | 95.2 (86.1, 100) | 90.9 (73.9, 100) | 32.3 (20.6, 43.9) |
| 3 | 2 of 73 | 2.7 (0, 6.5) | 3.8 (0, 9.1) | 100 (100, 100) | 100 (100, 100) | 29.6 (19.0, 40.2) |
| Duplicate samples, parallel (true prevalence = 53.4%) | ||||||
| Trace+ | 47 of 73 | 64.4 (53.4, 75.4) | 61.5 (46.3, 76.8) | 32.4 (16.6, 48.1) | 51.1 (36.8, 65.4) | 42.3 (23.3, 61.3) |
| 1+ | 23 of 73 | 31.5 (20.9, 42.2) | 30.8 (16.3, 45.3) | 67.6 (51.9, 83.4) | 52.2 (31.8, 72.6) | 46.0 (32.2, 59.8) |
| 2+ | 11 of 73 | 15.1 (6.9, 23.3) | 17.9 (5.9, 30.0) | 88.2 (77.4, 99.1) | 63.6 (35.2, 92.1) | 48.4 (35.9, 60.8) |
| 3 | 2 of 73 | 2.7 (0, 6.5) | 5.1 (0, 12.1) | 100 (100, 100) | 100 (100, 100) | 47.9 (36.3, 59.5) |
| Duplicate samples, series (true prevalence = 81.8%) | ||||||
| Trace+ | 16 of 22 | 72.7 (54.1, 91.3) | 72.2 (51.5, 92.9) | 25.0 (0, 67.4) | 81.3 (62.1, 100) | 16.7 (0, 46.5) |
| 1+ | 6 of 22 | 27.3 (8.7, 45.9) | 27.8 (7.1, 448.5) | 75.0 (32.6, 100) | 83.3 (53.5, 100) | 18.8 (0, 37.9) |
| 2+ | 3 of 22 | 13.6 (0, 28.0) | 16.7 (0, 33.9) | 100 (100, 100) | 100 (100, 100) | 21.1 (2.7, 39.4) |
| 3 | 0 of 22 | 0 (0, 0) | 0 (0, 0) | 100 (100, 100) | NA | 18.2 (2.1, 34.3) |
| Late-lactation cows | ||||||
| Single sample (true prevalence = 64.4%) | ||||||
| Trace+ | 54 of 73 | 74.0 (63.9, 84.0) | 80.9 (69.6, 92.1) | 38.5 (19.8, 57.2) | 70.4 (58.2, 82.5) | 52.6 (30.2, 75.1) |
| 1+ | 39 of 73 | 53.4 (42.0, 64.9) | 61.7 (47.8, 75.6) | 61.5 (42.8, 80.2) | 74.4 (60.7, 88.1) | 47.1 (30.3, 63.8) |
| 2+ | 28 of 73 | 38.4 (27.2, 49.5) | 44.7 (30.5, 58.9) | 73.1 (56.0, 90.1) | 75.0 (59.0, 91.0) | 42.2 (27.8, 56.7) |
| 3 | 11 of 73 | 15.1 (6.9, 23.3) | 19.1 (7.9, 30.4) | 92.3 (82.1, 100) | 81.8 (59.0, 1.05) | 38.7 (26.6, 50.8) |
| Duplicate samples, parallel (true prevalence = 49.3%) | ||||||
| Trace+ | 54 of 73 | 74.0 (63.9, 84.0) | 86.1 (74.8, 97.4) | 37.8 (22.2, 53.5) | 57.4 (44.2, 70.6) | 73.7 (53.9, 93.5) |
| 1+ | 39 of 73 | 53.4 (42.0, 64.9) | 69.4 (54.4, 84.5) | 62.2 (46.5, 77.8) | 64.1 (49.0, 79.2) | 67.6 (51.9, 83.4) |
| 2+ | 28 of 73 | 38.4 (27.2, 49.5) | 52.8 (36.5, 69.1) | 75.7 (61.9, 89.5) | 67.9 (50.6, 85.2) | 62.2 (48.1, 76.4) |
| 3 | 11 of 73 | 15.1 (6.9, 23.3) | 22.2 (8.6, 35.8) | 91.9 (83.1, 100) | 72.7 (46.4, 99.0) | 54.8 (42.5, 67.2) |
| Duplicate samples, series (true prevalence = 75.9%) | ||||||
| Trace+ | 25 of 29 | 86.2 (73.7, 98.8) | 86.4 (72.0, 100) | 14.3 (0, 40.2) | 76.0 (59.3, 92.7) | 25.0 (0, 67.4) |
| 1+ | 17 of 29 | 58.6 (40.7, 76.5) | 63.6 (43.5, 83.7) | 57.1 (20.5, 93.8) | 82.4 (64.2, 100) | 33.3 (6.7, 60.0) |
| 2+ | 14 of 29 | 48.3 (30.1, 66.5) | 54.5 (33.7, 75.4) | 71.4 (38.0, 100) | 85.7 (67.4, 100) | 33.3 (9.5, 57.2) |
| 3 | 8 of 29 | 27.6 (11.3, 43.9) | 31.8 (12.4, 51.3) | 85.7 (59.8, 100) | 87.5 (64.6, 100) | 28.6 (9.2, 47.9) |
1 NA = not calculable.
Characteristics of MLD and CMT to Identify IMI in LL Quarters and Cows
A total of 346 functional quarters were sampled from 90 LL cows (14 quarters were blind). The mean (SD; range) parity and DIM for LL cows was 2.0 (1.2; 1 to 6) and 387.7 (87.6; 277 to 721) days, respectively. Of the 346 LL quarters sampled, 34 (9.8%) were omitted due to contamination in both samples, 1 was omitted because of a missing milk culture result, and 20 were omitted because MLD results were missing. A further 8 quarters were omitted from this analysis because later culture of frozen samples indicated that they were positive for Mycoplasma spp., but culture of fresh samples for Mycoplasma spp. had not been completed. As such, it would have been inappropriate to consider any apparent no growth results from routine culture of these 8 fresh milk samples in the analysis for objective 1. This left 283 LL quarters for the final analysis of MLD and CMT test characteristics when using SS or DP definitions of IMI from milk culture. When applying the DS definition of IMI to these 283 LL quarters, an additional 131 (45.9%) quarters were omitted from the final analysis because one or the other of the paired S1 and S2 results were contaminated (n = 96) or S1 and S2 had discordant pathogen results (n = 56), leaving 152 samples for the DS final analysis. The TP of IMI in LL quarters using the SS, DP, and DS definitions was 25.8 (73 of 283), 21.2 (60 of 283), and 21.7% (33 of 152), respectively (Table 2). Gram-positive infections were the predominant type of IMI detected in LL quarters, with a similar proportion for all 3 definitions of IMI (SS = 91.1%, DP = 92.3%, DS = 84.2%; Table 2).
Seventy-three cows were included in the cow-level analysis when using SS or DP definitions of IMI from milk culture. However, when applying the DS definition of IMI, 44 (60%) of cows had to be omitted from the final analysis because 1 or more quarters for a cow had either a contaminated (n = 36) or discordant pathogen result (n = 8), thereby leaving only 29 LL cows for the final DS analysis. At the cow level, the TP of infection in LL cows using the SS, DP, and DS definitions was 64.4 (47 of 73), 49.3 (36 of 73), and 75.9% (22 of 29), respectively (Table 5).
MLD Performance in LL Quarters
Though some variation existed, the MLD test characteristics to identify IMI in LL quarters were within a similar range, whether using SS, DP, or DS definitions of IMI from milk culture as the referent test (Table 3). For example, when using the SS, DP and DS methods to define IMI, the Se estimates for the MLD test at threshold setting 1 were 69.9, 73.3, and 84.8%, respectively, and the Sp estimates at setting 1 were 60.0, 59.2, and 61.3%, respectively (Table 3). In reviewing the MLD test characteristics for LL quarters, we found small to moderate differences in estimates for Se, Sp, PPV, and NPV depending on which MLD threshold setting was used. For example, when considering MLD test characteristics when compared with DP interpretation of milk culture, in general Se was fair, ranging between 53.3 (setting 12) to 73.3% (setting 1), and Sp was fair, ranging between 59.2 (setting 1) and 79.8% (setting 12). Predictive value of a positive test was low, ranging between 32.6 (setting 1) and 41.6% (setting 12), whereas NPV was good, ranging between 86.4 (setting 12) and 89.2% (setting 1; Table 3).
CMT Performance in LL Quarters
When examining the CMT test characteristics for LL quarters, again the test characteristics did not vary greatly whether using SS, DP, or DS definitions of IMI from milk culture as the referent test (Table 4). For example, when using the SS, DP, and DS methods for defining IMI, the Se estimates for the CMT test at the cut-point trace were 63.0, 65.0, and 72.7%, respectively, and the Sp estimates at this same cut-point were 52.9, 52.5, and 52.9%, respectively (Table 4). In reviewing the CMT test characteristics for LL quarters it can be seen that Se, Sp, PPV, and NPV estimates varied moderately depending on which CMT cut-point was used to declare a positive test. For example, if considering the CMT test characteristics when compared with DP interpretation of milk culture, in general Se was low to fair, ranging between 11.7 (cut-point = 3) to 65.0% (cut-point trace), and Sp was low to excellent, ranging between 52.5% (cut-point trace) and 97.3% (cut-point = 3). Predictive value of a positive test was low, ranging between 26.9 (cut-point trace) and 53.8% (cut-point = 3), whereas NPV was good, ranging between 80.4 (cut-point = 3) and 84.8% (cut-point trace; Table 4).
MLD Performance in LL Cows
Though some exceptions were present, the MLD test characteristics to identify IMI in LL cows were very similar, whether using SS, DP or DS definitions of IMI from milk culture as the referent test (Table 3). For example, when using the SS, DP, and DS methods to define IMI, the Se estimates for the MLD test at threshold setting 1 were 80.9, 83.3, and 86.4%, respectively, and Sp estimates at setting 1 were 34.6, 32.4, and 42.9%, respectively (Table 5). In reviewing the MLD cow-level test results (Table 5) it can be seen that Se, Sp, PPV, and NPV estimates varied somewhat among the 4 threshold settings reported. For example, when considering the MLD test characteristics when compared with DP interpretation of milk culture, in general Se was fair to good, ranging between 69.4 (setting 12) to 83.3% (setting 1), and Sp was low, ranging between 32.4 (setting 1) and 56.8% (setting 12). Predictive value of a positive test was fair, ranging between 54.0 (setting 6) and 61.0% (setting 12), and NPV was also fair, ranging between 60.9 (setting 6) and 66.7% (settings 1 and 10).
CMT Performance in LL Cows
When examining the CMT test characteristics for LL cows, again the test characteristics generally did not vary dramatically whether using SS, DP, or DS definitions of IMI from milk culture as the referent test (Table 6). For example, when using the SS, DP, and DS methods for defining IMI, the Se estimates for the CMT test at the cut-point ≥1 were 80.9, 86.1, and 86.4%, respectively, whereas the Sp estimates at this same cut-point were 38.5, 37.8, and 14.3%, respectively (Table 6). In reviewing the CMT test characteristics for LL cows we found mild to moderate variation in Se, Sp, PPV, and NPV estimates depending on which CMT cut-point was used to declare a positive test (Table 6). For example, if considering the CMT test characteristics when compared with DP interpretation of milk culture, in general Se was low to good, ranging between 22.2 (cut-point = 3) and 86.1% (cut-point trace), and Sp was low to very good, ranging between 37.8 (cut-point trace) and 91.9% (cut-point = 3). Predictive value of a positive test was fair to good, ranging between 57.4 (cut-point trace) and 72.7% (cut-point = 3), and NPV was also fair, ranging between 54.8 (cut-point = 3) and 73.7% (cut-point trace; Table 6).
Repeatability of MLD Results at Both the Quarter and Cow Level
We found a very high Pearson correlation coefficient (r ≥ 0.98) between paired MLD test results in both EL and LL quarter samples for the quantitative measures TLC, N, L, M, and QSCORE (Table 7). Fair to good correlation existed (r = 0.50–0.69) when comparing N%, L%, and M% between the 2 MLD tests. When examining the final results (positive/negative) from the paired MLD tests from quarters, and at all possible threshold settings agreement between the paired MLD tests to determine a positive test was substantial to almost perfect, ranging between 0.67 (setting 1) and 0.95 (Settings 11 and 12) for EL quarters and ranging from 0.84 to 0.95 for settings 1 to 12 for LL quarters (detailed results for each setting is not shown). Similarly, when examining cow-level MLD results, we observed very good to almost perfect agreement between the duplicate MLD test results, with Kappa values ranging from 0.72 to 0.94 for EL cows, and from 0.82 to 1.0 for LL cows.
Table 7Leukocyte measures from paired milk leukocyte differential (MLD) tests for early- and late-lactation quarters
| Parameter | MLD test | Mean | Median | SD | Minimum | Maximum | R2 between test 1 and 2 |
|---|---|---|---|---|---|---|---|
| Early-lactation quarters (n = 306) | |||||||
| Total leukocytes/mL | 1 | 193,791.8 | 34,860.9 | 735,581.3 | 5,232.5 | 7,804,906.2 | 0.998 |
| 2 | 198,198.8 | 36,266.2 | 753,879.5 | 4,517.4 | 7,936,778.8 | — | |
| Total lymphocytes/mL | 1 | 28,051.1 | 7,392.1 | 79,946.4 | 0 | 632,561.9 | 0.980 |
| 2 | 27,510.2 | 7,433.4 | 75,701.8 | 0 | 550,867.2 | — | |
| Total macrophages/mL | 1 | 44,158.3 | 9,830.8 | 152,241.4 | 0 | 1,357,313.3 | 0.985 |
| 2 | 45,600.9 | 10,021.2 | 170,500.6 | 656.0 | 1,754,445.9 | — | |
| Total neutrophils/mL | 1 | 121,582.4 | 17,625.6 | 519,283.9 | 654.1 | 5,815,030.9 | 0.998 |
| 2 | 125,087.8 | 17,291.2 | 528,518.5 | 1,317.1 | 5,779,067.5 | — | |
| Lymphocytes (%) | 1 | 21.0 | 20.3 | 9.4 | 0 | 50.0 | 0.501 |
| 2 | 21.5 | 20.6 | 9.5 | 0 | 60.0 | — | |
| Macrophages (%) | 1 | 29.5 | 28.6 | 14.2 | 0 | 68.8 | 0.693 |
| 2 | 29.0 | 28.4 | 14.0 | 2.7 | 76.0 | — | |
| Neutrophils (%) | 1 | 49.6 | 50.0 | 12.5 | 12.5 | 83.3 | 0.562 |
| 2 | 49.5 | 49.1 | 12.7 | 13.3 | 80.1 | — | |
| QSCORE 1 QSCORE is a final score derived using a proprietary algorithm that considers the total leukocyte count, total neutrophil count, percentage neutrophils, total lymphocyte count, percentage lymphocytes, total macrophage count, and percentage macrophages (M. Hockett, Advanced Animal Diagnostics Inc., Morrisville, NC, personal communication). | 1 | 0.1111 | 0.00235 | 0.2780 | 8.8307 ×10−9 | 0.9999 | 0.991 |
| 2 | 0.1122 | 0.00199 | 0.2823 | 1.3195 ×10−8 | 0.9999 | — | |
| Late-lactation quarters (n = 283) | |||||||
| Total leukocytes/mL | 1 | 629,512.8 | 139,501.4 | 1,603,775.8 | 7,383.5 | 18,246,640.2 | 0.998 |
| 2 | 624,305.1 | 145,418.4 | 1,597,695.2 | 7,331.2 | 18,821,460.7 | — | |
| Total lymphocytes/mL | 1 | 101,976.4 | 21,647.5 | 254,017.1 | 656.0 | 3,350,738.7 | 0.991 |
| 2 | 100,756.5 | 23,326.8 | 249,418.0 | 637.8 | 3,214,333.2 | — | |
| Total macrophages/mL | 1 | 157,268.9 | 38,740.3 | 302,189.7 | 676.5 | 2,868,983.0 | 0.990 |
| 2 | 155,217.5 | 45,316.1 | 279,057.3 | 0 | 2,529,409.2 | — | |
| Total neutrophils/mL | 1 | 370,267.6 | 64,802.7 | 1,157,383.8 | 3,356.1 | 13,928,588.8 | 0.998 |
| 2 | 368,331.1 | 65,109.1 | 1,183,948.6 | 3,189.0 | 14,827,792.1 | — | |
| Lymphocytes (%) | 1 | 18.4 | 17.1 | 8.0 | 3.7 | 55.6 | 0.636 |
| 2 | 18.3 | 17.2 | 8.1 | 4.6 | 52.9 | — | |
| Macrophages (%) | 1 | 31.7 | 30.8 | 14.2 | 5.5 | 78.2 | 0.825 |
| 2 | 32.3 | 30.4 | 14.8 | 0 | 76.8 | — | |
| Neutrophils (%) | 1 | 49.9 | 49.2 | 13.1 | 12.7 | 83.5 | 0.817 |
| 2 | 49.4 | 48.0 | 13.7 | 13.0 | 82.7 | — | |
| QSCORE | 1 | 2.0146 | 1.95 | 0.6172 | 0.8344 | 4.0827 | 0.963 |
| 2 | 2.0033 | 1.93 | 0.6206 | 0.6673 | 4.1277 | — | |
1 QSCORE is a final score derived using a proprietary algorithm that considers the total leukocyte count, total neutrophil count, percentage neutrophils, total lymphocyte count, percentage lymphocytes, total macrophage count, and percentage macrophages (M. Hockett, Advanced Animal Diagnostics Inc., Morrisville, NC, personal communication).
DISCUSSION
Study Strengths and Limitations
The dairy industry could potentially benefit from having rapid on-farm tests to identify infected quarters or cows, either in early lactation or at dry off, for the purpose of making selective Ab treatment decisions; however, the utility of such tests, which considers such factors such as accuracy, speed, cost, and convenience, must be evaluated before adoption. This is the first independent study that we are aware of to describe the diagnostic test characteristics of a new automated MLD test. One novel aspect of the current study is that we used 3 different approaches to defining true IMI status from milk culture as the referent test (SS, DP, and DS), to investigate if the approach to defining IMI had a large effect on estimates of MLD or CMT diagnostic test characteristics. Additional strengths are that our study was conducted in 3 commercial Midwest dairy herds, and the prevalence of IMI in EL (25.2%) and LL (25.8%) quarters was similar to that reported in previous studies of large commercial dairy herds that also used SS culture of milk to define IMI (
Dingwell et al., 2004
; Arruda et al., 2013
; Johnson et al., 2016
). The fact that gram-positive organisms were the predominant infection type present in both EL and LL quarters is also consistent with other recent studies (Arruda et al., 2013
; Johnson et al., 2016
).One factor that could affect the generalizability of study findings is that all 3 herds had a high bulk tank SCC. As such, our study should be repeated in other low- to moderate-SCC herds to investigate if findings are reproducible.
An additional concern is the level of contamination in samples; however, the contamination rate for single samples, though higher than we would prefer, was not out of line with other studies. For example, of the 336 EL and in 346 LL quarters sampled, 30 (8.9%) and 34 (9.8%), respectively, were omitted from further analysis due to contamination in both samples. The primary concern is that this contamination rate is amplified when samples are interpreted in series (DS), when contamination or discordant cells in either sample disqualifies the quarter from inclusion in the final analysis. For the quarter-level analysis, this resulted in reducing the sample size from 306 to 160 samples in EL (47.7% loss) and from 283 to 152 in LL (45.9% loss). It is difficult to know if these losses are typical, as most mastitis studies using a DS approach to define IMI have failed to report these numbers. However, in one study by
Pinzón-Sánchez and Ruegg, 2011
, the investigators reported that 11.2% (16 of 143) of duplicate quarter samples collected before treatment for clinical mastitis had discordant outcomes. In a second study of dairy sheep and goats, when collecting 3 samples at approximately 15 d intervals, Rovai et al., 2014
reported agreement among sample results 87% of the time. In a third study that collected paired consecutive samples 34 d apart, 46% (497 of 918) of paired quarter samples were lost to analysis due to discordant results (- Rovai M.
- Caja G.
- Salama A.A.K.
- Jubert A.
- Lázaro B.
- Lázaro M.
- Leitner G.
Identifying the major bacteria causing intramammary infections in individual milk samples of sheep and goats using traditional bacteria culturing and real-time polymerase chain reaction.
J. Dairy Sci. 2014; 97 (24996276): 5393-5400
Middleton et al., 2004
). The results of the current study are consistent with the magnitude of losses reported by the latter study, though readers should be cautious when making comparisons, given that our study considered paired duplicate samples collected simultaneously whereas - Middleton J.R.
- Hardin D.
- Steevens B.
- Randle R.
- Tyler J.
Use of somatic cell counts and California Mastitis Test results from individual quarter milk samples to detect subclinical intramammary infection in dairy cattle from a herd with a high bulk tank somatic cell count.
J. Am. Vet. Med. Assoc. 2004; 224: 419-423
Middleton et al., 2004
considered paired consecutive samples. These losses were magnified when we completed the cow-level DS analysis because in the case of a no-growth quarter (i.e., no growth in both S1 and S2), if even 1 of the other 3 quarters had a discordant or contaminated result when comparing S1 and S2, then that entire cow record had to be omitted from analysis. Conversely, if IMI had been diagnosed in 1 quarter (i.e., concordant pathogen in S1 and S2), then that cow would be declared as infected and retained in the DS analysis despite the possible presence of discordant or contaminated results in any of the other quarters. This had the result of significantly reducing our sample size from 73 cows (EL and LL) to 22 cows (EL) and 29 cows (LL), and may also have introduced a bias that inflated our estimate of the true prevalence of IMI in the DS cow sample set, given that all cows with IMI in even 1 quarter would be retained in the data set, but we did have a greater opportunity to exclude no growth cows that might have been truly uninfected. The small final sample size resulted in much wider confidence limits surrounding the point estimates for test characteristics. Furthermore, if the TP was inflated in the DS data sets, then this could affect have the effect of improving the estimates for PPV for the 2 diagnostic tests being evaluated while reducing the estimates for NPV as compared with the PPV and NPV calculated for the other 2 approaches investigated (SS and DP). As such, we suggest interpreting results of the cow-level DS analysis with caution. Future MLD validation studies should consider reexamining this approach to analysis to investigate if results are repeatable.- Middleton J.R.
- Hardin D.
- Steevens B.
- Randle R.
- Tyler J.
Use of somatic cell counts and California Mastitis Test results from individual quarter milk samples to detect subclinical intramammary infection in dairy cattle from a herd with a high bulk tank somatic cell count.
J. Am. Vet. Med. Assoc. 2004; 224: 419-423
Test Performance in EL Quarters and Cows
When interpreting MLD and CMT results at the quarter level, and regardless of the referent test used (SS, SP, or DS), both tests had low Se (≤39.1% for MLD and ≤52.2% for CMT; Table 3, Table 4) but good to very good Sp to identify infection in EL quarters depending on the threshold setting (MLD) or cut-point (CMT) selected. Sensitivity improved if diagnosis was applied at the cow level (≤56.4% for MLD and ≤72.2% for CMT; Table 5, Table 6). The choice of cut-point had a much greater effect on CMT test characteristics than did the choice of threshold setting for the MLD test. Based on these results, if a producer were to apply either the MLD or CMT as the sole test to make quarter-level Ab treatment decisions in EL quarters or cows, and if selecting a cut-point that yields optimal Sp (as would be suggested for a fresh cow screening program), the resulting low Se (<30%) would result in the majority of truly infected quarters or cows going undiagnosed and therefore untreated; thus, the effectiveness of the fresh cow screening program would be decreased. Even though both tests had reasonably high Sp, at the quarter level the PPV estimates tended to be low (≤60% for most settings or cut-points) for both tests, with the exception of using a cut-point of 3 for the CMT test. This raises the concern that a high proportion of test-positive quarters could be treated unnecessarily, increasing program costs. These concerns could be partially addressed by interpreting these tests at the cow (versus quarter) level, as PPV estimates for both tests were higher at the cow level. This concern could also be avoided by selecting a very high cut-point (positive = 3) on the CMT test. To sum up, these results suggest limited utility in applying either the MLD or CMT as the sole test when trying to identify infection in EL quarters or cows for the purpose of treating with Ab, though cow-level interpretation might improve program effectiveness. Large-scale randomized field studies will be needed to investigate these hypotheses.
Test Performance in LL Quarters and Cows
When interpreting MLD and CMT results at the quarter level, and regardless of the referent test used (SS, SP, or DS), both tests had variable Se, depending on threshold or cut-point selection (≤84.8% for MLD and ≤72.7% for CMT), and variable Sp, ranging from fair (MLD) to near excellent (CMT), depending on the threshold setting (MLD) or cut-point (CMT) selected. Sensitivity of both tests improved if applied at the cow level (≤86.4% for MLD and ≤86.4% for CMT), though Sp was reduced. Negative predictive values tended to be higher when interpreting MLD or CMT results at the quarter (versus cow) level. As was true in EL, the choice of cut-point in LL quarters or cows had a much greater effect on CMT test characteristics than did the choice of threshold setting for the MLD test. Also, similar to the EL results, Se of either test tended to improve but the Sp was reduced if test results were interpreted at the cow (versus quarter) level. If a producer were to apply the MLD or CMT as the sole test to make cow-level Ab treatment decisions for a SDCT program, and if selecting a cut-point that yields maximum Se (as might be recommended for a SDCT program), then our results suggest that between 81 and 86% of infected cows would be identified with the MLD test if using the most sensitive threshold (1) and between 75 and 77% of infected cows would be identified if using the current factory setting recommended by AAD (6). Similarly, for the CMT test, between 81 and 86% of infected cows would be identified and treated if using the most sensitive cut-point for test interpretation (trace). However, if using these settings to maximize Se, then Sp will be poor (≤43%) for both tests, resulting in a high proportion of truly uninfected cows that would be treated unnecessarily, increasing costs of the program. Producers considering adopting either test for LL or EL screening programs will need to carefully consider the goals and priorities of a screening program [e.g., to maximize Se, Sp, or overall accuracy when deciding on the level of interpretation (quarter or cow) and when selecting the optimal cut-point for interpreting test results]. Although the results of our study give producers and veterinarians some insight into how these 2 tests will perform on farms, the present study should be repeated in lower SCC herds to determine if results are repeatable. Ultimately field studies evaluating application of the MLD or CMT tests in SDCT or fresh cow screening programs will be needed to describe effect of the program on udder health, convenience, and economics.
Other MLD evaluation studies are not yet available to compare against the current findings; however, our results are consistent with previous studies evaluating the CMT test. When using culture of a single milk sample (SS) as the referent test, Se of the CMT to identify IMI in quarters has been reported to be <57% (
Sargeant et al., 2001
; Middleton et al., 2004
). In the current study we did not compare the MLD or CMT test characteristics to other available rapid tests, such as SCC. Readers should always be cautious when making comparisons between studies, given that different findings may be potentially confounded by different conditions between studies. However, it is interesting to note that the MLD test characteristics observed in our study are numerically similar to test characteristics previously reported when using SCC to diagnose IMI at the quarter level. For example, when using a threshold of >200,000 cell/mL, an earlier study reported the Se of the SCC test for detecting IMI to be 64% at the quarter level when measured on day of dry off and when comparing SCC measures against culture results from DP milk samples (- Middleton J.R.
- Hardin D.
- Steevens B.
- Randle R.
- Tyler J.
Use of somatic cell counts and California Mastitis Test results from individual quarter milk samples to detect subclinical intramammary infection in dairy cattle from a herd with a high bulk tank somatic cell count.
J. Am. Vet. Med. Assoc. 2004; 224: 419-423
Pantoja et al., 2009
). By comparison, in the current study when using culture results from DP samples as the referent test, we estimated the Se of the MLD test to be 62.7% when using setting 6, which is the factory setting recommended by the manufacturer for use on farms. Future validation studies should compare MLD and SCC tests to describe relative diagnostic test characteristics under the same study conditions.Although not our primary aim, the current study afforded us the opportunity to evaluate the effect of choice of definition of IMI from milk culture (SS, DP, or DS) on estimates of MLD and CMT characteristics. We believe this is the first time this question has been investigated. Many mastitis researchers consider culture of multiple samples with series interpretation (DS) to be the preferred approach to defining IMI. However, previous test validation studies have used culture of a single sample (SS) as a referent test by which to assess the diagnostic test characteristics of other indirect tests, such as SCC or CMT (
Sargeant et al., 2001
; Dingwell et al., 2003
). Because culturing duplicate samples can become costly for large studies, culture of a single milk sample is a common practice in field studies. Dohoo et al., 2011b
described the relative diagnostic test characteristics of using culture of a single versus multiple milk samples to diagnose IMI. In that study, the authors reported that overall triplicate samples provided the best combination of Se and Sp, but compared with a single sample, provided only a modest gain in Sp and little or no gain in Se. In the current study, a high proportion of quarters (47.7% in EL; 45.9% in LL) were omitted from the DS analysis due either to contamination of 1 or both samples or discordant culture results. For this reason, it is valid to question how this may affect the validity of the results of using these in the SS and DP analysis. Certainly, many mastitis researchers will argue that the results of the DS analysis is the preferred and more valid referent test. However, and despite the aforementioned concerns about the significantly reduced sample size when using the DS approach to define IMI, it was very interesting to note that there were relatively small differences in estimates of Se and Sp for the MLD and CMT when comparing the SS, DP, and DS analyses. Numerically small improvements in Se estimates for the MLD and CMT tests were observed when using the DS approach to define IMI, but we found virtually no difference in Sp estimates for the MLD and CMT when comparing the 3 approaches. As such, and even though culture of duplicate milk samples might be ideal, our results suggest that researchers may arrive at similar estimates of test characteristics of a new test, whether using SS, DS, or DP as the definition of IMI from milk culture. With series interpretation, only animals that test positive to both tests are considered test positive. With parallel interpretation, animals that test positive to 1 test, the other test, or both tests are considered test positive. Series interpretation increases Sp but decreases Se; whereas parallel testing increases Se and decreases Sp (Dohoo et al., 2009
). As such, given that we anticipated SS and DP interpretation of milk culture would result in a higher relative Se but lower Sp to diagnose presence of IMI as compared with DS (Dohoo et al., 2009
), it was not unexpected that, when comparing MLD or CMT against each of the 3 referent culture methods, the relative Se of the MLD and CMT test was numerically highest when compared against the DS culture method (versus SS or DP methods). Conversely, if any differences were to be observed at all, we might have anticipated that the relative Sp of the MLD and CMT might be numerically lowest when compared against the DS culture method (versus SS or DP methods). However, after analyzing the results, we saw variable and relatively small numeric differences in Sp estimates for the MLD and CMT when comparing across the 3 referent culture methods.Repeatability of the MLD
When examining continuous measures for various leukocyte populations, we found very high correlation between paired MLD test results for both EL and LL quarter samples when duplicate tests were run using the same milk sample. Also, when examining qualitative test results (positive/negative), we observed excellent overall agreement between the paired MLD test results at both the quarter and cow level, indicating that the MLD test is very repeatable. As such, in future research studies it should not be necessary for investigators to run this test in duplicate.
CONCLUSIONS
Under the conditions of this study, the automated MLD test showed excellent repeatability. We noted minor differences in estimates of MLD or CMT characteristics when true IMI status was defined from milk culture of SS, DP, or DS. Estimates of diagnostic test characteristics were relatively comparable between the MLD and CMT tests when applied to EL and LL quarters and cows. When interpreted at the quarter level, the MLD and CMT tests demonstrated low Se and fair to very good Sp in EL quarters, and fair to good Se and Sp in LL quarters, depending on the cut-point selected. Sensitivity improved slightly but Sp declined when test results were interpreted at the cow level. Additional test validation studies as well as large randomized field studies are needed to evaluate the effect of adopting either the MLD or CMT in SDCT or fresh cow screening programs on udder health, Ab use, and economics.
ACKNOWLEDGMENTS
Funding was provided by Elanco Animal Health (Greenfield, IN). The authors are grateful to Advanced Animal Diagnostics (AAD; Durham, NC) for their in-kind and technical support of this project, specifically in loaning us the necessary MLD sampling equipment, readers, and test cassettes, as well interpreting and reporting back the MLD test results. The authors also thank the hard-working team of students who assisted with sampling, as well as the participating dairies.
REFERENCES
- Diagnosing intramammary infections: Evaluating expert opinions on the definition of intramammary infection using conjoint analysis.J. Dairy Sci. 2010; 93: 2966-2975
- Randomized non-inferiority clinical trial evaluating three commercial dry cow mastitis preparations: I. Quarter level outcomes.J. Dairy Sci. 2013; 96 (23628244): 4419-4435
- Evaluation of selective dry cow treatment following on-farm culture: Risk of postcalving intramammary infection and clinical mastitis in the subsequent lactation.J. Dairy Sci. 2014; 97 (24183691): 270-284
- Evaluation of the California mastitis test to detect an intramammary infection with a major pathogen in early lactation dairy cows.Can. Vet. J. 2003; 44 (12757133): 413-415
- Association of cow and quarter-level factors at drying-off with new intramammary infections during the dry period.Prev. Vet. Med. 2004; 63 (15099718): 75-89
- Screening and Diagnostic Tests.in: Veterinary Epidemiologic Research. 2nd ed. VER Inc., Charlottetown, Prince Edward Island, Canada2009: 91-127
- Diagnosing intramammary infections: Comparison of multiple versus single quarter milk samples for the identification of intrammmary infections in lactating dairy cows.J. Dairy Sci. 2011; 94 (22032374): 5515-5522
- Use of total and differential somatic cell counts from composite milk samples to detect mastitis in individual cows.Can. J. Comp. Med. 1981; 45 (7272844): 8-14
- Diagnosing intramammary infections: Evaluation of definitions based on a single milk sample.J. Dairy Sci. 2011; 94: 250-261
- Selective dry cow therapy: We can do this.in: Proc. NMC Regional Meeting. June 29–30, 2016. Appleton, WI. National Mastitis Council, New Prague, MN2016: 8-15
- Influence of dry period bacterial intramammary infection on clinical mastitis in dairy cows.J. Dairy Sci. 2002; 85 (12416812): 2589-2599
- Randomized non-inferiority study evaluating the efficacy of two commercial dry cow mastitis formulations.J. Dairy Sci. 2016; 99 (26585471): 593-607
- The selective treatment of clinical mastitis based on on-farm culture results I: Effects on antibiotic use, milk withholding time and short-term clinical and bacteriological outcomes.J. Dairy Sci. 2011; 94 (21854917): 4441-4456
- Immune cell differentiation in mammary gland tissues and milk of cows chronically infected with Staphylococcus aureus..J. Vet. Med. B Infect. Dis. Vet. Public Health. 2003; 50 (12710501): 45-52
- Use of somatic cell counts and California Mastitis Test results from individual quarter milk samples to detect subclinical intramammary infection in dairy cattle from a herd with a high bulk tank somatic cell count.J. Am. Vet. Med. Assoc. 2004; 224: 419-423
- Dynamics of somatic cell counts and intramammary infections across the dry period.Prev. Vet. Med. 2009; 90 (19409630): 43-54
- Risk factors associated with short-term post-treatment outcomes of clinical mastitis.J. Dairy Sci. 2011; 94 (21700025): 3397-3410
- Identifying the major bacteria causing intramammary infections in individual milk samples of sheep and goats using traditional bacteria culturing and real-time polymerase chain reaction.J. Dairy Sci. 2014; 97 (24996276): 5393-5400
- California Mastitis Test (CMT) Fact Sheet 1.http://milkquality.wisc.edu/wp-content/uploads/2011/09/california-mastitis-test-fact-sheet.pdf(Accessed Sep. 24, 2015.)Date: 2005
- Test characteristics from latent-class models of the California mastitis test.Prev. Vet. Med. 2006; 77 (16876270): 96-108
- Sensitivity and specificity of somatic cell count and California mastitis test for identifying intramammary infection in early lactation.J. Dairy Sci. 2001; 84 (11573781): 2018-2024
- Experiments and observations leading to development of the California mastitis test.J. Am. Vet. Med. Assoc. 1957; 130 (13416088): 199-204
- Using dairy herd improvement records and clinical mastitis history to identify subclinical mastitis at dry-off.J. Dairy Res. 2008; 75 (18474143): 240-247
Article info
Publication history
Published online: June 07, 2017
Accepted:
April 18,
2017
Received:
January 2,
2017
Identification
Copyright
© 2017 American Dairy Science Association®.
