Advertisement

Technical note: Evaluation of odor from vaginal discharge of cows in the first 10 days after calving by olfactory cognition and an electronic device

Open ArchivePublished:July 01, 2013DOI:https://doi.org/10.3168/jds.2013-6813

      Abstract

      The objective of this study was to determine test characteristics (i.e., intra- and interobserver variability, intraassay variability, sensitivity, and specificity) of an evaluation of odor from vaginal discharge (VD) of cows in the first 10 d postpartum conducted by olfactory cognition and an electronic device, respectively. In experiment 1, 16 investigators (9 veterinary students and 7 licensed veterinarians) evaluated 5 VD samples each on 10 different days. The kappa test revealed an agreement between investigators (interobserver) of κ = 0.43 with a Fleiss adjusted standard error of 0.0061. The overall agreement was the same for students (κ = 0.28) and veterinarians (κ = 0.28). Mean agreement within observers (intraobserver) was κ = 0.52 for all observers, and 0.49 and 0.62 for students and veterinarians, respectively. In experiment 2, the repeatability of an electronic device (DiagNose; C-it, Zutphen, the Netherlands) was tested. Therefore, 5 samples of VD from 5 cows were evaluated 10 times each. The repeatability was 0.97, determined by Cronbach’s α. In experiment 3, 20 samples collected from healthy cows and 20 of cows with acute puerperal metritis were evaluated by the 16 investigators and the DiagNose using a dichotomous scale (1 = cow with acute puerperal metritis; 0 = healthy cow). Sensitivity and specificity of olfactory evaluation was 75.0 and 60.1% compared with 92.0 and 100%, respectively, for the electronic nose device. The study revealed a considerable subjectivity of the human nose concerning the classification into healthy and sick animals based on the assessment of vaginal discharge. The repeatability of the electronic nose was higher. In conclusion, the DiagNose system, although imperfect, is a reasonable tool to improve odor assessment of VD. The current system, however, is not suitable as a screening tool in the field. Further research is warranted to adapt such electronic devices to practical on-farm screening tools.

      Key words

      Technical Note

      Acute puerperal metritis (APM) is an acute systemic illness due to an infection of the uterus, usually occurring within 10 d after parturition. It is characterized by an enlarged uterus and uterine discharge varying from watery red-brown to viscous and purulent fluid, which often has a fetid odor. Recently, a 3-grade classification has been suggested to improve diagnosis and therapy (
      • Sheldon I.M.
      • Cronin J.
      • Goetze L.
      • Donofrio G.
      • Schuberth H.-J.
      Defining postpartum uterine disease and the mechanisms of infection and immunity in the female reproductive tract in cattle.
      ). A cow showing an abnormally enlarged uterus and a purulent uterine discharge without any systemic signs of ill health is classified as having grade-1 metritis. At grade-2 and -3 metritis, these signs are accompanied with fever >39.5°C and signs of toxemia and a fetid watery red-brown uterine discharge. The categorization is based on the appearance of fever together with abnormal vaginal discharge (VD), indicative of a generalized infection caused by interactions between the host immune system and bacterial endotoxins (
      • Sheldon I.M.
      • Cronin J.
      • Goetze L.
      • Donofrio G.
      • Schuberth H.-J.
      Defining postpartum uterine disease and the mechanisms of infection and immunity in the female reproductive tract in cattle.
      ). Characteristics used in research and in the field to differentiate between a normal and abnormal VD include color, viscosity, and smell (
      • Sheldon I.M.
      • Dobson H.
      Postpartum uterine health in cattle.
      ;
      • Benzaquen M.E.
      • Risco C.A.
      • Archbald L.F.
      • Melendez P.
      • Thatcher M.-J.
      • Thatcher W.W.
      Rectal temperature, calving-related factors, and the incidence of puerperal metritis in postpartum dairy cows.
      ;
      • Sannmann I.
      • Arlt S.
      • Heuwieser W.
      A critical evaluation of diagnostic methods used to identify dairy cows with acute postpartum metritis in the current literature.
      ).
      Although VD and fever are plausible criteria and have been used in several research trials studying efficacy of different therapies, there is a lack of science-based evidence for their diagnostic value (
      • Sannmann I.
      • Arlt S.
      • Heuwieser W.
      A critical evaluation of diagnostic methods used to identify dairy cows with acute postpartum metritis in the current literature.
      ). Most recent studies have demonstrated that body temperature can be measured with high repeatability (
      • Burfeind O.
      • von Keyserlingk M.A.G.
      • Weary D.M.
      • Veira D.M.
      • Heuwieser W.
      Short communication: Repeatability of measures of rectal temperature in dairy cows.
      ) but is subject to certain variables, such as time of day, parity, and ambient temperature (
      • Burfeind O.
      • Suthar V.S.
      • Heuwieser W.
      Effect of heat stress on body temperature in early postpartum dairy cows.
      ). For a visual assessment of VD in cows suffering from clinical endometritis through vaginoscopic examination, sensitivity and specificity ranged between 96.3 and 99.6% and 90.1 and 96.7%, respectively. Intraobserver (κ = 0.55–0.60) and interobserver (κ = 0.44) repeatability of a VD score on a scale from 0 to 3, however, was only moderate (
      • Leutert C.
      • von Krueger X.
      • Plöntzke J.
      • Heuwieser W.
      Evaluation of vaginoscopy for the diagnosis of clinical endometritis in dairy cows.
      ). To our knowledge, the sensorial assessments of viscosity and smell of VD from cows with APM or clinical endometritis have not been studied yet.
      Odor of VD is associated with the bacterial growth density of potential pathogens in the uterus (
      • Williams E.J.
      • Fischer D.P.
      • Pfeiffer D.U.
      • England G.C.W.
      • Noakes D.E.
      • Dobson H.
      • Sheldon I.M.
      Clinical evaluation of postpartum vaginal mucus reflects uterine bacterial infection and the immune response in cattle.
      ) and seems intuitive to assess without the use of additional diagnostic tools. However, data on test characteristics for the evaluation of VD (intra- and interobserver agreement, sensitivity, and specificity) are lacking (
      • Sannmann I.
      • Arlt S.
      • Heuwieser W.
      A critical evaluation of diagnostic methods used to identify dairy cows with acute postpartum metritis in the current literature.
      ). No evidence exists whether the sensorial assessment of odor of VD is reliable enough to draw sound conclusions concerning the health status of the animal and the necessity of treatments.
      Science-based evidence exists both from accepted clinical (e.g., rectal palpation and temperature measurement) and advanced diagnostic methods (e.g., radiography and ultrasound) that the investigator is a relevant source of measurement errors (
      • Andermann P.
      • Schlögl S.
      • Mäder U.
      • Luster M.
      • Lassmann M.
      • Reiners C.
      Intra- and interobserver variability of thyroid volume measurements in healthy adults by 2D versus 3D ultrasound.
      ;
      • Burfeind O.
      • von Keyserlingk M.A.G.
      • Weary D.M.
      • Veira D.M.
      • Heuwieser W.
      Short communication: Repeatability of measures of rectal temperature in dairy cows.
      ;
      • Leutert C.
      • Suthar V.S.
      • Heuwieser W.
      Evaluation of transrectal examination of cervical diameter by palpation in dairy cows.
      ). Studies testing the repeatability of sensorial assessments of odors using a graded solution of phenol in liquid paraffin and olfactometer threshold tests found a correlation between observers (n = 57 and 98, respectively) varying from correlation coefficient = 0.43 to 0.9 (
      • Fordyce I.D.
      Olfaction tests.
      ;
      • Doty R.L.
      • McKeown D.A.
      • Lee W.W.
      • Shaman P.
      A study of the test-retest reliability of ten olfactory tests.
      ).
      Electronic sensor devices, so-called electronic noses (
      • Gardner J.W.
      • Bartlett P.N.
      A brief history of electronic noses.
      ), have made possible several beneficial applications to a variety of commercial industries, including the agricultural, biomedical, cosmetics, environmental, food, manufacturing, military, pharmaceutical, and regulatory industries, and various scientific research fields (
      • Wilson A.D.
      • Baietto M.
      Applications and advances in electronic-nose technologies.
      ).
      In dairy research, electronic devices have been used for the detection of substances indicating estrus or pathological conditions such as mastitis or respiratory infection in cattle (
      • Eriksson Å.
      • Persson Waller K.
      • Svennersten-Sjaunja K.
      • Haugen J.-E.
      • Lundby F.
      • Lind O.
      Detection of mastitic milk using a gas-sensor array system (electronic nose).
      ;
      • Knobloch H.
      • Schroedl W.
      • Turner C.
      • Chambers M.
      • Reinhold P.
      Electronic nose responses and acute phase proteins correlate in blood using a bovine respiratory infection.
      ;
      • Wiegerinck W.
      • Setkus A.
      • Buda V.
      • Borg-Karlson A.-K.
      • Mozuraitis R.
      • de Gee A.
      BOVINOSE: Pheromone-based sensor system for detecting estrus in dairy cows.
      ). These electronic devices are capable of detecting different types and sources of chemical species and mixtures of compounds present in the headspace volatiles of sampled air. Volatile organic compounds are commonly produced and released from organic sources as living microbes and multicellular organisms (
      • Barsan N.
      • Koziej D.
      • Weimar U.
      Metal oxide-based gas sensors research: How to?.
      ;
      • Wilson A.D.
      • Baietto M.
      Advances in electronic-nose technologies developed for biomedical applications.
      ).
      The overall objective of this study was to evaluate odor from VD of cows in the first 10 d postpartum by olfactory cognition and an electronic nose device. Specifically, we set out (1) to determine the intra- and interobserver variability of the human olfactory cognition, (2) to determine the repeatability of odor assessment conducted with an electronic nose, and (3) to establish sensitivity and specificity of olfactory cognition and the electronic nose device.
      Vaginal discharge samples were collected from a total of 45 cows (22 healthy and 23 with APM) in December 2011, held on a commercial dairy farm in Sachsen-Anhalt, Germany, housing 1,200 Holstein dairy cows. The cows were closely monitored after calving for 10 d postpartum by daily rectal temperature measurement and evaluation of VD (viscosity, color, and odor) according to
      • Sannmann I.
      • Arlt S.
      • Heuwieser W.
      A critical evaluation of diagnostic methods used to identify dairy cows with acute postpartum metritis in the current literature.
      , at 2, 5, and 10 DIM. Cows having fetid, reddish-brown, watery vulvar discharge in combination with a rectal temperature ≥39.5°C (i.e., fever) were characterized as having APM and sampled. All VD samples were collected with a gloved hand from the vagina after cleaning of vulva and perineum with dry paper towels. Approximately 5 mL of VD were transferred into sterile vials (Sarstedt AG & Co., Nürnberg, Germany) and stored at −20°C for later analysis. For repeatability testing, 50 mL of VD from 5 cows each (2 healthy and 3 with APM) was collected and stored in 5-mL aliquots at −20°C.
      Three experiments were conducted to determine (1) intra- and interobserver variability of olfactory cognition (i.e., human nose), (2) intraassay variation of an electronic device (i.e., DiagNose; C-it, Zutphen, the Netherlands), and (3) sensitivity and specificity by olfactory cognition and the electronic device, respectively, to diagnose APM based on odor alone.
      Sixteen investigators were enrolled in experiment 1, including 9 veterinary students in their fourth year and 7 licensed veterinarians working at the Clinic of Animal Reproduction, Freie Universität Berlin (Berlin, Germany). All investigators consented to participate (informed consent) and had the same information about the study design (i.e., nature of substance under investigation and definition of APM) but no information about the health status of the cows.
      A total of 10 appointments in 3- to 4-d intervals were made. All observers evaluated a set of 5 samples (2 healthy and 3 with APM) containing 5 mL of VD. The samples were newly randomized for each appointment with the random number function of Excel (Microsoft Office 2010; Microsoft Deutschland GmbH, Munich, Germany) and labeled with capital letters A to E. At each appointment, new 5-mL samples were incubated in a water bath at 38°C for 10 min before testing. To allow a semiquantitative assessment of odor, a 5-point scale (1 = penetrating, very fetid, 2 = fetid, 3 = slightly fetid, 4 = aromatic, and 5 = neutral) was used.
      In experiment 2, intraassay variation of the DiagNose was determined according to the descriptions of the manufacturer (see below). The system uses an array of 12 micro-hotplate-type metal oxide-sensor modules (i.e., 6 different sensor types in duplicate). The sensors consist of a heating element and a sensor element (a sintered metal oxide). The sensors are temperature modulated under software control and allow temperature cycling from 180 to 340°C. In this range, the metal-oxide sensors behave as semiconductors. When oxygen absorbs and ionizes at the sensor surface, the conductivity is low. Removal of oxygen due to reaction with other gases (redox reaction) results in a measurable change in conductivity. These types of redox reactions are dependent on the nature of the metal-oxide catalyst, the reacting gases, and the temperature (
      • Bruins M.
      • van Belkum A.
      • Bos A.
      The use of electronic nose devices in clinical microbiology.
      ).
      To determine the repeatability of the electronic device, the same 5 samples of VD (2 healthy cows and 3 with APM) were evaluated 10 times each. For calibration, the DiagNose was connected to a gas filter (Respirator RT, Budapest, Hungary) to allow a clean airflow through the system for 15 min. The frozen VD samples were incubated in a water bath at 38.0°C for 10 min with closed caps to ensure a constant vapor pressure and humidity over the fluid. According to the manufacturer’s instructions, a sample volume of 5 mL of VD per 10-mL vial was chosen to get an ideal volatile organic compound saturation within the headspace proportional to the amount of fluid. After thawing, the samples were thoroughly mixed (Reax Top vortex mixer; Heidolph Instruments GmbH & Co. KG, Schwabach, Germany) for 2 s. The cap of the vials was punctured by two 1.6 × 25-mm sterile injection needles (Fine-Ject; Henke Sass Wolf, Tuttlingen, Germany), connected to a Viton tube (2.0-mm diameter and 19.0-cm length) leading into the DiagNose system. This closed circuit allowed only air from the headspace of the sample to enter the DiagNose. After each measurement, the DiagNose system was cleaned by air filtered through active carbon (Carl Roth GmbH & Co. KG, Karlsruhe, Germany) for 10 min.
      In experiment 3, the investigators, identical to the ones described in experiment 1, evaluated 20 samples each, collected from healthy and APM cows, using a dichotomous scale (1 = cow with APM; 0 = healthy cow) to determine sensitivity and specificity using the diagnoses made in the barn as reference. The diagnoses of the investigators had been based on the evaluation of odor alone, whereas the diagnoses in the barn were based on the clinical examination of the cows and the presence of odor and a rectal temperature ≥39.5°C, as described above.
      Data were analyzed using SPSS (version 19.0; SPSS Inc., Munich, Germany) and MedCalc software (version 10.1.3.0; MedCalc Software bvba, Mariakerke, Belgium). The interobserver repeatability between 16 investigators was calculated using the Fleiss kappa test, extended with an additional syntax obtained from http://www.spsstools.net/Syntax/Matrix/CohensKappa.txt (
      • Leutert C.
      • von Krueger X.
      • Plöntzke J.
      • Heuwieser W.
      Evaluation of vaginoscopy for the diagnosis of clinical endometritis in dairy cows.
      ). The intraobserver repeatability was calculated with Cohen’s kappa using the same syntax. The results of the kappa test were interpreted according to the classification κ <0.00 = poor, 0.00 to 0.20 = slight, 0.21 to 0.40 = fair, 0.41 to 0.60 = moderate, 0.61 to 0.80 = substantial, and 0.81 to 1.00 = almost perfect agreement, as described by
      • Landis J.R.
      • Koch G.G.
      The measurement of observer agreement for categorical data.
      .
      Sensitivity and specificity were determined with the receiver operating characteristic (ROC) curve, calculated using the presence of fever (rectal temperature ≥39.5°C) and abnormal vaginal discharge at the day of extraction of VD as a reference standard. This combination of findings has been defined as APM by
      • Sheldon I.M.
      • Lewis G.S.
      • LeBlanc S.
      • Gilbert R.O.
      Defining postpartum uterine disease in cattle.
      . Sensitivity was calculated as the proportion of positive samples correctly diagnosed as APM positive by olfactory evaluation or the DiagNose. Specificity was calculated as the proportion of negative samples correctly diagnosed as negative by olfactory evaluation or the DiagNose.
      For the calculation of interobserver agreement, 5 samples were evaluated 10 times by 16 investigators (i.e., total of 800 evaluations). The kappa test revealed an agreement of κ = 0.43 between different observers (interobserver). The agreement was the same for students (κ = 0.28) and veterinarians (κ = 0.28). When transforming the 5-point scale into a dichotomous scale (1, 2, and 3 = APM; 4 and 5 = healthy), κ was 0.55. Mean agreement within each observer (intraobserver) was κ = 0.52 for all observers (n = 16), and 0.49 (n = 9) and 0.62 (n = 7) for students and veterinarians, respectively (Table 1).
      Table 1Interobserver and intraobserver agreement (κ from the Fleiss kappa test) of odor assessment considering a 5-point and a dichotomous (healthy vs. metritic) classification for a total of 16 investigators
      ObserverOdor assessment
      5-point scaleDichotomous scale
      Penetrating;

      very fetid
      FetidSlightly

      fetid
      AromaticNeutralHealthy

      vs. metritic
      Interobserver repeatability
       All (n = 16)0.310.230.150.320.390.55
       Veterinarians (n = 7)0.240.220.170.280.510.57
       Students (n = 9)0.330.210.170.380.30.54
      Intraobserver repeatability
       All (n = 16)0.440.360.250.430.480.54
       Veterinarians (n = 7)0.630.450.170.40.760.53
       Students (n = 9)0.410.330.220.430.370.55
      For the olfactory assessment of odor, we used a 5-point classification system. In a second step of the analysis, the ordinally scaled data were dichotomized, 1 of the 2 outcomes (cows with APM) being defined as the event of intervention. This was done to test whether the agreement would increase, allowing a more robust distinction between diseased and healthy cows. The threshold was chosen through a survey performed at the end of the study by all participants. Investigators were asked to state a threshold for treatment intervention based exclusively on the odor of VD. The survey revealed that none of the investigators would treat a cow with a VD score of 4 (i.e., aromatic) or 5 (i.e., neutral) with antibiotics. Vaginal discharge with an odor score of 1 to 3 caused the investigators to consider therapeutic interventions. The kappa test revealed a moderate overall agreement for the olfactory assessment. Agreement between observers (interobserver; κ = 0.43) was lower than within observers (intraobserver; κ = 0.52), which is in agreement with other studies evaluating clinical tests (
      • Thomsen P.T.
      • Baadsgaard N.P.
      Intra- and inter-observer agreement of a protocol for clinical examination of dairy cows.
      ;
      • Pedersen K.S.
      • Toft N.
      Intra- and inter-observer agreement when using a descriptive classification scale for clinical assessment of faecal consistency in growing pigs.
      ;
      • Leutert C.
      • von Krueger X.
      • Plöntzke J.
      • Heuwieser W.
      Evaluation of vaginoscopy for the diagnosis of clinical endometritis in dairy cows.
      ). Differences in the perception of odors between and within observers can be accounted for by various factors such as age, experience, and environment (
      • Doty R.L.
      • McKeown D.A.
      • Lee W.W.
      • Shaman P.
      A study of the test-retest reliability of ten olfactory tests.
      ;
      • Hadley K.
      • Orlandi R.R.
      • Fong K.J.
      Basic anatomy and physiology of olfaction and taste.
      ;
      • Orhan K.S.
      • Karabulut B.K.
      • Keles N.
      • Deger K.
      Evaluation of factors concerning the olfaction using the sniffin’ sticks test.
      ). The veterinarians were more experienced diagnosing APM compared with the students and had higher agreement within observers (intraobserver) Agreement between observers (interobserver), sensitivity, and specificity, however, did not differ between veterinarians and students. Even though experience had an influence on agreement, the ability to distinguish healthy cows from cows with APM did not differ between the 2 groups.
      In experiment 2, the raw data from the DiagNose were processed using a proprietary (C-it) multiway decomposition algorithm. In brief, each individual sensor gives a 3-dimensional matrix for each measurement (time of measurement vs. thermal cycle vs. amplitude) as a result. As the DiagNose unit contains multiple sensors, a set of matrices make up 1 measurement. Although all sensors are measured in parallel, any arbitrary selection and combination of sensors can be extracted for data analysis. For each measurement, a vector was generated containing the scores of the underlying multiway kernels of the compression. These score vectors were used for the subsequent pattern recognition, as they contain the maximum amount of information of the measurement. As there were duplicate sensors (2 of each type), a total of 80 patterns (20 healthy and 20 metritic cows × 2) were generated. With this data set, we performed a leave-2-out cross-validation (
      • Airola A.
      • Pahikkala T.
      • Waegeman W.
      • De Baets B.
      • Salakoski T.
      An experimental comparison of cross-validation techniques for estimating the area under the ROC curve.
      ). The models were derived using a nonlinear artificial neural network model (

      Malmgren, H. 2000. Artificial neural networks in medicine and biology. A philosophical introduction. Opening lecture at the Artificial Neural Networks in Medicine and Biology (ANNIMAB)-1 conference, Göteborg, Sweden. Accessed Apr. 15, 2013. http://gupea.ub.gu.se/handle/2077/19466

      ). The leave-2-out results are presented in Figure 1.
      Figure thumbnail gr1
      Figure 1Results of the leave-2-out cross-validation from the DiagNose system (C-it, Zutphen, the Netherlands), considering 20 healthy cows (black asterisks) and 20 cows with acute puerperal metritis (black squares). Each 2 consecutive points relate to 1 measurement, as there were 2 identical sensors in the measurement system. The artificial neural network was trained to give a 0 if the cow was healthy and a 1 if the cow was sick. Color version available in the online PDF.
      The artificial neural network was trained to give a 0 if the cow was healthy (i.e., no APM) and a 1 if the cow was metritic. An ROC curve was calculated (Figure 2) from the prediction results of the leave-2-out cross-validation (
      • Fawcett T.
      An introduction to ROC analysis.
      ). Sensitivity was 92.0% and specificity 100%, with an area under the curve of 0.99 (P = 0.00) when using 0.5 as the threshold value. To test the repeatability of measurements, the relative conductivity response (in microsiemens) of each sensor (n = 12) was used for further analysis. The relative response (Gr) of the sensor was calculated by Gr = (GmaxGmin)/Gmin, where Gmax and Gmin are the maximum and minimum conductivity response of the sensor, respectively. This method uses only 1 parameter (i.e., the relative sensor response) of the measurement at a given time and, thus, only a small part of the total available measurement information. Subsequently, we performed an ROC analysis with the results of the relative responses for each sensor pair. The PtE0 sensor (AS-MLX, Applied Sensors GmbH, Reutlingen, Germany) with sensitivity of 100%, specificity of 73.3%, and an area-under-the-curve value of 0.87 was the sensor with the highest values and, therefore, chosen for further analysis. The within-class correlation coefficient was 0.76 and 0.97 for single and average (n = 10) measures, respectively. The repeatability was tested using Cronbach’s α. The resulting α was 0.97 for the sensor described above (PtE0).
      Figure thumbnail gr2
      Figure 2Receiver operating characteristic (ROC) curve constructed from the leave-2-out cross-validation from the DiagNose (C-it, Zutphen, the Netherlands) measurements for 20 healthy cows and 20 cows with acute puerperal metritis. Color version available in the online PDF.
      Odor measurements from VD of healthy cows and cows with APM using the electronic device were different (P < 0.05 based on the area under the calculated ROC curve), as shown in Figure 1. This difference provides the first evidence that the DiagNose could be used as a screening tool to identify cows with APM based on an electronic assessment of odor from VD. In our study, however, the electronic device used was restricted to a laboratory setting. In a pretest with 18 VD samples, the Pearson correlation coefficient for native and frozen samples was calculated. Based on the satisfying results (r = 0.88, P = 0.01) we decided to store the VD samples after collection at −20°C until analysis. To reduce a potential bias as much as possible, we implemented a standardized thawing procedure and examined all samples at the same temperature (38°C) . Another potential factor that might have influenced the measurements could be the composition of the VD itself. Although all VD samples were thoroughly shaken, the fluids were not all homogenous. Vaginal discharge often consists of different compounds, such as blood, pus, and cell fragments (
      • Olson J.
      • Bretzlaff K.
      • Mortimer R.G.
      • Ball L.
      The metritis-pyometra complex.
      ). This composition could have contributed to limited agreement within observers (intraobserver), as it is possible that the composition of blood, cell fragments, and pus was not identical in all of the aliquots used.
      A direct comparison between the olfactory and electronic assessment is challenging because data from the olfactory assessment are ordinal, whereas those obtained by the electronic nose are continuous (microsiemens) and, consequently, different statistical tests were applied. The sensitivity and specificity of the electronic nose device, however, were higher than those of the olfactory assessment. Data of all 12 sensors resulted in twelve 3-dimensional matrices. These multidimensional data are not eligible for repeatability analysis. Therefore, we chose the best-performing sensor (PtE0) for the repeatability analysis, despite losing a considerable part of the total information available in the process. Notwithstanding, the repeatability was 0.97 (Cronbach’s α), which is excellent (

      George, D., and P. Mallery. 2002. SPSS for Windows: Step by Step: A Simple Guide and Reference: 11.0 Update. 4th ed. Allyn & Bacon, Boston, MA.

      ).
      Furthermore, repeatability of odor assessment using the electronic nose was high, whereas the olfactory assessment with multiple observers showed considerable variability. The intraobserver variability was lower than the interobserver variability. In agreement with previous studies evaluating palpation both in vivo and in vitro and ultrasound examinations, the experience of the observers did not influence the results (
      • Steyn M
      • Becker P.J.
      • L’Abbé E.N.
      • Scholtz Y.
      • Myburgh J.
      An assessment of the repeatability of pubic and ischial measurements.
      ;
      • Leutert C.
      • Suthar V.S.
      • Heuwieser W.
      Evaluation of transrectal examination of cervical diameter by palpation in dairy cows.
      ). Although several applications of electronic measuring systems for odors related to estrus, mastitis, and respiratory diseases in cattle have been described previously (
      • Eriksson Å.
      • Persson Waller K.
      • Svennersten-Sjaunja K.
      • Haugen J.-E.
      • Lundby F.
      • Lind O.
      Detection of mastitic milk using a gas-sensor array system (electronic nose).
      ;
      • Knobloch H.
      • Schroedl W.
      • Turner C.
      • Chambers M.
      • Reinhold P.
      Electronic nose responses and acute phase proteins correlate in blood using a bovine respiratory infection.
      ;
      • Wiegerinck W.
      • Setkus A.
      • Buda V.
      • Borg-Karlson A.-K.
      • Mozuraitis R.
      • de Gee A.
      BOVINOSE: Pheromone-based sensor system for detecting estrus in dairy cows.
      ), data on repeatability and test characteristics to compare with our data are not available.
      Sensitivity and specificity of olfactory evaluation was 75.0 and 60.1%, respectively, considering presence of fever and abnormal VD at the day of VD collection as a reference. These criteria have been used in several previous studies to distinguish between healthy and metritic cows (
      • Sheldon I.M.
      • Cronin J.
      • Goetze L.
      • Donofrio G.
      • Schuberth H.-J.
      Defining postpartum uterine disease and the mechanisms of infection and immunity in the female reproductive tract in cattle.
      ;
      • Benzaquen M.E.
      • Risco C.A.
      • Archbald L.F.
      • Melendez P.
      • Thatcher M.-J.
      • Thatcher W.W.
      Rectal temperature, calving-related factors, and the incidence of puerperal metritis in postpartum dairy cows.
      ;
      • Sannmann I.
      • Arlt S.
      • Heuwieser W.
      A critical evaluation of diagnostic methods used to identify dairy cows with acute postpartum metritis in the current literature.
      ). Because a sensory evaluation of odor of VD was part of the cowside diagnosis (i.e., healthy vs. APM), the sensitivity and specificity determined in the laboratory was biased by the inherent limitations of olfactory cognition by an investigator conducting the examination. As
      • Sheldon I.M.
      • Dobson H.
      Postpartum uterine health in cattle.
      pointed out, no gold standard exists for APM. Therefore, confounding through type I and type II errors cannot be fully excluded. This study was designed as a diagnostic accuracy cohort study, which is appropriate for the early investigation of an experimental diagnostic test to determine if further research is warranted, but such studies provide only relative accuracy of an experimental diagnostic test compared with a reference standard (
      • Rodger M.
      • Ramsay T.
      • Ferguson D.
      Diagnostic randomized controlled trials: The final frontier.
      ). Furthermore, the nature of VD even varies in healthy cows up to a certain degree. Abnormal VD without a fever could occur in healthy cows as a result of the opening of the cervix at 7 to 10 DIM (
      • Wehrend A.
      • Failing K.
      • Bostedt H.
      Cervimetry and ultrasonographic observations of the cervix regression in dairy cows during the first 10 days post partum.
      ) and cause type I errors by the assessment of VD alone. To minimize the risk of such errors, only VD samples collected at DIM 5 were used to avoid a potential bias caused by this natural phenomenon for the evaluation of sensitivity and specificity.
      The relatively low sensitivity (75.0%) and specificity (60.1%) for the olfactory assessment can have been caused by different reasons. It is possible that the color and viscosity of the VD samples distracted the investigators. All observers were familiar with the definition of APM, which includes odor, color, consistency, and a rectal temperature ≥39.5°C and, therefore, may have unconsciously included these attributes into their conclusion. This could be a potential bias, as color and consistency of lochia can differ widely in a physiological range (
      • Elliott L.
      • McMahon K.J.
      • Gier H.T.
      • Marion G.B.
      Uterus of the cow after parturition: Bacterial content.
      ;
      • Olson J.
      • Bretzlaff K.
      • Mortimer R.G.
      • Ball L.
      The metritis-pyometra complex.
      ;
      • Kumar N.S.
      • Singh D.V.
      • Umapathi V.
      • Prasad J.K.
      • Singh S.K.
      • Kumar A.
      Nature and properties of lochia collected from crossbred cows maintained under farm condition.
      ). Disregarding color and consistency may be an advantage of the DiagNose system measuring only the concentration of odorous gases. Furthermore, investigators did not have any information on rectal temperature of these cows, which might have biased the diagnosis as well.
      The sensitivity (92%) and specificity (100%) of the DiagNose system was higher than the olfactory assessment. Figure 1, however, shows 3 values obtained from metritic cows below the threshold of 0.5. These are either false-negative diagnoses by the DiagNose (i.e., type I error) or a misclassification of the cows as having APM at the time of VD collection by the authors.
      This is the first validation of a measuring system to electronically assess odor of VD from cows, with the objective to distinguish between healthy and metritic cows. Our data provide the first evidence that the DiagNose system, although imperfect, is a useful tool to improve odor assessment of VD. The current system, however, is not suitable as a screening tool in the field. Further efforts are warranted to (1) adapt such electronic devices to on-farm screening tools, providing just-in-time findings and (2) to determine test characteristics, considering fertility as a gold standard. Furthermore, our data show that the assessment of odor from VD by the human nose is highly subjective and might be a confounder of a correct diagnosis regarding the health status of the uterus.

      References

        • Airola A.
        • Pahikkala T.
        • Waegeman W.
        • De Baets B.
        • Salakoski T.
        An experimental comparison of cross-validation techniques for estimating the area under the ROC curve.
        Comput. Stat. Data Anal. 2011; 55: 1828-1844
        • Andermann P.
        • Schlögl S.
        • Mäder U.
        • Luster M.
        • Lassmann M.
        • Reiners C.
        Intra- and interobserver variability of thyroid volume measurements in healthy adults by 2D versus 3D ultrasound.
        Nuklearmedizin. 2007; 46: 1-7
        • Barsan N.
        • Koziej D.
        • Weimar U.
        Metal oxide-based gas sensors research: How to?.
        Sensors and Actuators B. 2007; 121: 18-35
        • Benzaquen M.E.
        • Risco C.A.
        • Archbald L.F.
        • Melendez P.
        • Thatcher M.-J.
        • Thatcher W.W.
        Rectal temperature, calving-related factors, and the incidence of puerperal metritis in postpartum dairy cows.
        J. Dairy Sci. 2007; 90: 2804-2814
        • Bruins M.
        • van Belkum A.
        • Bos A.
        The use of electronic nose devices in clinical microbiology.
        in: Hays J.P. Leeuwen W.B.van The Role of New Technologies in Medical Microbiological Research and Diagnosis. Bentham Science Publishers, Sharjah, United Arab Emirates2012: 90-110
        • Burfeind O.
        • Suthar V.S.
        • Heuwieser W.
        Effect of heat stress on body temperature in early postpartum dairy cows.
        Theriogenology. 2012; 78: 2031-2038
        • Burfeind O.
        • von Keyserlingk M.A.G.
        • Weary D.M.
        • Veira D.M.
        • Heuwieser W.
        Short communication: Repeatability of measures of rectal temperature in dairy cows.
        J. Dairy Sci. 2010; 93: 624-627
        • Doty R.L.
        • McKeown D.A.
        • Lee W.W.
        • Shaman P.
        A study of the test-retest reliability of ten olfactory tests.
        Chem. Senses. 1995; 20: 645-656
        • Elliott L.
        • McMahon K.J.
        • Gier H.T.
        • Marion G.B.
        Uterus of the cow after parturition: Bacterial content.
        Am. J. Vet. Res. 1968; 29 (): 77-81
        • Eriksson Å.
        • Persson Waller K.
        • Svennersten-Sjaunja K.
        • Haugen J.-E.
        • Lundby F.
        • Lind O.
        Detection of mastitic milk using a gas-sensor array system (electronic nose).
        Int. Dairy J. 2005; 15: 1193-1201
        • Fawcett T.
        An introduction to ROC analysis.
        Pattern Recognit. Lett. 2006; 27: 861-874
        • Fordyce I.D.
        Olfaction tests.
        Br. J. Ind. Med. 1961; 18: 213-215
        • Gardner J.W.
        • Bartlett P.N.
        A brief history of electronic noses.
        Sensors and Actuators B. 1994; 18: 211-220
      1. George, D., and P. Mallery. 2002. SPSS for Windows: Step by Step: A Simple Guide and Reference: 11.0 Update. 4th ed. Allyn & Bacon, Boston, MA.

        • Hadley K.
        • Orlandi R.R.
        • Fong K.J.
        Basic anatomy and physiology of olfaction and taste.
        Otolaryngol. Clin. North Am. 2004; 37: 1115-1126
        • Knobloch H.
        • Schroedl W.
        • Turner C.
        • Chambers M.
        • Reinhold P.
        Electronic nose responses and acute phase proteins correlate in blood using a bovine respiratory infection.
        Sensors and Actuators B. 2010; 144: 81-87
        • Kumar N.S.
        • Singh D.V.
        • Umapathi V.
        • Prasad J.K.
        • Singh S.K.
        • Kumar A.
        Nature and properties of lochia collected from crossbred cows maintained under farm condition.
        Ind. J. Anim. Prod. Manage. 2009; 25: 27-32
        • Landis J.R.
        • Koch G.G.
        The measurement of observer agreement for categorical data.
        Biometrics. 1977; 33: 159-174
        • Leutert C.
        • Suthar V.S.
        • Heuwieser W.
        Evaluation of transrectal examination of cervical diameter by palpation in dairy cows.
        J. Dairy Sci. 2013; 96: 1063-1070
        • Leutert C.
        • von Krueger X.
        • Plöntzke J.
        • Heuwieser W.
        Evaluation of vaginoscopy for the diagnosis of clinical endometritis in dairy cows.
        J. Dairy Sci. 2012; 95: 206-212
      2. Malmgren, H. 2000. Artificial neural networks in medicine and biology. A philosophical introduction. Opening lecture at the Artificial Neural Networks in Medicine and Biology (ANNIMAB)-1 conference, Göteborg, Sweden. Accessed Apr. 15, 2013. http://gupea.ub.gu.se/handle/2077/19466

        • Olson J.
        • Bretzlaff K.
        • Mortimer R.G.
        • Ball L.
        The metritis-pyometra complex.
        in: Morrow D.A. Current Therapy in Theriogenology. W. B. Saunders, Co., Philadelphia, PA1986: 227-288
        • Orhan K.S.
        • Karabulut B.K.
        • Keles N.
        • Deger K.
        Evaluation of factors concerning the olfaction using the sniffin’ sticks test.
        Otolaryngology. 2012; 146: 240-246
        • Pedersen K.S.
        • Toft N.
        Intra- and inter-observer agreement when using a descriptive classification scale for clinical assessment of faecal consistency in growing pigs.
        Prev. Vet. Med. 2011; 98: 288-291
        • Rodger M.
        • Ramsay T.
        • Ferguson D.
        Diagnostic randomized controlled trials: The final frontier.
        Trials. 2012; 13: 137
        • Sannmann I.
        • Arlt S.
        • Heuwieser W.
        A critical evaluation of diagnostic methods used to identify dairy cows with acute postpartum metritis in the current literature.
        J. Dairy Res. 2012; 79: 436-444
        • Sheldon I.M.
        • Cronin J.
        • Goetze L.
        • Donofrio G.
        • Schuberth H.-J.
        Defining postpartum uterine disease and the mechanisms of infection and immunity in the female reproductive tract in cattle.
        Biol. Reprod. 2009; 81: 1025-1032
        • Sheldon I.M.
        • Dobson H.
        Postpartum uterine health in cattle.
        Anim. Reprod. Sci. 2004; 82–83: 295-306
        • Sheldon I.M.
        • Lewis G.S.
        • LeBlanc S.
        • Gilbert R.O.
        Defining postpartum uterine disease in cattle.
        Theriogenology. 2006; 65: 1516-1530
        • Steyn M
        • Becker P.J.
        • L’Abbé E.N.
        • Scholtz Y.
        • Myburgh J.
        An assessment of the repeatability of pubic and ischial measurements.
        Forensic Sci. Int. 2011; 214: 210.e1-210.e4
        • Thomsen P.T.
        • Baadsgaard N.P.
        Intra- and inter-observer agreement of a protocol for clinical examination of dairy cows.
        Prev. Vet. Med. 2006; 75: 133-139
        • Wehrend A.
        • Failing K.
        • Bostedt H.
        Cervimetry and ultrasonographic observations of the cervix regression in dairy cows during the first 10 days post partum.
        J. Vet. Med. A Physiol. Pathol. Clin. Med. 2003; 50: 470-473
        • Wiegerinck W.
        • Setkus A.
        • Buda V.
        • Borg-Karlson A.-K.
        • Mozuraitis R.
        • de Gee A.
        BOVINOSE: Pheromone-based sensor system for detecting estrus in dairy cows.
        Procedia Comp. Sci. 2011; 7: 340-342
        • Williams E.J.
        • Fischer D.P.
        • Pfeiffer D.U.
        • England G.C.W.
        • Noakes D.E.
        • Dobson H.
        • Sheldon I.M.
        Clinical evaluation of postpartum vaginal mucus reflects uterine bacterial infection and the immune response in cattle.
        Theriogenology. 2005; 63: 102-117
        • Wilson A.D.
        • Baietto M.
        Applications and advances in electronic-nose technologies.
        Sensors (Basel). 2009; 9: 5099-5148
        • Wilson A.D.
        • Baietto M.
        Advances in electronic-nose technologies developed for biomedical applications.
        Sensors (Basel). 2011; 11: 1105-1176