If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Farm Technology Group, Wageningen University & Research, PO Box 16, Wageningen, 6700 AA, the NetherlandsSensors and Data Analysis Department, Lely Innovation, Cornelis van der Lelylaan 1, Maassluis, 3147 PB, the Netherlands
Farm Technology Group, Wageningen University & Research, PO Box 16, Wageningen, 6700 AA, the NetherlandsSensors and Data Analysis Department, Lely Innovation, Cornelis van der Lelylaan 1, Maassluis, 3147 PB, the Netherlands
Machine vision technology has been used in automated body condition score (BCS) classification of dairy cows. The current vision-based classifications use information acquired from a limited number of body regions of the cow. Our study aimed to improve automated BCS classification by including multiple body condition–related features extracted from 3 viewpoints in 8 body regions. The data set of this study included 44 lactating cows with their BCS evenly distributed over the scale of BCS from 1.5 to 4.5 units. The body images of these cows were recorded over 2 consecutive days using 3-dimensional cameras positioned to view the cow from the top, right side, and rear. Each image was automatically processed to identify anatomical landmarks on the body surface. Around these anatomical landmarks, the bony prominences and body surface depressions were quantified to describe 8 body condition–related features. A manual BCS of each cow was independently assigned by 2 trained assessors using the same predefined protocol. With the extracted features as inputs and manual BCS as the reference, we built a nearest-neighbor classification model to classify BCS and obtained an overall classification sensitivity of 0.72 using a 10-fold cross-validation. We conclude that the sensitivity of automated BCS classification has been improved by expanding the selection of body condition–related features extracted from multiple body regions.
Relationships between body condition score change, prior mid-lactation phenotypic residual feed intake, and hyperketonemia onset in transition dairy cows.
Effect of stocking rate and animal genotype on dry matter intake, milk production, body weight, and body condition score in spring-calving, grass-fed dairy cows.
). It varies throughout lactation and is a net result of fat storage and depletion. Abnormal and sudden variation in body condition can be a sign of metabolic failure caused by disorder, disease, or improper management (
Relationships between body condition score change, prior mid-lactation phenotypic residual feed intake, and hyperketonemia onset in transition dairy cows.
). Therefore, dairy farmers are advised to assess their cows' body condition regularly to prevent metabolic failure and maintain individual cow welfare.
Body condition is usually quantified by a BCS, which is assigned by trained assessors using a predefined scoring protocol consisting of, for example, a 5-point scoring system with 9 levels of fatness from lean to obese (
). To incorporate high-quality manual scoring in farm management, farmers must regularly hire experienced assessors or receive training to score their own cows, but both approaches are labor intensive and costly (
). Consequently, regular high-quality manual body condition scoring of individual cows is difficult to incorporate as a routine procedure in farm management.
Two sensor technologies have been applied to replace the costly manual scoring method: ultrasonography and machine vision. Ultrasonography has been used to measure the subcutaneous fat thickness of dairy cows directly (
), the procedure remains labor intensive because it is conducted with a handheld device. In contrast, machine vision can automate body condition scoring and this automation can be inexpensive. Vision cameras, including 2-dimensional (2-D;
) cameras, have been used to automatically extract body condition–related features. These features include bony prominences and body surface depressions, both of which are related to subcutaneous fat reserves and are quantified as variables of the automated body condition scoring.
In current machine vision–based studies, BCS is often assumed to be a continuous variable and is predicted with regression models. However, BCS is an ordinal variable because it classifies cows into different categories of body condition. Moreover, all machine vision–based studies are conducted using a single camera. The single camera views fewer body regions than those assessed by manual scoring. Recent studies have demonstrated the feasibility of using multiple 3-D cameras to record the entire body surface of the cow (
). We hypothesized that features can be extracted from all body regions assessed in manual body condition scoring using multiple 3-D cameras and that machine vision–based scoring will be improved if all of these features are included. Therefore, we aim to improve the 3-D vision-based BCS classification by expanding the selection of the automatically extracted body features from multiple viewpoints.
MATERIALS AND METHODS
Cow Selection
We aimed to build a balanced data set with 5 cows in each half-point BCS class. Two Dutch (farms A and B) and 2 German (farms C and D) farms were selected based on the expected body condition of the cows, which were suggested by local farm management advisors. Farm A was selected for its high proportion of lean cows, farms B and C were selected for their average cows, and farm D was selected for its obese cows. All farms were commercial freestall barns equipped with automatic milking systems (Astronaut A4, Lely, Maassluis, the Netherlands). On each farm, half of the herd was randomly selected and independently assigned a BCS by 2 trained assessors following a manual body condition scoring protocol (Supplemental Figure S1; https://doi.org/10.3168/jds.2018-15238). When a cow received the same BCS from both assessors and its BCS contributed to the balanced data set, it was selected for this study.
Data Collection
Data from 44 cows were collected twice on 2 consecutive days in October 2017. The cows were Holstein Friesian (n = 33) and Simmental (n = 11), from 34 to 451 DIM, with parities ranging from 1 to 11. On the days of data collection, all selected cows were gathered at the feed fences in a separate area for manual body condition scoring and image recording.
Manual Body Condition Scoring
On the days of data collection, 2 assessors independently assessed all 44 cows. Before cow selection and data collection, the 2 assessors were trained by an expert in how to use the manual body condition scoring protocol, as presented in Supplemental Figure S1 (https://doi.org/10.3168/jds.2018-15238). This method is based on the work of
and is used by advisors of GD Animal Health (Deventer, the Netherlands). The protocol is a guidance to assess the fatness of 8 body regions on a 5-point scale from lean to obese with half-point increments (i.e., 9 levels). The protocol focuses on the right side of the body to help the assessor visually assess (1) the bony prominences of the spinous and transverse processes and hook and pin bones, and (2) the body surface depressions above the transverse processes, the thurl area, the area between the spinous processes and hook bone, and the cavity between the tail head and pin bone (Figure 1). In addition, this protocol guides the assessor in assigning an overall BCS based on the conditions of all of the assessed body regions.
Figure 1Body regions assessed in the manual body condition scoring protocol.
A moveable setup for image recording was built such that it could be placed behind a cow to capture its 3-D body surface from different views with 3-D cameras. The setup consisted of an aluminum frame supporting 3 identical 3-D cameras (Realsense SR300, Intel, Santa Clara, CA) to record images from the top, right, and rear sides of the cow. The mounting positions and angles of each camera on the frame were selected to cover all body regions assessed in the manual scoring protocol (Figure 2). The cameras and the corresponding body regions that they covered are listed in Table 1.
Figure 2Image recording setup showing the mounting positions and angles of the 3 cameras.
Table 1Body condition–related feature extractions in their corresponding body regions, anatomy in vivo, camera view, anatomical landmarks, and rectangular regions of interest
Body region
Anatomy in vivo
Camera view
Anatomical landmark
Rectangular region of interest
Spinous processes
Bony projection of the posterior of each vertebra in the sacral region
Top
Vertebral column
11 columns with the spine trend line in the center and from the centerline of the sacral ligament to the end of the body
Spinous to transverse processes
Area between the tips of the spinous and transverse processes of each vertebra in the lumbar region
Top
Vertebral column, hook bone, sacral ligament
From the vertebral column to the center of the hook bone and 20 rows starting from the top edge of the sacral ligament in the cranial direction
Overhanging shelf
Area near the tips of the transverse processes
Side
Hook bone
25 columns starting from the edge of the hook bone in the cranial direction and 10 rows starting from the edge of the hook bone in the ventral direction
Hook bone
Tuber coxae
Side
Hook bone
21 rows and columns with the hook bone as the square center
Pin bone
Tuber ischii
Rear
Pin bone
21 rows and columns with the pin bone as the square center
Thurl area
Area where the pelvis connects to the femur bone through the hip joint
Top
Vertebral column, sacral ligament
From the vertebral column to the center of the hook bone and from the lower edge of the sacral ligament to the end of the body
Spinous processes to hook bone
Ligament connecting the spinous process and the tuber coxae ossis ilium
Top
Vertebral column, hook bone, sacral ligament
From the vertebral column to the center of the hook bone and 11 rows with the centerline of the sacral ligament in the center
Cavity between tail head and pin bone
Area between the coccyx and tuber ischii in the caudal region
Rear
Tail region, pin bone
From the tail head to the pin bone center and from the top of the tail to the pin bone center
The 3-D cameras were coded light cameras with a resolution of 640 × 480 pixels and field-of-view angles of 71.5° (horizontal) and 55.0° (vertical). The camera viewing distance ranged from 0.2 to 1.5 m, with a pixel size of 0.002 (horizontal) × 0.002 (vertical) × 0.001 m (depth) at a viewing distance of 1.0 m. Each camera was connected to a USB 3.0 port of a recording computer (NUC-6i3SYK, Intel).
After a cow underwent manual BCS assessment, the setup was manually moved behind the cow. When the cow was standing still, image recording was manually initiated. Five images were consecutively recorded at 2 Hz from the top-view camera, from the side-view camera, and finally from the rear-view camera. After recording, the cow was returned to the herd, and data collection continued for the next cow.
Each raw image provided a depth map with each image pixel containing a distance measurement from the center of the camera lens to the projection of the pixel on an object. The specific pixels projected the body surface of a cow, the floor, other cows in the camera view, or the frame of the image recording setup. Supplemental Figure S3 (https://doi.org/10.3168/jds.2018-15238) shows an example of a raw top-view image.
Because the 3 cameras were mounted at different locations in the image recording setup, all the images from different cameras were first transformed from their corresponding camera coordinate systems to that of the image recording setup. The transformation included rotation and orthogonal decomposition. First, the image was rotated by multiplying the rotation matrices (Equation [1]) with the camera-mounting angle about each axis. The top-view image was rotated 180° about the y-axis and 90° about the z-axis, the side-view image was rotated 160° about the y-axis and 90° about the z-axis, and the rear-view image was rotated 160° about the x-axis. Then, the image was orthogonally decomposed into X, Y, and Z matrices in the image recording setup coordinate system:
[1]
where θx, θy, and θz are the clockwise rotation angles and Rx(θx), Ry(θy), and Rz(θz) are the rotation matrices for rotating images about the x-, y-, and z-axes respectively, in 3 dimensions. An example of a Z matrix of a top-view image is shown in Supplemental Figure S4A (https://doi.org/10.3168/jds.2018-15238).
Background Subtraction
The background in an image was defined as the pixels that were not projected onto the body surface of the selected cow. The background included the floor, the bodies of other cows, and the recording setup frame. Based on the distance between the camera lens and the pixel projection, these background pixels were removed from all decomposed matrices to retain only the selected cow body surface. For a top-view image, the background pixels were identified from the Z matrix with the selection criteria shown in Equation [2]. The criteria were determined by estimating the range of body surface heights of all the cows in top-view images. An example of a top-view image after background subtraction is shown in Supplemental Figure S4B (https://doi.org/10.3168/jds.2018-15238). For side- and rear-view images, the background pixels were identified from the X and Y matrices, respectively, with the selection criteria shown in Equation [3]. After all of the background pixels in a matrix were identified, these pixels were discarded by assigning them a value of zero in the X, Y, and Z matrices:
[2]
[3]
where Zi,j is a pixel in the Z matrix, Xi,j is a pixel in the X matrix, i is the row number of the matrix, j is the column number of the matrix,
is the average of all the pixels in the X matrix, and SD(X) is the standard deviation of all the pixel values in the X matrix.
Noise Removal
After background subtraction, the matrix retained some pixels that did not belong to the cow body surface but had similar pixel values. These pixels were considered noise and were removed in the following steps. First, a binary matrix was converted from the Z matrix in the top-view image. The conversion assigned a value of 1 to the pixels of the cow body surface and a value of 0 to the background pixels. Second, the binary image was processed using the morphological operations of erosion, filling holes, and dilation with a 7- × 7-pixel disk-shaped structuring element. The operations and the size of the structuring element were determined by testing different combinations of operations and sizes and visually inspecting the results for all images of all cows. Third, the binary image was divided into groups of pixels with a value of 1 that were connected to each other. The size of each group was determined as the number of connected pixels, and the group with the largest size was considered the body surface of the cow. The remaining groups were considered noise and were discarded by assigning a value of 0 to each pixel in the groups. Fourth, the X, Y, and Z matrices were multiplied by the binary matrix to remove noise in all matrices of the top-view image. An example of a top-view image after noise removal is shown in Supplemental Figure S4C (https://doi.org/10.3168/jds.2018-15238). Finally, the same procedure was applied to the matrices of the side- and rear-view images.
Image Interpolation
The X, Y, and Z matrices of each image were combined by interpolating them into a 2-D mesh grid image (Supplemental Figure S4D; https://doi.org/10.3168/jds.2018-15238). For top-view image interpolation, the grid spacing was 0.005 m in the X and Y directions to guarantee at least 2 pixels in each grid and each direction for a proper interpolation. The boundaries in the X and Y directions of the mesh grid were the minimum and maximum values of the X and Y matrices, respectively. The value of each grid was calculated by triangulation-based linear interpolation of the Z matrix. This 2-D mesh grid image could be viewed as a matrix where the rows and columns were represented in the X and Y directions of the recording setup coordinate system and where the size of the matrix element was the same as that of the grid. For side-view image interpolation, the Y and Z matrices were used to produce the mesh grid, and the X matrix was interpolated. For rear-view image interpolation, the X and Z matrices were used to produce the mesh grid, and the Y matrix was interpolated.
Anatomical Landmark Identification
Vertebral Column
The vertebrae form a prominent ridge on the cow's body surface. In the lumbar and sacral regions, these vertebrae were viewed as the highest part of the body surface from the top-view Z matrix. To identify the vertebrae, we selected the highest element in each row of the matrix as part of the vertebral spinous processes and combined all these elements as the vertebral column.
Additionally, all the images were standardized because the cows could stand in different positions in the recording setup. We rotated the body surface of the cow in the x–y plane to align its vertebral column with the y-axis of the recording setup (Figure 3A). This rotation procedure included fitting the vertebral column with a linear trend line, calculating the angle θ between the trend line and the y-axis, and multiplying the X, Y, and Z matrices by the rotation matrices with rotation angles 0, 0, and θ about the x-, y- and z-axes, respectively (Equation [1]).
Figure 3Anatomical landmarks on one side of a cow's body in different camera-view images: vertebral column, centerline of the sacral ligament, and hook bone center in a top-view image after image rotation (A), hook bone center in a side-view image (B), and tail region and pin bone center in a rear-view image (C). Regions of interest 1 (spinous processes), 2 (spinous to transverse processes), 3 (thurl area), and 4 (spinous processes to hook bone) were selected from the top-view image (D). Regions of interest 5 (hook bone) and 6 (overhanging shelf) were selected from the side-view image (E). Regions of interest 7 (cavity between tail head and pin bone) and 8 (pin bone) were selected from the rear-view images (F).
The sacral ligament is a tendon plate that consists of connective tissues linking the spinous processes and the center of the hook bone. The visible part of the sacral ligament in the Z matrix was viewed as a group of rows, and the highest row was designated the centerline of the sacral ligament. To identify the highest row, we halved the top-view Z matrix into left and right regions by the rotated vertebral column. On the right side of the body, the row with the highest mean was considered the centerline of the sacral ligament (the solid line in Figure 3A).
Hook Bone
The palpable and visible part of the hook bone is the tuber coxae. When viewed from above the cow, the tuber coxae extend prominently from the body surface and are located at the iliac crest of the ilium (i.e., the craniodorsal part of the hook bone). The highest point on the tuber coxae projected onto the body surface was designated the center of the hook bone projection and was considered to be located on the centerline of the sacral ligament. To identify the center of this bone, we selected 20 elements on the centerline that were the furthest from the vertebral column in the lateral direction. Among these 20 elements, the one with the largest value was designated the highest point of the tuber coxae and the center of the hook bone (the dot in Figure 3A).
When viewed from the right side of the cow, the tuber coxae extend prominently in the lateral direction. The most distal point of the tuber coxae was designated the center of the hook bone. This point, however, was not the same as the center of the bone from the top view (i.e., the highest point of the ilium). In the side-view X matrix, this point was designated the element with the largest value. We selected the row and column with the highest means and designated their cross element as the center of the hook bone (Figure 3B).
Pin Bone
The palpable and visible part of the pin bone is the tuber ischii. When viewed from the rear of the cow, this structure extends prominently in the caudal direction. The most prominent point of the tuber ischii is the caudal point of the ischium, which we designated the center of the pin bone. In the rear-view Y matrix, this point was designated the element with the largest value other than the tail. Therefore, to identify the center of the pin bone, we first identified the tail region by selecting the column with the highest mean and its 5 adjacent columns on both the left and right sides and excluded them from the matrix. Then, the row and column with the highest means were selected on the right half of the matrix, and their cross element was designated the center of the pin bone (Figure 3C).
Feature Extraction
After identifying anatomical landmarks, we measured the bony prominence or the body surface depression around these landmarks and defined those morphological measures as the body condition–related feature extraction. Within each body region, a rectangular area was selected and defined as a region of interest (ROI). The size of the ROI was determined as the distance between the anatomical landmarks located in this ROI. When only one anatomical landmark was presented, the size was determined by estimating the maximum size of the ROI of all cows. The detailed ROI selections are shown in Figure 3D, E, and F for 3 camera views and the selection criteria are listed in Table 1. The ROI was designated a matrix and its averaged row was designated the average cross-sectional contour line of the ROI in the mediolateral direction for the top- and rear-view images and in the craniocaudal direction for the side-view images. The shape of the average contour line was then quantified as a body condition–related feature.
Spinous Processes
The bony prominences of the spinous processes were represented by the shape of the average contour line. This contour line was fitted with a second-degree polynomial regression line (Figure 4A) based on Equation [4]. The coefficient β2 was always negative because the spinous process was in the middle and always higher than the rest of the parts on the contour line. Additionally, the absolute value of the coefficient β2 determined the sharpness of the contour line. For an obese cow, the spinous processes were less prominent and the contour line was rounder than for a lean cow. Moreover, the absolute value of the coefficient β2 was less than that of a lean cow. Thus, β2 was defined as the spinous process feature:
y = β0 + β1·x + β2·x2,
[4]
where β0, β1, and β2 are the coefficients of the second-degree polynomial regression line, and x and y are the coordinates of the points on the average contour line.
Figure 4Three examples of automated feature extractions: second-degree polynomial fit of the average cross-sectional contour of the spinous processes (A), average height difference between the spinous and transverse processes (B), and the slope of the centerline of the thurl area (C).
The amount of subcutaneous fat accumulated above the transverse processes was represented by the shape of the average contour line (the solid line in Figure 4B). To quantify this shape, we first drew a straight line (the dashed line in Figure 4B) to connect the 2 ends of the contour line. The height differences between pairs of corresponding points on the 2 lines were averaged and defined as the feature of the area between the spinous and transverse processes.
Thurl Area
The centerline of the thurl area in the mediolateral direction was designated the average contour line of the ROI. The inclination of the contour line represented the amount of subcutaneous fat accumulated in the area where the pelvis connects to the femur bone through the hip joint. This inclination was quantified by fitting the contour line to a linear regression line (Figure 4C) based on Equation [5]. For an obese cow, a great amount of subcutaneous fat was accumulated on the thurl area, which caused minor inclination of the contour line. For a lean cow, little or no subcutaneous fat was accumulated on the thurl area, which caused a great inclination of the contour line. Coefficient β1 was the slope of the regression line and defined as the thurl area feature.
y = β0 + β1·x,
[5]
where β0 and β1 are the coefficients of the linear regression line, and x and y are the coordinates of the points on the average contour line.
Spinous Processes to Hook Bone
The amount of subcutaneous fat accumulated in the sacral ligament area was represented by the shape of the average contour line. This shape was quantified as the height difference between the hook bone center and the mean of the contour line. This height difference was defined as the feature of the area between the spinous processes and the hook bone.
Hook Bone
The bony prominence of the tuber coxae was represented by the shape of the average contour line. This shape was quantified using the same procedure as that for the spinous processes by fitting the contour line with a second-degree polynomial regression line based on Equation [4]. The absolute value of coefficient β2 was defined as the hook bone feature.
Pin Bone
The shape of the average contour line was quantified using the same procedure as that for the hook bone based on Equation [4], and the absolute value of coefficient β2 was defined as the pin bone feature.
Overhanging Shelf
Around the overhanging shelf, the bony prominence of the overhanging shelf was represented by the shape of the average contour line. Similar to the spinous to transverse processes, this shape was quantified by calculating the height difference between the hook bone center and the mean of the contour line from the side view. This height difference was defined as the overhanging shelf feature.
Cavity Between Tail Head and Pin Bone
The amount of subcutaneous fat accumulated in the cavity between the tail head and pin bone was represented by the shape of the average contour line. This shape was quantified by calculating the largest height difference between the pin bone center and the mean of the contour line from the rear view. The height difference was defined as the feature of the cavity between the tail head and pin bone.
Statistical Analysis
Intra- and Inter-Assessor Agreement and Correlation with Manual BCS
The quality of manual body condition scoring of the 2 assessors was quantified by the scoring agreement Cohen's kappa (κ) and the scoring correlation coefficient Spearman's rho (ρ). The agreement of scoring is defined as the identical half-point BCS that is assigned in a pair of scorings. Cohen's κ measures pairwise agreement of 2 assessors or 1 assessor across 2 d, including the possibility of agreement occurring by chance. The interpretation of the κ value was the one used in the study of
. Spearman's ρ is a pairwise rank correlation coefficient and is defined as the statistical dependence between the rankings of 2 manual scores without interference from the systematic scoring difference. The ρ value ranges from 1 (i.e., a pair set of scores with identical ranks) to −1 (i.e., a pair set of scores with completely opposite ranks). Both κ and ρ values were calculated for intra-assessor comparison of each assessor's scores between d 1 and 2, and for inter-assessor comparison of the 2-d average scores between assessor 1 and assessor 2.
Test-Retest Reliability of Automated Feature Extraction
Test-retest reliability of feature extraction is used to quantify the closeness of the results of successive extractions over 2 d performed under the same conditions. For each feature, test-retest reliability was assessed by calculating the Pearson correlation coefficient (r) of this feature across all the cows quantified between d 1 and 2. On each day, this feature of each cow was the average of the extractions from the 5 images.
Correlation Between Manual BCS and Features
For each body region, the correlation between the manual BCS and the automatically extracted feature on each day was quantified by Spearman's ρ. For each cow, a feature was the average of the extractions from the 5 images of 1 day, and the manual BCS was the average of the scores assigned by the 2 assessors on the same day.
Half-Point Overall BCS Classification
The overall BCS of a cow was classified with a k-nearest (k = 1) neighbor classification model. The inputs of the model were 8 automatically extracted features, and each feature of a cow was the average of the extractions from the 5 images. The reference of the model was the average of the manual BCS assigned by both assessors on that day. The model with 8 features as inputs was defined as the full model.
To test this model, we built a data set of 88 samples including the data of all 44 cows from 2 d. The model was tested on this data set using a 10-fold cross-validation. In each iteration of the cross-validation, 90% of the samples were used to train the model and 10% were used to test the model. For each testing sample, the model assigned a half-point BCS from the reference BCS of its closest neighbor sample in distance in the training data set. The distance between these 2 samples was the normalized Euclidean distance calculated in a Euclidean 8-space with all 8 inputs of a sample as the Cartesian coordinates.
After cross-validation, all the samples had a model-classified half-point BCS and a reference BCS from manual scoring. The samples were divided into 9 half-point BCS classes ranging from 1.0 to 5.0, where all the samples with the same reference half-point BCS were in one class. For each class, the samples with the model-classified BCS being identical to the reference BCS were defined as true positives (TP) and the samples with the model-classified BCS being different from the reference BCS were defined as false negatives (FN). The quality of the model classification in this reference BCS class was tested by sensitivity, which was defined as the fraction of the number of TP over the number of all the cows with the same reference BCS in this class and calculated based on Equation [6]:
[6]
where q is a half-point reference BCS that all the samples have in one class, Sensitivityq is the sensitivity of the model classification in the class with reference BCS equal to q, TPq is the number of samples with both model-classified and reference BCS equal to q, and FNq is the number of samples with the reference BCS equal to q and model-classified BCS different from q. The overall sensitivity of the nearest-neighbor classification model was calculated by averaging the sensitivity of all the half-point BCS classes.
For comparison with the full model, we built another nearest-neighbor classification model with inputs of features extracted from body regions of the spinous processes, hook bone, pin bone, and the cavity between the tail head and pin bone. These body regions have been used in other studies of automated body condition scoring using machine vision. This model with 4 features as inputs was defined as the partial model. This model was also validated by a 10-fold cross-validation and tested by calculating the sensitivity of each half-point BCS class.
RESULTS
We compared the intra- and inter-assessor agreements and correlation coefficients of the manual body condition scoring of the 2 assessors (Table 2). Assessor 1 scored 8 of 9 body regions, including the overall BCS, with moderate intra-assessor agreement (i.e., κ values between 0.40 and 0.59). Assessor 2 scored 3 body regions, including the overall BCS, with good intra-assessor agreement (i.e., κ values between 0.60 and 0.79) and 5 body regions with moderate intra-assessor agreement. Both assessors scored the body region of the spinous processes to the hook bone with the lowest κ value compared with the rest of the body regions. In comparing the 2 assessors, 2 of 9 body regions were scored with moderate inter-assessor agreement, 6 of 9 body regions were scored with fair inter-assessor agreement (i.e., κ values between 0.20 and 0.39), and overall BCS was scored with moderate inter-assessor agreement (κ = 0.48, P < 0.001). Additionally, both assessors scored 8 of 9 body regions, including the overall BCS, with high intra- and inter-assessor correlation coefficients (i.e., ρ > 0.90). The body region with the lowest intra- and inter-assessor ρ values was the overhanging shelf. In addition, the result of test-retest reliability of the automated feature extraction was the Pearson correlation coefficient between d 1 and 2 for the indicated feature (Table 2). Among all features, 7 of 8 were extracted with Pearson's r >0.90 (P < 0.001), and the overhanging shelf feature extraction had the lowest Pearson's r (0.89; P < 0.001).
Table 2Intra- and interassessor agreements and correlation coefficients for manual body condition scoring and the test-retest reliability of automated feature extractions
Scoring agreement, scoring correlation coefficient, and test-retest reliability
The correlations between the body features and manual BCS were quantified with Spearman's ρ (Table 3). The ρ values calculated for the manual BCS of the corresponding body regions were similar to those of the overall BCS. A comparison of the ρ values in all body regions revealed that the spinous processes had the highest absolute ρ values for both the manual BCS of this body region (0.89, P < 0.001) and the overall BCS (0.90, P < 0.001). In contrast, the thurl area had the lowest absolute ρ values for both the manual BCS of this body region (0.69, P < 0.001) and the overall BCS (0.72, P < 0.001).
Correlations between the features and the manual BCS of their corresponding body regions are presented in Figure 5. The spinous process feature had almost equal average values for the cows in the half-point BCS classes from 3.5 to 5.0. This feature varied less for obese cows than for lean cows. A similar trend was found for the pin bone feature. In contrast, the hook bone feature had almost equal average values for the cows in the half-point BCS classes from 1.0 to 2.0. This feature varied less for lean cows than for obese cows. A similar trend was found for the feature of spinous processes to the hook bone. For the feature of spinous to transverse processes, cows could be classified into 3 main groups: those within the half-point BCS classes from 1.0 to 2.5, those with the half-point BCS class 3.0, and those within the half-point BCS classes from 3.5 to 5.0.
Figure 5Box plots of manual BCS in the 8 body regions versus the corresponding automatically extracted features: spinous processes (A), spinous to transverse processes (B), overhanging shelf (C), hook bone (D), pin bone (E), thurl area (F), spinous processes to hook bone (G), and tail head to pin bone (H). In each panel, the central mark in each box indicates the median of all the feature values with certain manual BCS. The bottom and top edges of the box indicate the 25th and 75th percentiles of the feature values, respectively. The whiskers extend to the most extreme data points that are not considered outliers. The outliers in the feature values are plotted individually (+).
The results of half-point overall BCS classification using nearest-neighbor classification models are the numbers of cows in the confusion matrix. The full model included all 8 features as inputs and the overall manual BCS as a reference. After the 10-fold cross-validation, this model yielded an overall sensitivity of 0.72 for all samples in all the half-point BCS classes. In the confusion matrix (Table 4), the highest sensitivity was 1 for half-point BCS class 1.0, and the lowest sensitivity was 0.55 for half-point BCS class 3.5. The partial model included only the features of the spinous processes, hook bone, pin bone, and tail head to pin bone as inputs and the overall manual BCS as a reference. After the 10-fold cross-validation, the partial model yielded an overall sensitivity of 0.68 for all samples in all the half-point BCS classes. In the confusion matrix (Table 5), the highest sensitivity was 1 for half-point BCS class 1.0, and the lowest sensitivity was 0.43 for half-point BCS class 3.0.
with the numbers of samples classified in half-point BCS classes resulting from the k-nearest neighbor (k = 1) BCS classification model with all 8 features as inputs (i.e., the full model)
with the numbers of samples classified in half-point BCS classes resulting from the k-nearest neighbor (k = 1) BCS classification model with features of spinous processes, hook bone, pin bone, and the cavity between the tail head to pin (i.e., the partial model)
This study aimed to improve the automated BCS classification by expanding the inclusion of body condition–related features extracted from multiple body regions using 3-D cameras. The validation results of the BCS classification model with 8 features (i.e., the full model) were compared with the results reported in previous 2-D and 3-D vision-based studies and the results of our partial model with 4 features.
First, our full model yielded a BCS classification sensitivity of 0.72. This classification sensitivity is greater than that reported by
, who used 2-D vision to quantify the bony prominences on the body contour and obtained a sensitivity of 0.53. The differences between our results and those of
demonstrate 2 limitations of classifying BCS using 2-D vision. The first limitation is related to the assessed body regions. Studies using 2-D vision aimed to extract the 2-D body contour and to quantify the bony prominences of the hook bones, pin bones, and tail head. The surfaces of many body regions that are related to the BCS, however, cannot be quantified or included in BCS classification. The reason is that these surfaces are not located on the body contour, and thus surface depression cannot be recorded in 2-D images. The lack of sufficient BCS-related information caused the low sensitivity of the BCS classification using 2-D vision. The second limitation is related to the quantification of bony prominences from the body contours projected onto a 2-D plane. The body contour varies in the 2-D plane when a cow's posture varies, which can introduce extra measurement variation in the features extracted from fixed-position cameras and reduce the sensitivity of the BCS classification. To address these limitations of 2-D vision, we recommend using 3-D vision to extract body condition–related features for BCS classification.
Second, we evaluated the BCS classification model by calculating the sensitivity of the classification. Different from our statistical analysis, previous 3-D vision-based studies chose regression models to predict BCS and calculated the mean absolute error (MAE) between the model outputs and reference of all their samples. Considering BCS as an ordinal variable, classification sensitivity should be the valid statistical measure instead of MAE. However, to be able to compare our BCS classification performance to those of other studies, we have to calculate the MAE of our full model as well. Our full model yielded an MAE of 0.15 BCS units, which is less than the MAE of 0.28 BCS units reported by
. These error differences may be due to the differences in the body regions used to extract features. Previous 3-D vision-based studies recorded and analyzed cow body surfaces from the top view with only one 3-D camera. In top-view images, however, some important body regions are not completely visible, such as the overhanging shelf, hook bone, pin bone, and the cavity between the tail head and pin bone. Hence, features representing these body regions were neither extracted from these body regions nor included in the BCS classification. In contrast to the top-view recording, the body prominences and surface depressions in these regions are more visible from either the side or rear of a cow's body. Therefore, to improve the current 3-D vision-based BCS classification, features should be retrieved from more than one camera placed at different viewpoints.
Finally, the discrepancy in results between our study (i.e., the full model) and previous studies could be due to the different models selected. We selected a classification model because BCS is defined as an ordinal variable. The appropriateness of defining BCS as an ordinal variable can be observed in the boxplots (Figure 5), which show a variety of correlations between different features and manual BCS. Previous studies, however, considered BCS a continuous variable and used linear regression models to predict BCS. The model selections cause the results to incorrectly represent the BCS-predictive abilities of the extracted features. Therefore, it is not accurate to directly compare the BCS classification results among studies. For a valid comparison, we used the partial model to simulate the BCS classification by limiting the selection of assessed body regions in the current machine vision–based studies. The partial BCS classification model had lower sensitivity than our full model that included features in 8 body regions. The difference in BCS classification sensitivity between the full and partial models demonstrates the improvement in BCS classification when features are extracted in multiple body regions.
Multiple 3-D Camera System
To capture the BCS-related information that cannot be captured by using a single 3-D camera, we chose to use multiple 3-D cameras in this study. The use of multiple cameras can improve BCS classification because they mimic human assessors evaluating different body regions of a cow from different views. This multi-camera approach to cow morphological measurements and feature extraction is consistent with studies by
. Moreover, during our data collection, images from different cameras were recorded one at a time to prevent inter-camera interference. We also minimized the time that the cows were required to stand in the recording setup by recording only 5 images per camera per cow. The total recording time for one cow was approximately 1 min, including the time spent for initializing the cameras and saving the recorded images. During the 1-min image recording, the cow could still move in the setup, which can influence the quality of the image and feature extraction. In future studies, new types of 3-D cameras without inter-camera interference should be used for simultaneous image recording by multiple cameras, which will significantly reduce the recording time and improve the quality of feature extraction.
Cow Selection Influencing the Sensitivity of the BCS Classification
The design of this study included the selection of cows that could contribute to a heterogeneous data set with evenly distributed BCS in the samples. Our cow selection plan differed from those of previous studies, which collected homogeneous data sets of all cows in a herd. Depending on the herd, the number of cows in different BCS classes can vary substantially. For the underrepresented BCS classes (i.e., <2.0 units or >4.0 units), the sensitivities of the BCS classification can be biased. To prevent inducing a sensitivity bias, we built the data set with the numbers of cows evenly distributed in the BCS classes ranging from 1.5 to 4.5. Hence, the sensitivities of the classification for BCS classes 1.5 and 4.5 were more reliable than the ones calculated from the homogeneous data sets. Nevertheless, our study had fewer cows with extreme BCS values. Only one cow had a BCS of 1.0 and no cows had a BCS of 5.0 in the data set. Although lacking cows with extreme BCS values is common on commercial farms, we must use sufficient samples with extreme BCS values in future studies because cows in those less-represented BCS classes are also the ones that need extra attention in farm management.
Additionally, most of the selected cows in this study were Holstein Friesian (n = 33) with BCS ranging from 1.0 to 3.5. It was difficult to find many obese Holstein Friesian cows to contribute to an evenly distributed BCS data set. Therefore, we chose a farm with Simmental cows and selected 11 cows with BCS ranging from 3.5 to 4.5. By including 2 breeds in the data set, we created a breed variance in the BCS class of 3.5 units. We assumed the breed variance caused the lowest classification sensitivity (i.e., 0.55) in the BCS class of 3.5. Hence, we recommend including breed variance in future BCS classification models or building a breed-specific body condition scoring system.
Quality of Manual Body Condition Scoring
The 2 assessors in this study were trained together by an expert using a detailed protocol for manual body condition scoring (Supplemental Figure S1; https://doi.org/10.3168/jds.2018-15238). We found that the inter-assessor agreement for the overall BCS was moderate, and the correlation coefficient reached 0.95, which is greater than the value found in a study by
. Additionally, we found the same trend of moderate agreement with a high rank correlation coefficient for the intra-assessor scoring of both assessors. Therefore, the quality of the overall BCS assigned by our 2 assessors was acceptable. However, their scoring agreement and rank correlation coefficient indicated systematic differences in their scoring, which were reduced by averaging the BCS of the 2 assessors. The use of 2 assessors for reference BCS was consistent with the recommendation of
, who stated that more than one assessor should be used to obtain accurate BCS.
Future Work
This cross-sectional study is an intermediate step toward a fully automated system that can continuously measure dynamic variations in body fat reserves of cows. In this step, we aimed to find body features that were highly correlated with body condition. The manual BCS was used as the reference of this study because alternative solutions (e.g., ultrasonography) were not feasible to offer a comprehensive fat measurement on all the body regions we needed. Manual BCS, however, is an ordinal variable and its smallest increment is 0.25 BCS units (
). Furthermore, we had an increment of 0.5 BCS units based on the scoring abilities of our assessors. When an automated BCS classification uses manual BCS as a reference, the automated method is limited and cannot detect any BCS variation less than the increment of its reference. The automatically extracted features used in the automated classification, however, are continuous variables and can continuously represent body condition variations in different body regions without the limitation of the manual scoring increment. Therefore, in future work, we suggest conducting longitudinal studies focusing on detecting the variations of the body features found in this study. These variations are continuously variable and can be an important health indicator of dairy cows.
CONCLUSIONS
This study considered BCS an ordinal variable and included all extracted features from multiple 3-D cameras in a BCS classification model. We increased the sensitivity of BCS classification compared with that reported for current machine vision–based body condition scoring methods.
ACKNOWLEDGMENTS
This research was funded by Lely Industries N.V. (Maassluis, the Netherlands). The authors thank Daniel Herd and Janine Niehaus from Lely Deutschland GmbH (Waldstetten, Germany) for their contributions in arranging farm visits for data collection; A. C. M. Meeuwesen, Beatrice Cortesi, E. J. P. M. Leijdekkers, Friso van Ooststroom Brummel, and Pierre Natur from Lely Innovation (Maassluis, the Netherlands) for their contributions in data collection on the farms; F. A. Steenstra from the Animal Production Systems Department of Wageningen University and Research for her contribution in training the two assessors in body condition scoring; Jose Gomez Lopez (Madrid, Spain) for his contribution in developing the image recording algorithm; and four anonymous dairy farmers from Germany and the Netherlands, for cooperating with data collection on their farms.
Effect of stocking rate and animal genotype on dry matter intake, milk production, body weight, and body condition score in spring-calving, grass-fed dairy cows.
Relationships between body condition score change, prior mid-lactation phenotypic residual feed intake, and hyperketonemia onset in transition dairy cows.
30249-8&id=gr1.jpg){kind=link}
30249-8&id=gr2.jpg){kind=link}
30249-8&id=gr3.jpg){kind=link}
30249-8&id=gr4.jpg){kind=link}
30249-8&id=gr5.jpg){kind=link}