Training dairy heifers with positive reinforcement: Effects on anticipatory behavior

Dairy cattle are often restrained for veterinary procedures, but restraint can cause fear responses that can make the procedure challenging for both the animal and the human handler. Positive reinforcement training (PRT) is used in other species to reduce fear responses and there is now evidence that this can also facilitate handling in cattle. The objectives of this study were to test the effect of PRT on anticipatory and play behavior in dairy heifers. We predicted that heifers trained with PRT would show more anticipatory and play behaviors than control heifers in the period before gaining access to a chute. We used 20 heifers (5 ± 0.6 mo old) that had been habituated to the chute area and had previous experience with handling. Heifers were randomly assigned to 2 treatments: control (n = 10) and PRT (n = 10). Positive reinforcement training heifers were subjected to a training protocol that included standard farm handling techniques, as well as target training with food reinforcement. Control heifers were moved to the chute using standard farm handling techniques only. As predicted, PRT heifers performed more behavioral transitions (7.6 ± 0.77 vs. 4.4 ± 0.57 transitions for control heifers; F 1,9 = 21.99, P < 0.01), and specifically performed more locomotory play such as jumping (2.1 ± 0.30 vs. 0.4 ± 0.19 jumps; F 1,9 = 57.18, P < 0.01) and running (2.0 ± 0.40 s vs. 0.5 ± 0.16 s; F 1,9 = 20.73, P < 0.01). These results indicate that PRT results in heifers having a more positive emotional state in anticipation of handling, and support the use of training to improve the welfare of dairy cattle.


INTRODUCTION
Positive reinforcement training (PRT) can be used to shape the behavior of animals (Skinner, 1938), including companion animals (Reisner, 2016), laboratory animals (Laule et al., 2003;Daugette et al., 2012), and a variety of zoo species (e.g., tortoises: Weiss and Wilson, 2003;baboons: Dorey et al., 2009;lemurs: Spiezio et al., 2017).Positive reinforcement training is associated with benefits for animals including reduced fear (Jønholt et al., 2021), increased environmental choice and control (Bassett and Buchanan-Smith, 2007), and reduced risk of injury for animal handlers associated with unpredictable or aggressive animals (Minier et al., 2011).Predictability in the environment, as well as choice to participate in routine procedures, are positive for welfare (Bassett and Buchanan-Smith, 2007).Positive reinforcement training enhances environmental predictability by providing context for previously unknown stimuli (Sørensen et al., 2020).Opportunities to learn can also be positive for welfare, as evidenced by the finding that cattle will work to participate in training events (e.g., Meagher et al., 2020).Further, PRT decreases distress responses as measured using both behavioral and physiological methods (O'Brien et al., 2008;Rogge et al., 2013).For example, baboons participating in PRT have lower cortisol levels, have improved participation and rates of learning, and show fewer aggressive behaviors (O'Brien et al., 2008).Other work has shown how PRT can reduce stress responses to otherwise aversive procedures such as injections (Lambeth et al., 2006).Collectively, there is a growing body of evidence showing benefits of PRT for animals cared for by humans.However, little research to date has addressed this approach in farm animals, although one study with dairy heifers showed that PRT can reduce negative behavioral responses to otherwise aversive procedures (Lomb et al., 2021).
Anticipatory behaviors are those that occur after an animal is exposed to a conditioned stimulus but before given a reward (Berridge, 1996;Van den Berg et al., 1999); some scholars have suggested that these behaviors can be used to assess aspects of animal welfare (Clegg et al., 2018;Anderson et al., 2020), including positive affective states that play a part in reward balance (Spruijt et al., 2001).Pavlovian conditioning, which involves linking a previously neutral stimulus (such as a sound) with a relevant stimulus (such as food) to elicit a specific response (Pierce and Cheney, 2004), is commonly used to study anticipatory behavior in relation to rewards (Spruijt et al., 2001;Moe et al., 2009).Specific anticipatory behaviors measured in previous studies include body positioning (Wichman et al., 2012), behavioral transitions (Spruijt et al., 2001), latency to access rewards (Neave et al., 2021), and latency to participate in training events (Clegg et al., 2018).
Researchers have used anticipatory behaviors to assess sensitivity to rewards and draw inferences regarding welfare (e.g., Neave et al., 2021).However, as reviewed by Anderson et al. (2020), anticipatory behaviors may also be indicative of frustration, a negative emotional state associated with aggression, stereotypic behaviors, and behavioral transitions (de Almeida and Miczek, 2002) that are typically observed in response to aversive situations (Amsel, 1958).Due to this potential connection with negative states, other behaviors are used to understand the valence of observed responses.Play behavior, including locomotor play, is an indicator of positive emotional states in animals (Ahloy-Dallaire et al., 2018).Locomotor play in calves has been used as an indicator of positive emotions and welfare (Boissy et al., 2007).For example, play is increased in grouphoused calves (Größbacher et al., 2020) and reduced in calves after disbudding (Rushen and de Passillé, 2012;Mintline et al., 2013).
The objective of this study was to test the effect of training dairy heifers with PRT on anticipatory and play behaviors.We hypothesized that, compared with control heifers, animals trained with PRT would be more likely to volunteer to participate in sessions and would show more anticipatory behavior and play behavior.

MATERIALS AND METHODS
All heifers were trained from July to August 2021 at The University of British Columbia Dairy Education and Research Centre located in Agassiz, British Columbia.All procedures were approved by the University of British Columbia Animal Care Committee (protocol A18-0174-A003).

Animals and Housing
We used 20 Holstein dairy heifers (average age at enrollment: 5.0 ± 0.6 mo, range: 3.9-6.0mo), housed together in a single pen fitted with lying stalls and ad libitum access to TMR accessed via a feedbunk with self-locking headlocks.The number of heifers was determined based on the size of pen they would be housed in, number of appropriately aged animals, as well as feasibility of training a group of this size.All animals were visually assessed by the trainer before each training session; no health concerns were noted throughout the study.
Heifers were randomly assigned either PRT or control (i.e., standard farm handling) conditions.Standard farm handling includes using the heifer's flight zone as well as pushing to encourage the animal to move forward.Six of the heifers had been previously enrolled in another learning study; these animals were balanced between treatments (i.e., 3 animals allocated to each).Animals were trained in 2 groups of 10 heifers, each consisting of 5 PRT and 5 control heifers.

Training Sessions
The management of heifers during training sessions was similar to that described by Lomb et al. (2021).In brief, heifers were trained once per day, 4 times a week (Mon, Tue, Thu, Fri).To balance for order, the first group of 10 were brought to the waiting area on the morning of the first day and in the afternoon on the next day, and vice versa for the second group of 10 heifers.The morning session started at approximately 1100 h and the afternoon session started at approximately 1300 h.At the beginning of each training session, 1 of the 2 groups were brought to the waiting area in the home pen.Heifers were released individually from the pen to the back alley (Figure 1), and a standardized handling protocol (described below) was used to establish a handling routine.
To move the heifer, both handlers (trainer and assistant) walked along the back alley with the handler opening the pen gate.If any heifer exited the pen within 15 s after opening the gate, both handlers calmly remained standing at the gate and prevented the other heifers from exiting.If none exited the pen within 15 s, a heifer was selected haphazardly and calmly moved into the alley.Once in the alley, the gate to the home pen was closed, and the heifer was allowed 30 s to move to the start box.If she did not enter the start box within the allotted time, a handler gently guided her into the box.All heifers were kept in the start box for 1 min.The gate to the chute area was then opened, and treatment heifers began training sessions while control heifers were guided into the chute using standard handling procedures.

Training of PRT Heifers
In total, we conducted 28 training sessions each lasting 5 min.All training was performed by the primary trainer (J.Lee) who was always assisted by 1 of 5 trained assistants.Grain was used as a food reinforcement (400-g daily of Hi-Pro Medicated Heifer starter, Chilliwack, BC, Canada), fed from a black rubber tub (19.5 cm diameter; see Figure 1).Training took place over 3 stages following habituation (see Table 1).A black tub was placed at 5 different locations with the first placement (T1) 1 m from the chute entrance.Once heifers ate from the tub without stepping away, the marker word good was introduced and a small portion (approximately 50 g) of the daily grain allowance was added to the tub.After successfully associating the tub with the food reinforcement, a target (red round plastic circle attached to a clear stick used to position the target) was introduced at the T1 location.Heifers were trained to touch the target with their muzzle to receive a food reinforcement.After successfully learning how to nose touch the target and receiving the food reinforcement at T1, the black feed tub was moved to T2 and the series of training steps described above for the T1 location were repeated, culminating with the heifer entering the chute fully and successfully nose touching the target at T5.
Heifers had 3 consecutive successful trials at each target location (i.e., they touched the target promptly after it was presented and consumed the grain reinforcement at that target location), before the target was presented at the next location.Every target location was used at least once by each heifer.If a heifer failed to touch a target twice, failed to eat at that target location or backed away after consuming the grain, the target was then presented at the previous feeding station.Four of the PRT heifers did not meet the training criteria for one of the transitions so we added an intermediate location at the mid-point (between T3 and T4 and between T4 and T5).
The PRT heifers reached the final stage of training when they approached the target at T5 immediately after entering the training area and then remained calmly in the chute for 1 min, after which they were brought back to the home pen.Training sessions lasted on average 2.3 ± 0.1 min, and heifers required on average 20 training sessions to reach this learning criterion (minimum: 14, maximum: 25).Two heifers did not reach full training criteria by the 25th session; these heifers were still included in the analysis.

Control Heifers
The experience of control heifers was designed to be as similar as possible to that of PRT heifers, with the exception that they were not trained to nose touch a target nor were they provided a food reward.After the heifer was brought to the start box, the trainer remained standing behind the start box (at the fence of section B; Figure 1) while the assistant moved through the pen to the other end of the start box (beginning of section C) and opened it when 1 min had passed.During the first training phase (while PRT heifers learned to respond to the target), the time that control heifers spent in the training area was set to be equivalent to the PRT heifers.Specifically, the timing for each of the control heifers was determined by the timing of a randomly assigned PRT heifer (such that they served as a yoked control).In this phase, control heifers were left free in the training area with the chute gate closed.
Control heifers were moved into the chute when the PRT heifer with which they were yoked met the criteria to enter the chute.The chute door was open from the beginning of the session, and once heifers were released from the start box the handlers gently guided them into the chute.Heifers were encouraged verbally, with calm body movements, and with gentle physical pushing that prevented them from backing up.If the heifer had not moved into the chute within 5 min, she was brought back to the home pen.Once the control heifer was in the chute with all 4 feet, she was prevented from backing out of the chute for 1 min.The heifer was then brought back to the home pen.

Outcome Measures
Training sessions were video recorded using 2 cameras.The front camera (GoPro Hero 4, GoPro Inc.) was located 2.6 m above and in 3.3 m in front of the chute and provided a view of the entire back alley (including start box, pre-chute area, and chute).The back camera (Canon Vixia HF R82, Ota City, Tokyo, Japan) was located 2.1 m above and 9.3 m behind the start box and provided a view of the start box, pre-chute area, and the back end of the chute.During the final 4 d of training response measures (described below) were scored.

Behavior in the Start Box
To assess behavioral transitions, we noted 4 different types of behavior (Table 2) during the 1 min in the start box and scored every time heifers transitioned between behaviors.We also recorded 2 different play behaviors (time spent running and number of jumps) during the 1 min spent in the start box.One of the 2 people who scored behaviors was blind to treatment.Both observers scored all behaviors using a subsample of videos and achieved high interobserver reliability for each measure (Cohen's κ > 0.75).Then one observer scored all videos for behavioral transitions, and the second observer scored play behaviors; in both cases responses were recorded on a spreadsheet.

Statistical Analyses
Before analysis we averaged values for each heifer over the 4-d observational phase (Table 1).We assessed the effect of treatment (PRT vs. control) on behaviors during the observational phase using a generalized mixed linear model (PROC MIXED) in SAS Studio, with individual heifer as the experimental unit and specifying yoked pairs as a random effect.Diagnostic plots of residuals were used to verify assumptions for the mixed linear model.All data and the corresponding  SAS code are available for review (Heinsius et al., 2023; https: / / doi .org/ 10 .5683/SP3/ J2W1TB).

RESULTS AND DISCUSSION
PRT heifers showed more anticipatory behavior in the start box compared with control heifers.Specifically, PRT heifers transitioned between behaviors more frequently in the time before gaining access to the start box (7.6 ± 0.8 vs. 4.4 ± 0.6 transitions; F 1,9 = 21.99,P < 0.01; Figure 2).This indicates that when placed in the start box, PRT heifers were anticipating gaining access to the training area more so than control heifers.Previous work has shown that animals trained with PRT anticipate the training event based on environmental cues (Krebs et al., 2022), such as the handlers walking up to the pen, being in the start box, seeing the trainer in the chute area and seeing the assistant opening the gate.We also observed the control heifers exhibiting behavioral transitions.In the current study, the use of standard handling likely resulted in the experience being perceived as neutral by the control heifers, but on some farms handling events may be aversive, in which case they can become fear inducing (Rushen et al., 1999;Hemsworth, 2003).We would expect an even more pronounced effect of PRT under these conditions.Behavioral transitions alone do not tell us if the heifer considers the experience positive, so other behaviors were used to draw inferences regarding affective valence.
Although many studies have examined training methods, and many others that assessed emotional states, to our knowledge this study is the first to assess the effect of training on emotional state.In the current study we recorded the occurrence of locomotory play behaviors to better understand the valence of the emotional state experienced by the heifers.Jumping and running, as observed in this study, are both considered forms of locomotor play (Jensen et al., 1998;Jensen and Kyhn, 2000).Consistent with the prediction that PRT sessions were associated with positive anticipation, heifers in this treatment jumped more frequently (2.1 ± 0.30 vs. 0.4 ± 0.19 jumps; F 1,9 = 57.18,P < 0.01) and spent more time running in the start box (2.0 ± 0.40 s vs. 0.5 ± 0.16 s; F 1,9 = 20.73,P < 0.01), than did controls (Figure 2C).The results of our play behavior observations, an indicator of positive emotional states in calves (Boissy et al., 2007), suggests that PRT improved heifer emotional state before handling.
One practical limitation of training is that it takes time.In the current study, each heifer was provided 28 training sessions, a level of time investment likely impractical on many farms; future work should seek to improve the efficiency of training by examining methods used in other species, and working with cattle earlier in life (e.g., Bloomsmith et al., 2015).One promising approach is to automate training via novel applications of existing on-farm automation such as computerized feeders.In addition, new work is required to better understand the longer term benefits of training; for example, considering how long animals retain the initial training, and examining how an initial period of training facilitates later training events, and more generally, how this affects the human-animal relationship.Positive reinforcement training facilitates handling, likely making this process more efficient and more pleasant for handlers (and the animals); future work should attempt to assess these outcomes, including the time required for handlers to perform a task, their perception of the task, and their perception of the animal and the animal's experience.In the longer term, we expect that positive experiences with animals, and the reduced risk of frustrating or dangerous interactions, would also reduce the risk that handlers respond inappropriately, putting their animals and the reputation of the industry at risk.
Current welfare discussions use the presence of positive emotional states as an indicator of good welfare (Mellor et al., 2020).Positive reinforcement training has been shown to reduce aggression (Leeds et al., 2016), shape behaviors (Yin et al., 2008), and improve husbandry issues (Weiss and Wilson, 2003).A next step to better understand the effect of PRT on welfare All 4 feet on the ground.Repositioning is also included in this category.Rotating in a circle with only 2 legs moving is part of this category.This category can include exploration of the surrounding area.This means using their nose to explore the area around them.Any grooming or itching in place is also included in this category.

Lying
Lying down with head in an upright position.would be research to investigate the effect of different handling methods on emotional states.In addition, future dairy studies on PRT should include measures that assess improvement in the ease of handling animals, as this is likely to provide further welfare benefits for both workers and animals.

CONCLUSIONS
Training heifers using PRT increased anticipatory and play behavior during a waiting period before gaining access to a training session.These results suggest that PRT training can improve emotional states during what might otherwise be aversive events on farm.Further research should attempt to decrease the time required to train cattle.

Figure 1 .
Figure 1.Drawing of the back alley and chute, illustrating the location of the waiting area (A) behind the furthest most gate; the start box (B), between the furthest and second furthest gates, where anticipatory behavior was scored; the pre-chute area (C), after the start box but before the chute; and the chute (D) with black buckets used for food delivery in positive reinforcement training progressively positioned at T1(1 m away from chute entrance), T2 (outside the chute entrance), T3 (inside the chute entrance), T4 (halfway along the chute), and T5 (at the end of the chute).Illustration by Ann Sanderson (independent illustrator, Canada).
trained using positive reinforcement training techniques, specifically target training.Control = heifers moved using standard farm handling techniques.

Figure 2 .
Figure 2. Behavioral transitions (A), number of jumps (B), and time spent running (C) scored in the start box during the observational phase.Each bar represents interquartile range for data from a single heifer; in total, 20 ranges are illustrated, n = 10 from the control group and n = 10 from the PRT group.Data points form each of the 4 d of observation are shown separately.Data from each yoked pair of heifers appear adjacent to one another.The circles show the mean value and whiskers extending beyond the bars illustrate the range.

Table 1 .
Heinsius et al.: TRAINING DAIRY HEIFERS Stages of training for PRT and control heifers 1

Table 2 .
Heinsius et al.: TRAINING DAIRY HEIFERS Definition of behaviors used for scoring instances of behavioral transitions (adapted from Neave et al., 2021) Behavior Definition or criteria Behaviors directed at the fence Any touch directed toward the fence separating the heifer from the chute.This includes scratching, touching, headbutts, and biting.Sniffing/exploring for longer than 5 s would also be included in this behavior.Locomotion This category includes walking, running, jumping/hopping, and rolling.Inclusion in this category requires at least 2 steps.Standing