Yashan Dhaliwal - Bimodal Data Analysis for Lameness Detection in Cows Using Artificial Intelligence

Bimodal Data Analysis for Early Detection of Lameness in Dairy Cows Using Artificial Intelligence
Yashan Dhaliwal, Hanqing Bi, Suresh Neethirajan
Abstract
Lameness is a painful condition that causes dairy cows to alter their gait to minimize weight on the affected limbs. As the third most economically significant health issue in dairy herds after fertility and mastitis, lameness affects 35% of dairy cattle in Canada. This study leverages artificial intelligence (AI) to analyze bimodal data-visual (images and videos) and accelerometer data-for the early detection and prediction of lameness in dairy cows. In the first phase, facial biometric data from six Holstein cows was collected over 21 days at the Ruminant Animal Centre (RAC) at Dalhousie Agricultural Campus in Truro, Nova Scotia. Data was gathered tri-daily-morning, afternoon, and evening-to capture natural variations in facial expressions and movements. The tri-daily schedule encompassed capturing cow images and videos under various contextual settings: before and after exercise in the morning, before and after feeding at noon, and during rest in the evening. This phase resulted in a dataset comprising 7000 images and 1000 videos. Concurrently, accelerometer data from four cows was collected using Fibion Sens motion sensors.
We conducted a correlation analysis between the accelerometer data and images to identify patterns indicative of lameness. A DenseNet AI model was developed to process and analyze this bimodal data. Our results clearly show that integrating bimodal data-visual and accelerometer-enhances the early detection of lameness by revealing new features. These new features include:
Facial Tension Indicators - Changes in facial muscle tension and expressions that correlate with pain and discomfort.
Gait Abnormalities - Detailed analysis of walking patterns, including step length and symmetry.
Activity Variations - Differences in daily activity patterns, such as resting and movement times.
Behavioral Changes - Indicators of altered behavior, such as reluctance to move or changes in standing postures.
These features, not observable through a single data modality, enable more accurate and early detection of lameness. Our AI model allows for timely veterinary intervention and management adjustments, ultimately improving cow welfare and reducing economic losses in dairy farming.
Keywords: Lameness; Dairy Cows; AI; Facial Expression Analysis; Accelerometer Data; Early Detection; Bimodal Data; Predictive Model