Noninvasive Tracking of Every Individual in Unmarked Mouse Groups Using Multi-Camera Fusion and Deep Learning

 Feng Su1,2,3,4 · Yangzhen Wang5  · Mengping Wei1  · Chong Wang6  · Shaoli Wang7  · Lei Yang1  · Jianmin Li8  · Peijiang Yuan9  · Dong‑Gen Luo4  · Chen Zhang1,2,3
1 Department of Neurobiology, School of Basic Medical Sciences, Beijing Key Laboratory of Neural Regeneration and Repair, Capital Medical University, Beijing 100069, China 
2 Chinese Institute for Brain Research, Beijing 102206, China 
3 State Key Laboratory of Translational Medicine and Innovative Drug Development, Nanjing 210000, China 
4 Peking-Tsinghua Center for Life Sciences, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing 100871, China 
5 School of Life Sciences, Tsinghua University, Beijing 100084, China 
6 School of Biological Science and Medical Engineering, Beihang University, Beijing 100191, China 
7 The Key Laboratory of Developmental Genes and Human Disease, Institute of Life Sciences, Southeast University, Nanjing 210096, Jiangsu, China 
8 Institute for Artifcial Intelligence, the State Key Laboratory of Intelligence Technology and Systems, Beijing National Research Center for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China 
9 School of Mechanical Engineering and Automation, Beihang University, Beijing 100191, China

Abstract

Accurate and efficient methods for identifying and tracking each animal in a group are needed to study complex behaviors and social interactions. Traditional tracking methods (e.g., marking each animal with dye or surgically implanting microchips) can be invasive and may have an impact on the social behavior being measured. To overcome these shortcomings, video-based methods for tracking unmarked animals, such as fruit flies and zebrafish, have been developed. However, tracking individual mice in a group remains a challenging problem because of their flexible body and complicated interaction patterns. In this study, we report the development of a multi-object tracker for mice that uses the Faster region-based convolutional neural network (R-CNN) deep learning algorithm with geometric transformations in combination with multi-camera/multi-image fusion technology. The system successfully tracked every individual in groups of unmarked mice and was applied to investigate chasing behavior. The proposed system constitutes a step forward in the noninvasive tracking of individual mice engaged in social behavior.


Keywords
Noninvasive tracking; Deep learning; Multicamera; Mouse group; Social interaction