Double-Camera Fusion System for Animal-Position Awareness in Farming Pens

Shoujun Huo, Yue Sun (Corresponding author), Qinghua Guo, Tao Tan, J. Elizabeth Bolhuis, Piter Bijma, Peter H.N. de With

Research output: Contribution to journalArticleAcademicpeer-review

6 Downloads (Pure)


In livestock breeding, continuous and objective monitoring of animals is manually unfeasible due to the large scale of breeding and expensive labour. Computer vision technology can generate accurate and real-time individual animal or animal group information from video surveillance. However, the frequent occlusion between animals and changes in appearance features caused by varying lighting conditions makes single-camera systems less attractive. We propose a double-camera system and image registration algorithms to spatially fuse the information from different viewpoints to solve these issues. This paper presents a deformable learning-based registration framework, where the input image pairs are initially linearly pre-registered. Then, an unsupervised convolutional neural network is employed to fit the mapping from one view to another, using a large number of unlabelled samples for training. The learned parameters are then used in a semi-supervised network and fine-tuned with a small number of manually annotated landmarks. The actual pixel displacement error is introduced as a complement to an image similarity measure. The performance of the proposed fine-tuned method is evaluated on real farming datasets and demonstrates significant improvement in lowering the registration errors than commonly used feature-based and intensity-based methods. This approach also reduces the registration time of an unseen image pair to less than 0.5 s. The proposed method provides a high-quality reference processing step for improving subsequent tasks such as multi-object tracking and behaviour recognition of animals for further analysis.

Original languageEnglish
Article number84
Number of pages17
Issue number1
Publication statusPublished - Jan 2023


This work was supported in part by the Dutch NWO project IMAGEN (P18-19 Project 1) of the research programme Perspectief. This work was supported in part by the program of China Scholarship Council (No. 202106930006). The Volmer facility in Germany was offered by Topigs Norsvin in Helvoirt, the Netherlands. The author would like to thank all the participants in the project.

FundersFunder number
Nederlandse Organisatie voor Wetenschappelijk OnderzoekP18-19
China Scholarship Council202106930006


    • convolutional neural network
    • double-camera
    • fine-tune
    • image registration


    Dive into the research topics of 'Double-Camera Fusion System for Animal-Position Awareness in Farming Pens'. Together they form a unique fingerprint.

    Cite this