Using Siamese neural networks to identify individual animals

Madzingira, Tinotenda

Using Siamese neural networks to identify individual animals

Master Thesis

2022

Abstract

The ability to identify individual animals in ecology is essential for monitoring [Schneider et al., 2018]. It allows a researcher to determine whether an animal being observed is new to the researcher or has previously been observed. This in turn allows estimation of ecological metrics such as population density [Schneider et al., 2018]. Traditionally, this was done by capturing and physically tagging animals [Cross et al., 2014a]. Increasingly, animal observation is being conducted by photographic means creating the need to be able to identify individual animals from images [Weinstein, 2018]. The dissertation answers whether machine learning can determine if the animals in a pair of images come from the same individual animal. To answer this question, the animal of interest in each image must be (1) located and isolated and (2) compared to the other image in the pair. Mask-Regional Convolution Neural Networks (Mask - RCNN) [He et al., 2017] are used for the object detection and instance segmentation which answers (1). This is a modern deep learning approach which has been used for tasks such as identifying breast cancer tumors [Chiao et al., 2019] outside of ecology and to measure the size of whales [Gray et al., 2019b] in ecology. In addition to classifying the object and proposing a bounding box, Mask RCNN also proposes a mask for each object. The “mask” is a selection of pixels that belong to the object of interest which are highlighted in processed images. A ResNet-101 model with the Feature Pyramid Network (FPN) that has been trained on the MS Coco dataset [Lin et al., 2014] is used with v Abstract vi transfer learning. Finally, a Siamese Neural Network (SNN) is used to measure the similarity between the objects in each pair of images, which answers (2). A SNN is a pair of identical neural networks that share the same weights and whose outputs are connected to a distance computing function. SNNs are used to identify subtle differences in the feature space of its inputs. Each of the neural networks is given one of the images in the pair as an input and a distance measure is computed on their outputs which allows the inputs to be classified as similar or dissimilar based on a given threshold of the distance metric. The structure of the SNNs applied are inspired by the one proposed in [Dey et al., 2017] and the weights are trained separately on each dataset. The proposed approach is evaluated on two datasets: a set of approximately 25,000 images of 5,000 humpback whale individuals taken a cross a variety of locations, and a set of approximately 13,000 images of 300 individual harbor seals from 3 locations on the west coast of Scotland. The object detection models are evaluated using the intersection over union (IOU) approach while the SNNs are evaluated on F1 score, the harmonic mean of precision and recall. The proposed method is tested on unseen images for each dataset. The SNN models achieved F1 scores of 63.9%, 66.7% and 68.4% for the Humpback whale dataset, the right and the left fins of the bottlenose dolphins respectively. The object detection model achieved an intersection over union of 88.6% and 74.3% for the Humpback whale dataset and the bottlenose dolphin dataset respectively. Lastly the orientation model achieved an F1 score of 94.7%. All quoted results are evaluated on an unseen test dataset.

Keywords

statistical sciences

Reference:

Collections

Masters

Full item page