Viewpoint estimation in medical imaging

Hounkanrin, Mahouclo Anicet

Viewpoint estimation in medical imaging

Thesis / Dissertation

2024

Abstract

In medical imaging, the appearance of a certain body part on a radiograph depends not only on the position but also on the orientation of the X-ray imaging system with respect to the patient. Given a 2D image of a 3D scene, the problem of viewpoint estimation aims to determine the position and the orientation of the imaging sensor that resulted in that view. We investigate methods to solve the viewpoint estimation problem for medical images, notably the determination of orientation parameters. Machine learning models, particularly convolutional neural networks (CNNs), are developed to predict a human subject's orientation in a radiograph. Since deep learning models require data for training, we first generate a dataset of digitally reconstructed radiographs (DRRs) from a set of computed tomography (CT) scans using Fourier volume rendering (FVR). The dataset of DRRs is then used to train CNN models for viewpoint regression and classification. A label-softening strategy is used to improve the performance of the classification models. Meanwhile, a geometric structure-aware cost function is used to account for the geometric continuity of the viewpoint space. Several 3D rotation methods such as Euler angle, axis-angle, and quaternions are investigated for viewpoint representation. The results demonstrate that viewpoint estimation in medical imaging can be effectively solved using CNN-based classification and regression models. The geometric structure-aware cost function proves to be essential to the success of classification models for viewpoint estimation. The regression-based models, on the order hand, appear to be sensitive to the type of parametrization used to represent the viewpoints. In particular, the unit quaternion representation of 3D rotations proves to be more effective than other representations for viewpoint regression with CNN models. Moreover, we extend the proposed method to perform viewpoint estimation for natural images. The performance on the PASCAL3D+ dataset indicates that the application of the methods presented is not restricted to medical imaging.

Keywords

Electrical Engineering

Reference:

Collections

PhD / Doctoral

Full item page