Automatic Generation of Labelled mmWave FMCW Radar Datasets Using Collocated Optical Sensors and Computer Vision

Simple item page
dc.contributor.advisorPaine, Stephen
dc.contributor.authorBourn, William
dc.date.accessioned2026-07-02T08:35:43Z
dc.date.available2026-07-02T08:35:43Z
dc.date.issued2026
dc.date.updated2026-07-02T08:33:42Z
dc.description.abstractAdvancements made in computational power, the development of complex AI algorithms, and an industry push to create self-driving cars and autonomous systems has resulted in a large amount of research into computer vision. The majority of computer vision research is performed with optical sensors because radar and lidar sensors suffer from a paucity of data that makes classification tasks difficult, and lack easily implemented open-source datasets for general applications. Future research into radar-based computer vision would benefit from a method of automating the process of annotating data, which can be implemented using sensor fusion and existing image recognition algorithms. We propose a processing chain that uses fused spatio-optical and radar data to automatically produce class and localisation ground truth labels in radar data using a YOLO image recognition model to label objects in a projection of 3D optical space. We developed a processing pipeline to perform automatic labelling of radar data and a prototype data capture rig. We analysed the performance of the pipeline by processing data captured by the rig into annotated radar range-azimuth-Doppler frames for human classification and training a CNN classification model on the data. Furthermore, we analysed the performance of the approach to calibration in the processing chain. The CNN classifier achieved strong human classification performance, though was affected by localisation errors that were produced by the processing pipeline. Despite this, the strong classification performance indicates that the pipeline was able to successfully perform automatic annotation of radar range-azimuth-Doppler data with human target information. The processing chain has potential use in radar-based computer vision going forward, and the automatic labelling approach detailed in this report merits further development.
dc.identifier.apacitationBourn, W. (2026). <i>Automatic Generation of Labelled mmWave FMCW Radar Datasets Using Collocated Optical Sensors and Computer Vision</i>. (). University of Cape Town ,Faculty of Engineering and the Built Environment ,Department of Electrical Engineering. Retrieved from http://hdl.handle.net/11427/43453en_ZA
dc.identifier.chicagocitationBourn, William. <i>"Automatic Generation of Labelled mmWave FMCW Radar Datasets Using Collocated Optical Sensors and Computer Vision."</i> ., University of Cape Town ,Faculty of Engineering and the Built Environment ,Department of Electrical Engineering, 2026. http://hdl.handle.net/11427/43453en_ZA
dc.identifier.citationBourn, W. 2026. Automatic Generation of Labelled mmWave FMCW Radar Datasets Using Collocated Optical Sensors and Computer Vision. . University of Cape Town ,Faculty of Engineering and the Built Environment ,Department of Electrical Engineering. http://hdl.handle.net/11427/43453en_ZA
dc.identifier.ris TY - Thesis / Dissertation AU - Bourn, William AB - Advancements made in computational power, the development of complex AI algorithms, and an industry push to create self-driving cars and autonomous systems has resulted in a large amount of research into computer vision. The majority of computer vision research is performed with optical sensors because radar and lidar sensors suffer from a paucity of data that makes classification tasks difficult, and lack easily implemented open-source datasets for general applications. Future research into radar-based computer vision would benefit from a method of automating the process of annotating data, which can be implemented using sensor fusion and existing image recognition algorithms. We propose a processing chain that uses fused spatio-optical and radar data to automatically produce class and localisation ground truth labels in radar data using a YOLO image recognition model to label objects in a projection of 3D optical space. We developed a processing pipeline to perform automatic labelling of radar data and a prototype data capture rig. We analysed the performance of the pipeline by processing data captured by the rig into annotated radar range-azimuth-Doppler frames for human classification and training a CNN classification model on the data. Furthermore, we analysed the performance of the approach to calibration in the processing chain. The CNN classifier achieved strong human classification performance, though was affected by localisation errors that were produced by the processing pipeline. Despite this, the strong classification performance indicates that the pipeline was able to successfully perform automatic annotation of radar range-azimuth-Doppler data with human target information. The processing chain has potential use in radar-based computer vision going forward, and the automatic labelling approach detailed in this report merits further development. DA - 2026 DB - OpenUCT DP - University of Cape Town KW - computer vision KW - radar datasets LK - https://open.uct.ac.za PB - University of Cape Town PY - 2026 T1 - Automatic Generation of Labelled mmWave FMCW Radar Datasets Using Collocated Optical Sensors and Computer Vision TI - Automatic Generation of Labelled mmWave FMCW Radar Datasets Using Collocated Optical Sensors and Computer Vision UR - http://hdl.handle.net/11427/43453 ER - en_ZA
dc.identifier.urihttp://hdl.handle.net/11427/43453
dc.identifier.vancouvercitationBourn W. Automatic Generation of Labelled mmWave FMCW Radar Datasets Using Collocated Optical Sensors and Computer Vision. []. University of Cape Town ,Faculty of Engineering and the Built Environment ,Department of Electrical Engineering, 2026 [cited yyyy month dd]. Available from: http://hdl.handle.net/11427/43453en_ZA
dc.language.isoen
dc.language.rfc3066eng
dc.publisher.departmentDepartment of Electrical Engineering
dc.publisher.facultyFaculty of Engineering and the Built Environment
dc.publisher.institutionUniversity of Cape Town
dc.subjectcomputer vision
dc.subjectradar datasets
dc.titleAutomatic Generation of Labelled mmWave FMCW Radar Datasets Using Collocated Optical Sensors and Computer Vision
dc.typeThesis / Dissertation
dc.type.qualificationlevelMasters
dc.type.qualificationlevelMSc (Eng)
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
thesis_ebe_2026_bourn william.pdf
Size:
87.25 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
license.txt
Size:
1.72 KB
Format:
Item-specific license agreed upon to submission
Description:
Download
Collections
Masters