OpenUCT :: Browsing by Author "Verrinder, Robyn"

Browsing by Author "Verrinder, Robyn"

Now showing 1 - 17 of 17

Open Access
An aircraft and provide information about flight performance and local microclimate
(2013) Johnson, Bruce Edward; Verrinder, Robyn; Ginsberg, Samuel
The application of using Unmanned Aerial Vehicles (UAVs) to locate thermal updraft currentsis a relatively new topic. It was first proposed in 1998 by John Wharington, and, subsequently, several researchers have developed algorithms to search and exploit thermals. However, few people have physically implemented a system and performed field testing. The aim of this project was to develop a low cost system to be carried on a glider to detect thermals effectively. A system was developed from the ground up and consisted of custom hardware and software that was developed specifically for aircraft. Data fusion was performed to estimate the attitude of the aircraft; this was done using a direction cosine (DCM) based method. Altitude and airspeed data were fused by estimating potential and kinetic energy respectively; thus determining the aircraft’s total energy. This data was then interpreted to locate thermal activity. The system comprised an Inertial Measurement Unit (IMU), airspeed sensor, barometric altitude sensor, Global Positioning System (GPS), temperature sensor, SD card and a realtime telemetry link. These features allowed the system to determine aircraft position, height, airspeed and air temperature in realtime. A custom-designed radio controlled (RC) glider was constructed from composite materials in addition to a second 3.6 m production glider that was used during flight testing. Sensor calibration was done using a wind tunnel with custom designed apparatus that allowed a complete wing with its pitot tube to be tested in one operation. Flight testing was conducted in the field at several different locations over the course of six months. A total of 25 recorded flights were made during this period. Both thermal soaring and ridge soaring were performed to test the system under varying weather conditions. A telemetry link was developed to transfer data in realtime from the aircraft to a custom ground station. The recorded results were post-processed using Matlab and showed that the system was able to detect thermal updrafts. The sensors used in the system were shown to provide acceptable performance once some calibration had been performed. Sensor noise proved to be problematic, and time was spent alleviating its effects.
Open Access
An application of Hierarchical Temporal Memory (HTM)
(2011) Liddiard, Ashley; Tapson, Jonathan; Verrinder, Robyn
While a number of neuromorphic studies have been based on understanding and building the brain in software and hardware, a recent theory has been presented from a high level, top down approach, with the view of understanding how the human brain performs higher reasoning, and then designing software infrastructure based on that theory - namely Hierarchical Temporal Memory (HTM). Current opinion raises question marks as to whether this theory is sound, feasible, coherent, logical and correct. This thesis aims to, if not answer some of those questions, at least get closer to understanding the brain and therefore implementing HTM theory in software for further use in an embedded environment. This paper begins by reviewing the discovery of the sensory input and chemical makeup of the signals in the brain, and then introduces mathematical abstraction to form a software program node.
Open Access
An autonomous self-reconfigurable modular robotic system with optimised docking connectors
(2014) Muthui, Patrick; Verrinder, Robyn
Self-Reconfigurable Modular Robots are robotic systems consisting of a number of self-contained modules that can autonomously interconnect in different positions and orientations thereby varying the shape and size of the overall modular robot. This ground breaking capability is what in theory, makes self-reconfigurable modular robots more suitable for use in the navigation of unknown or unstructured environments. Here, they are required to reconfigure into different forms so as to optimise their navigation capabilities, a feat that is rendered impossible in conventional specialised robots that lack reconfiguration capabilities. However, the frequent development and use of self-reconfigurable modular robots in everyday robotic navigation applications is significantly hampered by the increased difficulty and overall cost of production of constituent robotic modules. One major contributor to this is the difficulty of designing suitably robust and reliable docking mechanisms between individual robotic modules. Such mechanisms are required to be mechanically stable involving a robust coupling mechanism, and to facilitate reliable inter-module power sharing and communication. This dissertation therefore proposes that the design and development of a functional low cost self-reconfigurable modular robot is indeed achievable by optimising and simplifying the design of a robust and reliable autonomous docking mechanism. In this study, we design and develop such a modular robot, whose constituent robotic modules are fitted with specialised docking connectors that utilise an optimised docking mechanism. This modular robot, its robotic modules and their connectors are then thoroughly tested for accuracy in mobility, electrical and structural stability, inter-module communication and power transfer, self-assembly, self-reconfiguration and self-healing, among others. The outcome of these testing procedures proved that it is indeed possible to optimise the docking mechanisms of self-reconfigurable modular robots, thereby enabling the modular robot to more easily exhibit efficient self-reconfiguration, self-assembly and self-healing behaviours. This study however showed that the type, shape, functionality and structure of electrical contacts used within the docking connectors for inter-module signal transfer and communication play a major role in enabling efficient self-assembly, self-reconfiguration and self-healing behaviours. Smooth spring loaded metallic electrical contacts incorporated into the docking connector design are recommended. This study also highlights the importance of closed loop control in the locomotion of constituent robotic modules, especially prior to docking. The open loop controlled locomotion optimisations used in this project were not as accurate as was initially expected, making self-assembly rather inaccurate and inconsistent. It is hoped that the outcomes of this research will serve to improve the docking mechanisms of self-reconfigurable modular robots thereby improving their functionality and pave the way for future large scale use of these robots in real world applications.
Open Access
A Comparison of ICA versus genetic algorithm optimized ICA for use in non-invasive muscle tissue EMG
(2014) Mulligan, Shaun R; Verrinder, Robyn; John, L
The patent developed by Dr. L. John [1] allows for the the detection of deep muscle activation through the combination of specially positioned monopolar surface Electromyography (sEMG) electrodes and a Blind Source Separation algorithm. This concept was then proved by Morowasi and John [2] in a 12 electrode prototype system around the bicep. This proof of concept showed that it was possible to extract the deep tissue activity of the brachialis muscle in the upper arm, however, the effect of surface electrode positioning and effectual number of electrodes on signal quality is still unclear. The hope of this research is to extend this work. In this research, a genetic algorithm (GA) is implemented on top of the Fast Independent Component Analysis (FastICA) algorithm to reduce the number of electrodes needed to isolate the activity from all muscles in the upper arm, including deep tissue. The GA selects electrodes based on the amount of significant information they contribute to the ICA solution and by doing so, a reduced electrode set is generated and alternative electrode positions are identified. This allows a near optimal electrode configuration to be produced for each user. The benefits of this approach are: 1.The generalized electrode array and this algorithm can select the near optimal electrode arrangement with very minimal understanding of the underlying anatomy. 2. It can correct for small anatomical differences between test subjects and act as a calibration phase for individuals. As with any design there are also disadvantages, such as each user needs to have the electrode placement specifically customised for him or her and this process needs to be conducted using a higher number of electrodes to begin with.
Open Access
Fusion of sensor information to measure the total energy of an aircraft and provide information about flight performance and local microclimate
(2013) Johnson, Bruce; Verrinder, Robyn; Ginsberg, Samuel
The application of using Unmanned Aerial Vehicles (UAVs) to locate thermal updraft currents is a relatively new topic. It was first proposed in 1998 by John Wharington, and, subsequently, several researchers have developed algorithms to search and exploit thermals. However, few people have physically implemented a system and performed field testing. The aim of this project was to develop a low cost system to be carried on a glider to detect thermals effectively. A system was developed from the ground up and consisted of custom hardware and software that was developed specifically for aircraft. Data fusion was performed to estimate the attitude of the aircraft; this was done using a direction cosine (DCM) based method. Altitude and airspeed data were fused by estimating potential and kinetic energy respectively; thus determining the aircraft's total energy. This data was then interpreted to locate thermal activity. The system comprised an Inertial Measurement Unit (IMU), airspeed sensor, barometric altitude sensor, Global Positioning System (GPS), temperature sensor, SD card and a realtime telemetry link. These features allowed the system to determine aircraft position, height, airspeed and air temperature in realtime. A custom-designed radio controlled (RC) glider was constructed from composite materials in addition to a second 3.6 m production glider that was used during flight testing. Sensor calibration was done using a wind tunnel with custom designed apparatus that allowed a complete wing with its pitot tube to be tested in one operation. Flight testing was conducted in the field at several different locations over the course of six months. A total of 25 recorded flights were made during this period. Both thermal soaring and ridge soaring were performed to test the system under varying weather conditions. A telemetry link was developed to transfer data in realtime from the aircraft to a custom ground station. The recorded results were post-processed using Matlab and showed that the system was able to detect thermal updrafts. The sensors used in the system were shown to provide acceptable performance once some calibration had been performed. Sensor noise proved to be problematic, and time was spent alleviating its effects. Results showed that the system was able to measure airspeed to within ± 1 km/h. The standard deviation of the altitude estimate was determined to be 0.94 m. This was deemed to be satisfactory. The system was highly reliable and no faults occurred during operation. In conclusion, the project showed that inexpensive sensors and low power microcontrollers could be used very effectively for the application of detecting thermals.
Open Access
Investigation into the use of the Microsoft Kinect and the Hough transform for mobile robotics
(2014) O'Regan, Katherine; Verrinder, Robyn; Nicolls, Fred
The Microsoft Kinect sensor is a low cost RGB-D sensor. In this dissertation, its calibration is fully investigated and then these parameters are compared to the parameters given by Microsoft and OpenNI. The parameters found were found to be different to those given by Microsoft and OpenNI therefore, every Kinect should be fully calibrated. The transformation from the raw data to a point cloud is also investigated. Then, the Hough transform is presented in its 2-dimensional form. The Hough transform is a line extraction algorithm which uses a voting system. It is then compared to the Split-and-Merge algorithm using laser range _nder data. The Hough transform is found to compare well to the Split-and-Merge in 2 dimensions. Finally, the Hough transform is extended into 3-dimensions for use with the Kinect sensor. It was found that pre-processing of the Kinect data was necessary to reduce the number of points input into the Hough transform. Three edge detectors are used - the LoG, Canny and Sobel edge detectors. These were compared, and the Sobel detector was found to be the best. The _nal process was then used in multiple ways - _rst to determine its speed. Its accuracy was then investigated. It was found that the planes extracted were very inaccurate, and therefore not suitable for obstacle avoidance in mobile robotics. The suitability of the process for SLAM was also investigated. It was found to be unsuitable, as planar environments did not have distinct features which could be tracked, whilst the complex environment was not planar, and therefore the Hough transform would not work.
Open Access
A magnetometer based payload for a PTOL UAV with application in geophysical surveys
(2014) Ocker, Darcy; Verrinder, Robyn; Gouws, Danie; Saunderson, Elda
Applying the principles of physics to studying the Earth has given rise to the field of geophysics, which has been recognised as a separate discipline since the 19th century. The practical implementation of this field has led to a separate branch, aptly named exploration geophysics. Exploration geophysics aims to measure various naturally occurring phenomena associated with the Earth in order to make predictions about what might lie beneath the Earth’s surface. One of the fundamental phenomena associated with the Earth is the magnetic field or geomagnetic field. By localising magnetic anomalies within the geomagnetic field one can make predictions or inferences about the localised geophysical makeup and potential ore bodies, hydrocarbon deposits or archaeological artefacts that might exist below the surface. The fundamental sensor used to perform these surveys is the magnetometer. The concept of an unmanned aerial vehicle (UAV) has been around since 1915, with the first manufactured UAV appearing in 1916. Subsequent to the realisation of the UAV in the 1950s by Ryan Aeronautical for military reconnaissance, the idea of using UAV platforms to perform dull, dirty and dangerous functions has become common-place in the military environment. The first practical use of a UAV came in the 1991 Gulf War. The subsequent appearance of UAVs in the civilian realm can largely be attributed to the advent of low cost, high power density, lithium based batteries in the 1990s and the growth of the radio controlled (RC) hobbyist market.
Open Access
Modelling and design of an autonomous sailboat for ocean observation
(2014) Kilpin, Geoffrey; Verrinder, Robyn
This study presents various aspects of the development of an autonomous sailboat for ocean observation, with specific focus on modelling and simulation. The potential value of such platforms for ocean observation is well established, with there being a number of expected advantages over existing solutions. A comprehensive literature review is presented, revealing that the modelling of sailboats is an existing field but that the modelling of small autonomous platforms appears to have been limited. This study develops three and four degree of freedom models of a small autonomous J-Class style sailboat. The sailboat is a prototype platform which is developed from its existing state as part of the study. Both models are validated against data logged during field tests, showing broad agreement with some limitations being noted. Results of simulations of the models are used to draw a number of conclusions regarding the appropriate design of the platform's wing-sail, the wing-sail's control requirements in diffirent wind conditions, potential modifications of the platform's design, and the control of its heading while sailing. Results are also used to inform the proposal of a novel 'variable draft sailing spar' as an alternative autonomous sailboat design.
Open Access
Multiple Mobile Robot SLAM for collaborative mapping and exploration
(2021) Dikoko, Boitumelo; Verrinder, Robyn; Boje, Edward
Over the past five decades, Autonomous Mobile Robots (AMRs) have been an active research field. Maps of high accuracy are required for AMRs to operate successfully. In addition to this, AMRs needs to localise themselves reliably relative to the map. Simultaneous Localisation and Mapping (SLAM) address the problem of both map building and robot localisation. When exploring large areas, Multi-Robot SLAM (MRSLAM) has the potential to be far more efficient and robust, while sharing the computational burden across robots. However, MRSLAM encounters issues such as difficulty in map fusion of multi-resolution maps, and unknown relative positions of the robots. This thesis describes a distributed multi-resolution map merging algorithm for MRSLAM. HectorSLAM, which is one of many single robot SLAM implementations, has demonstrated exceptional results and was selected as the basis for the MRSLAM implementation in this project. We consider the environment to be three-dimensional with the maps being constrained to a two-dimensional plane. Each robot is equipped with a laser range sensor for perception and has no information regarding the relative positioning of the other robots. The experiments were conducted both in simulation and a real-world environment. Up-to three robots were placed in the same environment with Hector-SLAM running, the local maps and localisation were then sent to a central node, which attempted to find map overlaps and merge the resulting maps. When evaluating the success of the map merging algorithm, the quality of the map from each robot was interrogated. Experiments conducted on up to three AMRs show the effectiveness of the proposed algorithms in an indoor environment.
Open Access
Proof of concept data logger for non-active power
(2013) Martindale, Christine; Verrinder, Robyn; Gaunt, C Trevor
This dissertation discusses a metering concept design that is based on equipment cheaper and smaller than a laptop which is able to meet the requirements of power measurements, such as those for non-active power, that need relatively high frequency, simultaneous sampling. An Analog Devices ADE7878 energy measurement IC is used for measurement of a three phase system instantaneous voltage and current. A STMicroelectronics STM32F4 ARM Cortex M4 is used for the digital signal processing. The software uses direct memory access and high speed data capture to allow enough time to perform the new general power theory proposed by Gaunt and Malengret [1]. The results are tested to ensure there is no data corruption and basic calibration is performed. The system has an upper limit of 2 000 Hz sampling frequency at which power calculations and instantaneous, simultaneously sampled voltage and current values can be output in binary format to an SD card without loss or corruption of data. This project considers only the ability of the system to accurately perform the power calculations.
Open Access
Sea-state interaction based dynamic model of the Liquid Robotics' Wave Glider: Modelling and control of a hybrid multi-body vessel
(2018) Rampersadh, Gevashkar; Verrinder, Robyn; Boje, Edward
A new class of unmanned marine research vessels makes use of wave propulsion to minimise energy requirements during voyages. Existing models of these hybrid sea-surface and underwater craft have not considered if the platform’s interaction with the immediate surrounding sea could be incorporated to allow for more accurate navigation and path planning. To this end a detailed three-dimensional model of one such vessel, the Liquid Robotics’ Wave Glider, has been developed in this study. The multi-body system is described using DenavitHartenberg parametrisation and a Lagrangian approach is used to generate the equations of motion for the body. Physical dimensions are derived from platform measurements and from the product specification sheet, hydrodynamic factors are derived from a SolidWorks model of the system, and added mass components are determined from empirical data. Finally, the dynamic model is verified for a given sea state and multiple sea states are tested to investigate the effect on the model’s performance. The developed Wave Glider model is shown to have a realistic response when hydrodynamic factors, added mass and hydrodynamic damping forces, are included and to sea states in terms of the hydrostatic restorative response. The wave-driven propulsion provided by the hydrofoils is shown to have dependence on the sea state by running the model in an open-loop simulation. Following the model validation, a control system is developed for the Wave Glider model to allow yaw attitude control of the glider using the controllable glider rudder input. The control system is generated making use of quantitative feedback theory (QFT) methods to provide robust control for the under-actuated system. The control scheme is shown to provide suitable performance for sea states that result in variable glider velocities. The model’s performance, in terms of the average velocity, is shown to have dependence on the direction of the sea state by running the model in an open-loop simulation for multiple sea states with sinusoidal waves approaching the Wave Glider model from different directions.
Open Access
SHARC Buoy: Robust firmware design for a novel, low-cost autonomous platform for the Antarctic Marginal Ice Zone in the Southern Ocean
(2021) Jacobson, Jamie Nicholas; Verrinder, Robyn; Mishra, Amit; Vichi, Marcello
Sea ice in the Antarctic Marginal Ice Zone (MIZ) plays a pivotal role in regulating heat and energy exchange between oceanic and atmospheric systems, which drive global climate. Current understanding of Southern Ocean sea ice dynamics is poor with temporal and spatial gaps in critical seasonal data-sets. The lack of in situ environmental and wave data from the MIZ in the Antarctic region drove the development of UCT's first generation of in situ ice-tethered measurement platform as part of a larger UCT and NRF SANAP project on realistic modelling of the Marginal Ice Zone in the changing Southern Ocean (MISO). This thesis focuses on the firmware development for the device and the design process taken to obtain key measurements for understanding sea ice dynamics and increasing sensing capabilities in the Southern Ocean. The buoy was required to survive the Antarctic climate and contained a global positioning system, temperature sensor, digital barometer and inertial measurement unit to measure waves-in-ice. Power was supplied to the device by a power supply unit consisting of commercial-grade batteries in series with a temperature-resistant low dropout regulator, and a power sensor to monitor the module. A satellite modem transmitted data through the Iridium satellite network. Finally, Flash chips provided permanent data storage. Firmware and peripheral driver files were written in C for an STMicroelectronics STM32L4 Arm-based microcontroller. To optimise the firmware for low power consumption, inactive sensors were placed in power-saving mode and the processor was put to sleep during periods of no sampling activity. The first device deployment took place during the SCALE winter expedition in July 2019. Two devices were deployed on ice floes to test their performance in remote conditions. However, due to mechanical and power errors, the devices failed shortly after deployment. A third device was placed on the deck of SA Aghulas II during the expedition and successfully survived for one week while continuously transmitting GPS coordinates and ambient temperature. The second generation featured subsequent improvements to the mechanical robustness and sensing capabilities of the device. However, due to the 2020 COVID-19 pandemic, subsequent Antarctic expeditions were cancelled resulting in the final platform evaluation taking place on land. The device demonstrates a proof of concept for a low-cost, ice-tethered autonomous sensing device. However, additional improvements are required to overcome severe bandwidth and power constraints.
Open Access
Thesis Report for the Design and Qualification of a Modular Scientific Drone
(2023) Borrageiro, Mauro; Verrinder, Robyn
Data acquisition using small unmanned aircraft systems (sUAS) or drones for scientific research purposes continues to grow as a new norm across a wide spectrum of disciplines. Consumer sUAS often boast high quality stabilised video capture capabilities and easy-to-use platforms. However, given the need for stringent data provenance and rigour in scientific data capture processes, these platforms are often limiting for academic research purposes due to closed source firmware and communication links, thus requiring sensor and mechanical infrastructure duplication on deployed platforms. This paper presents the LANDRS open source Modular Science Drone; an open source flexible multirotor design, intended to provide the high quality and easy user experience of consumer platforms in a form that enables advantageous tight integration of custom sensors with full access to necessary data and metadata capture process provenance. The Science Drone was designed to operate as either a quadcopter or hexacopter for greater accessibility and versatility, with an operational flight time in the range of 15 to 20 minutes and the ability to lift a 5kg payload. The design used readily available components to configure the powertrain, command and control and power subsystems. The structural frame of the drone was designed and qualified in CAD simulation software with an e↵ort to use simple manufacturing techniques such as 3D printing. The result was a simple to use, easy to replicate, and well documented drone focused on accessibility for research. The presented design was performance tested and validated in a number of typically expected flight behaviours for scientific research drones to determine the endurance and heavy lift capabilities compared to literature referenced commercial and custom drones. The dissertation that follows outlines the design decisions and verification methods for the overall multirotor system and associated subsystems. It subsequently, reports the performance results of the quad- and hexacopter configurations in comparison to the set-out requirements before concluding with suggested future work.
Open Access
UCT SHARC Buoy V3.0: Waves-in-ice measurement in the Antarctic MIZ
(2024) Noyce, Michael; Verrinder, Robyn; Vichi Marcello
The Antarctic Marginal Ice Zone is an area of key scientific interest as the climatic processes in the region affect the global climate. Sea ice in particular has been difficult to model due to its high variability, as a result there is a pressing need to collect in situ data to further develop and validate models of the annual and seasonal sea ice cycles. The UCT SHARC Buoy has been developed to collect and measure in situ wave data that characterises the interaction between ocean waves and sea ice. It aims to do this accurately and with a platform that is cost-effective enough to deploy at scale. To achieve this, the use of low-cost inertial measurement units (IMUs) to measure wave activity on a mock ice floe was validated in a controlled wave tank setting. Following this, the existing SHARC Buoy instrument was upgraded to collect and process high-frequency IMU data and tested for reliability. Finally, the devices were deployed and collected on the SCALE Winter Cruise 2022. The time series were analysed to verify that the instrument's measurements and local calculation of wave parameters were accurate. Several inertial phenomena were noted in the inertial time series and characterised, specifically the collisions between ice floes which are of importance in the region
Open Access
Using regions of interest to track landmarks for RGBD simultaneous localisation and mapping
(2019) Harribhai, Jatin I; Nicolls, F; Verrinder, Robyn
The simultaneous localisation and mapping (SLAM) algorithm have been widely used for autonomous navigation of robots. A type of visual SLAM that is popular among the researchers is RGBD SLAM. However processing immense image data to identify and track landmarks in RGBD SLAM can be computationally expensive for smaller robots. This dissertation presents an alternate method to reduce the computational time. The proposed algorithm extracts features from a region of interest (ROI) to track landmarks for RGBD SLAM. This strategy is compared to the traditional method of extracting features from an entire image. The ROI algorithm is implemented via a pre-processing algorithm, which is then integrated into the RGBD SLAM framework. The pre-processing pipeline implements image processing algorithms in three stages to process the data. Stage one uses a ROI algorithm to detect ROIs in an image. For visual SLAM such as RGBD SLAM, objects that are highly detailed are used as landmarks. Hence the ROI algorithm is designed to detect ROIs containing highly detailed objects. Stage two extracts features from the image and stage three uses feature matching algorithms to re-identify a ROI. Once a ROI has been successfully re-identified, it is stored and categorised as a landmark for RGBD SLAM. Scale invariant feature transform (SIFT), speeded up robust features (SURF) and orientated FAST and rotated BRIEF (ORB) are three feature extraction algorithms that are used in stage two. The outcomes from this study revealed that the pipeline was able to successfully create a database of landmarks which can be re-identified in subsequent frames. In addition, the results showed that when the pipeline is configured such that SURF features are used with a bigger ROI, RGBD SLAM produced more accurate results in determining the position of the robot compared to the traditional method of extracting features from an entire image. However, this strategy requires more computational time. The findings further revealed that this strategy still out performs the traditional method when the number of features extracted is reduced. This indicated that this strategy performs more robustly compared to the traditional method in environments that can contain few features. The method presented in this study did not improve the computational time of RGBD SLAM but did improve the accuracy in localizing the robot.
Open Access
Vision-Based automatic translation for South African Sign Language (SASL)
(2024) Setshekgamollo, Mokgadi; Verrinder, Robyn; Tsoeu, Mohohlo
There are more than four million South Africans who are deaf and hard of hearing (DHH). However, most people with hearing abilities neither understand sign language nor know how to sign. This creates a communication barrier between the deaf and hard of hearing and people with hearing, to the disadvantage of the DHH. In 2018, South African Sign Language (SASL) became an official subject in South African schools and in 2023 it became South Africa's 12th official language. However, these implementations do not impose it on institutions and service providers. Although some provisions are made to cater to the needs of DHH people in the form of sign language interpreters, such interpreters are not always readily available. Sign language interpreters are also costly, charging in excess of R500.00 per hour. In this research, we developed the first vision-based Neural Sign Language Translation model for SASL as a first step towards bridging the communication gap. To this end, we recorded a sizeable parallel SASL and English corpus with the help of six sign language interpreters, three of which are native signers and constitute around 90% of the dataset. The dataset comprises 5047 sentences in the domain of government and politics recorded in a studio setting with a uniform green background. At an average of 3.83 seconds per segment, this equates to around five hours of sign language data. We conducted comprehensive experiments using various visual feature extraction architectures as well as translation architectures. We found that recurrent translation models outperform transformer models. We also investigated the impact of pretraining our feature extractor on a Continuous Sign Language Recognition task and fine-tuning on the SASL dataset and found that this is effective for improving feature extraction. Our best models achieved a BLEU-4 score of 1.35 on the SASL test set, a comparable performance to the How2Sign dataset with a best BLEU-4 score of 1.73 but much lower than on the RWTH-PHOENIX-Weather 2014T dataset which produced a BLEU-4 score of 13.23 without gloss supervision. Our experiments also showed that annotating fingerspelled words as individual letters improves the performance of the model. Our model might benefit from the collection of more data and the addition of gloss annotation. Our results on the SASL dataset are very poor and still very far from practical, indicating that more resources and experiments are required before we can remove the language barrier between the hearing and the Deaf. This would be most effectively achieved by working in collaboration with the Deaf Community to produce high quality datasets, annotations, and models.
Open Access
Vision-Based automatic translation for South African Sign Language (SASL)
(2024) Setshekgamollo, Mokgadi; Verrinder, Robyn; Tsoeu, Mohohlo
There are more than four million South Africans who are deaf and hard of hearing (DHH). However, most people with hearing abilities neither understand sign language nor know how to sign. This creates a communication barrier between the deaf and hard of hearing and people with hearing, to the disadvantage of the DHH. In 2018, South African Sign Language (SASL) became an official subject in South African schools and in 2023 it became South Africa's 12th official language. However, these implementations do not impose it on institutions and service providers. Although some provisions are made to cater to the needs of DHH people in the form of sign language interpreters, such interpreters are not always readily available. Sign language interpreters are also costly, charging in excess of R500.00 per hour. In this research, we developed the first vision-based Neural Sign Language Translation model for SASL as a first step towards bridging the communication gap. To this end, we recorded a sizeable parallel SASL and English corpus with the help of six sign language interpreters, three of which are native signers and constitute around 90% of the dataset. The dataset comprises 5047 sentences in the domain of government and politics recorded in a studio setting with a uniform green background. At an average of 3.83 seconds per segment, this equates to around five hours of sign language data. We conducted comprehensive experiments using various visual feature extraction architectures as well as translation architectures. We found that recurrent translation models outperform transformer models. We also investigated the impact of pretraining our feature extractor on a Continuous Sign Language Recognition task and fine-tuning on the SASL dataset and found that this is effective for improving feature extraction. Our best models achieved a BLEU-4 score of 1.35 on the SASL test set, a comparable performance to the How2Sign dataset with a best BLEU-4 score of 1.73 but much lower than on the RWTH-PHOENIX-Weather 2014T dataset which produced a BLEU-4 score of 13.23 without gloss supervision. Our experiments also showed that annotating fingerspelled words as individual letters improves the performance of the model. Our model might benefit from the collection of more data and the addition of gloss annotation. Our results on the SASL dataset are very poor and still very far from practical, indicating that more resources and experiments are required before we can remove the language barrier between the hearing and the Deaf. This would be most effectively achieved by working in collaboration with the Deaf Community to produce high quality datasets, annotations, and models.

Browsing by Author "Verrinder, Robyn"

Results Per Page

Sort Options