Modelling attrition in the Eastern Cape public health system using multilevel survival analysis and machine learning methods

Perrie, Cailin

Modelling attrition in the Eastern Cape public health system using multilevel survival analysis and machine learning methods

Thesis / Dissertation

2023

Abstract

The size of South Africa's public health workforce is influenced by many factors including, but not limited to, inter-facility transfers, emigration, voluntary exits, illness, death and retirement. Understanding the rate at which public health workers exit or move within the public health system (i.e. the attrition rate), is essential for adequately formulating effective workforce policies and strategies. South Africa's public health system budget currently accounts for an annual 5% attrition rate for health facilities in general. This rate does not consider fluctuations in attrition rates between cadres, across facilities, or across districts. Presently, there are no guidelines or models for predicting attrition within the Eastern Cape (EC) public health care system from an individual, cadre, facility, or district level. As a result, staffing levels are determined entirely by the discretion of facility or departmental managers. The purpose of this investigation was, therefore, to explore and utilize human resource (HR) data within South Africa's public healthcare system, with specific focus on the EC province, to predict attrition rates within and across cadres, health facilities, and districts. The study places a large focus on using the findings of the study to improve budgeting and health care staffing levels. The study thus aims to develop predictive models that are capable of handling data that is hierarchical in nature, use these models to identify level specific factors that both negatively and positively impact annual attrition rates, and compare predictive models to determine the most effective model for predicting attrition rates in the EC public health sector. The study further aims to perform a historical data analysis on the HR data to identify areas of high concern regarding attrition. Based on a preliminary and historical exploratory data analysis (EDA) of the EC province's public heath HR data, the annual attrition rates between 2010 and 2020 have consistently exceeded this budgeted 5%, with the annual attrition rate in some years reaching as high as 15.65%. The preliminary analysis further indicated that attrition rates are subject to high variation when computed at different levels (i.e. cadre and facility level groupings) as well as across different years. Consequently, the Eastern Cape Department of Health (EC DOH) have been historically and holistically under budgeting for attrition. Additionally, by catering for attrition at a provincial level only, the department has been neglecting the effects that within and between-group variation in attrition has on budget formulation. The historical EDA further identified several cadres that consistently experienced high levels of attrition namely, the Medical Services, Nursing, and Primary Health Care cadres. The job titles that fall within these cadres (i.e. Medical Specialists, Clinic Specialists, and Nurses) are considered i critical to the functioning of any health facility as they are responsible for providing medical care to patients. The historically high attrition levels obtained in these cadres are, therefore, alarming as they suggest that the EC province can expect to consistently see the same or a degrading level of patient care in the years to come. The findings from the historical EDA, and the potential risks associated with over or under-budgeting for attrition, suggest that there is a financial incentive for the EC DOH to develop models capable of accurately predicting future attrition rates within and between multiple levels within the EC province. The application of both statistical and machine learning (ML) modelling techniques were thus explored in this investigation, however, only one statistical modelling method (multilevel discrete-time event models) and three ML modelling methods (multi-layer perceptron neural networks, generalized linear mixed-model trees, and tree-based mixed effect models) were explored. This was due to their potential ability to handle and, effectively model, the complex multilevel and longitudinal HR data available for use in this study. Unfortunately, all multilevel machine learning models explored failed to converge, resulted in excessive computational time forcing an abort, or simply resulted in poor model performance when evaluated on unseen data. Based on these findings, and within the limitations of the study scope, it is accepted that these three modelling methods are unable to outperform traditional multilevel statistical methods at this time. The multilevel discrete-time event models, however, are able to handle the complex data used in this investigation. Based on model performance metrics, the best multilevel discrete-time event model developed in this investigation is considered feasible for use in attrition prediction for the EC DOH. The model is further capable of being used to determine time-indicator and healthcare worker level variables influencing attrition. Overall, the insights gained from this investigation can be used to help guide intervention planning, optimize HR capacity planning processes and, in turn, improve overall budgeting for the EC health system. The findings and limitations of this investigation, however, open up opportunities for future work both as improvements to, or extensions of, the data preparation processes as well as model formulations and optimizations. Such follow-up work may include the exploration of different attrition definitions and the impact that has on the investigations findings, exploring methods for reducing HR healthcare data integrity issues, and provisioning or implementing re-sampling techniques, different cadre grouping strategies, or virtual machines to improve the performance of the machine learning models proposed.

Keywords

Statistical Sciences

Reference:

Collections

Masters

Full item page