Predictive models for computing clusters: a case study based on Ilifu

Cloete, Jacobus Marthinus

Predictive models for computing clusters: a case study based on Ilifu

Thesis / Dissertation

2026

Publisher

University of Cape Town

Department

Department of Statistical Sciences

Faculty

Faculty of Science

Abstract

This thesis investigates the feasibility of simulating job waiting times on a High-Performance Computing (HPC) cluster using a simplified discrete event simulation approach. We develop SimPE, a model that uses a Processor Equivalent (PE) metric to reduce the complexity of multi-resource job scheduling to a single dimension. The model is tested against a real Slurm Training Cluster and a Slurm Simulator for various scheduling policies including First-In-First-Out (FIFO), static and dynamic priority, and backfill scheduling. Results show that SimPE effectively models FIFO scheduling with waiting time estimates within 12% of the actual values. For static priority scheduling and dynamic priority scheduling, the model maintains similar levels of accuracy. However, for backfill scheduling, SimPE had more aggressive backfilling which led to much lower waiting time estimates. When modelling jobs with multiple resource requirements, SimPE accurately estimates when one resource dominates, but becomes too conservative when different resources are equally important. This research shows that while PE-based simulation can effectively model certain aspects of cluster behaviour, careful thought is needed when applying it to more complex scheduling scenarios. The findings help to increase our understanding of simple cluster simulation approaches and their practical limitations.

Keywords

Data science

Reference:

Collections

Masters

Full item page