Developing a tool for eliciting users  moral theories for automated moral agents

Seakgwa, Kyle

Developing a tool for eliciting users moral theories for automated moral agents

Thesis / Dissertation

2024

Publisher

University of Cape town

Department

Department of Computer Science

Faculty

Faculty of Science

Abstract

In recent work, Rautenbach and Keet have developed a model of a system, which they name Genet, that allows the user to choose which moral theory their automated moral agent will follow. What remains unclear, however, is how the users will make this choice, given that most of them will not have the vocabulary to classify themselves in the moral philosophical terms used by Genet. This issue is what this thesis is meant to address. This was done by building three high fidelity prototypes and then conducting online user evaluations of them. Each of these prototypes implemented an algorithm that was designed based on the elicitation approach of one of three fields: cognitive science, human computer interaction and knowledge engineering. Each of these aimed to computationally determine a user's preferred moral theory, by availing itself of a human-in-the-loop component enabled by discipline specific elicitation stimuli and rules to classify the user. These prototypes were then evaluated from a usability perspective using the System Usability Scale (SUS), and from an accuracy perspective, to determine which is most validly able to elicit users' moral preferences in the form required by Genet. This latter evaluation was done using validation measures based on existing approaches to validation in moral psychology. It was observed that all the prototypes performed equally well in terms of usability, with each having an acceptable SUS score. However, each of the prototypes also performed equally inaccurately in terms of the validity of the moral theory categorizations made. While this evaluation was carried out with only a small sample size (n=20) and thus has limited generalizability, as the first study to compare and computationally implement different moral theory elicitation approaches, the present study contributes to evidence for (or at least fails to falsify) problems with the project of making the design of Automated Moral Agents dependent on elicitation of a user's one preferred moral theory. A positive claim that the data collected here does support is that, at least for some potential users, even computational elicitation tools that use empirically validated measures of moral theory preferences (like those from cognitive science) do not allow one to predict the moral judgements they will make.

Keywords

Information Technology

Reference:

Collections

Masters

Full item page