Evolution of Mental Models of Interactive Machines: A Formal Approach

Abstract

The rapid proliferation of technology has resulted in numerous new devices, which are accessible to end users. This challenges users to quickly become familiar with any new device that they encounter. While the user learns to use a new device, her understanding of it evolves gradually. We use the term user model to refer to a finite state machine (FSM) representing the user’s understanding of a machine.In contrast to the user model, we use the term target model to refer to the FSM representing the behavior of the machine, which the end user is trying to learn. In order to learn how to operate a machine correctly, there is a line of thought within Human Computer Interaction (HCI) that the user interface must communicate the underlying target model to the user in such a way that the user’s understanding of the machine is behaviorally indistinguishable from the target model. The user may attain such a model if her multiple user models progressively approach the target model, while interacting with the device. Relatively little attention has been paid to how user models evolve as a part of a user’s learning process, which is the focus of this thesis. In this thesis, we build on the past work on FSMs for representing mental models, and propose a metric for comparing the differences between FSMs. We apply this metric to represent the process of how a user learns to use a new device by tracing the evolution of user models towards a target model. We study user learning through a progressive comparison of the user model and target model over time by using bisimulation. (Bi)simulation relations allow behavioral comparisons between two models; they indicate whether the two models are behaviorally equivalent, or not. However during learning, user models may be incomplete, erroneous and contradictory, making it important to quantify the amount of learning. This translates to the problem of determining the proximity between the two models. (Bi)simulation relations by themselves do not provide such a metric to measure proximity as they only capture the notion of order, not measure. To quantify the gap between a user model and a target model, we introduce edit distance for measuring behavioral proximity between them. We propose an algorithm to compute edit distance between two models and employ the heuristic procedure on experimental data for computing edit distance between target and user models. The data is organised into two experiments depending on the device the user interacted with: (a) a simple string generator device and (b) a close to real-world vehicle transmission model. The results indicate that the edit distance modulo bisimulation measure of a user model and user learning as measured extensionally through participant responses to given tasks have a strong downhill linear relationship (correlation co- efficient = -0.76). This validates the proposed metric as edit distance converges with progressive user learning, increases for erroneous learning, and remains unchanged indicating no learning. Thus, the work demonstrates that user learning can be witnessed, captured, and measured formally, allowing for a better understanding of how users learn to use an unfamiliar device. The proposed representational technique provides an intensional description of the process involved in learning a new device. The experimental studies indicated that our proposed edit distance metric allows us to examine several questions of interest to the HCI community about the learning process. For example, deciding if the two user models are behaviorally close, or do successive user models for all users converge at some instance, etc. The key here is not to show a mere convergence of the user models, but to capture and represent the process of their evolution to show that the convergence (or divergence) follows a user’s ability to learn the system. Such a representation allows designers to identify the problem areas in a user interface by differentiating instances when a user model is improving from instances where it is not.