Self Adaptation of Machine Learning Enabled Systems Through QoS-Aware Model Switching

Abstract

Machine Learning (ML), particularly deep learning, has seen vast advancements, leading to the rise of Machine Learning-Enabled Systems (MLS). However, numerous software engineering challenges persist in propelling these MLS into production, largely due to various run-time uncertainties that impact the overall Quality of Service (QoS). These uncertainties emanate from ML models, software components, and environmental factors. Self-adaptation techniques present potential in managing runtime uncertainties, but their application in MLS remains largely unexplored. As a solution, this thesis proposes Machine Learning Model Balancer, a novel concept focusing on managing uncertainties related to ML models by using multiple models in runtime. Subsequently, the thesis introduces AdaMLS, an novel approach that leverages the Machine Learning Model Balancer concept for continuous adaptation. AdaMLS extends the traditional MAPE-K loop, employing lightweight unsupervised learning for dynamic model switching, thereby ensuring consistent QoS in dynamic environments. The effectiveness of AdaMLS is demonstrated through an object detection use case, showcasing its ability to effectively mitigate run-time uncertainties and surpass both naive approaches and standalone models in terms of QoS. We further developed SWITCH, an exemplar to demonstrate the practical application of our research in self-adaptation of MLS. The discussion on SWITCH highlights its role as a tool designed to enhance self-adaptive capabilities in MLS through dynamic model switching in runtime. SWITCH is developed as to cater to a broad range of ML scenarios. It features advanced input handling, real-time data processing, and logging for adaptation metrics, supplemented with an interactive real-time dashboard for system observability. Through its architecture and user-friendly interface, SWITCH not only demonstrates adaptability and performance but also serves as a valuable platform for researchers, practitioners, and students to explore self-adaptation in MLS. Beyond the primary focus on ensuring optimal QoS, we also explore the application of the Machine Learning Model Balancer concept in two main areas. Firstly, the EcoMLS approach leverages this concept to enhance the sustainability of MLS. By optimally balancing energy consumption with model confidence through runtime ML model switching, EcoMLS marks a significant step towards sustainable, energyefficient ML solutions. Secondly, RelMLS approach to systems, where machine learning is deployed in streaming mode. Through experiments and implementation, this approach has shown promising results in object detection use case, adopting a software architecture-based solution to dynamically switch between models based on contextual reliability. This thesis encapsulates a comprehensive exploration of the Machine Learning Model Balancer concept, from its theoretical introduction to practical applications in diverse MLS scenarios. Through AdaMLS, SWITCH, EcoMLS, and RelMLS implementations, we demonstrate the potential of our approaches in enhancing both the QoS and sustainability of MLS. Our findings suggest that the judicious application of model switching and self-adaptation techniques can significantly mitigate run-time uncertainties, paving the way for more resilient, efficient, and adaptable MLS.