A Multidimensional Evaluation of a School Disengagement Predictive Model Across Technical, Usability, and Acceptance Dimensions
The aim of this study is to conduct a comprehensive evaluation of the proposed predictive model by validating its technical performance, usability, and acceptance.
School disengagement is a persistent challenge that can lead to adverse academic and social outcomes if not identified early. Although predictive and data-driven approaches have been proposed to support early intervention, many existing systems lack comprehensive evaluation across technical performance, usability, and user acceptance, limiting their adoption in real school environments. This gap motivated the development and evaluation of a school disengagement predictive model implemented through the MyBuddy mobile and web-based systems.
A six-phase methodology was adopted in this study to systematically develop, implement, and evaluate the proposed school disengagement predictive model through the implementation of MyBuddy. The methodology comprised six sequential phases: factor identification, model design and development, model verification, model refinement, model validation, and performance evaluation. Initially, the predictive model was developed based on 13 contributing factors identified through systematic literature reviews. Following expert verification, the model was refined to incorporate an additional factor, resulting in a final model consisting of 14 contributing factors.
The verified model was subsequently implemented through the development of MyBuddy, which was deployed as both web- and mobile-based applications. A comprehensive evaluation was then conducted to assess both the technical quality of the system and the effectiveness of the predictive model. Technical evaluation involved 27 information technology experts who assessed the system’s functionality, reliability, and overall performance. Subsequently, 27 intended users evaluated the system for usability, interface design, perceived usefulness, and user satisfaction. Finally, predictive performance was evaluated using data from 50 secondary school students across two schools by comparing MyBuddy’s predictions with counsellors’ assessments using confusion matrix analysis. This comprehensive methodology enabled a rigorous evaluation of the predictive model’s conceptual validity, technical robustness, usability, and predictive performance in supporting the early identification of students at risk of school disengagement.
The findings contribute to sustainable educational innovation and align with Sustainable Development Goal 4 (Quality Education) by supporting inclusive and timely intervention strategies.
The findings indicate that the proposed model can effectively support the early identification of students at risk of school disengagement by classifying students into low-, medium-, and high-risk categories. The prediction results showed reasonable agreement with school counsellors’ professional assessments, supporting the role of MyBuddy as a decision-support tool rather than a standalone diagnostic system. In addition, the usability and user acceptance evaluations demonstrated positive responses from school counsellors, particularly regarding ease of use, interface clarity, and suitability for routine use in school counselling via a mobile-based platform.
These findings indicate that the system is accurate, practical, user-friendly, and well-suited for integration into real-world school counselling workflows.
Future work will focus on improving model accuracy through larger, more diverse datasets, algorithm enhancements, and broader validation across varied educational contexts.
The study demonstrates the feasibility of integrating predictive analytics with user-centred design to support early identification of school disengagement, for use in schools and by the Ministry of Education.
Future work should focus on improving the model through larger and more diverse datasets, including longitudinal data, and by integrating advanced analytical techniques to enhance predictive performance. Broader stakeholder involvement and system integration with existing school platforms are also recommended to support scalability, adoption, and long-term impact.
Back