Towards A User-Centric Framework For Measuring The Quality of A Tertiary Education Recommendation System In South Africa

Recommender systems (RS) have gained traction in education for guiding learners in selecting tertiary institutions and qualification pathways. The quality of these RS re-mains underexplored, particularly in contexts where mismatches between recommen-dations and learner’s capabilities can have far-reaching consequences. This study aims to develop a user-centric framework for evaluating the quality of tertiary education RS, using Design Science Research (DSR) methodology. Using mixed methods, the study iteratively integrates survey and interview data. Structural Equation Modelling (SEM) tests hypotheses rooted in existing theoretical frameworks for RSs in general. Results from two iterations of the ongoing study reveal critical relationships between system attributes, such as Perceived Recommendation Quality, System Usefulness, Ease of Use, and Overall Satisfaction, while highlighting the importance of recommendation diversity. While situational or personal characteristics did not significantly moderate constructs like trust and utility as expected. The study findings highlight the importance of a user-centric approach to designing RS in tertiary education, providing actionable insights for developers, institutions, and policymakers. Future iterations will integrate system-generated data to enhance the robustness of interaction measures, paving the way for more effective and equitable RS in the educational landscape.