The Need for a Comprehensive Strategy to Evaluate Search Engine Performance in the Classroom
Published in The 3rd KidRec Workshop co-located with ACM Interaction Design and Children (IDC) Conference 2019, 2019
Recommended citation: Anuyah, O., Green, M., Milton, A., & Pera, M. S. (2019). "The Need for a Comprehensive Strategy to Evaluate Search Engine Performance in the Classroom " The 3rd KidRec Workshop co-located with ACM Interaction Design and Children (IDC) Conference. http://ashleemilton.github.io/files/comprehensiveKidRec2019.pdf
Given how ingrained Search Engines (SEs) are in educational environments, it is essential to evaluate their performance in response to inquiries that pertain to the classroom setting. In this position paper, we discuss the limitations of relying solely on traditional Information Retrieval metrics and usability studies. We argue in favor of a new comprehensive strategy for assessment that integrates other aspects (i.e., the search context) with new and existing measures, in order to better quantify positive and negative factors that influence SEs outcomes targeting children.