Evaluating Information Retrieval Systems for Kids
Published in The 4th KidRec Workshop co-located with ACM Interaction Design and Children (IDC) Conference, 2020
Recommended citation: Milton, A. & Pera, M. S. (2020). "Evaluating Information Retrieval Systems for Kids " The 4th KidRec Workshop co-located with ACM Interaction Design and Children (IDC) Conference. http://ashleemilton.github.io/files/evaluatingKidRec2020.pdf
Evaluation of information retrieval systems (IRS) is a prominent topic among information retrieval researchers–mainly directed at a general population. Children require unique IRS and by extension different ways to evaluate these systems, but as a large population that use IRS have largely been ignored on the evaluation front. In this position paper, we explore many perspectives that must be considered when evaluating IRS; we specially discuss problems faced by researchers who work with children IRS, including lack of evaluation frameworks, limitations of data, and lack of user judgment understanding.