Exploring Sources of Bias to Improve the Universal Design of Mathematical Problem-Solving Tests

Advancement of testing of mathematical problem-solving skills calls for open-ended, realistic tasks particularly susceptible to bias, compromising the score validity and fairness of tests. Informed by universal design principles, this study framed 360 prototype items developed for the Problem-solving Measures Grades 6-8 Computer Adaptive Tests as a bounded case to illuminate sources of bias in middle-school mathematics assessment. Experts' shared understandings of potential sources of bias were collected using an iterative Delphi technique. Four themes in sources emerged from a thematic analysis of 201 sources of bias identified: assumed lived experiences, language use, item presentation, and stereotypes. These sources potentially placed subgroups of students from certain sociocultural backgrounds (e.g. culture, gender, race/ethnicity, socioeconomic status) at a disadvantage by challenging them in ways unrelated to mathematical problem solving. Recommendations for minimizing potential sources of bias in test development to achieve equitable K-12 mathematics problem-solving tests are discussed.