A cognitive diagnostic investigation of inequality in mathematics mastery in England

Many education studies use subject-level measurements for achievement, such as scores from PISA or national-level assessments. This approach implies disadvantage affects all areas of a subject equally. However, fine-grained studies have found achievement gaps below the subject level. For example, in mathematics, studies have found girls performed better on calculation questions, while boys performed better on algebra and geometry. As a result, policies or interventions using subject-level score as the sole indicator of mathematics ability would overlook differences between genders on specific topics.
Skills rather than topics
Most modern testing is based on Item Response Theory (IRT). Traditional IRT statistical models assume that a test is unidimensional (i.e. the test measures a single, overall ability) and that questions should function the same for all sociodemographic student groups.
Cognitive diagnostic models (CDM) bypass these assumptions of IRT since they allow multidimensionality. Simply put, CDM are designed by identifying specific skills required to successfully solve each test question. The model estimates each student's mastery of the skills and the difficulty of each skill. The information can then be used to identify groups with similar mastery characteristics.
Studies show that CDM can produce better model-fit compared with IRT (Ma et al., 2020; Yamaguchi & Okada, 2018). Also, true to the goal of cognitive diagnosis, CDM have been used to identify curriculum weaknesses, such as finding that students in the USA need extra support for reading data from tables and graphs (Lee et al., 2011).
Focus
Mathematics is an especially pertinent topic in England considering the government is considering making maths compulsory in post-16 education. Important information for England can be found by investigating groups' mathematics mastery with CDM; for example, identifying whether particular sociodemographic groups struggle with a specific area of the...