Research Article

The cognitive reflection test and students’ achievements in mathematics and physics

Daniel Doz 1 * , Josip Sliško 2
More Detail
1 Faculty of Education, University of Primorska, Koper, SLOVENIA2 Faculty of Mathematical and Physical Sciences, Benemérita Universidad Autónoma de Puebla, Puebla, MEXICO* Corresponding Author
European Journal of Science and Mathematics Education, 12(1), January 2024, 85-96, https://doi.org/10.30935/scimath/13832
Published Online: 25 October 2023, Published: 01 January 2024
OPEN ACCESS   2429 Views   748 Downloads
Download Full Text (PDF)

ABSTRACT

The cognitive reflection test (CRT) assesses an individual’s capacity to restrain impulsive and intuitive responses and to engage in critical reflection on mathematical problems. The literature indicates that several factors influence students’ performance on CRT, including gender, age, and prior knowledge of mathematics. In this study, our objective was to investigate the correlation between CRT scores and students’ achievements in both mathematics and physics. We conducted our research with a sample of 150 Italian high school students, and the findings revealed a positive predictive relationship between CRT scores and students’ performance in both mathematics and physics. Furthermore, we employed an ordinal logistic regression to evaluate the impact of CRT scores, gender, and school level on students’ achievements in mathematics and physics. The results showed that both CRT scores and school level had statistically significant effects on predicting these achievements. In contrast, gender emerged as a statistically significant factor only in predicting students’ mathematics achievements.
Keywords: achievements, cognitive reflection test, mathematics, ordinal logistic regression, physics

CITATION (APA)

Doz, D., & Sliško, J. (2024). The cognitive reflection test and students’ achievements in mathematics and physics. European Journal of Science and Mathematics Education, 12(1), 85-96. https://doi.org/10.30935/scimath/13832

REFERENCES

  1. Akpotor, J., & Egbule, E. (2020). Gender difference in the scholastic achievement test (SAT) among school adolescents. World Journal of Education, 10(1), 97-101. https://doi.org/10.5430/wje.v10n1p97
  2. Bartlett, J. E., & Charles, S. (2021). Power to the people: A beginner’s tutorial to power analysis using Jamovi. PsyArXiv. https://doi.org/10.31234/osf.io/bh8m9
  3. Blacksmith, N., Yang, Y., Behrend, T. S., & Ruark, G. A. (2019). Assessing the validity of inferences from scores on the cognitive reflection test. Journal of Behavioral Decision Making, 32(5), 599-612. https://doi.org/10.1002/bdm.2133
  4. Brañas-Garza, P., Kujal, P., & Lenkei, B. (2019). Cognitive reflection test: Whom, how, when. Journal of Behavioral and Experimental Economics, 82, 101455. https://doi.org/10.1016/j.socec.2019.101455
  5. Bull, R., & Lee, K. (2014). Executive functioning and mathematics achievement. Child Development Perspectives, 8(1), 36-41. https://doi.org/10.1111/cdep.12059
  6. Campitelli, G., & Gerrans, P. (2014). Does the cognitive reflection test measure cognitive reflection? A mathematical modeling approach. Memory & Cognition, 42(3), 434-447. https://doi.org/10.3758/s13421-013-0367-9
  7. De Smedt, B., Janssen, R., Bouwens, K., Verschaffel, L., Boets, B., & Ghesquière, P. (2009). Working memory and individual differences in mathematics achievement: A longitudinal study from first grade to second grade. Journal of Experimental Child Psychology, 103(2), 186-201. https://doi.org/10.1016/j.jecp.2009.01.004
  8. Epstein, S. (1994). Integration of the cognitive and the psychodynamic unconscious. American Psychologist, 49(8), 709-724. https://doi.org/10.1037/0003-066X.49.8.709
  9. Etcheverry, P. T., Ignjatov, J. S., & de Lourdes Juárez, E. (2020). Influencia de la escolaridad en el desarrollo del razonamiento lógico y la reflexión cognitiva en estudiantes de bachillerato [Influence of schooling on the development of logical reasoning and cognitive reflection in high school students]. UNIÓN-Revista Iberoamericana de Educación Matemática, 16(60), 212-232.
  10. Fang, S. C., Hsu, Y. S., & Lin, S. S. (2019). Conceptualizing socio-scientific decision making from a review of research in science education. International Journal of Science and Mathematics Education, 17(3), 427-448. https://doi.org/10.1007/s10763-018-9890-2
  11. Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149-1160. https://doi.org/10.3758/BRM.41.4.1149
  12. Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19(4), 25-42. https://doi.org/10.1257/089533005775196732
  13. Frosch, C., & Simms, V. (2015). Understanding the role of reasoning ability in mathematical achievement. In Euroasianpacific joint conference on cognitive science. In Proceedings of the EuroAsianPacific Joint Conference on Cognitive Science (pp. 633-638). https://doi.org/10.13140/RG.2.1.1107.2727
  14. Gette, C. R., & Kryjevskaia, M. (2019). Establishing a relationship between student cognitive reflection skills and performance on physics questions that elicit strong intuitive responses. Physical Review Physics Education Research, 15(1), 010118. https://doi.org/10.1103/PhysRevPhysEducRes.15.010118
  15. Gilmore, C., Keeble, S., Richardson, S., & Cragg, L. (2015). The role of cognitive inhibition in different components of arithmetic. ZDM, 47(5), 771-782. https://doi.org/10.1007/s11858-014-0659-y
  16. Gómez-Chacón, I. M., García-Madruga, J. A., Vila, J. Ó., Elosúa, M. R., & Rodríguez, R. (2014). The dual processes hypothesis in mathematics performance: Beliefs, cognitive reflection, working memory and reasoning. Learning and Individual Differences, 29, 67-73. https://doi.org/10.1016/j.lindif.2013.10.001
  17. Gómez-Veiga, I., Vila Chaves, J. O., Duque, G., & García Madruga, J. A. (2018). A new look to a classic issue: Reasoning and academic achievement at secondary school. Frontiers in Psychology, 9, 400. https://doi.org/10.3389/fpsyg.2018.00400
  18. Hyde, J. S., Fennema, E., & Lamon, S. J. (1990). Gender differences in mathematics performance: A meta-analysis. Psychological Bulletin, 107(2), 139-155. https://doi.org/10.1037/0033-2909.107.2.139
  19. Kryjevskaia, M., Stetzer, M. R., Lindsey, B. A., McInerny, A., Heron, P. R., & Boudreaux, A. (2020). Designing research-based instructional materials that leverage dual-process theories of reasoning: Insights from testing one specific, theory-driven intervention. Physical Review Physics Education Research, 16(2), 020140. https://doi.org/10.1103/PhysRevPhysEducRes.16.020140
  20. Lem, S., Kempen, G., Ceulemans, E., Onghena, P., Verschaffel, L., & Van Dooren, W. (2015). Combining multiple external representations and refutational text: An intervention on learning to interpret box plots. International Journal of Science and Mathematics Education, 13(4), 909-926. https://doi.org/10.1007/s10763-014-9604-3
  21. Liu, X., & Koirala, H. (2012). Ordinal regression analysis: Using generalized ordinal logistic regression models to estimate educational data. Journal of Modern Applied Statistical Methods, 11(1), 242-254. https://doi.org/10.22237/jmasm/1335846000
  22. Louis, R. A., & Mistele, J. M. (2012). The differences in scores and self-efficacy by student gender in mathematics and science. International Journal of Science and Mathematics Education, 10(5), 1163-1190. https://doi.org/10.1007/s10763-011-9325-9
  23. Ma, X., & Xu, J. (2004). The causal ordering of mathematics anxiety and mathematics achievement: A longitudinal panel analysis. Journal of Adolescence, 27(2), 165-179. https://doi.org/10.1016/j.adolescence.2003.11.003
  24. McFadden, D. (1977). Quantitative methods for analyzing travel behavior of individuals: Some recent developments. https://elischolar.library.yale.edu/cgi/viewcontent.cgi?article=1706&context=cowles-discussion-paper-series
  25. Primi, C., Donati, M. A., Chiesi, F., & Morsanyi, K. (2018). Are there gender differences in cognitive reflection? Invariance and differences related to mathematics. Thinking & Reasoning, 24(2), 258-279. https://doi.org/10.1080/13546783.2017.1387606
  26. Riegle-Crumb, C., & Moore, C. (2014). The gender gap in high school physics: Considering the context of local communities. Social Science Quarterly, 95(1), 253-268. https://doi.org/10.1111/ssqu.12022
  27. Ring, P., Neyse, L., David-Barett, T., & Schmidt, U. (2016). Gender differences in performance predictions: Evidence from the cognitive reflection test. Frontiers in Psychology, 7, 1680. https://doi.org/10.3389/fpsyg.2016.01680
  28. Shtulman, A., & McCallum, K. (2014). Cognitive reflection predicts science understanding. In Proceedings of the Annual Meeting of the Cognitive Science Society.
  29. Skaalvik, E. M., Federici, R. A., & Klassen, R. M. (2015). Mathematics achievement and self-efficacy: Relations with motivation for mathematics. International Journal of Educational Research, 72, 129-136. https://doi.org/10.1016/j.ijer.2015.06.008
  30. Sliško, J. (2017). Self-regulated learning in a general university course: Design of learning tasks, their implementation and measured cognitive effects. Journal of European Education, 7(2), 12-24.
  31. Speirs, J. C., Stetzer, M. R., Lindsey, B. A., & Kryjevskaia, M. (2021). Exploring and supporting student reasoning in physics by leveraging dual-process theories of reasoning and decision making. Physical Review Physics Education Research, 17(2), 020137. https://doi.org/10.1103/PhysRevPhysEducRes.17.020137
  32. Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate? Behavioral and Brain Sciences, 23(5), 645-665. https://doi.org/10.1017/S0140525X00003435
  33. Stieger, S., & Reips, U. D. (2016). A limitation of the cognitive reflection test: Familiarity. PeerJ, 4, e2395. https://doi.org/10.7717/peerj.2395
  34. Szaszi, B., Szollosi, A., Palfi, B., & Aczel, B. (2017). The cognitive reflection test revisited: Exploring the ways individuals solve the test. Thinking & Reasoning, 23(3), 207-234. https://doi.org/10.1080/13546783.2017.1292954
  35. Toplak, M. E., West, R. F., & Stanovich, K. E. (2011). The cognitive reflection test as a predictor of performance on heuristics-and-biases tasks. Memory & Cognition, 39(7), 1275-1289. https://doi.org/10.3758/s13421-011-0104-1
  36. Toplak, M. E., West, R. F., & Stanovich, K. E. (2014). Assessing miserly information processing: An expansion of the cognitive reflection test. Thinking & Reasoning, 20(2), 147-168. https://doi.org/10.1080/13546783.2013.844729
  37. Travers, E., Rolison, J. J., & Feeney, A. (2016). The time course of conflict on the cognitive reflection test. Cognition, 150, 109-118. https://doi.org/10.1016/j.cognition.2016.01.015
  38. Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty. Science, 185(4157), 1124-1131. https://doi.org/10.1126/science.185.4157.1124
  39. Warner, P. (2008). Ordinal logistic regression. Journal of Family Planning and Reproductive Health Care, 34(3), 169-170. https://doi.org/10.1783/147118908784734945
  40. Williamson, K. E., & Willoughby, S. D. (2012). Student understanding of gravity in introductory astronomy. Astronomy Education Review, 11(1), 10105. https://doi.org/10.3847/AER2011025
  41. Wood, A. K., Galloway, R. K., & Hardy, J. (2016). Can dual processing theory explain physics students’ performance on the force concept inventory? Physical Review Physics Education Research, 12(2), 023101. https://doi.org/10.1103/PhysRevPhysEducRes.12.023101
  42. Zhang, D. C., Highhouse, S., & Rada, T. B. (2016). Explaining sex differences on the cognitive reflection test. Personality and Individual Differences, 101, 425-427. https://doi.org/10.1016/j.paid.2016.06.034
  43. Zhou, C., Kuttal, S. K., & Ahmed, I. (2018). What makes a good developer? An empirical study of developers’ technical and social competencies. In Proceedings of the 2018 IEEE Symposium on Visual Languages and Human-Centric Computing (pp. 319-321). IEEE. https://doi.org/10.1109/VLHCC.2018.8506577