Exploring Metacognition in Math with Generative AI: A Pilot Study of the MAI-C in Higher Education

This study introduces the Metacognitive Awareness Inventory for Critical AI Use (MAI-C), an adapted version of the original MAI, designed to assess how students critically and ethically engage with Artificial Intelligence (AI) in educational settings. To evaluate its effectiveness, a preliminary implementation was carried out in an undergraduate mathematics course at the University of Salerno, where students employed generative AI during educational activities. Students collaborated in small groups to explore a mathematical property, subsequently submitting the same problem to ChatGPT and performing a comparative analysis between their own reasoning and the AI-generated responses. At the end of the course, participants completed a qualitative questionnaire aimed at uncovering their perceptions, experiences, and interpretations related to AI use. Afterwards, the MAI-C was administered to quantitatively measure their metacognitive awareness in the context of critical AI use. The analysis of responses to the 30-item inventory, structured across six metacognitive dimensions, revealed generally high levels of awareness, particularly in monitoring, evaluation, and self-regulation. These findings suggest that integrating generative AI in educational contexts can enhance students’ metacognitive control, critical reflection, and evaluative reasoning within problem-solving processes. Although limited to a single institutional setting, this exploratory study constitutes an initial step toward validating the revised MAI-C and provides empirical evidence of its pedagogical relevance. Future research will extend the investigation to a broader and more diverse student population, aiming to refine the instrument and confirming its reliability and transferability across different disciplinary domains.