This paper introduces ETQ-AI (Effective Teaching Questionnaire—Artificial Intelligence), a conversational AI framework designed to support teacher professional development by identifying pedagogical misconceptions and fostering critical reflection. Building on Evidence-Informed Education (EIE) principles, the system integrates a 50-item diagnostic questionnaire with an automated, Gemini-driven Explainable AI (XAI) feedback layer. When teachers complete the diagnostic form, they receive personalized feedback and can engage in dialogic interactions with Google Gemini to explore the underlying meta-analytic evidence (e.g., Hattie’s effect sizes, Cognitive Load Theory) or argue personal objections. We evaluated system fidelity against an expert panel benchmark (𝑁=64 items) and conducted a pilot evaluation with 123 educators (70 primary teachers, 40 secondary teachers, 13 school leaders) recruited at an EIE conference. Gemini demonstrated high alignment with expert evaluations (97%), matching or exceeding static baseline feedback in user satisfaction (Mean = 4.1/5 vs. 3.8/5). Qualitative thematic analysis of 34 participant objections revealed key areas of tension, particularly around contextual constraints and pedagogical subjectivity. While the findings validate the technical proof-of-concept, system usability, and user engagement of ETQ-AI as an interactive scaffolding interface, we emphasize that this pilot study measures user perceptions and conversational patterns rather than direct pre/post shifts in critical reflection or classroom instructional practice.
A Conversational AI Framework for Teacher Training
Sergio, MIRANDA
;
2026
Abstract
This paper introduces ETQ-AI (Effective Teaching Questionnaire—Artificial Intelligence), a conversational AI framework designed to support teacher professional development by identifying pedagogical misconceptions and fostering critical reflection. Building on Evidence-Informed Education (EIE) principles, the system integrates a 50-item diagnostic questionnaire with an automated, Gemini-driven Explainable AI (XAI) feedback layer. When teachers complete the diagnostic form, they receive personalized feedback and can engage in dialogic interactions with Google Gemini to explore the underlying meta-analytic evidence (e.g., Hattie’s effect sizes, Cognitive Load Theory) or argue personal objections. We evaluated system fidelity against an expert panel benchmark (𝑁=64 items) and conducted a pilot evaluation with 123 educators (70 primary teachers, 40 secondary teachers, 13 school leaders) recruited at an EIE conference. Gemini demonstrated high alignment with expert evaluations (97%), matching or exceeding static baseline feedback in user satisfaction (Mean = 4.1/5 vs. 3.8/5). Qualitative thematic analysis of 34 participant objections revealed key areas of tension, particularly around contextual constraints and pedagogical subjectivity. While the findings validate the technical proof-of-concept, system usability, and user engagement of ETQ-AI as an interactive scaffolding interface, we emphasize that this pilot study measures user perceptions and conversational patterns rather than direct pre/post shifts in critical reflection or classroom instructional practice.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
