Free or Systematic? Structured AI Guidance Improves Tutor Knowledge-State Estimation Quality and Self-Regulated Learning in Programming Education
Abstract
Generative AI can accelerate programming learning, but friction-free, answer-oriented help may weaken learners' self-regulation. We compare unstructured answer-oriented assistance with systematic scaffolding that provides step-by-step hints, knowledge-state feedback from a Graph-enhanced Interactive Knowledge Tracing (GIKT) model, and difficulty-matched practice. We conduct two within-subject crossover studies across Java and Python programming courses. Students alternate between conditions by concept unit with counterbalanced starting conditions. The systematic scaffolding condition delivers interactive guidance through a Learn-Practice-Evaluate-Support loop, providing Socratic prompts and step-by-step hints without revealing final solutions, along with personalized practice recommendations and diagnostic views. We analyze primary outcomes of self-reported transfer (Likert 1--6) and tutor knowledge-state estimation quality, quantified by Expected Calibration Error (ECE) under time-split evaluation, using linear mixed-effects models. We demonstrate that systematic scaffolding yields higher self-reported transfer with convergent trends in exam subscores, and substantially better prediction calibration (ECE: 0.13 vs. 0.24, effect size d=1.64). Process analyses reveal increased planning and monitoring engagement alongside reduced time pressure and frustration, transforming engagement behaviors into productive learning processes while enhancing the diagnostic value of interaction traces. Our findings demonstrate that systematic scaffolding outperforms unstructured answer-oriented assistance, establishing it as the superior approach for AI-supported programming education.