Measuring the Application of Various Open Source AI Models for Intro Computer Science Course Integration

Researcher(s)

  • Magnus Culley, Computer Science, University of Delaware

Faculty Mentor(s)

  • John Aromando, Computer & Information Systems, University of Delaware

Abstract

Intro to Computer Science courses have an unfavorable student to teacher ratio, causing some students to not get the help they need. It might be possible to supplement this shortcoming by integrating AI to provide some personalized guidance during coursework. The goal is to create an effective system that benefits both the students and teachers by making the troubleshooting process of coding less frustrating for novices. Previous research regarding AI in the education field has both cautioned and encouraged its use, but was lacking when discussing differences in models. We tested various open source lightweight models in a test suite that resembled coursework of an intro CS course which simulated common errors that students made. To score well, the model had to guide the student, not correct them, reference the provided textbook, and be consistent. At this time, research has not concluded but it will determine if an integrated AI system could be viable for future intro CS courses.