Yogesh Kumari defends her M.Tech Thesis

Abstract: Providing feedback to all the students individually, for a programming assignment is a very tedious and error-prone task and can take a lot of instructor's
or TA's time. This challenge of giving individual feedback manually, becomes more prominent with increase in "Massive Open Online courses (MOOCs)", where a typical classroom has around thousands of students. Thus, there is a need of an automatic mechanism which can provide students a better feedback. Also, students often complain about the grading schemes used for grading their assignments. This grading scheme is generally based only on the number of failing test cases, which students don't nd fair enough to judge the correctness of their submissions. Because of redundancy in the test cases used for grading, student's submission fail for more than one similar test cases. It is possible that single fault leads to many failing tests and thus student's find this
grading based on failing test cases biased because of redundant tests and each test given equal weightage. There are two types of techniques which are generally used for providing feedback to students, one is peer-feedback and the other is test case based feedback.

We propose a test case based technique to provide feedback, as peer-feedback would require involvement of other students and TAs and thus can not be fully automated. We also propose a grading function which will grade the student submissions on the scale of correctness. Feedback to each student submission consists of a set of failing test cases which are redundant and expected to uncover one fault each and also a set of faulty lines in order of their ranks of contributing to the fault in the submission.Each test case in the feedback covers one failing PathClass for a submission. These techniques of automatic feedback and grading will help in saving time of TAs and instructor, provide consistent feedback to all the students and the students will be graded on test cases with different probabilities of failing. This probability of failing is calculated based on class performance for the test cases. We have experimented with various problem defnitions from "Data Structures and Algorithms" course offered in IIIT Delhi. For the grading part we graded student submission based on our grading function and asked them to fix their faults and resubmit the assignments. 40% of the students with grades more than average could fix their submissions, thus validating that their submissions were indeed closer to being correct and small changes help them achieve full correctness. For feedback part, on an average the number of failed test cases given were reduced from 100% to 24.27% thus not giving the redundant test cases to students and giving only test cases which target a bug in the submission.