In an effort to bring a more human dimension to the online-education experience, Department of Electrical Engineering and Computer Science Associate Professor Rob Miller has developed a new computer system that will help provide students with feedback on their homework assignments and create more interaction between students, teachers and alumni.
Called Caesar, the system was developed by Miller, a principal investigator at the MIT Computer Science and Artificial Intelligence Lab (CSAIL), and two of his graduate students, Mason Tang and Elena Tatarchenko, to address the challenge of how to facilitate instructor feedback to the hundreds of students taking his introductory computer science course each semester. Many of the students taking the course, “Elements of Software Construction” (MIT course 6.005), are new to the subject matter, and Miller thought they would benefit from more hands-on guidance. In particular, he wanted to find a way to critique the thousands of lines of code that his students write as part of each of their homework assignments.
Miller’s own research focuses on human-computer interaction and crowd computing — the process of distributing complex tasks to a group of people over the Internet, allowing them to tackle smaller, more specific portions of the overall task. Miller used this approach in developing Caesar, devising a system that allocates small chunks of code written by his students to a diverse group of computer science students, teaching assistants and 6.005 alumni, who then review the work. Several reviewers are assigned to look at each student’s work, so every student receives feedback from a variety of sources.
“What we are trying to do is to learn how to use a crowd of people with mixed expertise in an intelligent way; one that helps students and ‘the crowd’ expand their knowledge and improve on their expertise,” Miller says.
The Caesar system has three components: the code selector, the task router and the reviewing interface. Once students have turned in an assignment, the code selector divides their work into chunks and prioritizes the chunks that need review, based on features of the code that suggest it will need attention. The task router then assigns these chunks to a diverse group of reviewers.
By distributing the evaluation process across a large pool of reviewers, Miller hopes to provide students with useful feedback that will improve their work. At the same time, he hopes that this type of crowd-sourced code reviewing can serve as a new learning platform for students at MIT and elsewhere.
The reviewing process itself takes no more than three days, a much shorter timeframe than traditional methods for giving feedback on student assignments. The quick speed with which the evaluation is completed allows students to receive feedback before they tackle their next assignment.
The code-reviewing process also teaches students a skill that could serve them well down the road, as software companies need developers who can track down bugs and other glitches in code.
Like Facebook and other social networks, Caesar provides opportunities for other kinds of interaction among its users. Reviewers can agree or disagree with fellow reviewers’ comments via an “upvote” or “downvote,” a process similar to the “like” feature on Facebook, and can also leave comments for both students and other reviewers.
Miller hopes that by linking students, alumni and TAs with different backgrounds and programming experience, he can increase learning opportunities for all parties. “Every time you interact with a new person, it’s an opportunity to learn, whether it’s a student demonstrating a new technique to a TA, or an alum providing a student with a valuable piece of industry advice,” Miller says.
Looking to the future, Miller believes that crowd-sourcing tools such as Caesar will become increasingly important to the success of online education. “Such systems are able to draw on a diverse and multitalented pool of individuals, and could potentially make the online classroom a more vibrant, interactive place,” Miller says.
Caesar is already being adopted in other MIT programming courses, and plans are in the works to use it in edX, the online-learning initiative founded by Harvard University and MIT. Miller believes that the system could also be adapted beyond academia to industry and other fields.
“Ultimately, I believe that crowdsourcing is going to develop hand-in-hand with automatic mechanisms for making online education work, because there are so many aspects of learning that require a human touch,” Miller says.
Called Caesar, the system was developed by Miller, a principal investigator at the MIT Computer Science and Artificial Intelligence Lab (CSAIL), and two of his graduate students, Mason Tang and Elena Tatarchenko, to address the challenge of how to facilitate instructor feedback to the hundreds of students taking his introductory computer science course each semester. Many of the students taking the course, “Elements of Software Construction” (MIT course 6.005), are new to the subject matter, and Miller thought they would benefit from more hands-on guidance. In particular, he wanted to find a way to critique the thousands of lines of code that his students write as part of each of their homework assignments.
Miller’s own research focuses on human-computer interaction and crowd computing — the process of distributing complex tasks to a group of people over the Internet, allowing them to tackle smaller, more specific portions of the overall task. Miller used this approach in developing Caesar, devising a system that allocates small chunks of code written by his students to a diverse group of computer science students, teaching assistants and 6.005 alumni, who then review the work. Several reviewers are assigned to look at each student’s work, so every student receives feedback from a variety of sources.
“What we are trying to do is to learn how to use a crowd of people with mixed expertise in an intelligent way; one that helps students and ‘the crowd’ expand their knowledge and improve on their expertise,” Miller says.
The Caesar system has three components: the code selector, the task router and the reviewing interface. Once students have turned in an assignment, the code selector divides their work into chunks and prioritizes the chunks that need review, based on features of the code that suggest it will need attention. The task router then assigns these chunks to a diverse group of reviewers.
By distributing the evaluation process across a large pool of reviewers, Miller hopes to provide students with useful feedback that will improve their work. At the same time, he hopes that this type of crowd-sourced code reviewing can serve as a new learning platform for students at MIT and elsewhere.
The reviewing process itself takes no more than three days, a much shorter timeframe than traditional methods for giving feedback on student assignments. The quick speed with which the evaluation is completed allows students to receive feedback before they tackle their next assignment.
The code-reviewing process also teaches students a skill that could serve them well down the road, as software companies need developers who can track down bugs and other glitches in code.
Like Facebook and other social networks, Caesar provides opportunities for other kinds of interaction among its users. Reviewers can agree or disagree with fellow reviewers’ comments via an “upvote” or “downvote,” a process similar to the “like” feature on Facebook, and can also leave comments for both students and other reviewers.
Miller hopes that by linking students, alumni and TAs with different backgrounds and programming experience, he can increase learning opportunities for all parties. “Every time you interact with a new person, it’s an opportunity to learn, whether it’s a student demonstrating a new technique to a TA, or an alum providing a student with a valuable piece of industry advice,” Miller says.
Looking to the future, Miller believes that crowd-sourcing tools such as Caesar will become increasingly important to the success of online education. “Such systems are able to draw on a diverse and multitalented pool of individuals, and could potentially make the online classroom a more vibrant, interactive place,” Miller says.
Caesar is already being adopted in other MIT programming courses, and plans are in the works to use it in edX, the online-learning initiative founded by Harvard University and MIT. Miller believes that the system could also be adapted beyond academia to industry and other fields.
“Ultimately, I believe that crowdsourcing is going to develop hand-in-hand with automatic mechanisms for making online education work, because there are so many aspects of learning that require a human touch,” Miller says.
