Beck's ITS Best Paper Award
Prof. Neil Heffernan reports:
I would like to congratulate our colleague Joe Beck, who won the Best Paper Award at the Intelligent Tutoring Systems Conference in Montreal this summer. There were seven papers nominated for best paper out of a field of 70 accepted full papers. So getting nominated was an accomplishment in and of itself . None of my papers were nominated, but Joe had two!
I have listed both paper's abstracts below and given links to the full version as well. They are both impressive new mathematical/algorithmic techniques to improve educational data mining. He also showed how he could use these new techniques to produce publishable new discoveries.
It was a good summer month for Joe, as the week before, he was the Conference Co-Chair for the First International Educational Data Mining Conference. The Educational Data Mining working group also announced a new Journal on Educational Data Mining. Joe is the recognized leader of this new and growing field, having Chaired or Co-Chaired workshops on the topic for years (for a list of past events see http://www.educationaldatamining.org/events.html ), so he was a natural pick for one of the associate editors for the new journal.
All in all, a remarkable accomplishment!
References:
Beck, J. E., Chang, K.-m., Mostow, J., & Corbett, A. (2008) Does help help? Introducing the Bayesian Evaluation and Assessment methodology. ITS2008: 9th International Conference on Intelligent Tutoring Systems, Montreal, pp. 383-394. <http://www.cs.cmu.edu/%7Elisten/pdfs/beck%20Does%20help%20help.pdf>. Best paper award.
Abstract: Most ITS have a means of providing assistance to the student, either on student request or when the tutor determines it would be effective. Presumably, such assistance is included by the ITS designers since they feel it benefits the students. However, whether-and how-help helps students has not been a well studied problem in the ITS community. In this paper we present three approaches for evaluating the efficacy of the Reading Tutor's help: creating experimental trials from data, learning decomposition, and Bayesian Evaluation and Assessment, an approach that uses dynamic Bayesian networks. We have found that experimental trials and learning decomposition both find a negative benefit for help--that is, help hurts! However, the Bayesian Evaluation and Assessment framework finds that help both promotes student long-term learning and provides additional scaffolding on the current problem. We discuss why these approaches give divergent results, and suggest that the Bayesian Evaluation and Assessment framework is the strongest of the three. In addition to introducing Bayesian Evaluation and Assessment, a method for simultaneously assessing students and evaluating tutorial interventions, this paper describes how help can both scaffold the current problem attempt as well as teach the student knowledge that will transfer to later problems.
____
Beck, J. E., & Mostow, J. (2008) How who should practice: Using learning decomposition to evaluate the efficacy of different types of practice for different types of students. ITS2008: 9th International Conference on Intelligent Tutoring Systems, Montreal, pp. 353-362. <http://www.cs.cmu.edu/%7Elisten/pdfs/beck%20its%20learning%20decomp.pdf>. Nominated for best paper award.
Abstract: A basic question of instruction is how much students will actually learn from it. This paper presents an approach called learning decomposition, which determines the relative efficacy of different types of learning opportunities. This approach is a generalization of learning curve analysis, and uses non-linear regression to determine how to weight different types of practice opportunities relative to each other. We analyze 346 students reading 6.9 million words and show that different types of practice differ reliably in how efficiently students acquire the skill of reading words quickly and accurately. Specifically, massed practice is generally not effective for helping students learn words, and rereading the same stories is not as effective as reading a variety of stories. However, we were able to analyze data for individual student's learning and use bottom-up processing to detect small subgroups of students who did benefit from rereading (11 students) and from massed practice (5 students). The existence of these has two implications: 1) one size fits all instruction is adequate for perhaps 95% of the student population using computer tutors, but as a community we can do better; and 2) the ITS community is well poised to study what type of instruction is optimal for the individual.
Maintained by webmaster@cs.wpi.eduLast modified: August 11, 2008 20:40:18