Presenting Helpers in MOOCs

Description

Students in Massive Open Online Courses often do not receive the help they need from their course discussion boards. A social recommendation algorithm can select appropriate helpers for a student's question, but how do we present these helpers in such a way as to encourage help seeking? I answer this question by applying Expectancy Value Theory in this field experiment.

I looked at the effect of badges (expectation that there will be help), expertise about the potential helper (belief that the help will be accurate, good quality), and up & downvoting (a social cost of seeking help), and its impact on the number of helpers a student invites to his question thread in the forum. This resulted in a 2X2X2 experiment. Results showed a significant interaction between badges and up & downvoting. Knowing that their question will be voted on caused students to ask less helpers to their forum thread. However, if the potential helpers were shown with badges, this negative effect was mitigated. Future work involves aligning our experimental manipulations with Expectancy Value Theory, as well as exploring the reduction in negative effects of other types of voting (i.e., upvoting only, voting on the question only, voting on the comments only, etc.).

Up and down voting is often used in discussion forums as a way to reward and engage users for participating in the question and answer community. It can also be used to elevate higher quality responses or more popular questions in large discussion forums. However, this experiment has shown that some formulations of up and down voting may actually inhibit help seeking. Going forward, I plan to offer empirically-derived design recommendations for course designers implementing discussion forums in their online courses.

Documentation

Adobe PDF link Howley, I., Tomar, G., Yang, D., Ferschke, O., & Rosť, C. (2015). Alleviating the negative effect of up and downvoting on help seeking in MOOC discussion forums. In Proceedings of 17th International Conference on Artificial Intelligence in Education (AIED 2015).

Skills

experimental design, statistical analysis, ANOVA, MOOCs, field experiment

An interaction between Badges and Voting
The effect of topic match percentage on whether a Helper Was Selected or not
Experimental Manipulations for the Quick Helper Experiment