Experimental Condition Assigner


GitHub logo Github: UberHowley/Self-Efficacy-Condition-Assigner

Description

For a field experiment in a classroom I needed to assign participants to one of three conditions, and assign them to one of three types of pairs (homogenousx2, heterogeneous) based on their self-reported self-efficacy. Students arrived at class, completed a questionnaire, and while they took their pretest, my condition assigner computed each student's self-efficacy, determined the median (to create "high" and "low" self-efficacy categorizations), created pairs, and evenly assigned types of pairs to a condition (targeting high/low/neither self-efficacy). I wrote the condition assigner in Java.

Much of my past research looks at how we can use dialogue tutors (conversational agents) to facilitate collaborative learning. Sometimes, this is done through varying the automated tutor's social behavior, other times we use the tutor to teach specific concepts (like here), but in this case I wanted our intelligent tutor to target a specific student in a group for opportunities to participate. I wanted to see how asking questions of a student, according to their self-efficacy (the belief that "I can succeed") would impact their participation in the group.

In order to answer this question, I would need to (1) pseudo-randomly assign participants to pairs according to their self-efficacy (high-high, low-low, high-low), (2) pseudo-randomly assign pairs to a targeting condition (target high, target low, neither), and (3) output these assignments in a format that our Bazaar dialogue tutor can interpet. The pseudo-random assignments were necessary, because our sample size was not incredibly large, so in order to ensure balanced assignments, those assignments would have to be partially supervised. All of this had to be done after students completed their online prequestionnaire, but before they completed their pretest.

Using Java in Eclipse, I wrote software that would first read-in the Comma Separated Values (CSV) file of student responses to the prequestionnaire and calculate each student's self-efficacy by averaging ~3 self-report responses. Once I had the class' self-efficacy scores, the software would perform a median split and assign each student to be either high SE (above the median) or low SE (below median). From this point, the software could then assign participants to heterogeneous/homogenous pairs. User inputs were used to specify how many of each pairing to make, as we performed this experiment in several classes and the user input allowed the software to more evenly assign pairs across classes. Since pairs were assigned, each pair could be assigned to one of three conditions. Pair assignment was balanced across the three targeting conditions based on pair composition so that there would be an even number of H-H, L-L, and H-L pairs assigned to each condition. Pair assignments and condition assignments were then written to a CSV file to be interpreted by the Bazaar Dialogue Tutor software, and a summary output file was also created to help in the case of debugging and tracking pseudo-random assignments.

Documentation

GitHub logo Github: UberHowley/Self-Efficacy-Condition-Assigner

This software was used in the implementation of this experiment:
Adobe PDF link Howley, I., Adamson, D., Dyke, G., Mayfield, E., Beuth, J., & Rosť, C. P. (2012, January). Group composition and intelligent dialogue tutors for impacting studentsí academic self-efficacy. In Intelligent Tutoring Systems (pp. 551-556). Springer Berlin Heidelberg.

Skills

software engineering, experimental design
Experimental Condition Assigner