Philip Guo (Phil Guo, Philip J. Guo, Philip Jia Guo, pgbovine)

Demographic Differences in How Students Navigate Through MOOCs

edX research paper

Millions of students from around 200 countries are now learning on MOOC platforms such as edX. Do student demographics such as age and country of origin affect how they navigate through these courses?

To explore this question, Katharina Reinecke and I performed an empirical study and published the following paper at the new ACM Learning at Scale conference:

Philip J. Guo and Katharina Reinecke. Demographic Differences in How Students Navigate Through MOOCs. ACM Conference on Learning at Scale, March 2014.
[BibTeX citation]

This blog post summarizes our paper.

Methodology

We extracted demographic information and navigation patterns of over 110,000 students taking four edX courses in Fall 2012, 10% of whom earned certificates of completion.

We focused on demographic variables such as age, education level, and country of origin. One interesting variable is the student-teacher ratio of the country of origin. This is a widely-used indicator of educational quality and level of individualized student attention. (However, it is also correlated with economic indicators such as per-capita GDP and median household income, so any observed effects might not be due simply to pedagogical quality.)

One interesting form of navigation is a backjump, which occurs when a student navigates backwards from a resource to another one that was released earlier in the term (e.g., jumping from Lecture 6 to Lecture 3).

Main Findings

EdX students come from diverse backgrounds and navigate through the courseware in many different ways. In particular:

  1. Certificate-earning students are, on average, more highly educated than the general student population.

  2. Certificate-earning students view only 78% of learning sequences (e.g., lectures or assessments). This means that they skip 22% of the course content, on average, and yet still earn a certificate.

  3. Certificate earners engage in lots of non-linear navigation behavior, often jumping backward to revisit earlier lectures.

  4. Backjumps from assessments to lectures are more common than lecture-to-lecture backjumps. One possible explanation is that students are opportunistically working backward to learn old content in order to pass the assessments.

  5. Older students participate more actively in discussion forums.

  6. Older students visit and repeat more learning sequences than younger students.

  7. Students from countries with lower student-teacher ratios (e.g., the U.S. and Eastern European countries) visit and repeat more learning sequences than those from countries with higher student-teacher ratios (e.g., India, Kenya). See the graph below for details.

  8. However, the effect of age is stronger than that of country: Older students from countries with higher student-teacher ratios behave more like their similarly-aged counterparts in countries with lower student-teacher ratios.

This graph shows the mean number of backjumps per visited sequence for all certificate-earning students in the 30 countries with the most edX students. There is a negative correlation between student-teacher ratio and backjumps. One possible explanation is that students from countries with a higher student-teacher ratio are more accustomed to a linear, teacher-led instructional style and hence do not make as many backjumps when taking MOOCs.

Design Implications

Right now, MOOC user interfaces try to be “one size fits all,” but we have found that students navigate quite differently based on both demographics and enrollment intent. Here are some interface design ideas that could help MOOCs serve a broader audience:

  • To encourage greater engagement with learning sequences, MOOCs should augment certificates (which indicate only pass/fail) with richer metrics such as discussion forum participation, peer ratings, or time-on-task. When properly designed, these new reward mechanisms could motivate students to cover more sequences and not just do the minimum required to get a certificate.

  • MOOC user interface designers could experiment with adaptive interfaces that adjust based on, say, a student's age or country of origin. Prior educational technology research has shown that certain types of students prefer a more linear structure and explicitly stated learning goals while others prefer more freedom in navigation.

  • Social dashboards (e.g., “other learners usually spent 45 minutes on this part”) could motivate students and keep them on track as they progress through a several-month-long MOOC. However, these systems must be carefully designed so as not to alienate students from certain demographics.

One ideal to strive for is to present each student with a personalized user interface catered to their individual preferences and learning needs.

Read the full paper for details.

Created: 2014-03-12
Last modified: 2014-04-06
Related pages tagged as research: