This is the first in a series of articles that will tackle common objections to and arguments against using massive open online courses for training.
Have you seen how people use public libraries these days?
They pick up books, skim through them, and then put them back on the shelf without reading them. Sometimes they even check out books and don’t read them. Sometimes they just photocopy a few pages or a chapter, or look up a reference. Sometimes they don’t use the books at all, but instead participate in a discussion group or even watch a film! In fact, a 2012 study found that only a bare majority of people who go to a public library actually borrow printed books.
Since people who go to libraries aren’t all borrowing books—and even when they are borrowing books they probably aren’t all reading them—public libraries are failures.
By now, I expect you are rolling your eyes. And for very good reason—the assertion that public libraries are failures is ridiculous. But these are the very same arguments often used to suggest that MOOCs are failures. The fact that only between 5 and 10% of people who sign up for MOOCs actually complete them has led some to conclude that MOOCs are not engaging, that people don’t like them, and that they are not effective forms of instruction. However, the research that has been done on MOOCs shows that this argument is not valid, because completion rates are not useful measures of what really happens in a MOOC.
Let’s take a closer look at what the completion rates of MOOCs really mean.
The completion rates usually reported for MOOCs are straightforward: divide the number of people who complete the MOOC (e.g., 5,000) by the number who sign up (e.g., 100,000). Regardless of the fact that 5,000 people is still a huge number in the context of most courses, 5% just doesn’t look very good.
The problem with this calculation is that it doesn’t take into account learner intention. Those 100,000 people who enroll may never intend to finish in the first place. Many of them don’t even show up for the first class. Like people who sign up for a free library card, the people who enroll in MOOCs do so with a variety of intentions in mind—maybe they want to see what it’s like (taking a book off of the shelf and skimming through it), maybe they are only interested in one or two modules (photocopying a chapter or two), or maybe they just want to be reminded when the MOOC will start and plan to make their decision at that time (putting their name on the waiting list for a popular book).
The more informative calculation isn’t based on the number of people who enroll, but the number who enroll intending to complete the course. For a study released last month, Harvard’s Justin Reich examined exactly that, and the results look much different.
Reich surveyed students from nine HarvardX courses about their intentions to complete their MOOCs. What they found was that, depending on the course, only 40 to 78% (average: 58%) of respondents intended to complete the course at all, and of those, 9 to 36% (average: 22%) actually did. Another interesting finding was that among those who didn’t intend to complete the course, between 6% and 10% went on to earn a certificate, showing that learners’ intentions can change—in both directions. Finally, the study showed that attrition happens early in the course: if learners are engaged early, they are more likely to see it through to the end.
Admittedly, these numbers are not high, but there is still more to the story. Reich’s study looked at a range of academic courses, most of which learners probably weren’t taking to improve their job skills (for example, courses on ancient Greek heroes and the letters of Paul).
So let’s look more closely at courses that learners are taking for professional development. One way to measure this is by exploring the completion rate for people who sign up for verified certificates, which cost around $50 each and are starting to be considered as valid credentials in the workplace. In an interview at Disrupt SF 2014, Coursera co-founder Daphne Koller said that among learners who embark on the verified certificate track, the completion rate is between 80 and 90%. That looks a heck of a lot better than 5 to 10%!
What does all of this mean?
It means that completion rates aren’t valid measurements of MOOC success. We can’t measure the success of a public library based only on the number of people who go into the library and then, regardless of their intentions, leave with a book which they then read cover to cover. That would be silly. It is equally silly to judge the success of a MOOC based on the overall percentage of people who complete the course. For corporate trainers, the more relevant metric is the 80-to-90% completion rate of those who enroll in the verified certificate track.
Different learners have different needs, and that’s something we should embrace, not fight against. Some learners need a whole course, some need a module or two, and some just want to skim. As I’ve written before on this blog, MOOCs can accommodate different learners’ needs in ways that instructor-led training and traditional elearning simply can’t.
Public libraries aren’t failures, and neither is online learning. They are both ways of providing personalized learning and engagement opportunities for a diverse group of people. There may not yet be an agreed-upon metric for measuring MOOC success, but until there is one, let us not condemn them on the basis of something as meaningless as low completion rates.
Copyright Bryant Nielson. All Rights Reserved.
Comments are closed.