Q & A with Anant Agarwal

Two years ago this month, Harvard and MIT launched edX, a non-profit virtual learning platform to educate people across the globe and shed light on how students learn. To date, the joint venture has attracted dozens of partners, from top-tier research universities to international organizations, as well as more than two million users. Yet questions about the efficacy of online courseware, edX’s rapid growth, and the platform’s financial sustainability remain.

Anant Agarwal, an MIT computer science professor who has served as CEO of edX since its establishment, sat down with The Crimson to recount the challenges of creating courses for an online learning environment, discuss the non-profit’s business model, and speculate about what the future might hold for edX. The interview has been edited for clarity and concision.

The Harvard Crimson: At the time of edX’s launch, there was a lot of speculation into what the platform might look like and what it might do to transform higher education. What was your vision for edX at the beginning?

Anant Agarwal: So when we started edX,  our vision for edX had three legs to the stool, so to speak. The first part of our mission was that we wanted to increase access to learners all over the world, millions of students everywhere,  and with our very first course, we reached 155,000 students in 162 countries, and that gave us initial confidence that maybe this would work. Today, two years out, we are at 2.3 million students in 196 countries around the world, and so in terms of the growth and number of students taking courses, it’s certainly exceeded our expectations.

The second leg of the stool for our mission is improving education on our campuses. In other words, how do you improve the quality of education on campuses and online? And our goal was to look at new ways of innovating and learning around multiple dimensions. So for example, one aspect of it was [the question of] how do you create very rich interactives, how do you create virtual laboratories so you get much a better experience for a student taking a course online. Similarly, we wanted to be able to allow students to do things like be able to get instant feedback. When you offer assessments and exercises, students can get feedback, and that is a really good thing. So we want to use that technology of instant feedback to improve the learning experience whether you’re on campus or somewhere 10,000 miles away. For learners even on campus, normally you submit homework, and the homework might come back to you graded hopefully later, sometimes never, and that decreases the motivation for completing the homework, figuring out where you’re going wrong, and trying to do better, but if you can do instant feedback, you can do a lot better. Students get a lot more motivated, and learners are telling us that on campus when they are using the edX platform, they are working much harder on homework because students have a natural competitive streak in them. They want to get things right, and when they know something’s wrong, they keep working at it to get things right. In fact, when I taught a blended class at MIT in our very first course, many students would come to me jokingly and say, “Professor Agarwal, you have me working much harder on homework than I’ve ever worked before.” I said, “what do you mean?” They said, “We can keep trying to get something right so we just keep working hard. I’m just spending way too much time on this platform.” I would joke to them, “well is that a good thing or a bad thing?” So I think creating instant feedback and instant grading and simulation-based laboratories helps more students on campus and worldwide, and combining the best of online and in-person [education], we want to improve the campus experience as well.

By Helen Bae

The third leg of the stool was research. We wanted to do research in learning to see how students learn and thereby improve the whole learning experience itself. So when we started, those were the three big aspects of our mission—access, improving education quality on campus and beyond, and research. Two years into it, I feel we’ve made a decent dent into all three of these.

THC: That being said, in the first two years, what were the biggest challenges that you saw in getting edX off the ground?

AA: Oh my goodness. I’m not even sure where to start. There were so many challenges. Frankly, when MIT and Harvard came together to do this thing there was a huge speculative experimental nature to it. We knew where we wanted to go but had no idea how to get there, so it was like jumping off a cliff saying, “we want to get down,” but then looking around and saying, “well, who’s got the parachutes?” But we knew where we wanted to go. The beauty of doing this was that by committing to doing it, you knew that you had to succeed. You had to bring your parachutes along the way, you had to experiment and build things along the way. Failure was not an option, you had to succeed. So we built a number of things as we went along.

I’ll give you a number of examples. One challenge was that for many of the humanities courses—we want to strongly support the humanities—[we needed platforms for] open response and being able to assess students in things beyond multiple choice. You want them to be able to write essays, descriptive answers and so on. When we started edX, we knew that was a challenge but had no idea how to do it, but a year into it, our team built a number of technologies to be able to support that. They built peer-grading. They built A.I. assessment and a number of techniques that they are experimenting with to grade some of these things, and we’ve already gone from version one to version two, so version two of “Open Response Assessment,” called “ORA2” came out three months ago, so we’ve been building and refining a lot of these things. So building an open response was a challenge, so we continued to refine and improve that.

I think a second one was [the question of] how do you do things like answer questions. We knew the moment that we saw 150,000 students taking a course, we had this amazing “Oh-my-god moment.” So the first response we had, which I discussed with our team, our instructors, and our TAs was “seven by twenty-four.” We were going to sit down and answer questions on forums seven by twenty-four, so we thought that was a really big challenge. There, we had a huge epiphany one night. I was up at 2 o’clock at night, answering questions on a discussion forum. I type pretty slowly, and when I was typing an answer, I noticed that a student from Pakistan had jumped in with an answer. It wasn’t quite the right answer, so I tried to correct the answer, and before I could type the right answer, someone else had jumped in. I sat back fascinated, and students kept discussing it and by 4 a.m., they had figured out the right answer. Then I could go in and say that’s the right answer.

THC: So that’s sort of a “Wikipedia” model for answering questions.

AA: Exactly. It’s almost like a Wikipedia-style approach to answering questions, where students discuss themselves. The beauty of this, by the way, unlike Wikipedia, is that learning is in the journey. Oftentimes, when someone shows you the answer, that may not be as good as discovering the answer for yourself. If you’ve read ‘The Zen and the Art of Motorcycle Maintenance,’ which has been a huge influence in my life, it’s not where you get to, it’s how you get there, so as students work together to get the answer, many have told us that they are actually learning by teaching. So that was a huge learning experience for us. It was a huge challenge, but resulted in the way students answer each others’ questions and learn in the process.

When we first started edX, authoring courses was a real challenge. We had to use XML, which is a programming approach to writing problems and questions, and you had to be a computer scientist in order to be able to author a course. Frankly, at that time, we had somewhat of the naive view that we could help author courses and would build a huge support team to help people author courses. So we would have people that could work with professors and help them author courses where the computer science side of authoring courses would be done by us. We very quickly found that it was very expensive, it would not scale, and that professors and schools wanted much more control over how they did things, so we rapidly, rapidly created a huge initiative called “Studio.” We built an authoring platform as part of the whole system. It’s like GarageBand for courses. It makes it much easier for mere mortals to author courses, and frankly that has been a game-changer. Whether you’re teaching a humanities course or a computer science course or an engineering course, it’s become significantly easier to author courses. Frankly, I think we have a long way to go, but we’ve come a long way. I think that was a huge challenge, and I think we’ve made a dent in authoring courses as well.

THC: On edX’s website, there’s a list of guiding principles, the first of which is, “not for profit,” and the last of which is “financially sustainable.” These two ideas are not necessarily at odds with each other, but are edX’s current sources of revenue sufficient to sustain it long-term?

AA: I’m glad you said that being a non-profit does not mean non-revenue. It does not mean non-sustainable. The difference between a startup and a non-profit startup is that get their investments from venture capitalists. Then, they expect a return on investment, so you have to make profits to return on investments for your investors. At edX, our investment comes from Harvard and MIT. Many of our university partners have contributed substantially as well, and philanthropic foundations contribute. At the same time, we want to be sustainable, so we want to be able to generate revenue so that we can not only sustain ourselves, but more importantly, sustain our university partners. Our university partners are spending anywhere from $50,000 to $250,000 or more to create courses and so right now, they are spending that money to put those courses on edX and also to use those courses on their own campuses. That said, we need to find revenues to sustain their efforts as well so the ecosystem can continue. There’s only so long that courses will keep putting courses on edX without seeing some way of sustaining themselves.

THC: I know initially, there was a $30 million investment from Harvard as well as a matching $30 million from MIT, so a total of about $60 million. How much of that initial investment is remaining two years in?

AA: I want to point out that it’s MIT and Harvard, but many other universities have contributed and many of the foundations have also contributed. I would say that we still have significant portions of all the investments that are still available to us and so, we have a significant amount of runway for several years. We are pretty comfortable that we have the time to reach sustainability, and so we are quite happy with that.

We started out, frankly, with not that many ideas about how to generate revenue that would sustain us, so we have been experimenting for a couple of months. So one model that we used at the beginning was that we would issue certificates for a fee. Right from day one, we said we would charge for certificates and we would charge the order of magnitude of the cost of a textbook for these certificates. So the course would be free, but we would charge for certificates. That was one approach, so we launched verified certificates last year, where a student can take a course for free—they can even get a free honor code certificate. However they would have to pay a fee that could range from $50 to $250 for a verified certificate where we use webcams to compare the face to the photo ID to make sure they are who they say they are. We charge a small fee to do that. So it turns out that that can be quite a healthy source of revenue. However, our experiments have shown that that alone is not going to be sufficient. So, we expect that it’s going to be like a Swiss-army knife, where we need more than a single blade in our arsenal, but that is one blade.

We are experimenting with a number of other approaches. A second source of revenue is called course licensing. As you know, we have open-sourced our platform, so a number of countries have adopted it. The Middle East has adopted our platform and created edRawq. The Queen Rania Foundation. China has launched a platform called XuetangX. France has launched a platform called “FUN”—France Université Numerique. So many of them are coming to us, [asking to license courses from edX.] So a second line of business is that they pay us a fee to license courses, so we are generating a revenue for our partners and for ourselves from licensing courses.

A third approach is executive education or professional education where we create a line of business where you charge for certain types of courses and there’s no free version of certain types of courses and professional ed is one example of a class of courses. We did a pilot with a big data course in partnership with MIT and that partnership went extremely well. We are looking to create executive education as well.

These are three examples of businesses and we believe that these three, perhaps with one or two others that you will discover along the way, give us confidence that we will be sustainable along with our partners.

THC: Is there a quantitative breakdown of edX’s finances in terms of how much revenue has been brought in from things like the verified certificates and course licensing?

AA: Right now, we haven’t provided a detailed breakdown of where revenue is coming from how much the revenue is. It’s still the early days. It’s still very experimental, but I think as we solidify this and get more comfortable with the number and put our future plans together, I think that’s when we can feel a lot more comfortable providing these numbers at that time.

THC: edX saw a major change of leadership in the hiring of COO Wendy Cebula, which was actually on the same day as the hiring of Richard Levin as the [CEO] of Coursera [edX’s major competitor]. What do these two hires indicate about the directions in which edX and Coursera are going in?

AA: At edX, we expanded our leadership team. We were looking for a really strong, business-oriented COO and president, [one] who worked with a large company, someone who understands how to execute on a business, which is why we had an international search and we identified Wendy Cebula and are absolutely delighted that she came on board edX. Previously, she was on the leadership team that was running a billion dollar company [Vistaprint], so we brought her in because we really believe that financial sustainability is a critical, critical aspect of what we have to do. There are three legs of our mission, of course, but at the same time, you want to be sustainable, and bringing in Wendy Cebula was a clear and strong indication that we view sustainability as extremely important, not just for edX, but for our partners. In fact, edX views the sustainability of our partners as one of our corporate goal, so we will use that as a corporate goal to say, “Look, all of us have to be sustainable, not just edX.”

And second is that we want to continue to execute very effectively internally and oftentimes, non-profits have been dinged for less than stellar execution in comparison to a for-profit company, and from day one, we have run edX as a startup company, but a non-profit startup. We want to continue in that vein. Like any startup company, we want to be super innovative. We want to be hardworking. We want to be extremely effective and efficient. We want to continue to be that and so, we brought in Wendy to really continue on a path of extremely effective execution.

THC: Do you believe that virtual education could ever replace face-to-face interactions, which have been the norm for teaching since the dawn of humanity essentially? If so, would that be a worthwhile development?

AA: I really believe that online learning, and virtual education is another tool in our arsenal. What tools have we given teachers for the past 500 years? We’ve given teachers chalk, blackboards, textbooks, and dare I say, the PowerPoint. We haven’t given teachers a whole lot of tools. I think clicker technology is a tool that some people have begun to use, but there’s not a whole lot of tools that technology has given teachers. We believe that online learning is another tool in our educational arsenal. I don’t believe that online learning will replace in-person learning. We believe it will augment it. So for the students who do not have access to school, who do not have access to great teachers, it can suffice for giving them an education. But certainly for students who already have access to teachers and access to learning, online learning can improve the quality of their education by creating models in which we combine the best of online learning  and in-person. That is why I like to say that online learning is like a rising tide that will lift all boats—it’ll increase access to those who don’t have access, and will improve learning for those who do have access. It really could help everybody. I really don’t see it as a replacement. I see it as creating another extremely powerful tool in our educational arsenal.

THC: The finding of a set of working papers this spring showed that relatively few people who registered for a HarvardX or MITX a course actually viewed all the chapters—much less received a certificate of completion. In a physical classroom or a lecture-hall setting, seeing these high attrition rates would likely be somewhat shocking to a teacher. Why is edX not held to the same standards that a physical classroom would be?

AA: First of all, I think edX should be held to the same standards as a physical classroom. However, the metric—the completion rate—is not comparing apples to apples. I would like to answer the question in two ways. First, let’s talk about the metric. The metric says that if you have a 100,000 students who registered, all they did was express interest in the course. All they had to do was click on something and they showed up for the course and 6 percent passed the course. So in the denominator, you have students who simply clicked a link expressing interest in taking a course. If you take a course in a classroom, at Harvard and MIT, the pass rate might be upwards of 95 percent. We are comparing that, in the denominator, to students that have registered for the course, and are taking the course for credit.

I used to teach a course on parallel computing at MIT, and it was typical that on the first day of class, 80 students would show up, so people were shopping around for courses. But if you look at the number of students who pass the course in the end it would be on the order of 40 students, but you don’t say the pass rate is 50 percent, so how do we get 96 percent as a pass rate? 80 students showed up as auditors, but of those, only 45 would end up taking the course for credit, of which 42 or 43 students might pass the course. But we don’t count the pass rate as a fraction of the number of people who showed up on the first day of the course.

So I think we need to think about metrics, so a better metric is a pass rate as a fraction of students who signed up for a verified certificate and are paying a small fee. And if you look at the proportion of students who pass the course as a fraction of those who signed up for a verified certificate, it is on the order of 60 percent. So I think we should compared apples to apples.

That said, we do want to improve the completion rate for courses. We want more people to sign up for certificates and we want more people to pass the course once they sign up for a certificate. We’re looking at a number of ways to improve that as well.

—Staff writer Michael V. Rothberg can be reached at mrothberg@college.harvard.edu. Follow him on Twitter @mvrothberg.