#MassiveTeaching experiment falls on deaf ears?

Alright, #MassiveTeaching (or under its official name: "Teaching Goes Massive: New Skills Need") on Coursera is over, that's all he wrote (and then deleted, and someone else recovered). All joking aside, I decided to participate in the final assignment/test of the course which ultimately turned out to be a Level 1 evaluation. I've included the three questions in my previous blog post about this. What I neglected to include in my previous blog post was a fourth question which went something like this: Would you recommend this course to someone else?

The requirement to "pass" the assessment was to grade 4 submissions. I ended up "Grading" 14 submission because I was really surprised at the results.

Of these 14 graded assignments, 8 of them were positive (57%) and these respondents claimed that they would, or have already recommended the course to their colleagues. One person commented that this course would be good for professional development and it was completed by them in mere hours. Another person got their previous professors "hooked" on this course (really?) The reasons for recommending the course are that learners felt that their peers would learn a lot about MOOCs, especially topics of pedagogy, copyright and technology. From a delivery point of view, others commented that they liked Paul-Olivier Dehaye's (few) videos (which were done in common craft style). One person would tentatively recommend the course to peers if Dehaye included more content for newbies.

Next up, of the 14 assignments I graded, 2 participants (14%) were unsure if they would recommend the course to peers. One response seemed to indicate that the whole drama around the course was a really a detractor, but there was a question about the efficacy of the course as an xMOOC. The respondent seemed to indicate that they got a lot out of the forum conversations and not Dehaye's work. The other response said that they liked MOOCs because they didn't have to structure their own learning, and the minimalism of this course was really not the format for them.

Finally, 4 of the 14 responses were negative (28%), in other words they would not recommend the course to their peers. The most common issue was the drama surrounding the course, and Dehaye's conduct. The interesting thing is that one of the respondent only learned materials through the videos, like going through some sort of OCW, which sharply contrasts with the comments another person made, in the unsure category, that stated that they only really learned something from the interactions.

Now this to me is interesting. Granted this is only a small cross-section of the responses but I was amazed at how positive the responses seemed to be overall. some people in the forums claim that they have graded over 40 to see what other said in their responses and they reported similar results. According to some sleuthing in the forums, there were 107 total responses to the final assignment. According to others the responses (yes/no recommendation) seem to be neck to neck depending on how you categories the responses. For instance do you could "yes only if..." as a "yes" or something else? What about responses like "your mileage may vary?"


From the 14 that I saw, it seemed to me that most people seemed to be new to MOOCs and as such had either no previous hands-on experience with MOOCs (only what they read in the press), or had minimal experience. This means, to some extent, that people's outlook on this #massiveteaching MOOC was predicated on three things:

  1. When you enrolled for the MOOC: If you enrolled past week 2 and missed the drama of content being deleted, you may have never noticed anything wrong (except for some angry people in the forums).
  2. Previous experience with MOOCs: Your previous experience might determine if you've been satisfied or not satisfied with this MOOC. Whether you are just watching the videos or whether you are attempting to participate in the forums.
  3. Paying attention: I got the sense that some people weren't paying attention to the MOOC. They seemed to indicate that they learned things about materials that were never released to the learners, like the Week 3 materials that never cam our way. I don't know how some people "learned" about the aspects of MOOCs that were never discussed neither in Dehaye's videos, nor in the forums. I wonder if these cases were cases of certificate hunters (like pokemon, gotta catch all MOOC certificates :) )

 

At the end of the day it seems to me that Dehaye's experiment fell on deaf ears. If people weren't perturbed enough to not recommend these rogue tactics I am not sure what to make of learners. Are they sheep willing to follow anyone in a position of authority? Are they not paying attention (mental image of student facebooking during lecture)? Are they trying to make lemonade out of a lemon? I really don't know. And for Dehaye, was all this production and drama all for naught? Your thoughts?

 

Comments

Maybe: This was not a representative sample of all enrolled students. Many either quit or didn't want to give any more information. Those who completed the survey were the ones who wanted to see others' responses, (like you) and may have fabricated their own answers to confuse the data collectors, knowing Coursera and UZH would see the results.
Interestingly the 25 super course participants in the Google+ group from 06/27 to 07/10 seem to be having evolving ideas about this course too!

https://plus.googleapis.com/communities/107715582635536052413
Thank you for sharing :) I was wondering where these groups were since I don't recall seeing anything on Twitter about it, or on the course itself. It's interesting that these 25 have created their own niche (which doesn't seem affiliated with the course).
I agree. I don't think this sample is necessarily representative (even the 107 students who submitted their final evals). It's hard to get into the minds of those who submitted evaluations. I can only taken them at face value and assume that these are honest responses and not some way to mess with data collectors :)
If you want to pursue it further, I have a list of all 107 submission responses to the 4th question - one line max of each response (some went longer in the submission) - I could send it to you. (I was going to re-check a few longer ones but Coursera shut the site mere hurs after the "evaluation period" ended, we can't even see any added comments on our own submissions!)

By my count the yes and no responses were roughly even, and a number of the ones I count as no said "not unless the structure changes" or "yes but only if the structure changes." I count both as no.

By far the majority would like a good MOOC on how to make a MOOC.

Several said they know nothing at all about online classes so this was all new info to them and important for teachers to learn about. (Content way too short and too elementary for some is all new to others, of course.)

A few of the yes said the content was worthless but they liked the prof's style. (And a few said yes because any course by a famous university is good for the resume.)
On the left side they say their purpose is to continue the discussion outside the [Coursera ?] forums. On 07/03 if you click down arrow to expand Dehaye's post there are 13 comments following his request to them for a "joint statement" for him. That's where it appears their opinions are evolving from June 27, Also on Jly 3, Dehaye posts separately that he can't get into the FaceBook Group...with no responses!
Someone actually said that they would recommend it due to the University? wow!


You are right about the course shutting down. Even up until Monday the course archive was visible to me, but I just checked and the course archive has been removed from me being able to access it. Too bad! I guess they didn't want people to keep looking at the train wreck :)


I wish I had the energy and foresight to collect all responses (to all four questions). Even if I had, though, I suspect that I would also need a log of how many people were originally registered, how many people dropped (when they dropped) and a complete record of the forums to do some forensic pedagogical analysis ;-) Oh well. One MOOC "done" I suspect that we will be talking about this one for a while! :)
Three people specified "I would recommend this course because it is prepared by a famous university" or this free course from a recommended university is a golden opportunity" as the whole response to "would you recommend it." (That second one was submitted twice - in fact the entire submission was duplicated, two submissions with consecutive Coursera submission numbers word for word the same. Interesting?)

I have 45 of the complete responses (I have an obsessive habit of keeping copies of what I grade, never to look at them again), can send to anyone interested but it's not all 107 and far from all who were involved at some point.

But this course is dead, we are probably all on to other things now.
If anyone is still interested, Paul-Olivier is actively posting about it in the forum of "Unethical Decision Making in Organizations" (fall 2014) in a thread he started called "Keep on teaching on Coursera, once you see it as evil"
I added comments about this post and the previous ones in the series here: http://paulolivier.dehaye.org/posts/erosion-of-thick-legitimacy-by-coursera.html

Popular posts from this blog

Academic Facepalm (evaluation edition)

Discussion forums in MOOCs are counter-productive...well, sort of...

Latour: Third Source of Uncertainty - Objects have agency too!