Week 7 - Evaluation (OLDSMOOC)

It's week 7 in OLDSMOOC, and as we are windowing down we are tackling the topic of Evaluation. I will be switching tracks again, from the Blended Mobile Learning course (that I've been working on for a while), and going back to the idea of offering the course as a cMOOC. Going through OLDSMOOC I've gotten some good ideas about how to implement my own cMOOC.  I've been thinking a lot about the recommended paths that are available in some weeks (the short and the long path). This, in combination with badging, and deliverable, is making me think about the assessment aspect of the MOOC; but let's not get sidetracked, let's talk about evaluation.

In terms of evaluation decisions, what immediately comes to mind are these:
  • Should the content for this learning design be expanded, reduced, or remain the same? One of the tricky things about MOOCs is that you will always have critics, since there will be instances (many of them) where the MOOC does not hit the sweet spot for quite a few people.  How does one address these issues, and how does one stay explain the scope of the MOOC to the stakeholders?
  • What is the best way to make the materials open to anyone who may want to facilitate this course? I am thinking about this course as a way to also contribute back to the community, thus I want to make the OER as accessible as possible.
  • What strategies can we identify to maintain our learning design in the future? If the MOOC is going to run more than once, then, like open source software, there needs to be some way of maintaining the learning design, and updating it
  • Should the MOOC offer assessment instruments?  If so, what are meaningful assessment instruments and, when implement, will they have met their goal?
  • What is our engagement goal? Learning does not happen in a vacuum, so how do we encourage people to engage with one another without being overbearing? You can't force engagement and cooperation onto others.
  • How did facilitators work out? How was communication between facilitators and participants?  What worked and what did not?
  • Does one facilitator work? Or should we employ many more facilitators?
  • What is the "sweet spot" for MOOCs? 4 weeks? 6 weeks? 8 weeks? 12 week? If students are taking this for credit (like they did with CCK) then their academic semester is defined, and they need to "slog through" it, but what about the open learners?
  • What role should learner supplied resources take in the course if the course is offered for credit? In MobiMOOC 2011 there were participants that posted resources like there was no tomorrow (this was a good thing in my opinion), but what do other participants feel about this?
 As I was writing this, I decided to mingle the specific question in with the decisions to be made.

Your thoughts?

Comments

Apostolos, you have identified an interesting set of evaluation decisions and questions. As you noted, you have mixed decisions and questions together. This is OK, but I generally promote the idea of identifying decisions at one level and then having one or more questions
aligned with each decision to be made. Evaluation planning is difficult to be sure. When you are wearing your evaluator hat, you may find yourself in the position of having to persuade your collaborators of the value of
anticipating difficult decisions, asking certain questions, or addressing specific issues by engaging in evaluation. Unwilling or unable to confront the complexities involved in most evaluation contexts, some people may demand
direct and simple answers to complex questions. But direct and simple answers are extremely rare in the context of learning design, and “it depends” and
other conditional statements are part of even the best evaluations. A thorough evaluation plan will expose as many of these conditionals as possible up-front, but the trick is doing so without having the team decide to abandon evaluation altogether.

One of your decisions is: “Does one facilitator work? Or should we employ many more facilitators?” My
experience in online teaching is that you need at least two facilitators, and most MOOCs have several (as the OLDS MOOC does). The biggest issue is that one
person can have an adverse event (e.g., illness, death of a family member, hurricane, etc.) occur during the course, and be unable to keep up with the demands of the course. Here is a link to a MOOC focused on Learning Analytics and Knowledge that my friend, George Siemens, kicked off earlier this month: https://www.canvas.net/courses/learning-analytics-and-knowledge You’ll notice that he has six world class facilitators for eight week course.


Good luck with your project and your upcoming PhD studies.

Popular posts from this blog

Academic Facepalm (evaluation edition)

Discussion forums in MOOCs are counter-productive...well, sort of...

Latour: Third Source of Uncertainty - Objects have agency too!