Quality Assurance in Online/Blended Learning

"7 Measures of Quality in Online Learning" by Giulia Forsythe is licensed under CC BY 2.0

“7 Measures of Quality in Online Learning” by Giulia Forsythe is licensed under CC BY 2.0

I’m sure many people are familiar with the old idiom of “putting the cart before the horse” and the idea that a conventional process or timeline has been ignored.  It’s definitely the first phrase that comes to my mind when I think about quality assurance in online and blended learning with regards to my personal experiences.

I’ve been “doing” online teaching since my first year as a tenure-track faculty member at my institution.  And I’ve now added blended teaching to that.  Both opportunities came about very informally and were more a product of my own interests rather than any institution-wide mandate or effort.  The conversation went something like this: “Here is this modality of teaching that better suits the needs of some of our students and gives us some unique flexibility in course delivery.  Go figure out how to make it work.”  I went.  I figured.  In those early days of online teaching for me, probably 80% of what I did was self-taught.  There may have been models out there for me to emulate, but I simply didn’t have time to find them or use them in between implementing my online classes and teaching my colleagues about what I was doing (and supporting them in their efforts to do the same).  There was little formal institutional structure or guidance for what I was doing (and what did exist was largely “start up” support with little ongoing support).  The conversation of quality assurance seemed very much an afterthought, and clear, institutional standards or guidelines for developing and implementing online/blended learning were (and to a degree, still are) nonexistent.

Almost as long as I’ve been teaching online (and blended), I’ve struggled to find meaningful ways to measure what’s happening in my courses and whether it’s working.  Our institution’s standard student course evaluation is ill-suited to the online or blended environment and doesn’t capture the things I really want to know about my students’ experiences.  Do my instructional materials “work” for them?  Are they able to process and internalize the course content independently?  Do they feel supported by their peers and by me as the instructor?  Do they feel a connection to others in the class?  Are the assignments clear?  Do they see the connections between the course goals and the activities of the course?  What about the designed environment “works”?  What doesn’t?

Along the same lines, I love this quote from Chapter 5 of the BlendKit Reader

While a tour of an unoccupied kindergarten classroom and an inventory of its resources might provide some indication of the nature of the teaching and learning that occur there, it is the lived experiences of the students and teachers, their actual interactions, in which teaching and learning are made manifest. Limiting the scope of blended or online course quality to considerations of the designed environment results in a significant blind spot.

I currently rely on a number of indicators to gauge how well the class is going.  I look at the number of questions I get from students that are the result of poor organization or clarity on my part (“I can’t find _____!”).  I watch student performance on course activities and assignments.  I read students’ reflections on course content for depth of understanding and thought.  I consider the feedback on those institutional evaluations (as limited as they may be, they has guided some changes in my practice).  And I’m currently working with some colleagues to develop a peer review structure for online/blended teaching.  However, none of this is currently institutionalized at my university, and I’d love to see the idea of quality assurance for online and blended more formalized.  How great would it be for my colleagues who are new to online/blended teaching to have an established, research-based process of development and evaluation to follow?

I sense we’re on the cusp of some of these conversations at my university, and this is exciting to me.  There are great things happening in online and blended classes, and I’d love to help find ways to capture that.

Here are some links to some existing resources and rubrics related to evaluating online and blended teaching (mostly just for my own selfish future reference…):

Beg, Borrow, and Steal: Open Educational Resources

"Open Way." by Rak Tia is licensed under CC BY 2.0

“Open Way.” by Rak Tia is licensed under CC BY 2.0

It’s in a teacher’s nature to beg, borrow and steal.  So the concept of Open Educational Resources (OER) is an appealing one for this former third grade teacher, particularly when I consider my blended and online classes.  I create a lot of my own content based on presentations, screencast lectures, and activities.  I’ve used several different tools to create that content, including SnagIt, Camtasia, and SoftChalk.  I also try to identify material and resources from other sources to include in my online modules so that students are not just listening to me all the time.

In blended courses, it’s so important to get the content right and to be consistent with the structure.  Of equal importance is the need to connect what’s happening in the online portion of the course with what’s happening face-to-face.  Being clear about the purpose, structure, and routine of the blended class is vital to this effort.

I like to use the online component of my blended class to deliver the background knowledge and information students need to complete the in-class activities and assignments.  Additionally, there is a reflective component that comes after the in-class portion through which students contemplate what they’ve learned and how it will impact their future practice.

The promise of OER is an enticing one.  I love the idea of this repository of useful information, presentations, materials, and activities that I can draw from and use to enhance my students’ experiences.  However, I have always found it far faster to create my own materials that are focused on what is relevant to the course than to spend time searching for OER materials to include.  So I was grateful to have the “forced opportunity” to explore OER in greater depth through BlendKit.

My exploration was made much easier by UCF’s Diigo annotated list of links to OER repositories.  I eagerly started searching to see what I could find.  The answer?  Not much, unfortunately.  I ended up confirming my suspicions that my current approach (creating my own content and doing my own searches for links, media, and resources that relate) was really the most efficient.

Several of the OER directories I explored were so poorly curated that I found it difficult to find anything of value.  Even advanced searches filtered for specific content, format and purposes rarely yielded resources I could actually use (or that even matched the filters I had set).  I was overwhelmingly underwhelmed by what I found on several of the sites.  I was feeling quite discouraged about the usefulness of OER in my own practice, until I found the OERCommons!

The OERCommons is everything OER should be.  The content is clearly categorized and organized, and it’s easy to search.  The search results are detailed and relevant.

LOC Public Domain Western Litho. Co. Los Angeles.

For example, I searched “World War I” and found a wide variety of lessons, instructional materials, and assessments, including a lesson called, “Reading Like a Historian: US Entry into World War I”.  The lesson was tagged with appropriate K-12 content standards, had a succinct abstract describing what was included, and had clear guidelines for use.  There is enormous potential here, not only for my use at the university level, but also for my students, who are future teachers.

So, as is pretty typical of the web and online resources, there’s good stuff there.  You just have to dig past some of the not so good stuff to find it…

Assessment in Online Environments

Assessment is a vital component of the learner’s experience.  It provides them with important feedback related to their progress, whether it’s formative feedback along the way or summative feedback at the conclusion of a course.

"151-365 Teacher's desk and iPad" by PaulSh is licensed under CC BY 2.0

“151-365 Teacher’s desk and iPad” by PaulSh is licensed under CC BY 2.0

The instructor’s role is to ensure that quality assessments are utilized in an effective way and that students are given timely, specific feedback related to goals.  The challenge is accomplishing this in an online or blended course.  I’ve talked about this topic a bit here already, but assessment is important.  So I’m going to talk about it some more!

As a teacher of future teachers, not only am I responsible for administering effective assessments, I am also responsible for modeling effective assessment practices.  My goal is to expose my students to a variety of effective assessment practices that they can incorporate in their future classrooms, whether it’s a course that includes assessment as stated course content or not.

I believe assessment takes on an even more vital role in an online environment.  In such an environment, the instructor is not immediately present to provide informal or impromptu feedback as they can in a face-to-face environment.  Additionally, the online environment can lead to students feeling isolated from not only their classmates, but the instructor as well.  Quality assessment in online environments can help students feel connected to the instructor and monitor their progress toward course goals.

"Still alive..." by Thibaud Saintin is licensed under CC BY 2.0

“Still alive…” by Thibaud Saintin is licensed under CC BY 2.0

I don’t believe in taking shortcuts when it comes to assessing my students.  Sure, it’s easy to create a multiple choice quiz in my learning management system that is automatically graded for students.  Beyond the actual creation of the quiz, there isn’t much more required of me as the instructor.  It takes more time to create and grade a reflective blog entry, a performance assessment, or a group project.  But the feedback opportunities and the individualized input from the instructor are infinitely more valuable to the students and to me, as the instructor, in terms of monitoring students’ progress.

In my online and blended courses, I rely heavily on performance-based assessments to gauge my students’ learning.  Personally, I found O’Reilly and Kelly’s (2008) table of  “Assessment strategies and disciplines that may commonly use them” to be woefully limited in terms of the discipline of teacher education.  To assess my students, I routinely engage them in writing (essays, reflective blog entries, written feedback on classmates’ projects, creation of wikis, etc.), creation of media (websites, eportfolios, multimedia presentations, screencasts, technology-based teaching tools, etc.), and collaborative projects (group wikis, team-based presentations, online synchronous team debates, etc.).  One of my online courses has a traditional final exam with a blend of multiple choice, fill in the blank, matching, and essay responses administered through the LMS.  The rest of my online and blended courses utilize almost entirely performance-based or written assessments that are graded using comprehensive rubrics that are shared ahead of time with students.

I spend a LOT of time grading and providing feedback in my online courses.  My feedback to the students takes many forms.  Sometimes I comment on or “like” their blog posts.  Other times I fill in a rubric and add personalized comments.  For some assessments, I screencast feedback for each individual student.  All of these efforts take time.  But in my opinion, it’s time well spent on connecting with my students, informing their learning process, helping them grow, and modeling effective strategies for their future in the field of education.

The Human Need for Interaction

Whether students are taking courses face-to-face, online, or in a blended format, they all share a basic need for human interaction.  In fact, I would argue that’s a basic human need of people in just about any situation.  Our innate desire is to process our experiences with those around us.  And at least for me, when I discuss things I’m learning with others, my understanding is often deeper and my retention is improved.

Part of the instructor’s role is to help this interaction to happen, regardless of the course format.  Generally speaking, students want to feel part of a learning community (though there are always exceptions).  Some colleagues and I have actually studied this in some of the online courses we offer at our institution.  Specifically, we studied the sequence of online courses in one of our master’s programs.  What we found was that students appreciated the sense of community and collaboration developed through the interactive portions of the online courses.  (Our research was published as one chapter in a very pricey text titled Building Online Communities in Higher Education Institutions.)

Those interactive portions of our online classes always included some form of introductory activity designed to help students get to know each other (with pictures!), similar to those described by Kelly & Cox (2008).  We also routinely incorporated online discussions through various tools and platforms, both related to collaborative projects and individual responses to course readings.  Students generally reported feeling connected to others in the class through these activities, and recognized the value of these connections.  My goal, in transitioning to online (and blended) instruction, has always been to replicate the richness of the relationships and dialogue from my face-to-face classes in the online, asynchronous environment.

"Online Mentoring" by Alec Couros is licensed under CC BY 2.0

“Online Mentoring” by Alec Couros is licensed under CC BY 2.0

Online discussion forums alone do not create authentic interaction.  In creating interactive opportunities, whether they’re happening face-to-face or online, instructors must ensure that expectations for interaction are clear and thoroughly explained to students.  Additionally, students need feedback on their participation, both from the instructor and their peers.  If an environment is established where students feel comfortable sharing their thoughts and ideas, rich dialogue and interaction can follow, even in an online environment.

Ever since my first experiences with the field of education as an undergraduate student, through my harrowing first years of teaching elementary school, and into my years working with students in higher education, my educational philosophy has centered around creating genuine relationships with and among my students.  To me, that’s what this interactive piece is all about…building relationships within the context of learning.

Reflections on Formative Feedback

Because I’m certifiable (and also a little bit addicted to professional development), I’m currently participating in a week-long workshop through the Online Learning Consortium called “Fundamentals: Giving Effective Feedback” (this is in addition to the MOOC I’m also working on about designing blended learning).  I’m always looking for ways to improve my practice, particularly related to online teaching.  I want to know I’m doing everything possible to ensure my students are getting as much (if not more) out of my online classes as they would from a face-to-face class.

One of the components of online teaching that I think is most important is effective (and timely) feedback.  Hence, my participation in this workshop.

In providing feedback for students in my online courses, I strive to ensure that it is timely and specific. In some cases, my students are submitting assignments weekly, and I try to have feedback to them within 1-2 days of submission so they can incorporate that feedback in their assignments for the following week. I also use a variety of tools to ensure that the feedback is rich and detailed.

One of the concerns I have about the feedback I provide is related to how the students are internalizing and utilizing that feedback. I don’t currently have any guarantee that students are viewing and incorporating the feedback. And even if they are, it’s a very one-sided process. It’s all me giving feedback and no guided process of them processing or thinking about the feedback. I’m relying on them doing all of that independently, and the reality is that I don’t know if students know how to do that effectively.

One new strategy I would like to try is asking students to reflect on the feedback I’ve given them. In an article by Grant Wiggins about the seven keys to effective feedback, it was suggested that the instructor provide students with feedback and then ask the students to reflect on it. The wording suggested was, “Given the feedback, do you have some ideas about how you might improve?” It’s such a simple question, but it so clearly places the responsibility on the student to think about the feedback and come up with an action plan. It’s currently something I’ve been doing for them, and this isn’t nearly as powerful as having them do it themselves.

Since I’m not currently using a tool such as Skype where I can verbally ask the students this question, I’m thinking about adding a component to subsequent assignments that begins with a brief reflection on the feedback they received on the prior assignment and what they are going to do differently as a result. This will fit nicely with what they are already doing and will add an element of reflection based on feedback that should be valuable to them.

I’m looking forward to trying this with my current group of students and continuing to learn more about providing effective feedback!

My “Blend”

In my role as a faculty member, I teach a variety of courses in several different formats. I’ve taught traditional on-ground courses, fully online courses, and hybrid courses with varying degrees of online content.  As I participate in BlendKit2015, I am reflecting on all of these differing formats with particular attention to the blended (or what I have previously referred to as “hybrid”) approaches.

I have been what some might consider an “early adopter” of online teaching in my department.  As such, I’ve had to lead the fight for online learning in many cases.  I often find myself advocating for the possibilities presented by online learning to those who have either had bad experiences themselves or heard horror stories of bad experiences from others.  I’ve spent a great deal of time and effort sharing my online philosophy with others and explaining the benefits of the online format.  I’m gaining ground, but it’s a continual uphill battle.  So to entertain (and publicly acknowledge) the possibility that a blended approach is at least equally promising as online (and perhaps more so) feels risky!  But I’m finding that a blend offers the best of both worlds (online and f2f), if done properly.

This semester, I am teaching a class in a blended format (approximately 1/3 online, 2/3 f2f) that I have previously only ever taught fully online (normally I develop courses in the opposite order!).  It has been a fascinating experience (all 3 weeks of it so far), and one that has required immense amounts of cognitive energy to prep.  But I’m having fun with it, and I’m really enjoying the class (hopefully the students who are in the class and might be reading this blog entry will agree with me!).  It’s been a unique and refreshing challenge to rethink the delivery of the course and find ways to take advantage of the human interaction that is often so difficult to replicate in my fully online courses.  And I’m learning a lot from the experience.

As the course is one that is already established, the course description, goals, objectives and learning outcomes are clear.  In addition, the components of the previously-developed, fully online class have naturally lent themselves to the online portions of this blended class.  What’s been left in terms of planning is the in-class experience, which I am trying to capitalize upon by integrating application, higher-order thinking, problem solving and collaborative activities.  While I initially felt somewhat guilty for not having every minute of every session planned in advance, I am realizing now that this leaves space for an emergent process that allows for modification of the course activities based on students’ needs and interests.

The following thoughts from George Siemens (included in this week’s reading from the BlendKit Reader) describe pretty accurately what I’m trying to achieve when I think about designing blended learning experiences for my students:

By recognizing learning as a messy, nebulous, informal, chaotic process, we need to rethink how we design our instruction…Instead of presenting content/information/knowledge in a linear sequential manner, learners can be provided with a rich array of tools and information sources to use in creating their own learning pathways.  The instructor or institution can still ensure that critical learning elements are achieved by focusing instead on the creation of the knowledge ecology.  The links and connections are formed by the learners themselves.

I think learning should be a bit messy and chaotic.  Isn’t is so much more memorable that way??

Fun with Creative Commons

Copyright and Creative Commons can be confusing!  The important thing to know is that, if you are using something (anything!) created by someone else, you should give credit where credit is due.  And tread carefully.  People post (or re-post) images and content on the internet every day, often in violation of copyright.  “I didn’t know!” isn’t an acceptable excuse either!  Educate yourself and make good decisions.

Creative Commons is an easier (not necessarily easy!) way to include content in your online endeavors without violating copyright.  However, even with Creative Commons content, it’s still important to give credit, or attribution, to the original creator.  This is where things get a little tricky, particularly considering you’re at the mercy of the widely varied CC licensing levels and the information shared by the content’s original creator.

This page contains some very helpful information regarding attribution with CC licensed materials.  Here is an example:

13540663395_803678f33b_mBumblebee Trying to Find a Way Out in a Psychiatric Ward of Bohunice Hospital” by Kojotisko is licensed under CC BY 2.0

The above attribution contains the original title of the image as a link to the image’s original location (in this case, Flickr Creative Commons), a link to the original creator’s profile page (also on Flickr) and a link to the creative commons license information (on CreativeCommons.org).  Most of this information can be found on the page with the image, sometimes included as a link itself.

Sometimes, you are missing some of the information.  Perhaps the image doesn’t have a title (use “Photo” instead, still linked to the original location).  Perhaps the original creator doesn’t have a profile linked (just use the creator’s name without a link).  You may need to investigate a bit to find some of the information.  However, the more complete your attribution, the better.  And if you aren’t sure whether the material is licensed under creative commons, it’s probably better not to use it (here’s lookin’ at you, Google Image Search!).

Happy New Year!

The whole “new year” in January thing has always felt a little strange to me.  As a perpetual student and teacher, the true new year celebration has always happened at the end of August for me.  Nevertheless, January does mark the start of a new year for many, and it still feels a little like a fresh beginning!

Today, I am starting another round of EDUC 407 with two new groups of students.  It’s always a little anticlimactic starting a new class when it’s fully online.  You miss out on the formal introductions and “getting to know you” activities of a traditional, face-to-face class.  That said, it’s still exciting to think of these students getting started and anticipating a new class!

I’m surrounded by people who are bemoaning this day…back to work, back to school, back to routine, back to responsibility.  However, I really can say I love what I do, and I am eager to get back to it!  Here’s to a great 2015!

FREE Professional Development Opportunity!

I’m on a state email list about the Common Core implementation, and included in an email this morning were two opportunities for professional development for teachers.  They’re both MOOCs (massive open online courses), which is a format I have mixed feelings about.  However, it might be worth looking into if the topics pertain to you!

The course I thought sounded interesting is called “Disciplinary Literacy for Deeper Learning”.  This course is open to anyone; however, it is specifically designed for 6th-12th grade teachers of English, science, history-social studies, or math.  It runs from September 29th through November 9th.  Here is a description of what participants will learn:

“You will:
  • Strengthen your understanding of what it takes to engage students in deeper learning through disciplinary literacy practices;
  • Explore the Model for Inquiry-Based Disciplinary Literacy and how it can promote deeper learning for your students;
  • Engage in disciplinary literacy practices (e.g., close reading, digital learning, constructing and supporting claims);
  • Experience multiple opportunities for personalized application of disciplinary literacy to your teaching context;
  • Design, critically evaluate, and share Inquiry-Based Disciplinary Literacy lessons;
  • Take action to become a teacher leader by engaging others in disciplinary literacy practices.”

There is also an intriguing focus on digital literacy.  If the MOOC is well done, it might be worth looking into.

We are so fortunate to live in an era of technology where the professional development can come to us!  The learning opportunities are rich and varied.



You can read more and register for the MOOC here:  https://courses.mooc-ed.org/dldl1/preview

Links, links, and more links! (Plus a plug for Twitter…)

I have an inconsistent relationship with Twitter.  I go through phases where I ignore it entirely for months at a time, and then something happens and my interest is renewed.  I suspect part of the problem is it feels a little disjointed because I follow such a variety of people and accounts.  However, I consistently find Twitter, when I read it, to be a source of thought-provoking information and perspectives.

Currently, my Twitter feed is dominated by the Los Angeles Kings (playoff season and all that) and EdTech chatter.  The EdTech chatter is what I find most fascinating.  And I frequently come across posts or links to articles that I find compelling and relevant to EDUC 407.  So I thought I would post a few of the most recent links here, in case any of you following me finds yourself with a whole bunch of extra time and nothing better to do… (Ha, ha, very funny.  I know.)

Top iPad Apps for Student Creativity (Edutopia) I find Edutopia to be a great source for teachers for technology and PBL.  This article talks about how the tendency, with iPads (or mobile devices in general) is to focus on apps where the students practice basic content (think alphabet, phonics, and entry-level math).  Instead, the true power of the mobile device is the opportunity for students to create.  This article highlights apps that help students create their own content in new and exciting ways.  Can’t wait to try some of these with my own guinea pigs…I mean…kids.

Let It Marinate: The Importance of Reflection and Closing (Edutopia) Again with the Edutopia, I know.  But this one is a great take on the power of reflection, something I believe in wholeheartedly.  Blogging is an amazing way to reflect on learning experiences.

Skype Loves Bringing Groups Together (Skype Blog) This one is more of an “information item”.  Up until recently, Skype required at least one user to have a paid premium account in order to allow group skyping.  Apparently, they have realized the errors of their ways and decided to change this policy.  Now, even if everyone in the group has a lowly free account, you can still group skype.  It’s about time.

If you aren’t using Twitter as a source for professional reading and inspiration, I encourage you to try it.  It’s easy to curate who you follow to create a feed that focuses on your interests (professional, personal, or both!) and offers timely and relevant content.  For a preview, check out some of the hashtags listed below:

#edtech  #edtechchat  #edusocmedia  #edcamp

And here’s an article on getting started with Twitter (for educators!).  Try it!  I think you’ll like it!