Jared Stein – Education, Technology, Culture, and the Internet

Chris Lott, commenting on his institution’s acquisition of Elluminate video-conferencing platform as a teaching tool concludes,

The real issue with any of these tools isn’t finding one that works, it is learning– and then teaching colleagues– how to teach in a way that takes advantage of the capabilities and doesn’t merely replicate the lecture mode in a distributed format. That’s deadly. As I always say, the only thing more deadly than the PowerPoint drone and lecture model is that same model through a mediating tool like Elluminate…About Elluminate

I agree that when video-conferencing avoiding replication of ineffective lecture modes is important; however, always the devil’s advocate (well, at least 50% of the time), have to ask, first, if this is a blanket condemnation of the lecture mode regardless of delivery method, or if the technology itself interferes with, disturbs, or detracts from the traditional lecture (which may in a classroom actually be effective).

I’m guessing most folks in educational technology or instructional design lament the continued use of straight lecture format (“sage on the stage”) regardless of delivery environment, but I, having been a student in more than a handful of damn good and several quite memorable lectures (from which I still retain a significant amount of information), causing me to assert that straight lecture is not de facto a bad thing.

At this point I have absolutely no empirical evidence that suggests straight lecture is or can be highly effective, but now (from this small comment tacked onto the end of a technology tool review, no less) I’m inspired to look into it. Comparitively speaking how effective is lecture for learning? What makes lecture more or less effective? There has to be a good deal of research on this already. (Any recommendations on salient books/articles are now being accepted!)

I can name some ailments of lectures delivered via video conferencing that I have witnessed. In both edtech informational sessions and in vendor presentations delivered via video conferencing, presenters do tend to follow a simple, generic pattern (much informed by PowerPoint) which centers on providing text-and-talk-heavy information in tedious spurts with brief pauses for “questions” (which, in and of themselves, often occur too late or at moments so ill-planned moments that they actually increase the presentations/presenter’s anesthetizing qualities). Any questions posed rarely lead to real dialogue or discussion; rather, questions are merely a challenge that the presenter must overcome before being allowed to continue with his/her script.

And though these ailments can certainly be present in a live, in-person classroom-style lecture or presentation, my instinct tells me that there’s something about phsyical human presence that either reinforces the delivery of the information, or provides for better audience attention through either overt or more subtle person-to-person engagement.

The next question, then, would be how does technology deliver methods affect the effectiveness of lecture? Video conferencing in particular should be examined, though of course some common attributes will need to be agreed upon so as to include a live fiber-optic system like we have at UVSC, or an Internet-delivered system like Elluminate.

I would hypothesize that a boring lecture in the classroom becomes worse when video-delivered, either because it becomes (a) less interesting for lack of physical presence (for whatever reason…engagement?), or (b) less compelling to one’s attention when technology-delivered (possibly because of the presence or availability of more distractors, or because of the absence of social pressure to show interest/passively participate).

It would be interesting to brainstorm presentation effectiveness on tech-delivered platforms with some ed researchers and public speaking experts. How does one leverage the live delivery method so that the end results are superior to static information delivery? How does one construct information to affect better learning? How does one engage with the audience and make an impact that may stimulate memory a la the affective factor?

Looking ahead to conferences in 2008, I’m already planning on attending ITC’s elearning conference in Florida in February as a pure particpant/observer. And of course I’ll be co-hosting and probably presenting at our Teaching w/ Technology Idea Exchange in June. There are a few other conferences that I’m interested in presenting at:

• Distance Teaching & Learning in Madison, Wisconsin (proposal deadline: Jan 15, 2008). I could do an online course showcase or two. Japanese comes to mind, and I’m pretty proud of my new version of Web Essentials. Or I could just push for a regular session proposal, but what topic? “Authentic Applications of Social Networking Tools”? or the yet-unfounded “LMS-Less”?
• Collaboration 2008 – The Southwest Vista User’s Group in Salt Lake City, Utah (proposal deadline: Jan 11, 2008). No idea on a topic here, as I’m not a BB Vista advocate, but I should try to present to support the Utah cause.
• WCET’s 20th Annual Conference is held in Phoenix, AZ November 5-8 this year. As far as I know, the call for proposals is not yet open, and I don’t see a proposal deadline. WCET sessions usually annoy me because each speakers get approx 15 minutes to talk, sharing a 50 minute session with other presenters who may or may not have similar ideas, and may or may not develop synergy. I would love to do a pre-conference at WCET this year, but am not sure how to go about suggesting one to the WCET folks.
• C()SL is sure to have another OpenEd 2008 conference, at which I’d love to present our still-alpha OpenMod for moodle. But will that ever be completed?

I think more than anything I simply need to motivate myself to come up with a good presentation and submit by said proposal due date(s). If needs be, I can use a vague title and determine the specific content as the months pass. I also am considering collaborating with colleagues on a presentation, though I myself often disdain presentations with multiple and unnecessary “support” presenters.

Which leads me to consider the fact that attending a conference can be quite different depending on whether I go by myself or with comrades. I’m a loner by nature, so the solo experience is wonderful in that I tend to learn a lot and reinforce my internal motivation to strive for excellence. I also tend to explore ideas fairly broadly. And I’m always concerned with the bang-for-buck factor of conferences, and so going by myself means the department has more money to spread around at the other conferences.

When I’m with colleagues or comrades, however, the experience is substantiated by the affective factor, and my exploration of ideas tends to be deeper as we discuss possibilities and scenarios together. We also seem to develop a stronger team relationship, and return from such conferences more socially engaged, which is, of course, a natural agent for productivity and innovation in the workplace.

Regardless, I’ll sure to be practicing my own peculiar style of session attendance, which has gained the dubious label of “The Jared Method” by Mr. Hugentobler, where I pop into one session, grab the printed materials, listen to the opening lines as I scan the materials to evaluate the session, then (often) pop out to investigate another session in like manner. If colleagues are involved I can often verify or correct my first impression of the session later in the day.

I am a bit miserable about a series of questions that I whipped up for a survey device at the request of an instructor who teaches a Distance Education course.

Not only do I disagree with the instructor’s desired objectives in using this survey (she essentially hopes to prove that the reason students are failing her online course is because they are under-prepared or have wrong assumptions about online education–of course it couldn’t have anything to do with the fact that the course has nearly no media-enhanced learning, no student-student contact, and very little student-instructor interaction), I disagree with the questions that I wrote.

Of course anyone who has written survey questions with a mind to gain accurate and insightful information on the participants knows what a challenge the task is from the get-go; I don’t kid myself that it’s no easy endeavor, but I also think there has to be a better way.

Among my primary objectives in writing the questions were the following ideas:

• Keep the survey short, so that students would actually do it.
• Have some redundancy to check for accuracy and inhibit prejudicial responses.
• Avoid asking questions that dare students to label themselves “dumb”.
• Avoid questions that tempted students into labeling themselves “smart”.

But the primary objective was essentially this: after reading a good number of “student readiness” surveys online I wanted to avoid asking questions that gauged a student’s willingness to partake in a lonesome independent study course. “Independent study” is not equivalent to modern “distance education” in the Stein dictionary (in fact, even “distance education” is not equivalent to “distance education” in the Stein dictionary, but that’s another story). And so though several of my questions are based on the questions asked in other “distance education” surveys, I purposefully steered away from presumptive questions like:

Feeling that I am part of a class is:
a. Not particularly necessary to me.
b. Somewhat important to me.
c. Very important to me.

As if being “part of a class” is somehow mutually exclusive from distance learning! And it’s not that I’m opposed to independent study types of courses; in fact, I myself greatly enjoy and grow in isolation, but I recognize that’s not necessarily the norm.

Then there are questions that perpetuate instructors’ presumptions that they can get back to distance students at their leisure:

My comfort level with waiting a few days to receive instructor feedback is..
Low   Moderate   High

While it may be an unfortunate reality in distance education programs that instructors do often delay responding to students (I recommend a 24 hour turn around at the latest), we certainly don’t want to encourage that behavior, nor do we want to discourage student expectations of their instructors.

Finally, I also have disagreements with the term “student readiness” in general, as that tends to automatically place the blame for student failure at the feet of the students. Jared Spool, a Web usability expert whom I greatly admired, once inspired me to make the following provocative paraphrase, There are no user errors, only
design errors. And while I recognize that this statement is not universally true, it does challenge the designer (in this case, the instructor or the instructor’s instructional designer) to reconsider blaming the user (aka student) for failing to complete the task.

My Questions

Even though I have a pretty good insight into what I think is wrong with so many “student readiness” surveys, I still had a hard time making my fundamentally different. But I’ll share them here anyway, with the hopes that some brainy folks can offer better suggestions to achieve the same general objective: determine if our students are adequately prepared–both mentally and technically–for an online course experience.

(Note: these questions are randomized in the final survey to mask redundancy.)

Options: Strongly Agree | Agree | Disagree | Strongly Disagree
1. I often get things done ahead of time.
2. I can work independently and meet deadlines without being reminded.
3. I learn best through live classroom discussions.
4. I am comfortable engaging in class discussions on the Web.
5. If given clear instructions, I am confident that I can complete the assignment independently.
6. I often need to have instructions for an assignment clarified or explained more than once.
7. As a reader, I sometimes need help to understand the text.
8. When I need help understanding the subject, I am comfortable e-mailing an instructor to ask for clarification.
9. When I don’t understand something I’ve read, I ask the instructor to explain it as soon as possible.
10. I am very competent using e-mail and Web sites.
11. I am a skilled writer.
12. I don’t always comprehend what I read.
13. I expect to spend less time on an Distance Education course than a regular on-campus course.
14. I often put things off until the last minute
15. I expect a Distance Education course to be easier than a regular on-campus course.

If you hate these questions, give me something better.

And if you like them, you can download them here (This survey is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.):

• DE Readiness Survey (survey).zip Blackboard Vista IMS Survey version
• DE Readiness Survey (quiz).zip Blackboard Vista IMS Quiz version (scored for a “perfect” student)
• DE Readiness Survey.txt Moodle GIFT format (scored for a “perfect” student)

(Originally posted on our Distance Ed unit’s IDS ‘blog.)

I and several folks on our team are currently in what our sister-in-arms Janel would rightly call “crisis mode” for a certain online course we’re producing. We have been for the past 10 weeks. It’s not a comfortable place to be, and much of the additional stress is my responsibility–in part through miscalculating the length of the project’s development cycles, and in part by acquiescing to the instructors’ wishes for an “on-time” delivery. However, now I’m beginning to reflect on the course of this project, and in fielding a number of complaints throughout the process I have returned to an article I read some years ago on rapid prototyping in instructional design:

Tripp, S. D., & Bichelmeyer, B. (1990). Rapid protoyping: An alternative instructional design strategy. Educational Technology, Research and Development, 38(1), 31-44.
(You can read a summary of the Tripp Bichelmeyer article here.)

The number-one complaint I’ve heard from folks on the team (myself included!) is that the scope of the project is not clear, and both the instructor and the instructional designer (me) are asking for changes to the tools and the learning media “mid-stream”–that is, as the students progress. But when I analyze those complaints, they lose a lot of their legitimacy.

Unlike other types of design, instructional design has as its highest priority the effectiveness and usability of the learning media for the outcome of learning. What instructional designers deliver to learners is not a product, but an experience. Unlike other media, effective learning media does not entertain, it engages. Technology-facilitated learning media has the potential of providing maximum return for minimal effort. And yet it is still a fairly a new “art”. Though there are principles and best practices, and though instructional designers likely know more about learning and cognition than ever before, there are no easy formulas for designing instruction, and no instructional design will “fit” all instructors, let alone all learners.

Instructional designers design and produce learning media with the assistance of skilled human resources, such as programmers, graphic designers, videographers, etc. In order to provide the optimum learning experience, instructional designers must evaluate the learning media for usability and effectiveness continually, and as early as possible, with the option of immediately revising, rewriting, recreating, or adding to the learning media in a cyclic pattern. As one cycle proves effective and usable, the next cycle begins, based upon the best-practices of the previous, and so on.

Even as the learning media goes into production, instructors should receive formative feedback from learners in each lesson. Formative feedback provides instructors with information as to whether or the not the students are learning, how efficient that learning is, and whether or not the tools that facilitate the learning are frustrating and over-encumbering. When a learner is over-encumbered by extraneous tasks, such as manipulating the technology, learning is inhibited, if not impossible. Such encumbrances must be eliminated as they are discovered.

Scope, therefore, is something that is only recognizably accurate in hindsight for an instructional designer. Scope is a forecast, not a definition.

This idea of instructional design through rapid prototyping may be most necessary when the learning outcomes are skills-based. If you remove the severe time constraints that we’ve been on, this online course is a perfect example of effective instructional design through rapid prototyping. It is also a perfect example of how it is nearly impossible to define scope all at once; the learners have four skills that they must develop, and each skill is dramatically different in how it’s learned and how it’s practiced. Not only that, but as the learners progress through the course, the learning outcomes change, their needs change, and the tools that will best facilitate their skills must also change.

Is there a point at which an instructional design is complete? Not necessarily. Sometimes we finish a project when we’ve shown that we’ve met our learning objectives, sometimes the conclusion is based on more arbitrary factors. Budget, time, personality conflicts—all of these are capable of providing sufficient justification for the conclusion of any given project. But the point is that we are open, we are zen in the way we accept change, we are focused on the learners their experiences, and we base our success on effectiveness more than efficiency.

And though we in Distance Education pride ourselves in outwitting the limitations of time and space, efficiency must still be a principle concern to any director and any project manager. At the same time the process of rapid prototyping and a focus on the learner’s experience will help us take learning media from bad to better to best. I don’t think the two are necessarily at odds; by practicing the process we can only learn from experience, and our results can surely become better, faster and more efficient. The pay-off is clear; it’s merely the mind-sets that are muddled.