20 Questions With Karl Kapp

This past year the American Society for Training and Development published a survey of the eLearning community to see how long, on average, it took to develop one hour of courseware. The results were astonishing. We sat down with one of the survey’s authors, Karl Kapp to discuss his findings and to try to peer more deeply into the state of the industry that they suggest. Karl is a consultant, scholar, and expert on the convergence of learning, technology and business operations. He received his Doctorate of Education in Instructional Design at the University of Pittsburgh in Pittsburgh, PA. Dr. Kapp is a full professor of Instructional Technology at Bloomsburg University.

Describe briefly the survey you conducted for ASTD.

Karl: Back in 2003, I wanted to explore how long it took to develop one hour of instruction for a variety of types of tools and instruction including stand up and e-learning because a lot of people were asking that question. The “one hour” metric seemed to be the standard measurement so I scoured the literature and found a little information but not much. So I created a survey and gathered data on the amount of time it took to develop one hour of instruction. Now, 6 years later I wanted to see if the data was still holding. Does it now take longer or shorter? I teamed with Robyn Defelice, an alumni of our program, and we re-did the study. We surveyed ASTD members, our own alumni and anyone else who wanted to contribute. Our 2009 study got about 47 respondents and since that time has grown to over 60. We looked at the high end and low end to develop stand-up instruction, virtual classroom instruction, e-learning and simulation development.

What were the differences you found between 2003 and 2009?

Karl: Time went up in most categories but the ranges tended to hover around the 2003 numbers. In other words, we see slight increases but not huge differences in the original ranges in most cases. Time for developing one hour of e-Learning without a template did decreased for moderate interactivity and high interactivity but e-learning developed with a template seemed to increase in several categories. I think it is important to remember that the numbers are a result of self-reporting and not direct observation so the numbers may be skewed when compared to direct observation but I don’t have those direct observation numbers. Perhaps my next study on the subject.

Yeah, when we look at the table in your survey, as you said, we see the time to develop increased in nearly every category. Did that surprise you?

Karl: Actually, it did surprise me a bit. My co-author and I have been trying to come up with viable explanations of why that may be.

One reason we divined from the study was the interaction with Subject Matter Experts (SMEs) is becoming more difficult and more time consuming. The SMEs do not have time so much of the instructional designer/developer’s job is now focused on hunting down an expert to get answers or getting sign-off on content, or agreement on objectives so we think those elements are taking more time than in 2003. Therefore, streamlining the interaction with a SME could help reduce time.

Another reason could be that instructional design (Analysis, Design, formative evaluation) still is a time consuming and lengthy process and most of those elements of the design process are untouched by technology so those elements haven’t been reduced. This is a common complaint of the instructional design process…it takes too much time. However, done properly, I’ve never seen the process fail. Everyone wants to rush the process but the results of rushing the process aren’t always good.

So, the two biggest reasons for the increase are the increased difficulty working with the SMEs and the fact that many elements of the process have not been improved with the technology, only the development step in the process.

Over those six years did the quality of the courseware improve? Was it more rich, engaging, technologically sophisticated?

Karl: Boy, do I wish I could answer “Yes” to this question. Ok, well, in some areas the instruction is much more sophisticated and today, there are some truly engaging and artistic pieces of e-learning. Unfortunately, those are in the minority. Today, many people are simply placing slides online, adding a little audio and presto, they think they have e-learning. [This type of e-learning (Powerpoint on steroids) is what gives way to the impression that our numbers are too large. ]

While, many template-based tools now have some interactive elements built-in and some neat interactions, they are not leveraged as much as they could be. So, yes the overall bar is being raised slightly form 2003 but not enough. In 2003, the focus was mostly on the technology, today, it has shifted a little more toward design. Good instructional design takes time, slapping something together does not. The technology makes the development a little easier than the past but the increase in the difficulty of working with SMEs and the need to develop sophisticated instruction has kept the time requirements high.

Were changes in web technology and trends reflected in the newer courseware or were you still seeing GIFs and Frames and such?

Karl: To some degree, new software is reflected. However, while there is a wide range of options for creating instruction the two most frequently used development software reported in the survey are MS Word and MS Powerpoint with Abobe Flash and Abode Captivate bringing up the third and fourth positions.

What do you mean in your survey when you classify courseware made with and without a template?

Karl: “Without a template” is straight html programming of the online learning. With a template means having a structure into which you placed the navigation buttons, the content and the images. A template provides structure so that the person entering the content doesn’t have to know any programming or html.

It seems as though, here too, while we’d expect templates to cut time to develop, we see mostly increases. Why is that?

Karl: I think this is because templates can do more and provide more choices than they did in 2003. In 2003, the only real choices for interactions were clickable maps, multiple choice questions and drag and drop exercises if the programmer could create them. Today, in addition to the items mentioned you have interactive timelines, a variety of question types, animations sequences, and other elements that require more time to gather the material and design the instruction to work properly with those interactions. Designing the content to fit available pre-programmed interactions can take time. So that has elongated the time even though templates are involved.

Describe the ADDIE model and how it guided your survey.

Karl: ADDIE is the Analysis, Design, Development, Implementation and Evaluation process used to design instruction. It is a generic design model. Many experts have created specific models to design instruction but ADDIE is the generic term for those types of models. Analysis means that you find out who the instruction is for and they type of content, design is adding instructional strategies, development is creating the instruction, implementations is making sure learners can gain access to instruction and evaluation is seeing whether or not the instruction works.

What percentage of the courseware being developed in 2003 was online vs CD-ROM? 2009?

Karl: Good question, unfortunately, I don’t know the answer. But I can say that in 2003 the web was not as robust so the interactions, use of video and audio and sophistication wasn’t as commonly available as it is now. In fact, the lack of options for web-based learning in 2003 may account for the times not changing much. Back then, you had to work a lot harder just to make the simple stuff work. Today, the simple stuff is all done with templates. I mean in 2003, you even had to program forward and backward buttons, now those are preset along with many other buttons like Home, Help, etc.

How good a yardstick is the time to develop one hour?

Karl: Using one hour is surprisingly controversial. People say “what is one hour of instruction” or that “one hour of instruction is a mythical measurement” We define one hour as the “amount of time it would take an average person to progress through the instruction moving at a comfortable pace.” How accurate it is, I am not sure, but I do know that the industry needs a standard unit of measure and one hour is a good standard. It is like the metric system. You might not agree that the metric system is the best method for measuring but if we both used a meter stick, we’d both get the same measurement. We might not like metrics but the measurement is consistent and I think that is how the industry should view “one hour of instruction” it is not perfect but we can all use it to measure.

What’s the biggest impediment to the broad adoption of online training in your experience?

Karl: I think two things are 1) lack of familiarity. People that have never taken online learning tend to not like it. They’ve grown up learning in a classroom and don’t have the experience of online learning. Or 2) the first experience with online learning is horrible. Boring content, delivered with a lack of interactivity and motivation. They become soured on e-learning. We have the power to create good e-learning if we take the time to develop it correctly.

Doesn’t it seem as though we’re going backwards as an industry when we look at these results?

Karl: Actually, I hope it is a look forward. I hope it means that more careful consideration is going into the design and that the focus is on creating effective instruction and not just shortening the delivery time. Focus on time is a production aspect but not a learning aspect. I hope, but don’t know for sure, that the time used to develop one hour of instruction is focused on creating one good hour and not just cranking out what my friend Marc Rosenberg calls “Shovelware” which is just courses created quickly and with little thought to development. I sincerely hope people are spending more time on design.

Do you see anyone in the marketplace addressing these development times directly.

Karl: A number of vendors are working on rapid development tools but to me, the real time is spent in design and working with SMEs, not sure how you can speed up either of those two processes and, in fact, I see working with SMEs becoming more time consuming as they are less and less willing to give up there time because they are more and more overburdened.

Doesn’t this seem like an enormous opportunity for improvement and thus potential profit? Better mousetrap and all?

Karl: Yes and no. One thing I hope these numbers highlight is that good instructional design takes time. I think the focus on rapid development leads us down a dangerous path. Instead we should focus on good results. If I can create a learning piece in ten hours but no one learns from it, I’ve wasted ten hours. If it takes 20 hours but learning occurs, isn’t that worth it? So a focus solely on reducing the time it takes is, I think, counterproductive. We need to focus on effective instruction. Too often the need for speed trumps the actual learning. We need to be careful not to blindly focus on reducing development time at the expense of quality learning.

Finally, I’d like to ask your readers to weigh in on the survey. We have reopened the survey so that we can continue to obtain input from our peers on this topic. Take a few minutes and input your own ideas, thoughts and time frames concerning the development of one hour of training. Visit the survey here.