ADDIE, and the Systems Approach: a Critical Analysis

In yesterday's post I looked at a systems approach to instructional design and development, and suggested that this approach was an emergent property of the process whereby instructor-led courseware developers in the 1990's attempted to apply previously-effective training development techniques to the new domain of e-learning.

As an example, I looked at the ADDIE design appraoch and suggested that it could be characterized as a set of heuristics (or conceptual framework) to enable learning professionals to output content commensurate with the systems approach.

Now read on...

In this context, we can say that ADDIE will only deliver results consistent with the expertise of the content designer, as it relies on their (the designers') skills and experience in transferring information and knowledge to "deliver" learning to the recipient. In E-Learning Strategies: how to get implementation and delivery right first time, Don Morrison remarks that

...like many ISD models it fails to give clear guidance about what makes good learning content. You can follow ADDIE to the letter and still deliver mediocre content.

(2004, p.255)

It can be said that ADDIE (and similar systems' approaches to learning content development) are closer in philosophy and practice to a project management methodology than a pedagogy. In this respect they can be a useful approach when developing "one-shot" training content in environments where repetitive worker activities are the norm.

Take this scenario as an example:

1. Production workers have just started using the new WidgetMaker 300-0 tool: it can create 3o widgets per hour.
2. As it's operating principles are very similar to the the older WM 299-8 (20 widgets per hour) model, the organization decides that workers only need a 2-day familiarization course before beginning production with the new tool.
3. A new production target (30 WPH) is set for workers to reflect the increased efficiency of the tool.
4. After three months, production is still 5WPH short of target. It turns out that workers are using the old production process on the new machine, and with the expectation of attaining 30WPH are actually working harder than ever, but are still 50% short of the difference between the new and old hourly production rate. Much disgruntlement all round.
5. It seems that there's an extra gadget on the handle of the WM300-0 that nobody noticed because of the cursory nature of the 2-day familiarization course.
   
6. Time for some training...

Table 1 Systems Approach Methodology applied to learning intervention

ISD Process Phase	Systems Inputs
	People
Analyze	Organization sees deficiency in workers skill-base or competency set (causes lower productivity from the new widget-making tool, the WidgetMaker 300-0, than had been planned)
	Material
Design / Develop	Two-day delta training developed
	Technology
Implement	Training on new process delivered to learners
	Time
Implement	ILT / WBT course delivered (Learners taken out of productive work to take training)
Evaluate	Workers complete certification on tool
	Outputs
Outputs	Workers' productivity increased	Delta training integrated into updated operators' training program

As this example illustrates, treating learning like a project leads to "training outcomes" equivalent to project deliverables. While these training deliverables may have value in and of themselves, they have limited value for workers in the longer term. We could even say that the outputs are valid; workers have increased productivity and the next training course will have extra content to address the issue of lack of competency on the WidgetMaker 300-0. The efficiency of the system has been improved, and future trainees on the WidgetMaker 300-0 will not experience the productivity "hit" experienced by their colleagues. From a project perspective, it makes sense to fold the delta training into the course for the tool.

• There are a number of negative consequence associated this approach
• The delta training no longer exists as a discrete knowledge asset for the organization
• Over time, specific knowledge of that particular learning intervention will be lost, half hidden among the overall learning goals of the WidgetMaker 300-0. Useless in effect, should another learner require remediation on that particular aspect of the tool
• Without this discrete knowledge resource at workers' disposal, the only way to remediate their specific deficiencies on the tool is to require them to re-take the complete How to use the WidgetMaker 300-0 course. Again.

Our field of educational communications is founded on the premise that communicating content to students will result in learning. In educational communications, information or intelligence (in many different forms) is encoded visually or verbally in the symbols systems employed by each technology. During the "instructional" process, learners perceive the messages encoded in the medium and sometimes "interact" with the technology. Interaction is normally operationalized in terms of student input to the technology, which triggers some form of answer judging and response from the technology in the form of some previously encoded (canned) message. Technologies as conveyors of information have been used for centuries to "teach" students by presenting prescribed information to them which they are obligated to "learn."

(Jonassen, 2001)

Rather, Jonassen argues technology (and in particular so-called Read/Write Web technology) should enable learners to develop their skills in a constructivist learning environment (or CLE).

As a quick reminder, here's the Cliff Notes version of Constructivism:

• We construct our own understanding of the world we live in
• Knowledge not received from outside, but by fitting new information together with what we already know we construct knowledge in our head
• Learning is the process of adjusting our mental models to accommodate new experiences
• Constructivist theorists support that people learn best when they actively construct their own understanding

With this in mind (no pun intended. Well... maybe), we'll be looking at CLEs tomorrow.

____________________________

References:

Exclusive Interview with Professor David Jonassen (2001) IN: elearningpost [Internet] Available from: http://www.elearningpost.com/articles/archives/
exclusive_interview_with_professor_david_jonassen [Accessed 12th January 2007]

Morrison, D. (2004) E-Learning Strategies: how to get implementation and delivery right first time Chichester: John Wiley & Sons, Ltd.

--

E-learning ecosystems and the failure of ADDIE

Onwards with our investigation of the e-learning ecosystem and learning & development in organizations. Today's post looks at the similarly-sounding but fundamentally different Systems Approach to instructional design and learning, and why the doubtful provenance of ADDIE combined with a misunderstanding of the role of content delivery channels have failed a generation of e-learners.

Now read on...

I have suggested that a growing number of researchers and learning professionals (Dillon & Hallett, Jonassen, Rosenberg etc) are recognizing the negative affect of what Brown, Collins, & Duguid (1989) call the "breach between learning and use" (p.1) of training interventions in organizations. We have seen that both instructor-led and 'traditional' self-paced learning courseware are relatively limited in their application, and are most effective in a restricted range of circumstances. Brown, Collins, & Duguid elaborate:

Many teaching practices implicitly assume that conceptual knowledge can be abstracted from the situations in which it is learned and used. [The authors] argue that this assumption inevitably limits the effectiveness of such practices. Drawing on recent research into cognition as it is manifest in everyday activity, the authors argue that knowledge is situated, being in part a product of the activity, context, and culture in which it is developed and used.

(p.1)

To meet these conditions, we can say that it is appropriate to adapt both pedagogical approaches and the technology resources now available to enhance learners performance.

Marc J. Rosenberg concurs:

Although e-learning began as a new way to deliver training, it cannot remain that way because it is no longer able to adequately support all the learning needs of individuals and organizations by itself - if it every was. E-learning has moved in a new, somewhat unanticipated direction that is not always reminiscent of an instructional framework. To be more influential, e-learning must be reinvented. While continuing to provide a viable instructional option in a formal learning setting, it must also move toward informational and collaborative solutions that focus more prominently on the specific jobs people do. It must move beyond courseware and classrooms and into work. To reinvent e-learning is, in many ways, to reinvent learning itself.

In essence, this means transforming workplace learning, so that learning activities and resources are situated around the learner, their work environment, and their tasks, enabling learners to construct their own knowledge in the context of what resources they need to carry out their work effectively. As David Jonassen says:

In constructivist learning environments, technologies are used to situate learning tasks in a variety of contexts. With video, very rich and engaging contexts can be created.

He asserts that in the traditional organizational approach that

[u]nfortunately, most e-learning replicates the worst features of face-to-face instruction. So, it may be cheaper to "deliver" knowledge over the Internet, but it will not be more effective.

This is commonly known as the systems-based approach to instructional design & development (ISD). A system is a set of elements or components that must integrate to perform a specific function. Every job in an organization is used by the organizational ecosystem to produce a product or output. The product or output is the means by which a organization generates its assets and remains self-supporting.

There are four inputs necessary in every system to produce a product or output:

• People: The workers making up a group and linked by a common activity.
• Material: The raw products which go into the system.
• Technology: The technique for achieving a practical purpose or goal.
• Time: The measured period during which an action or process begins and ends.

In learning and development, this systems-based approach is epitomized by the ADDIE conceptual framework (see Figure 1), most notably refined by Dick & Carey in The Systematic Design of Instruction (1996).

Figure 1. The ADDIE ISD Heuristic

The ADDIE approach has been one of the core tenets of instructional design for the best part of two decades, but curiously, it may not exist! In his article In Search of the Elusive ADDIE Model (2003), Michael Molenda undertook a Livingstonian attempt to discover the source for the original reference to the ADDIE model. Molenda’s research uncovered no original reference for the ADDIE model. This lack of an original reference led Molenda to write,

I am satisfied at this point to conclude that the ADDIE Model is merely a colloquial term used to describe a systematic approach to instructional development, virtually synonymous with instructional systems development (ISD). The label seems not to have a single author, but rather to have evolved informally through oral tradition. There is no original, fully elaborated model, just an umbrella term that refers to a family of models that share a common underlying structure.

(p.1)

Hence my suggestion above that ADDIE is more properly labeled a conceptual framework; I would go so far as to say that it could even more accurately be described as a set of heuristics or "rules of thumb" to enable instructional designers from a instructor-led tradition to develop learning content by adhering to the precepts of the systems approach (see Table 1).

Table 1 The ADDIE Model

Phase	Activity
Analysis	During analysis, the designer identifies the learning problem, the goals and objectives, the audience’s needs, existing knowledge, and any other relevant characteristics. Analysis also considers the learning environment, any constraints, the delivery options, and the timeline for the project.
Design	A systematic process of specifying learning objectives. Detailed storyboards and prototypes are often made, and the look and feel, graphic design, user-interface and content is determined at the Design stage.
Development	The actual creation (production) of the content and learning materials based on specifications instantiated during the Design phase.
Implementation	During implementation, the plan is put into action and a procedure for training the learner and teacher is developed. Materials are delivered or distributed to the student group. After delivery, the effectiveness of the training materials is evaluated.
Evaluation	This phase consists of (1) formative and (2) summative evaluation. 1. Formative evaluation is present in each stage of the ADDIE process. 2. Summative evaluation consists of tests designed for criterion-related referenced items and providing opportunities for feedback from the users. Once they have undergone formative and summative evaluation, learners are encouraged to review and revise the courseware as necessary, until they have successfully passed the proscribed tests

The transactional nature of the systems approach assumes that the very act of communicating information to the supplicant (sorry, learner) results in the output of "learning."

Adhering to this content development methodology, instructors consider that the channels used to "deliver" knowledge are cognitively neutral and merely replicate and modalities of the classroom - voice-over narration equating to instructor explanation, the screen being equivalent to the overhead projector or blackboard, computer-mediated interaction being the essentially the same as teacher-student interaction, and so on.

By balancing the cognitive load across the learning modalities, it is supposed that knowledge can be effectively delivered, and the student will "learn." A corollory of this is that the learning delivery channels (visual, audial, text-based, and so forth) themselves have no affect upon the learner's interpretation of the content, so to all intents and purposes, the "sage on the stage" transforms into the "guide by the side."

_____________________

References:

Brown, J. S. Collins, C. & Duguid, P. (1989) Situated Cognition and the Culture of Learning [Internet] Educational Researcher 18(1), pp. 32-42, Jan-Feb 1989. Available from: http://tiger.coe.missouri.edu/%7Ejonassen/courses/CLE/index.html [Accessed January 12th 2007]

Dick, W. & Carey, L. (1996). The Systematic Design of Instruction (4th Ed.). New York: Harper Collins College Publishers.

Dillon, P. & Hallett, C. (2001, October). Powering the leap to maturity: The eLearning ecosystem. Cisco Systems white paper.

Exclusive Interview with Professor David Jonassen (2001) IN: elearningpost [Internet] Available from: http://www.elearningpost.com/articles/archives/
exclusive_interview_with_professor_david_jonassen [Accessed 12th January 2007]

Molenda, M. (2003). In Search of the Elusive ADDIE Model. [Internet] Available from: http://www.indiana.edu/~molpage/
In%20Search%20of%20Elusive%20ADDIE.pdf Accessed 12 May 2008

Rosenberg, M.J. (2001) What Lies Beyond E-Learning? learningcircuits.org e-zine [Internet] Available from: http://www.learningcircuits.org/2006/March/rosenberg.htm Accessed 14th April 2007

--

E-Learning along the curve: adapting to knowledge workers learning needs in organizations

As I discussed in the previous blog entry, formal, structured approaches to learning (such as to both instructor-led and traditional CBT-type training interventions), which by their nature are long in duration, relatively generalized in terms of subject matter, and are best deployed to novices in the relevant discipline or skill area. As such, they represent the best value for organizations when to knowledge worker is not expected to be a full contributor in their role. In this context, I would suggest that organizations can justify using these learning solutions for situations such as

• entry-level or new hire orientation and competency building
• internal transfers to a new discipline (i.e. the worker moves from Production to QA)
• retraining on a new production system (i.e. a new type of widget-making tool or a new software system)
• employee career advancement (i.e. from individual contributor to manager)

Psychology tells us that the learning curve obeys what is called a power law (Ritter & Scholler, 2002).

As such they are often said to conform to "the power law of practice". Cognitive psychology has shown that the power law of practice is ubiquitous, and cognitive modeling has explained both the general speedup and variability in performance.

(The Learning Curve, p.2)

So, as a worker learns a task, skill, or process ("progresses along the learning curve"), their competency improves and productivity increases with some variations but broadly within generally accepted parameters.

As workers advance from their neophyte status, they begin to attain what Marc J. Rosenberg (2006) calls "performer" status; the are transitioning from being Novice to Competent, along a path that will enable them to become Experienced, until they achieve Expert (or Master) status.

We can also say that once a worker reaches a certain level of competence, their learning needs are met less by generic courses and curricula, and more by specific, even personalized, learning interventions such as task/skill practise and coaching, access to knowledge and performance resources, and collaboration and problem solving (see Figure 1).

Figure 1. Levels of mastery and appropriate learning strategies (after Marc J. Rosenberg, 2006)

As an extension of this (and effectively demonstrated by Rosenberg), organizations failing to move beyond the classroom or traditional CBT-type courseware for their ongoing learning and development needs, are probably impeding the development of their workers, as well as negatively affecting their (the organization's) own potential (see Figure 2).

Figure 2. Advantages of workflow learning (after Marc J. Rosenberg, 2006)

In order to provide effective learning and performance support to workers after they become competent, organizations must strive to develop their workers' skills as employees undertake their regular workplace activities. It is my view that this level of performance support can only be provided through access to networked knowledge assets.

And that is what I'll be discussing tomorrow.
______________________

References:

Dillon, P. & Hallett, C. (2001, October). Powering the leap to maturity: The eLearning ecosystem. Cisco Systems white paper.

Ritter, F. E., & Schooler, L. J. (2002). The learning curve. In International encyclopedia of the social and behavioral sciences. 8602-8605. Amsterdam: Pergamon.
[Internet] Available from: http://www.iesbs.com/ Accessed 27 May 2008.

Rosenberg, M. J. (2006) Beyond e-Learning. San Francisco, CA: John Wiley & Sons, Inc.

--

Learning Evaluation and Strategy: Using an e-learning readiness survey

During this series of posts on evaluating non-formal learning programs, I have mentioned carrying out an e-learning readiness survey without characterizing or discussing how to implement such a research instrument.

This was deliberate; in my view e-learning readiness surveys represent an alpha and an omega of evaluation: on one level they are the starting point for any evaluation of an organization's learning initiative, on another level they define an organization's ability to implement an effective e-learning strategy. As such, they are a bridge between the theory and the practice of implementing a learning program.

Readiness surveys enable the learning practitioner to understand and measure both the effectiveness of organizational learning and to identify the critical factors for success when developing learning programs.

I have found Marc J. Rosenberg's E-Learning Readiness Survey to be a very effective instrument to evaluate both the effectiveness of an organization's learning strategy, and one of the foundations of any serious evaluation of the effectiveness of learning programs.

The questions are grouped into seven areas of understanding:

1. business readiness
2. the changing nature of learning and e-learning
3. value of instructional and information design
4. change management
5. reinventing the training organization
6. the e-learning industry
7. your personal commitment

The questions provided in this survey represent some of the most important strategic issues organizations face when transitioning to e learning. Certainly there are additional questions and issues that deserve attention; add your own, organization-specific items as required.

Downloads:

Marc J. Rosenberg's E-learning Readiness Survey

References:

Rosenberg, M. J. (2006) Beyond e-Learning. San Francisco, CA: John Wiley & Sons, Inc.
--

Measuring the learning effect of non-formal learning programs

I have suggested in a previous blog entry that non-formal learning (NFL) is located on a continuum between so-called informal and formal learning (see Figure 1)

Figure 1 the Learning Continuum

and that learning is situated in a larger socio-cultural context - a cognitive web - that provides the knowledge-centric framework within which learners operate (see Figure 1.2).


Figure 1.2 Knowledge-centric View of Knowledge (after Marc J. Rosenberg)

Similarly, NFL shares characteristics with both informal and formal learning, but is differentiated enough from both of these approaches to merit it's own category (see Table 1)

Table 1 Definition of learning types

Learning Type	Description
Formal Learning	Learning typically provided by an education or training institution, structured (in terms of learning objectives, learning time or learning support) and leading to certification. Formal learning is intentional from the learner’s perspective [my italics].
Non-formal Learning	Learning that is not provided by an education or training institution and typically does not lead to formalised certification. It is, however, structured (in terms of learning objectives, learning time or learning support). Non-formal learning is intentional from the learner’s perspective [my italics].
Informal Learning	Learning resulting from daily life activities related to work, family or leisure. It is not structured (in terms of learning objectives, learning time or learning support) and typically does not lead to certification. Informal learning may be intentional but in most cases it is non-intentional (or “incidental”/ random) [my italics].

Given the non-formal nature of the learning (and by extension a paucity of the type of assessment metrics available when evaluating the efficacy of formal training programmes) I am going to explore the challenge of assessing non-formal learning in the context of Kirkpatrick's 4-level model, particularly between Level 1 (Reaction) and Level 3 (Behaviour); as Kirkpatrick & Kirkpatrick (2006, p.22) assert “learning has taken place …when one or more of the following occurs: Attitudes (sic) are changed. Knowledge is increased. Skill is improved. One or more of these changes must take place if a change in behaviour is to occur.”

More tomorrow...


References:

Kirkpatrick, D. & Kirkpatrick, P. (2006) Evaluating Training Programs. 3rd ed. San Francisco, CA: Berrett-Koehler Publishers, Inc.

Rogers, A. (2004) Looking again at non-formal and informal education - towards a new paradigm [Internet] Available from: http://www.infed.org/biblio/non_formal_paradigm.htm [Accessed 30th January 2008]

Rosenberg, M. J. (2001) e-Learning: Strategies for Delivering Knowledge in the Digital Age London: McGraw-Hill.


See Also:

Holford, J. Patulny, R. & Sturgis, P. (2006) Indicators of Non-formal & Informal Educational Contributions to Active Citizenship [Internet] Available from: http://crell.jrc.ec.europa.eu/active_citizenship.htm [Accessed 24th February 2008]

--