New York trip

Unfortunately no time to post on e-learning today: I'm off on my holidays for a week - going to New York for a week of rest and relaxation.

More on evaluating NFL using Kirkpatrick's Four-level model

I'm going to begin this post by mentioning Donald Clark's view on Donald Kirkpatrick's model - the post has the tongue-in-cheek title Donald Talks Bollocks, and it gives an alternative view on the four levels; have a read, it's good stuff.

However, as Mr. Clark points out, rejecting Kirkpatrick doesn't provide a answer to how else we should evaluate the effectiveness of learning (particularly non-formal learning) so in lieu of another approach I will carry on discussing how I think Kirkpatrick can be used to assess the effectiveness of NFL initiatives.

Pay attention, here comes the science bit...

I've already discussed why I think Kirkpatrick's guidelines for evaluating learning is incompatible with the assessment of non-formal learning (particularly in the context of NFL), and today I'll discuss how you can begin to counter these deficiencies. Like Steven Kerr, CLO at Goldman Sachs, Kirkpatrick Level 3 is the most interesting to me; in the context of NFL, Level 2 cannot be used as an instrument for evaluation: with no summative assessment at the end of an NFL session, you cannot evaluate knowledge transfer by traditional testing methods.

To understand why NFL cannot be assessed at Level 2, look at what Michael Eraut says in Non-formal learning, implicit learning and tacit knowledge (2000). Eraut makes a clear distinction between his interpretation of the term ‘non-formal learning’ and what others including Scribner (1988), Conner (2002) and Cross (2003) would call ‘informal learning’ – what Eraut terms ‘incidental learning’ or ‘the acquisition of knowledge independently of conscious attempts to learn and the absence of explicit knowledge about what was learned’ (Reber, 1993, quoted by Eraut 2000, p.12)

This is, he argues, because most workplace learning takes place outside formal learning contexts, and informal learning carries with it connotations of

so many other features of a situation, such as dress, discourse, behaviour, diminution of social differences – that its colloquial application as a descriptor of learning contexts may have little to do with learning per se.

(2000, p.12).

Not only does the term carry unwanted and confusing implications, but it is too vague to be of any real utility. For Eraut, an analysis of learning must focus on activities and the outcomes that that contribute to significant changes in capability or understanding. In a sense, Eraut does not define non-formal learning; rather, he defines the characteristics of formal learning (p.12) as:

1. A prescribed learning framework
2. An organised learning event or package
3. The presence of a designated teacher or trainer
4. The award of a qualification or credit
5. The external specification of outcomes.

The implication of this categorisation is that any learning that does not exhibit all of these characteristics should be classed as either informal or non-formal learning. Some reviewers (Colley, Hodkinson & Malcolm, 2002) make the point that Eraut does not make clear what the status is of learning in situations that meet some, but not all, of his ‘formal’ criteria. My interpretation of his characterisation is that the very nature of a formal activity - “following or according with established form, custom, or rule” (Merriam-Webster Online, 2007) validates Eraut’s description.

I am particularly interested in exploring the relationship between Level 1 (Reaction) and Level 3 (Behaviour); as Kirkpatrick & Kirkpatrick (2006, p.22) assert “learning has taken place …when one or more of the following occurs: Attitudes (sic) are changed. Knowledge is increased. Skill is improved. One or more of these changes must take place if a change in behaviour is to occur.” In this context, I believe a mixed method of quantitative data-gathering and a more naturalistic or qualitative information collection approaches can be employed to capture data that will assist in evaluating the success of a non-formal learning initiative.

And it is this approach that I will be discussing tomorrow.

References:

Colley, H. Hodkinson, P. Malcolm, J. (2002) non-formal learning: mapping the conceptual terrain. a consultation report [Internet] Available from: http://www.infed.org/archives/e-texts/colley_informal_learning.htm [Accessed 28th January 2008]

Conner, M. (2002) Informal Learning: more than “What we do between classes”. [Internet] Available from: http://www.productivity-solutions.net/documents/
informallearning-mconner-071202.pdf [Accessed 19th October, 2006]

Cross, J. (2004) An informal history of eLearning. On the Horizon [Internet] 12(3). pp.103-110. Available from: http://www.emeraldinsight.com/Insight/viewPDF.jsp?
Filename=html/Output/Published/EmeraldFullTextArticle/
Pdf/2740120301.pdf [Accessed 20th February, 2007]

Kirkpatrick, D. & Kirkpatrick, P. (2006) Evaluating Training Programs. 3rd ed. San Francisco, CA: Berrett-Koehler Publishers, Inc.

Scribner, S. (1988) Mental and manual work: An activity theory orientation. In E. Tobah, R. J. Falmagne, M. B. Parlee, L. M. Martin & A. S. Kapelman (Eds.), Mind and Social Practice: Selected Writings of Sylvia Scribner. (pp 367-374).Cambridge: CUP

Whitney, K. (2004) Steve Kerr: Managing the Business of Learning [Internet] Available from: http://www.clomedia.com/content/
templates/clo_cloprofile.asp?articleid=582&zoneid=4 [Accessed 28th February 2008]

--

E-Learning and economic downturns...

I'm getting some interesting feedback from a variety of sources on my blog post Recession and the challenge to e-learning. Steve McKenzie over at Eduspaces has commented that he considers e-learning to be a "recession buster ...on balance costs for institutions and individuals can be saved."

I completely agree with Steve's assessment; the challenge as I see it, is for e-learning practitioners to address traditional/institutional ways of thinking in organizations about how to leverage the benefits of e-learning - break the habits of a lifetime in a sense.

Most decision makers (i.e. C-level executives) in organizations are from a generation that would not have extensively used e-learning during their formative years. In my experience, they become much more risk-averse during economic downturns and push back on what they see as innovation in tough times.

I'm reminded of a conversation I had with a VP about two years ago. I was working on a proposal to introduce an on-demand pre-classroom training module for a ILT course - a bit of blended learning to reduce the workload on instructors to ensure that learners were of an appropriate skill level to actually attend the course in question. His response to the proposal was along the lines of "it's all very well being leading edge, but we don't have to be bleeding edge."

Now, this was hardly an all-singing, all-dancing 3D PLE we were proposing here, just a little 30-minute Flash-based presentation. I think that the assertion this VP made represents that we, as learning professionals, have to be cognisant of more conservative attitudes that exist in the workplace, and we need to develop strategies to counter such lines of argument.

--

Evaluating non-formal learning in the context of the Four-Level Model

Donald L Kirkpatrick first published his ideas on evaluating learning in 1959 in a series of articles in the US Training and Development Journal. The articles were subsequently included in his book Evaluating Training Programs (originally published in 1975; I have the 2006 edition).

In this text he outlined and further developed his theories on evaluating culminating in the Four-Level Model, arguably the most widely used and popular approach for the evaluation of training and learning. Kirkpatrick's Four-level model is now considered an industry standard across the HR and training communities (see Table 1.1).

Table 1.1 Kirkpatrick's Four-level Model

Kirkpatrick’s Model	Learning Effect
Level 1: Reactions.	Evaluate participants' satisfaction with the learning intervention.
Level 2: Learning and Level 3: Behaviour.	What do participants know they didn't know before? How are they using knowledge in their jobs? What is the learning and performance effect of the intervention?
Level 4: Organisation-level benefits.	Has the development of higher levels of domain knowledge improved organisational productivity?

According to the model, evaluation should always begin with Level One, and as time and budget allows, should move sequentially through Levels Two, Three, and Four. Information from each prior level serves as a base for the next level's evaluation. Each successive level represents a more precise measure of the effectiveness of the training program, however, each level also requires a more rigorous and time-consuming analysis. As we shall see, the characteristics of NFL mean that not all of the four levels can be applied to the evaluating of this approach to learning.

In Chapter 5 of Evaluating Training Programs (2006) Kirkpatrick and Kirkpatrick stress the significance of measuring learning, because “no change in behaviour can be expected unless one or more of the learning objectives have been accomplished” (p.42). They set out “helpful” guidelines for the measurement of learning:

Table 1.2 Guidelines for evaluating learning

Guidelines for evaluating learning

Use a control group if practical

Evaluate knowledge skills and/or attitudes both before and after the program

Use a paper-and-pencil test to measure knowledge and attitudes

Use a performance test to measure skills

Get a 100 per cent response

Use the results of the evaluation to take appropriate action

This approach is incompatible with the assessment of non-formal learning (particularly in the context of e-learning) in a number of ways:

1. The singular nature of each non-formal learning event means that no control group can exist to measure a difference between it and the “experimental group” (p.43).
2. Similarly; a significant number of the learners that access NFL do so asynchronously, via a number of different learning channels and over a broad time span in a relatively ad hoc manner. This “just enough, just in time” aspect of non-formal learning is one of its strengths, but it was unrealistic to attempt to measure learning using an experimental method in such an environment.
   
3. The lack of summative assessment in non-formal learning precludes both pencil-and-paper and performance testing to measure learning.
   
4. The distributed nature of access to these NFL interventions, over both time and location makes getting a 100 per cent learner response rate practically impossible.

I'll be discussing approaches to overcoming these challenges in my next posting.


References:

Kirkpatrick, D. & Kirkpatrick, P. (2006) Evaluating Training Programs. 3rd ed. San Francisco, CA: Berrett-Koehler Publishers, Inc.

--

Measuring the learning effect of non-formal learning programs

I have suggested in a previous blog entry that non-formal learning (NFL) is located on a continuum between so-called informal and formal learning (see Figure 1)

Figure 1 the Learning Continuum

and that learning is situated in a larger socio-cultural context - a cognitive web - that provides the knowledge-centric framework within which learners operate (see Figure 1.2).


Figure 1.2 Knowledge-centric View of Knowledge (after Marc J. Rosenberg)

Similarly, NFL shares characteristics with both informal and formal learning, but is differentiated enough from both of these approaches to merit it's own category (see Table 1)

Table 1 Definition of learning types

Learning Type	Description
Formal Learning	Learning typically provided by an education or training institution, structured (in terms of learning objectives, learning time or learning support) and leading to certification. Formal learning is intentional from the learner’s perspective [my italics].
Non-formal Learning	Learning that is not provided by an education or training institution and typically does not lead to formalised certification. It is, however, structured (in terms of learning objectives, learning time or learning support). Non-formal learning is intentional from the learner’s perspective [my italics].
Informal Learning	Learning resulting from daily life activities related to work, family or leisure. It is not structured (in terms of learning objectives, learning time or learning support) and typically does not lead to certification. Informal learning may be intentional but in most cases it is non-intentional (or “incidental”/ random) [my italics].

Given the non-formal nature of the learning (and by extension a paucity of the type of assessment metrics available when evaluating the efficacy of formal training programmes) I am going to explore the challenge of assessing non-formal learning in the context of Kirkpatrick's 4-level model, particularly between Level 1 (Reaction) and Level 3 (Behaviour); as Kirkpatrick & Kirkpatrick (2006, p.22) assert “learning has taken place …when one or more of the following occurs: Attitudes (sic) are changed. Knowledge is increased. Skill is improved. One or more of these changes must take place if a change in behaviour is to occur.”

More tomorrow...


References:

Kirkpatrick, D. & Kirkpatrick, P. (2006) Evaluating Training Programs. 3rd ed. San Francisco, CA: Berrett-Koehler Publishers, Inc.

Rogers, A. (2004) Looking again at non-formal and informal education - towards a new paradigm [Internet] Available from: http://www.infed.org/biblio/non_formal_paradigm.htm [Accessed 30th January 2008]

Rosenberg, M. J. (2001) e-Learning: Strategies for Delivering Knowledge in the Digital Age London: McGraw-Hill.


See Also:

Holford, J. Patulny, R. & Sturgis, P. (2006) Indicators of Non-formal & Informal Educational Contributions to Active Citizenship [Internet] Available from: http://crell.jrc.ec.europa.eu/active_citizenship.htm [Accessed 24th February 2008]

--

Thoughts on Evaluating Non-Formal Learning

In previous posts, I've suggested that knowledge workers under time-pressure and with high demands on their skills are motivated to continually educate themselves on new topics. In small- to mid-sized enterprises, for example, workers are required to learn new skills, behaviours and attitudes in the context of their functional tasks. As they do not necessarily have the time to take formal training courses, they take advantage of the range of knowledge resources at their disposal, ranging from searching the Web, corporate knowledge bases, and other information repositories, as well as discovering information incidentally through social interaction with credible colleagues to assist their ability to “frame the problem” (Argyris & Schön, 1974).

As organisations are at root, cognitive enterprises, and the sum of the knowledge of the organisation is expressed collectively through the behaviour, skills and attitudes of its employees, the organisation is itself a cognitive entity. One of the ways that such an entity continually constructs itself is through the learning processes its members engage in.

Situated, experiential learning, delivered:

1. synchronously in a social setting where workers can engage in a dialogue with More Knowledgeable Others, subject matter experts and their peers,
   
   or,
   
   
2. asynchronously accessed when the individual learner has a requirement to acquire information and knowledge relevant to a task at hand

in relevant, context-specific formats, whether in a classroom, live online, on-demand over the web, or via an easily accessible format such as a DVD-ROM to suit a range of learning styles and environments, creates the conditions for “meaningful learning” (Jonassen, 2001) to occur.

Providing workers with a knowledge-centric learning and performance architecture, and structured, goal-oriented content, gives them the motivation or intention to learn, enables them to process “raw” information into actionable knowledge on an ongoing basis, and regularly inhabit a scaffolded environment which encourages the learner’s development. Digitally-based technologies are the optimal mediator for this learning process, which is called ‘non-formal’ in recognition of its flexibility and accessibility for learners, and also because of its pedagogical structure and outcomes-focus.

Over the next week or so, I'm going to look at ways to quantify the learning effect of non-formal learning its target audience beginning with an overview of that stalwart of evaluation, Donald Kirkpatrick.

References:

Argyris, C. and Schön, D. (1974). Theory in practice: Increasing professional effectiveness, San Francisco: Jossey-Bass.

Jonassen, D. H. (1996) Computers in the Classroom: Mindtools for Critical Thinking. Upper Saddle River, NJ. Prentice Hall Inc.

--

A recurring theme: Recession in E-learning

It seems that I touched the zeitgeist last week when I posted on the looming recession in the US economy. Here are links to some of the more interesting posts others in the industry have been saying on the subject:

• Training for a Recession - Allen's Learning Blog
• US Economy and e-Learning in 2008 - G-Cube Solutions
• Classrooms 2.0; Learning in a Recession? - Elliott Masie

Brent Verhaaren on the Allen Learning Blog outlines some very helpful tips for training departments to anticipate and (hopefully) negate the the impact reduced revenue on them, including:

• Do the Numbers
• Talk to the Right People
• Tighten Your Belt
• Be Consistent
• Look to Past Recessions

And most importantly there No Time Like the Present to evaluate the potential affect a downturn on your organisation.

Finally, another e-learning blog from Ireland...

Despite the fact that Ireland is home to more commercial e-learning organisations per capita than anywhere else on the planet, there are very few blogs representing the views of the Irish e-learning community being maintained. More about that directly, but first...

Here's a quick look back at the the contribution of Irish-owned e-learning companies to the industry:

Back in the 1980's "Computer-Based Training" was the coming thing, and Irish-based CBT Group (CBT Systems in North America) were one of the originators of the industry. They specialised in self-study courses that users could take at their own PCs. CBT developed a library of more than 600 titles for client/server, Internet, intranet, and mainframe platform deployment.

By the mid-to-late 1990's, there was an explosion of Irish entrepreneurs exploiting gaps in the on-line learning market:

• Prometheus developed the "CompuTutor" range of products for the MOUS and MSCE audience.
• Transware PLC founded to meet the localisation needs of e-learning organisations.
  
• KnowledgeWell and Riverdeep begain making inroads into the K-12 sectors.
• Electric Paper began development of the first courseware for the ECDL qualification.
• EMG developed the skoool.ie initiative in association with The Irish Times and Intel (Ireland) Ltd.

By 1999, the CBT Group had transformed itself into SmartForce, "the e-learning company." Following convergence in the marketplace after the dot-com crash, a new raft of e-learning organisations emerged:

• EP, Creative Learning Media, A-V Edge and MindLeaders have all merged into the ever-expanding ThirdForce.
• In 2002, SmartForce merged with another virtual classroom leader, Ireland-based SkillSoft Corporation.
• Also in 2002, Barry O'Callaghan led an MBO of Riverdeep. Since going private, the organisation has acquired the assets of The Learning Company, Broderbund, Edmark, and most recently (2007) merged with Houghton Mifflin, to become the fastest-growing e-learning consultant in the US.

In 2008, for Ireland to hold on to it's hard-earned status as a global e-learning leader, it must maintain the momentum that's been built up over the past 20 years.

Certainly some signs are good: companies like Intuition, InnerWorkings and CompuPharma continue to innovate in the fields of finance, adaptive learning environments for programmers , and pharmaceutical training respectively. The National College of Ireland and the Royal College of Surgeons Ireland in particular are leading some very interesting initiatives in adult learning and post-graduate skills development. Specialities in adaptive learning and other third-generation e-learning solutions are being investigated in Trinity College, Dublin and the University of Limerick.

However, unlike our neighbours in the UK (who's government-backed education policy has the mantra "Education, education, education"), we in Ireland have a government who's lack of vision concerning this sector is downright embarrassing, if not ignorant.

A salutary lesson might illustrate my point at this stage of the post.

We are all agreed that e-learning is about channelling digitally-mediated content via networks like the internet. Yes? Therefore, without an effective high-speed network in place the development and delivery of high-quality learning products will be hindered.

In Ireland, our broadband is a mess. It's been said that we are a first-world country with third-world comms infrastructure. In fact it's worse than that; the Eamon Ryan, the minister whose responsibility it is to develop the comms infrastructure doesn't know the difference between high-speed internet availability and quality of service - as an example, here's an excerpt from a recent Sunday Times article:

John McElligott, the managing director of eBay in Ireland, has called on the minister to take a “quantum leap” by investing in NGNs, bringing Irish broadband speeds closer to the 100 megabits per second (Mbits/s) enjoyed in some countries. Irish download speeds for residential broadband users are a maximum of 12 Mbit/s. Most of the country operates on speeds of 1-2 Mbit/s. Ryan said McElligott was “right to be ambitious”, but questioned his thinking. “John runs eBay. When he starts running a construction company, he can advise me on digging holes to every home in the country,” Ryan said.

I would guess that if you're outside of Ireland, dear reader, your jaw is dropping at the supreme irony of a leading digital development hub and world-leading economy, and the steam-powered infrastructure we have to negotiate on a daily basis.

But that's not why we're here today. We're here today to welcome a new, Irish, e-learning blog to the blogosphere: welcome Brian Mulligan and the E-learning Ireland blog!

Click on the link under the E-learning Blogs panel to the right of this post to check out what his blog is about.

--

Recession and the challenge to e-learning - addendum

Coincidentally following yesterday's post, news that ThirdForce has warned that growth in its business this year will fall short of what it previously expected broke in Ireland this morning.
See the full report on Irish national broadcaster RTÉ.

Recession and the challenge to e-learning

A colleague in an area of business other than the learning industry asked me a few days ago; "Do you think that e-learning will survive a recession?"

My initial response to her was "there's no recession... yet!" Later, I reflected on the implications of the current economic slowdown (whether or not it turns into a recession) for the e-learning industry.

The dialectic I am presenting here can be summarised in the following manner: will the positive economic, organisational, and social value of e-learning outweigh traditional human responses to recessionary times? What strategies can we use to ensure the survival of and even the growth of e-learning as an industry in these changing times?

So what's changed in the e-learning industry since the last recession in 2001? I've outlined some discussion points below:

1. First of all: Traditional business practise
2. Developments in Infrastructure & Hardware
3. The E-learning Hype curve
4. Refinements in Content Development Methodologies
5. The Rise if the Read/Write Web
6. The Playstation Generation - Digital Natives in the workplace
   

Scroll down to find out more about each of these points.

--

Traditional business practise
Historically, when a slowdown or company rationalisation occurs, the first against the wall are the folks in the PR, marketing, and training departments. Typically, individuals and organisations revert to previously-learned behaviours in tough times; this usually means going through the process of carrying out tried-and-tested, though not necessarily logical responses to the problems put in front of them. The rationale is as follows:

• Profits are down
• Revenue projections aren't great for the next 12/18 months
• We need to cut our costs
• We need to keep the guys that make the widgets (we need to have product to sell)
• We need to keep the managers of these people (or productivity will go down)
• We need to keep Human Resources in place to manage everyone obviously (it's just a coincidence that I - that is the decision-maker - work in HR!)
• What about those training people? High travel costs for the ILT guys, they pull people out of work for 3 days to go on courses. Large budgets spent on implementing and maintaining an LMS/LCMS, third-party e-learning libraries, custom courseware etc, but they do seem to add value to the organisation. And let's face it, they don't really improve the quality of our product, because they never convince us with their ROI metrics...

• Outcome: tea and sympathetic chat, and the Training team get their pink slips / P45s.

Sadly, I reckon that this will be strategy undertaken by a significant number of organisations over the next year or so. However....

...and it's a big however.

Let's look at the evidence for factors that have changed in the e-learning industry since 2001 and the recession following the Dot-Com crash.

Infrastructure & Hardware
More-or-less general availability of high-speed internet access just wasn't there in 2001. To take the example of the company I worked for at the time, our high-quality courseware was developed in Authorware and Director and delivered to customers on CD-ROM for distribution via their intranet or accessed directly from the disk. Our on-line courseware was a 'lite' version of the CD material - not out of choice, but because of the limited functionality that could be provided to a user via a 56k connection.

Over-compressed images, poor animation, and very poor audio - hardly the immersive learning solution that e-learning flattered to promise at the time. Assuming the learner could access the content successfully, the chances were that the PC (for it was always a PC) that they were using to view their content was processing and displaying the date at a rate that we wouldn't find acceptable on a PDA now (screen-size excluded). Pentium or pre-Pentium processors, 8-bit sound cards, 16 colours, 800x600 pixel displays. And so on.

In short, we could see the potential, but our imaginations exceeded the available technology.

The e-learning hype curve
This brings me neatly to the e-learning hype curve (see Figure 1). Kevin Kruse described 2001 as the year that

...brought the harsh, steep slope of unfulfilled promises. Several high-profile providers shut their doors while many more announced large-scale layoffs in the face of missed revenue targets and crashing stock prices. E-learning advocates retreated to the more defensible ground of "blended learning. This year [went] down as the Trough of Despair.

Figure 1. The E-learning Hype Cycle

I would suggest through familiarisation and use, learners expectations are more reasonable about what can be achieved (and perhaps more importantly how it can be achieved through digitally mediated delivery). Given this environment, organisations are now more willing to invest in e-learning as part of their overall training strategy. But is it perceived as a necessity or a luxury?

I can't answer that question right now. I suspect that I will be able to answer it 12 months from now, because there will be evidence as to whether decision makers consider e-learning to be a core requirement that effectively meets organisations' training needs.

Content development methodologies
This kind of ties in with this month's Learning Circuits Big Question about instructional design. Without going into the history of this too much, the development of (relatively) easy-to-use authoring tools like Captivate, Articulate (and a whole raft of others), Rapid E-Learning development methodology and the disintermediation principle means that e-learning has fewer up-front costs associated with it than at the turn of the century.

Similarly, if it's done correctly at the outset, ongoing maintenance and support costs are lower than they ever have been. By developing content with smaller, more flexible teams, the value proposition of e-learning has been enhanced, and the total cost of ownership has been significantly reduced. Outside of e-learning, the take-up of podcasting and streamed media on sites like Blogger and YouTube demonstrates that this ease-of-use of tools and technologies has extended into the community at large.

The Read/Write Web
Who would have thought in 2000 that blogging, social networking, wikis and podcasts would be as big a part of life as they are now? At the start of the century, the Web (and e-learning) could at best be described as a half-duplex medium; it was pretty much all one-way traffic. The development of information platforms has facilitated knowledge-sharing, folksonomies, social interaction, and, key to all this, reciprocity.

We now live in a multiplex world of many voices and ideas, mediated by the internet. At the forefront in using these web technologies is the e-learning industry. By using these tools to develop content I feel we can demonstrate quite effectively that e-learning has a value now that it did not have a decade ago. I would assert that this is particularly true if you take a social-constructivist approach to learning. By the way, I'm happy to entertain debates about the role of formal as opposed to non-formal and informal learning in this environment.

The Playstation Generation
Concomitant with the read/write web is the Playstation generation that have grown up over the last number of years. In his seminal essay Digital Natives, Digital Immigrants (2001), Marc Prensky declares:

...students have changed radically. Today’s students are no longer the people our educational system was designed to teach.

Today’s students have not just changed incrementally from those of the past, nor simply changed their slang, clothes, body adornments, or styles, as has happened between generations previously. A really big discontinuity has taken place. One might even call it a “singularity” – an event which changes things so fundamentally that there is absolutely no going back. This so-called “singularity” is the arrival and rapid dissemination of digital technology in the last decades of the 20th century.

Today’s students – K through college – represent the first generations to grow up with this new technology. They have spent their entire lives surrounded by and using computers, videogames, digital music players, video cams, cell phones, and all the other toys and tools of the digital age. Today’s average college graduates have spent less than 5,000 hours of their lives reading, but over 10,000 hours playing video games (not to mention 20,000 hours watching TV). Computer games, email, the Internet, cell phones and instant messaging are integral parts of their lives.

Extending from this, there is a generation of workers who are comfortable with and practised in the concepts and use of e-learning - take a look at the resources available on www.skoool.ie, an initiative for second-level students in Ireland. I was involved in the development of the first iteration of this site, and it's changed a lot (for the better) since we took those first steps creating it 8 years ago. Similarly in third-level education, there have been significant developments in on-line learning, and I think that it's fair to say that it has become quite embedded in the pedagogy employed by universities: tools like Moodle enable students to upload coursework, take tests, build their own knowledgebases and wikis, and have on-line discussions through a single point of access.

This generation is in the workplace right now. It will expect to learn new skills as their careers develop using the tools that they have always learned on in the past: that is, by using e-learning.

Conclusion
I've left loads of stuff out here; this is a blog entry, not an essay. But consider other factors including transport costs and training in the era of $100 a barrel oil and the value of virtual classrooms; the ROI of e-learning as opposed to traditional methods; even the impact of traditional ways of teaching on the environment ("e-learning" becomes "eco-learning" anyone?).

I may well return to this topic again. But at this point I want to put the question out there: how do you think e-learning will fare in the next 18 months, particularly if there are tough times ahead?


References:

Kruse, K. (2002) The State of e-Learning: Looking at History with the Technology Hype Cycle. [Internet] Available from: http://www.e-learningguru.com/articles/hype1_1.htm [Accessed 12th February 2008]

Prensky, M. (2001) Digital Natives, Digital Immigrats. [Internet] Available from: http://www.marcprensky.com/writing/
Prensky%20-%20Digital%20Natives,%20Digital%20
Immigrants%20-%20Part1.pdf [Accessed 14th February 2008]

--

"M-Learning - Wherefore art thou?" - My response

Donald Clark pondered the question "Wherefore art thou M-Learning?" on his Plan B blog recently.

Here's my response:

I believe that part of the resistance to a broader take-up of m-learning is the issue of plain old content formatting. Given that we still have to jump through any number of hoops to ensure content is delivered as specified in different browsers on the Windows platform (never mind MacOS, Linux etc.) is it any wonder that training and content providers are reticent about developing courseware for the range of what Douglas Adams called "Personal Electronic Things"? Associated issues include designing UIs that work on 2x1 inch screens, bandwidth considerations, interoperability, tracking learner usage, providing and measuring tests, and so on.
So there's all that, and the fact that I have yet to see any really engaging m-learning materials.

--

LCB February's Big Question - ISD on the precipice of a crossroads

This month's Learning Circuits Blog Big Question is "For a given project, how do you determine if, when and how much an instructional designer and instructional design is needed?"

---

I wonder how many of you are familiar with this situation? A PowerPoint presentation and demonstration script for some end-user training on a certain process arrived in my inbox the other day. The content was created by one of our “star” Subject Matter Experts who has an unrivalled amount of domain expertise in the particular area under investigation, but has a rudimentary understanding of how to communicate his knowledge, not extending much beyond presenting at user conferences and assisting the Sales team in driving application demos, as well as a small amount of experience in using content authoring tools (mainly to support the above).

The demo script is a beautiful piece of work – a two-column table outlining the process being described on the left hand side, with a breakdown of the onscreen interactions and a voice-over narration script describing what’s happening onscreen on the right. In the VO script, the text also discusses why the process works the way it does, and includes references to the legislation underlying the process and links to where the user may find out more about the guidelines surrounding the process. The demo is about 25 minutes in duration.

The PowerPoint presentation is 93 slides in length, a mixture of bullet points, images and animations, with an average of three animated media elements per slide. The voice-over narration script provided with the PPT runs to about 200 words per slide – in all, nearly 150 minutes of content. Added to the 25-minute demo, it’s the best part of three hours of content, all to be delivered over the Web.

A request in the accompanying e-mail asks “By the way, can you make it so the user can jump into the appropriate part of the demo (if they wish) from the relevant slide in the PPT?”

---

From an instructional design perspective, this is the “precipice of the crossroads” as Little Carmine would say. Without access to instructional design expertise at some level, SMEs cannot transfer their knowledge effectively. Similarly, can we expect e-learning professionals to create high-quality courseware from the content and media assets that they are provided with if there is no instructional design to map the course?

By nature and practise, I am a strong advocate of Ted Cocheu’s Disintermediation in Learning principle. Like a lot of e-learning practitioners however, I am constrained by the realities of delivering courseware and all this entails, from delivery channel bandwidth considerations, through structuring the content according to certain pedagogical principles, to ensuring end-users / learners’ are tracked on an LMS.

I am suggesting that the formal role of “Instructional Designer” as we knew it a decade ago is becoming redundant. The ISD skillset is - except in certain cases - being subsumed into the skills possessed by most learning professionals. I would further assert we are moving towards a time where being an expert “in” ISD, or digital media, or EPSS development (or whatever), and working in a team to create courseware is being superseded by the learning professional acting more as a consultant, providing strategic and operational guidance to individuals who occasionally contribute to learning projects, but whose professional competence lies elsewhere.

The table below is an example of how this approach manifests itself, where the e-learning professional specifies all aspects of the course; I typically use this type of outline at the planning stage of courseware development. The key aspect of this model is that the learning practitioner needs to have a broad and deep understanding of all aspects of the courseware lifecycle.

Table 1 Typology of a course

Function	Approach
Theoretical Framework	Social-Constructivist Model
High level ISD Process	ADDIE & RIO/RLO Model
Strategy	Based on Blooms Taxonomy of Educational Objectives
Structure	Based on Gagné’s Conditions of Learning
Content	Uses a multimedia mix of text, slides, images, animation, Flash and video demonstrations
Delivery	The events are hosted as live Subject Matter Expert (SME)-led events, presented in the following format
Synchronous	Asynchronous
Live audience participate in mentoring event	On-demand after live event
Synchronously streamed on the web	DVD-ROMs available to Knowledge Workers. Each DVD-ROM contains one series of 10±2 lessons

Having made such a statement (and before instructional designers sputter, outraged, into their hot beverage of choice), I must now temper it by saying that I believe the primary exception to what I have just outlined is the COTS courseware industry, which will continue to produce content according to the proven production model that has worked so effectively for the creation of generic training content for the last 20 years or so.

Returning to my SME and his demo and PPT – I worked with him to develop an appropriate instructional design that met the end-users' needs, without doing a 'rip and replace' on the hard work he had already put in to developing the content.

We didn’t need an instructional designer.

But someone had to know about instructional design.

Creating a structure for non-formal learning events

Non-formal learning initiatives such as the Information Sessions are a great way for organisations to enable staff to find out about (current and proposed) procedures, standards, and best industry practise implemented in any project they are might be involved in. Both the initiative as a whole and the individual events themselves adhere to a comprehensive project plan. There are three phases in the project lifecycle, and each discrete Information Session event is a ‘micro-project’ that broadly adheres to project parameters of the project-at-large (see Table 1).

Table 1 Information Sessions Project Outline

Click image to view full-size version

--

Non-formal learning in action: Information Sessions

As discussed in my last blog entry, learner intention - specifically deliberative learning (where the worker schedules time to learn) is a key definer of non-formal learning. As such, the development of an educationally sound theoretical framework, pedagogical approach and instructional design process (are essential in create a learning environment (both synchronous and asynchronous) conducive to the requirements of knowledge workers. At the same time, the content model must make effective use of available technologies, with the express purpose of countering the effects of cognitive overload (learners had consistently remarked on an imbalance of text to other media types – occasionally too much, but also, interestingly, in some instances not enough text on-screen to elucidate the topic, with the presenter “talking to, or around” the content where textual reinforcement would have assisted the participants understand the concepts being discussed (see Table 1).

I have developed the Information Sessions according to well-established learner-centric instructional principles. Specifically, they use a Constructivist theoretical model and a fusion of the ARCS Model and a Systems Design approach to the instructional design of content. A modality of Gagné’s Assumption is used as the basis for the Information Session structure, and the actual presentation format adheres to Gagné’s Events of Instruction (1-5, 7). Content is developed according Rapid eLearning principles. Information Session presenters are guided by the production team through the content creation process.

Table 1 Information Sessions High-level Typology

Function	Approach
Theoretical Framework	Social-Constructivist Model
High level ISD Process	ADDIE & RIO/RLO Model
Strategy	Based on Blooms Taxonomy of Educational Objectives
Structure	Based on Gagné’s Conditions of Learning
Content	Uses a multimedia mix of text, slides, images, animation, Flash and video demonstrations
Delivery	The events are hosted as live Subject Matter Expert (SME)-led events, presented in the following format
Synchronous	Asynchronous
Live audience at Information Session location	On-demand post live event
Synchronously streamed on the web	DVD-ROMs available to KWs. Each DVD-ROM contains one series of 10±2 Information Sessions

An important aspect of Information Sessions is the simultaneous use of face-to-face presentation with synchronous Web delivery, as well as post-event asynchronous distribution of content via the Web and on DVD-ROM, ensuring the widest-possible distribution of content and ensuring just-in-time access to knowledge and domain expertise across an organisation.

--

Typology of non-formal learning

Similar to Rogers’ learning continuum (2004), Eraut establishes a matrix to identify varying types of non-formal learning, based on the timing of the stimulus (past, current, future) and the extent to which such learning is implicit, reactive or deliberative. A key component of the matrix is the intention of the learner’s activity. Eraut makes a distinction between implicit learning, which has a metacognitive component, deliberative learning (where the worker schedules time to learn) and reactive learning (where learning is explicit but almost takes place spontaneously, in response to recent, current or imminent situations but without any time being set aside for it). I would contend that these categories align closely with Kolb’s 4-Stage Experiential Learning Cycle. Similarly, a link can be made between the synchronous and asynchronous delivery mechanisms discussed in an earlier blog entry and Eraut’s Timing of Stimulus category.

Table 1 Michael Eraut’s typology of non-formal learning (2000, p.129)

References:

Eraut, M. (2000) Non-formal learning, implicit learning and tacit knowledge, in F. Coffield (Ed) The Necessity of Informal Learning: Policy Press. Bristol

Rogers, A. (2004) Looking again at non-formal and informal education - towards a new paradigm [Internet] Available from: http://www.infed.org/biblio/non_formal_paradigm.htm [Accessed 30th January 2008]

--