Seels and Glasgow Model: Discovering Instructional Design 20

And so my friends, we reach Number 20 in our journey of discovery around Instructional Design. That's a month's solid blogging, investigating one of the fundamental domains associated with our profession.

What have we learned? More of that anon, but for now I'm going to cover the the Seels and Glasgow Model in this E-Learning Curve Blog series on a systems approach to instruction design.

Now read on…

In a 2008 article called Can we reinvent e-learning? I asserted that

ADDIE emerged from the principles of project management, and resembles the philosophy and practice to this discipline's methodology more than a pedagogy. Treating learning like a project leads to "training outcomes" equivalent to project deliverables.

In my view this is no bad thing: the reality is that Learning & Development is a pragmatic discipline, tasked with facilitating individuals in their endeavors to learn, educators would be poorly served if the theoretical, methodological, and pedagogical epistemologies of the domain did not at least tacitly acknowledge the practical challenges associated with implementing learning programs.

Barbara Seels and Zita Glasgow's Model (see Figure 1) reflect this assertion; they situate their understanding of ISD and their model on the thesis that design occurs in the context of project management (p. 177).

Figure 1. The Seels and Glasgow Model
[Click to enlarge]

Their model is distributed across the three phases of project management:

1. Needs Analysis Management
2. Instructional Design Management
3. Implementation Management

This distribution allows a learning program (or project) to be planned, resourced, and managed much as any other project in an organization is arranged.

In this model, the first phase (Needs Analysis) includes the establishment of the instructional goals, requirements, and context for the courseware. Next, the Instructional Design phase begins when Needs Analysis is completed: this second phase consists of six activities:

1. task analysis
2. instructional analysis
3. objectives and tests
4. formative evaluation
5. materials development
6. instructional strategy and delivery systems

- all of which are linked via feedback and interaction communications channels. In Phase Three of the model (Implementation and Evaluation) the development and production of materials, training delivery, and summative evaluation are undertaken.

As is usual in a systems-based approach to ID, the phases in this model can are typically applied in a linear fashion, but they are often applied iteratively. As Gustafson and Branch highlight, the steps in the instructional design phase are interdependent and concurrent, and multiple iterations of this process may occur during this part of the development lifecycle (2001, p.43).

In this sense - and reflecting on my ADDIE/PM remarks, we can say that this is a product-oriented approach to content development. According to Chen

Developing an instructional project involves skill sets ranging from project management and interface design to sound preparation and programming...Design teams represent various fields of expertise (producers, instructors, editors, etc.).

(2007 pp.2-3)

Managing potentially large teams and and hundreds (if not thousands) of media assets needs substantial resources and commitment, and requires strong project management to stay on time and budget at the appropriate quality of outputs. To support this objective, Seels and Glasgow focus on the importance of well-designed materials, the need to identify and understand communication patterns within organizations, develop strategies for diffusion of innovations, and the importance of supporting learners.

Interestingly, Seels and Glasgow also include the concept of diffusion of innovations in their model:

The strategies that lead to diffusion are most effective if used during all the phases of a project.

(1998, p. 178)

They consider that when their model is applied ,the phases are generally linear in nature but

it is not necessary to complete a step before proceeding, and the order can be changed so that steps can be performed concurrently.

(1998, p 179)

We can say that this model successfully aligns to the systems philosophy epitomized in ADDIE quite will, while acknowledging the needs and limitations of the practical application of instructional design. Much like 3PD, formative evaluation via a feedback mechanism (multiple iterations rather than recursion) is a distinguishing (but not unique) characteristic of Seels and Glasgow's approach.

Next time: What have we learned? The implications of Instructional Systems Design for E-Learning
___________

References:

Chen, I. (2007) Instructional Design Methodologies. In: Kidd, T. & Song, H. (Eds.). Handbook of Research on Instructional Systems and Technology. IGI Global

Seels, B. & Glasgow, Z. (1990). Exercises in instructional Technology. Columbus OH: Merrill Publishing Co.

Seels, B., & Glasgow, Z. (1998). Making Instructional Design Decisions. (2nd ed.) Upper Saddle River, NJ: Merrill.

--

Gagne and the Events of Instruction: Discovering Instructional Design 19

In yesterday's post, I looked at the ASSURE instructional design model, which was originally developed by Heinich et al in the 1990's, and is now popularly and widely-used in both classroom and e-learning environments.

Given the ASSURE Model's constructivist epistemology and successful implementation in early 21st Century educational contexts, you may be surprised to learn that the ASSURE methodology has it's roots very firmly located the venerable, behaviorist-influenced (and occasionally criticized) Events of Instruction, devised by Robert M. Gagne.*

Now read on…

According to Kevin Kruse (2006)

Robert Gagne is considered to be the foremost researcher and contributor to the systematic approach to instructional design and training. Gagne and his followers ...focus [...] on the outcomes - or behaviors - that result from training.

I would assert that familiarity with Gagne’s work, and educators’ drive to continually investigate the New have perhaps led to a certain disregard (in some quarters) for the substantial contribution Gagne made to our discipline. For example, to characterize Gagne as a Behaviorist is, in my view, to underestimate the sophistication of his theories, the elegance of his models, and the relevance of his work today. Indeed, Walter Wager (2004) states that

Gagne didn't feel that the behaviorist theories were adequate to explain human learning. Rather, Gagne should be considered one of the early cognitive psychologists.

(p.296)

As I have previously indicated, his work still influences theorists and learning practitioners today. During his career, Gagne primarily concerned himself with understanding "the process of learning" (1972, p.1). In his life, he was central to the development of five instructional theories:

1. the five domains of learning
2. events of instruction
3. conditions of learning
4. role of the media
5. integrated goal theory (Wager, 2004)

Gagne's text The Conditions of Learning (first published in 1965) attempted to identify and describe the cognitive processes that occur in learning: the eponymous ‘conditions of learning.’ His philosophy was influenced by the concepts of cognitive mapping, as well as the information processing interpretation of the events that occur when (adult) learners are presented with various stimuli. In The Conditions of Learning, Gagne argued that that internal and external conditions of learning must be created to stimulate the desired learning response.

To understand the sequence of activities needed to support learning, Gagne suggested that tasks for

acquiring the intellectual skills needed should be organized according to complexity.

(Hriko, 2008, p.353)

He argued that information underwent a series of internal processes before being stored in long-term memory; he developed a nine-step process called the Events of Instruction to represent the manifestation of the external factors that influenced the acts associated with the process, which "correlate to and address the conditions of learning" (Hriko, 2008 p.353). Table 1 shows these instructional events in the left column and describes the associated mental processes in the right column.

Table 1. Nine Events of Instruction (after Gagne, 2004)

Instructional Event	Internal Mental Process
1. Gain attention	Stimuli activates brain's receptors
2. Inform learners of objectives	Creates level of expectation for learning
3. Stimulate recall of prior learning	Retrieval and activation of short-term memory
4. Present the content	Selective perception of content
5. Provide "learning guidance"	Semantic encoding for storage long-term memory
6. Elicit performance (practice)	Responds to questions to enhance encoding and verification
7. Provide feedback	Reinforcement and assessment of correct performance
8. Assess performance	Retrieval and reinforcement of content as final evaluation
9. Enhance retention and transfer to the job	Retrieval and generalization of learned skill to new situation

More…

* Yes, his name is Robert Gagné (with an acute aigu ), but English speakers typically don’t enter accents into Google, and I’m nothing if not pragmatic…
___________

References:

Gagne, R. M., (1972). Domains of learning. Interchange 3(1),pp.1-8.

Gagne, R. M., Wager, W. W., Golas, K. and Keller, J.M. (2004). Principles of Instructional Design (5th.Ed.). Wadsworth Publishing Co Inc.

Kruse, K. (2006). Gagne’s Nine Events of Instruction: An Introduction. E-Learning Guru. Internet: Available from: http://www.e-learningguru.com/articles/art3_3.htm Accessed 12 June 2009

Hriko, M. (2008) Gagne's Nine Events of Instruction. In: Tomei, L.A., Morris, R. (Eds.), Encyclopedia of Information Technology Curriculum Integration. Information Science Reference

Wager, W. (2004) Robert M. Gagne. In: Kovalchick, A., and Dawson, K. (Eds.), Education & Technology: An Encyclopedia. Santa Barbara, CA: ABC-CLIO

--

ASSURE Model: Discovering Instructional Design 18

As we saw when we investigated the Three-Phase Design Model, a number of stakeholders including subject matter experts, educationalists, and technical experts need to work together to design and develop learning programs and educational courseware.

The ASSURE Model is a constructivist approach to training design developed by Robert Heinich and Michael Molenda of Indiana University and James D. Russell of Purdue University in the 1990's.

ASSURE is an acronym derived from the key verb descriptor of the tasks associated with the approach. As such, the model proposes a six-step guide for planning and delivering instruction; while not specifically designed for e-learning, in practice it the methodology seems to align to the requirements of designing courseware for that modality.

The steps (or tasks) are described in detail in Table 1:

Table 1 ASSURE Model components (after Human Resource Development website)

A	Analyze Learners	Prerequisite skills or knowledge What courses are taken prior to this one? What knowledge is assumed? Learning Styles of the students - This model emphasizes teaching for different learning styles. Motivations - Why is the learner taking the course?
S	State Objectives	Statements describing what the learner will do as a result of instruction. Things to keep in mind as you write your objectives are: Focus on the learner, not the teacher Use behaviors that reflect real world concerns Objectives are descriptions of the learning outcomes and are written using the ABCD format. Audience: Who is the audience? Specifies the learner(s) for whom the objective is intended. Behavior: What do you want them to do? The behavior or capability needs to be demonstrated as learner performance, an observable, measurable behavior, or a real-world skill. Use an action verb from the helpful verbs list if you have difficulty doing this. Condition: Under what circumstances or conditions are the learners to demonstrate the skill being taught? Be sure to include equipment, tools, aids, or references the learner may or may not use, and/or special environmental conditions in which the learner has to perform. Degree: How well do you want them to demonstrate their mastery? Degree to which the new skill must be mastered or the criterion for acceptable performance (include time limit, range of accuracy, proportion of correct responses required, and/or qualitative standards.)
S	Select Methods, Media and Materials	You need to decide what method you will primarily use to support the learning objectives: for example: lectures, online collaboration, group work, a field trip, etc. What media you will use: photos, multimedia, video? Are you using store bought materials, getting an outside resource to provide materials, modifying something you already have, or develop something from scratch? Selection Criteria: Media Selection - Media should be selected on the basis of student need. - We must consider the total learning situation. - Should follow learning objectives. - Must be appropriate for the teaching format. - Should be consistent with the students' capabilities and learning styles. - Should be chosen objectively. - Should be selected in order to best meet the learning outcomes. - No single medium is the total solution. - Does it match the curriculum? - Is it accurate and current? - Does it contain clear and concise language? - Will it motivate and maintain interest? - Does it provide for learner participation? - Is it of good technical quality? - Is there evidence of its effectiveness (e.g., field-test results)? - Is it free from objectionable bias and advertising - Is a user guide or other documentation included?
U	Utilize Media and Materials	Plan how you are going to implement your media and materials. For each media type and/or materials listed in the Select step, describe in detail how you intend to implement them into your lesson to help your learners meet the lesson's objective. Do this for each item. In order to utilize materials correctly there are several steps to creating good student-centered instruction. 1. Preview the material- Never use anything in class you have not verified and validated. 2. Prepare the material- Make sure you have everything you need and that it all works. 3. Prepare the environment- Set up the classroom so that whatever you’re doing will work in the space you have. 4. Prepare the learners- Give the students an overview, explain how they can take this information and use it and how they will be evaluated up front. 5. Provide the learning experience- Teaching is simply high theatre. Showmanship is part of the facilitator's job. Teaching and learning should be an experience not an ordeal.
R	Require Learner Participation	Describe how you are going to get each learner "actively and individually involved in the lesson. Ex: games, group work, presentations, etc. All activities should provide opportunities to manipulate the information and allow time for practice during the demonstration of the skill.
E	Evaluate and Revise	Describe how you will evaluate and measure whether or not the lesson objectives were met. Were the media and the instruction effective? Evaluate student performance: How will you determine whether or not they met the lesson's objective? The evaluation should match the objective. Some objectives can be adequately assessed with a pen and paper test. If the objectives call for demonstrating a process, creating a product, or developing an attitude, the evaluation will frequently require observing the behavior in action. Evaluate media components: How will you determine the media effectiveness? Evaluate instructor performance: How will you determine whether or not your own performance as instructor/facilitator was effective?

While the learner's needs, the instructional goals, and the availability or desirability of particular media are the drivers for the design and development, Gunter and Baumbach (Education and Technology: An Encyclopedia, 2003) state that

The instructional goals should be the focus, not the goal. When integrating technology, it should always be viewed as a tool that assists... the learning needs of the student. The teacher becomes a mentor and colearner, who is actively engaged in enabling students to access, analyze, apply, and create information electronically.

(pp.193-194)

More…

___________

References:

Gunter, G. & Baumbach, D. (2004). Curriculum Integration. In: Education and Technology: An Encyclopedia. (Kovalchick, A. & Dawson, K. Eds). ABC-CLIO; illustrated edition

Heinich, R., Molenda, M., Russell, J., & Smaldino, S. (2002). Instructional Media and Technologies for Learning (7th Ed.). New Jersey: Merrill Prentice Hall.

Human Resource Development [Internet] Available from: http://itchybon1.tripod.com/hrd/id15.html Accessed 10th June 2009

--

Factors Influencing Learning Design: Discovering Instructional Design 17

One of the more interesting outcomes of the Three-Phase Design (3PD) model has been that, while in and of itself, it describes a strategic context from which to build and maintain online teaching and e-learning environments, it does not focus on more granular aspects of the design process proper.

According to Rod Sims (2008) there are six factors are essential to achieving engaging, interactive and memorable learning experiences (see Figure 1). The 3PD Model supports these factors by establishing the build-enhance-maintain process as core to successful project implementation.

Figure 1. Six factors influencing e-learning design (after Sims, 2008)
[Click to enlarge]

Sims called this model Proactive Design for Learning (PD4L): the six factors that enable the design of effective online teaching and learning are:

1. Theory-based, ensuring that decisions are based on contemporary approaches to teaching and learning.
2. Innovative and relevant (incorporating elements of proactive evaluation documented by Sims, Dobbs & Hand, 2002).
3. Team-based, with team members having the relevant and appropriate competencies to engage with and complete the design tasks (Sims & Koszlaka, 2008).
4. Emergent, allowing (where appropriate) the interactions between course participants to establish and introduce course content (Irlbeck, Kays, Sims & Jones, 2006).
5. Interactive, enabling participants to actively explore the relevance and application of the course content (Allen, 2003; Sims, 2006).
6. Personalized, such that participants are able to apply their own context and situation to the learning outcomes (Sims & Stork, 2007).

People (and organizations) do not adopt new ideas at the same time. Some adopt ideas when they are first introduced; others wait for varying periods of time; some never adopt an idea. In The Diffusion Process (1957), Bohlen and Beal maintain that

...the time span over which people adopt ideas will vary from practice to practice.

(p.4).

The authors' research indicated that complexity of practice is a significant factor in determining the value of a diffused idea or technology in organizations. They defined the following categories of complexity:

• Change in material and equipment
• Improved practice
• Innovation
• Change in enterprise
• Cost

Three-Phase Design and it's subsequent iterations are representative of educators' responses to the challenges and opportunities afforded by the introduction, diffusion, and adoption of Web-based technologies in education: traditional approaches to instructional design do not necessarily fit the requirements of online learning. Of particular note in this context is the emergence of Constructivism as a theoretical framework for the development of online learning programs. In the PD4L Model, for example, Sims cites

theories including the social formation of the mind (Vygotsky, 1978), meaningful learning (Ausubel, 1968), situated cognition (Clancey, 1997), constructivism (Driscoll, 2005) and connectivism (Siemens, 2004).

(p.9)

He continues:

Together with a pragmatic, interpretivist epistemology, the PD4L model focuses on creating teaching and learning environments where relevant, meaningful knowledge is constructed by the individual.

When compared to the purely Functionalist (in the anthropological sense of the term) methodology of ISD, we can see that models like Sims and Jones' are attempting to accommodate the power and flexibility afforded by digitally mediated technologies in the context of acquisition of skills, knowledge construction, and a more experiential view of learning, that the traditional systems-based approach.

More…
___________

References:

Bohlen, J. M., Beal, G. M. (1957). The Diffusion Process, Special Report No. 18 (Agriculture Extension Service, Iowa State College) 1: 56-77. [Internet] Available from: http://www.soc.iastate.edu/extension/presentations/publications/comm/Diffusion%20Process.pdf [Accessed 3rd November 2008]

Malinowski, B. 1990. A Scientific Theory of Culture. Reissue edition. Chapel Hill: University of North Carolina Press.

Rogers, E. M. (2003) Diffusion of Innovations, (5th ed.). Simon & Schuster International.

Sims, R. (2008). From three-phase to proactive learning design: Creating effective online teaching and learning environments, In: J. Willis (Ed), Constructivist Instructional Design (C-ID): Foundations, Models, and Practical Examples.

Sims, R., Dobbs, G., & Hand, T. (2002). Enhancing quality in online learning: Scaffolding planning and design through proactive evaluation. Distance Education, 23(2), 135-147.

Sims, R. & Jones, D. (2003). Where practice informs theory: Reshaping instructional design for academic communities of practice in online teaching and learning. Information Technology, Education and Society, 4(1), 3-20.

--

Aligning Learning Theory with Instructional Design

As you will know if you read yesterday's post, the proliferation of learning theories over the last century or so has led to a broad range of philosophies and ideas for learning professionals to choose from when undertaking the development of a learning program, as you can see from Figure 1, which is a simplified timeline of the philosophies and disciplines that influenced learning theories.

Figure 1 Influences on Learning Theories (after Stahl, G. 2003)

The heterogeneous nature of learning theories, with sometimes subtle and occasionally significant divergences in their character usually serves only to confuse an already complex domain: even gathering an understanding of the key terms associated with the subject (see Figure 2) can be an overwhelming task for those new to the discipline of instructional design.

Figure 2 Tag Cloud of Learning Theory Terms

So how do you choose a learning theory for your instructional design?

In their 1993 article Behaviorism, cognitivism, constructivism: Comparing critical features, Ertmer and Newby use Dale H. Schunk's definitive questions as well as two more of their own to enable those engaged in instructional design to distinguish between learning theories at the highest level. Schunk (1991) defines five questions to distinguish learning theories:

1. How does learning occur?
2. Which factors influence learning?
3. What is the role of memory?
4. How does transfer occur?
5. What types of learning are best explained by the theory?

Ertmer and Newby include two more questions for instructional designers:

1. What basic assumptions / principles are relevant to instructional design?
2. How should instruction be structured to facilitate learning?

Based upon these criteria, we can say that instructional design can be characterized as being effective in the contexts described as below:

Learning Program	Learning Theory / Instructional Design Approach
Introductory learning	A behaviorist/cognitivist approach works best. Instruction is predetermined, sequential and criterion-referenced
Advanced learning	A cognitivist/constructivist approach works best. Tasks require an increased level of processing (schematic organization, analogical reasoning etc)
Expertise development	A constructivist approach works best. Tasks associated with subject matter expertise demand high levels of analysis and problem-solving (i.e. situated learning, cognitive apprenticeships, and social negotiation)

__________

References:

Ertmer, P. A., Newby, T. J. (1993). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly, 6 (4), 50-70.

Schunk, D. H. (1991). Learning theories: An educational perspective. New York: Macmillan.

Stahl, G. (2003). Building Collaborative Knowing: Elements Of A Social Theory Of CSCL, IN J.W. Strijbos, P.Kirschner & R. Martins (ed.), What we know about CSCL in higher education, Amsterdam: Kluwer.

--

Informal and Non-Formal Workplace Learning

One of the central components of the impact of learning (and specifically the development of information workers’ expertise in organizations) is the context within which the learning takes place. A central pillar of this discussion is the type or format of the learning taking place. In the literature, it is apparent that a dichotomy exists between the paradigms of formal, goal-directed training programs and informal – “learning at the watercooler” (Grebow, 2002) or what Michael Eraut (2000) describes as incidental learning that takes place almost as a side effect of work:

it is difficult to make a clear distinction between formal and informal learning as there is often a crossover between the two

(McGivney, 1999, p.1).

Another complexity in the discussion is where is non-formal learning located in relation to the diametric opposites? For much of the forty years since the term ‘non-formal learning’ was first coined (Coombs, 1968, p.1.) there has been a great deal of debate in the literature as to the nature of formal, informal and non-formal learning; the components of each of the paradigms, their boundaries and their overlaps. The locus of this debate is centered on arguments for “the inherent superiority of one or the other” (Colley, Hodkinson & Malcolm, 2002, p.2).

I support Alan Rogers’ (2004) view that a “new paradigm” for learning exists, in which “most programs [are] partly formal and partly informal” going from formal to informal and from informal to formal in both directions along a continuum (see Figure 1) . “Both forms of education are important elements in the total learning experience” (Looking again at non-formal and informal education - towards a new paradigm, 2004).

Figure 1 the Learning Continuum

Similarly, Hodkinson & Hodkinson argue that focusing on the extent to which learning is planned and intentional may be a way of by-passing the distinction between formal, non-formal and informal altogether.” (Colley, Hodkinson & Malcolm, 2002).

More to follow…

__________

References:
Colley, H., Hodkinson, P., & Malcolm J. (2002) Non-formal learning: mapping the conceptual terrain. a consultation report [Internet] Available from: http://www.infed.org/archives/e-texts/colley_informal_learning.htm [Accessed 28th January 2009]
Coombs, P. (1968) The World Educational Crisis, New York, Oxford University Press.

Eraut, M. (2000) Non-formal learning, implicit learning and tacit knowledge, in F. Coffield (Ed) The Necessity of Informal Learning: Policy Press. Bristol

Grebow, D. (2002) At the Water Cooler of Learning [Internet] Available from: http://agelesslearner.com/articles/watercooler_dgrebow_tc600.html [Accessed 30th February 2009]

McGivney, V. (1999) Informal learning in the community: a trigger for change and development NIACE. Leicester.

Rogers, A. (2004) Looking again at non-formal and informal education - towards a new paradigm [Internet] Available from: http://www.infed.org/biblio/non_formal_paradigm.htm [Accessed 30th January 2008]

--

M-Learning via the iPhone 4 – some approaches and technologies

Last time, I investigated mobile learning (m-learning) in the context of its parent domain e-learning and I outlined some types and characteristics of devices that enable the delivery of m-learning to users. In this post, I will look at the some of the learning theories that support learning distributed via mobile devices.

Now read on…

In their 2004 paper Literature Review in Mobile Technologies and Learning (Naismith, Lonsdale, Sharples & Vavoula), consider the importance of taking an approach to m-learning that

moves away from the dominant view of mobile learning as an isolated activity to explore mobile learning as a rich, collaborative and conversational experience, whether in classrooms, homes or the streets of a city. …how we might draw on existing theories of learning to help us evaluate the most relevant applications of mobile technologies.

(p.1)

Table 1. Applying Learning Theories to M-Learning (after Naismith et al, 2004)

Learning Theory	Activity Type	Description
Behaviorist	Activities that promote learning as a change in learners’ observable actions	In the behaviorist paradigm, learning is thought to be best facilitated through the reinforcement of an association between a particular stimulus and a response. Applying this to educational technology, computer-aided learning is the presentation of a problem (stimulus) followed by the contribution on the part of the learner of the solution (response). Feedback from the system then provides reinforcement.
Constructivist	Activities in which learners actively construct new ideas or concepts based on both their previous and current knowledge	In the constructivist approach, learning is an active process in which learners construct new ideas or concepts based on both their current and past knowledge. Learners are encouraged to be active constructors of knowledge, with mobile devices now embedding them in a realistic context at the same time as offering access to supporting tools. The most compelling examples of the implementation of constructivist principles with mobile technologies come from a brand of learning experience termed ‘participatory simulations’, where the learners themselves act out key parts in an immersive recreation of a dynamic system.
Situated	Activities that promote learning within an authentic context and culture	Situated learning posits that learning can be enhanced by ensuring that it takes place in an authentic context. Mobile devices are especially well suited to context-aware applications simply because they are available in different contexts, and so can draw on those contexts to enhance the learning activity. The museum and gallery sector has been on the forefront of context-aware mobile computing by providing additional information about exhibits and displays based on the visitor’s location within them.
Collaborative	Activities that promote learning through social interaction	Collaborative learning has sprung out from research on Computer-Supported Collaborative Work and Learning (CSCW/L) and is based on the role of social interactions in the process of learning. Many new approaches to thinking about learning developed in the 1990s, most of which are rooted in Vygotsky’s socio-cultural psychology (Vygotsky 1978), including activity theory. Though not traditionally linked with collaborative learning, another theory that is particularly relevant to our consideration of collaboration using mobile devices is conversation theory (Pask 1976), which describes learning in terms of conversations between different systems of knowledge. Mobile devices can support Mobile Computer Supported Collaborative Learning (MCSCL) by providing another means of coordination without attempting to replace any human-human interactions, as compared to say, online discussion boards which substitute for face-to-face discussions (Zurita et al 2003; Cortez et al 2004; Zurita and Nussbaum 2004).
Informal and non-formal	Activities that support learning outside a dedicated learning environment and formal curriculum	Research on informal and lifelong learning recognizes that learning happens all of the time and is influenced both by our environment and the particular situations we are faced with. Informal learning may be intentional, for example, through intensive, significant and deliberate learning ‘projects’ (Tough 1971), or it may be accidental, by acquiring information through conversations, TV and newspapers, observing the world or even experiencing an accident or embarrassing situation. Such a broad view of learning takes it outside the classroom and, by default, embeds learning in everyday life, thus emphasizing the value of mobile technologies in supporting it.
Learning and teaching support	Activities that assist in the coordination of learners and resources for learning activities	Education as a process relies on a great deal of coordination of learners and resources. Mobile devices can be used by teachers for attendance reporting, reviewing student marks, general access of central school data, and managing their schedules more effectively. In higher education, mobile devices can provide course material to students, including due dates for assignments and information about timetable and room changes.

As yet there is no comprehensive ‘grand theory of mobile learning’ - nor do I expect that there will be any time soon. As I move forward through this series of posts, one of the areas I will consider is integrating pedagogy for the use of mobile devices that in a number of areas. In much the same way as we have many categories of devices, we will discover that there are many ways to integrate learning on mobile devices using a number of instructional designs, developmental approaches and delivery models. I support the view that one of the great strengths of m-learning (and indeed e-learning) is it's facility to transcend traditional learning environments like the classroom or training center, and to combine different elements in ways that are appropriate to the learning activities to be supported.

More…

______________________

References:

Cortez, C., Nussbaum, M., Santelices, R,. Rodríguez, P., Zurita, G., Correa, M. and Cautivo, R. (2004) Teaching science with mobile computer supported collaborative learning (MCSCL). Proceedings of the 2nd International Workshop on Wireless and Mobile Technologies in Education. JungLi, Taiwan: IEEE Computer Society, 67-74

Naismith, L., Lonsdale, P., Vavoula, G. and Sharples, M. (2004) Literature Review in Mobile Technologies and Learning. NESTA Futurelab Series, Report 11. [Internet] Available from: http://www.futurelab.org.uk/research/lit_reviews.htm Accessed 15 February 2009.

Pask, AGS (1976) Conversation Theory: Applications in Education and Epistemology. Amsterdam and New York: Elsevier

Vygotsky, L. S. (1978) Mind in society. Edited by Cole, M. John-Steiner, V. Scribner, Souberman, E. Cambridge, MA, Harvard University Press

Zurita, G, Nussbaum, M and Sharples, M. (2003) Encouraging face-to-face collaborative learning through the use of hand-held computers in the classroom. Proceedings of Mobile HCI 2003, Udine, Italy: Springer-Verlag, 193-208

Zurita, G., Nussbaum, M (2004) Computer supported collaborative learning using wirelessly interconnected hand-held computers. Computers & Education, 42(3): 289-314

--

Teaching organizations how to learn. Part 2 – Conditions for learning

As discussed in a previous post, Driscoll (1994) outlines five conditions for learning (p.382-3). Very much like the multiple approaches and interpretations that exist in constructivism, a number of conditions must be met for the approach to be implemented. The conditions are:

1. Providing complex learning environments that incorporate authentic activity. Constructivists argue that learners should learn to solve the types of complex problem they will face in real life. Learning how to do this is difficult unless complex and authentic learning environments are available to the learners.
2. Providing for social negotiation as an integral part of learning. Bruner (1986) explains that learning is a cultural interchange between group members. Collaboration creates an opportunity for learners to share their understandings with others and to have others do the same with them. This provides multiple perspectives to each learner, and this negotiation process between peers should lead to enhanced understanding.
3. Support multiple perspectives and the use of multiple modes of representation. Because learning skills, behaviors and knowledge can be diverse and complex, constructivists believe that to achieve complete understanding the learner must examine the material from multiple perspectives. If they are not supported in this endeavor, the learner will achieve only a partial understanding of the material. Multiple modes of representation allow the learner to view the same content through different sensory modes.
4. Nurture reflexivity. Duffy and Cunningham (1996) characterize reflexivity as “the ability of students to be aware of their own role in the knowledge construction process.” (p.172). It could also be described as the learner taking ownership of their own thinking and learning processes. Driscoll (1994) assets that reflexivity and by extension critical thinking are central attributes in the constructivist methodology, as it enables learners to understand how and why cognition creates meaning. This enables learners to attain goals such as reasoning, understanding multiple perspectives, and expressing and defending their own beliefs.
5. The last condition Driscoll describes is to “emphasize student-centred instruction.” Bruner (1966) calls this “discovery learning”. By obtaining knowledge by themselves, learners select and transform information, construct knowledge, and make decisions in the context of a cognitive structure that provides meaning and organization to experiences and allows the individual to “go beyond the information given”. Students are actively engaged in determining what and how they will study or gain understanding.

These principles and conditions position the constructivist approach to learning as an appropriate orientation for learning sans frontiers; using technologies like the Internet, websites and virtual learning environments, applying collaborative learning, problem-based learning and goal-based mechanisms, making Open Source Software and Course- and Content Management Systems accessible to learners, and using e-learning applications like online conferencing and collaboration tools could be the foundation for these multiple constructivist conditions for learning. (Duffy & Cunningham 1996, Driscoll 1994, Schank 1994)

These characteristics provide an appropriate framework for knowledge workers to learn (and for the learning intervention), given that their ongoing development is based in the context of already-established cognitive schemata (from the learners’ perspective), the knowledge and skills are applied to solve real-world problems, and their expertise (behaviors) are typically used in collaboration with their peers to enhance the performance of organizations.

___________

References:

Bruner, J. S. (1986). Actual minds, possible worlds. Cambridge, Mass: Harvard University Press.

Driscoll, M. P. (1994). Psychology of learning for instruction. Boston, MA. Allyn & Bacon.

Duffy, T. M. & Cunningham, D. J. (1996) Constructivism: Implications for the design and delivery of instruction. IN: Jonassen D. H. (Ed) Handbook of Research for Educational Communications and Technology (pp.170- 198). New York: Simon & Shuster Macmillan.

Schank, R. (1994) Active Learning Through Multimedia, IEEE Multimedia, 1(1), pp.69-78.

--

Teaching organizations how to learn. Part 1

Chris Argyris, Professor Emeritus at Harvard Business School, has made significant contributions to the development of organizational learning theory and experiential learning.

Any company that aspires to succeed in the tougher business environment ...must first resolve a basic dilemma: success in the marketplace increasingly depends on learning, yet most people don’t know how to learn. What’s more, those members of the organization that many assume to be the best at learning are, in fact, not very good at it.

(Argyris, C. 1991)

Over the next couple of weeks (taking digressions where appropriate) I will examine the significance of his work (focusing in particular on single-loop and double-loop learning) on organizational learning systems.

Today however, I will begin by reminding you of the basis for Chris Argyris’ work: Constructivism.

In her comprehensive reference text Psychology of learning for instruction, Marcy Driscoll describes constructivism as having

…multiple roots in psychology and philosophy, among which are the cognitive and developmental perspectives of Piaget, the interactional and cultural emphases of Vygotsky and Bruner, the contextual nature of learning, the active learning of Dewey, the epistemological discussions of von Glasersfeld and the paradigm and scientific revolutions of Thomas Kuhn.

(1994, p.375)

Constructivism is an approach to learning based on the premise that cognition, or learning, is the result of mental construction: it is an active process in which learners construct new ideas, skills and behaviors based upon their prior and current knowledge, behavior and skill assets. The learner transforms information, constructs knowledge, and makes decisions based upon extant cognitive structures or mental models. These cognitive structures - what Roger Schank calls “scripts” in his Dynamic Memory Model (1982) - provide meaning and organization to experiences and allow the individual to go “beyond the information given” (Bruner, 1974). Even though there are numerous interpretations of constructivism, several central concepts inhabit all constructivist theories. Cunningham and Duffy (1996) identified two major similarities that are the foundation of all constructivist thought. They are that “learning is an active process of constructing rather than acquiring knowledge” and “instruction is a process of supporting that construction rather that communicating knowledge” (1996, p.172).

Bruner’s 1966 text Toward a Theory of Instruction described the key principles of constructivism (p.225):

Table 1 Principles of constructivism

Principle		Definition
Readiness		Instruction must be concerned with the experiences and contexts that make the student willing and able to learn
Spiral organization	Structure.	The content must be structured so that it can be grasped by the learner.
	Sequence.	Material must be presented in the most effective sequences.
Generation		“Going beyond the information given” - Instruction should be designed to facilitate extrapolation and or fill in the gaps

Extending from these basic constructivist principles as well as from the work of other key figures in the constructivist school, Driscoll (1994) outlines five conditions for learning (p.382-3). Very much like the multiple approaches and interpretations that exist in constructivism, a number of conditions must be met for the approach to be implemented. It is useful to elaborate briefly on these conditions, as they are relevant to the learning approach discussed in the rest of this series of posts.

…and I will do this tomorrow.

____________________

References:

Argyris, C. (1991) Teaching Smart People How to Learn. Harvard Business Review Reflections 4(2). [Internet]. Available from: http://www.velinperformance.com/downloads/chris_argyris_learning.pdf [Accessed 12 November 2008]

Bruner, J. S. (1974) Going Beyond the Information Given. New York: Norton.

Cunningham, D. J. & Duffy, T. M. (1996) Constructivism: Implications for the design and delivery of instruction. IN: Jonassen. D. H. (Ed.), Handbook of Research for Educational Communications and Technology (pp. 170- 198). New York: Simon & Shuster Macmillan.

Driscoll, M. P. (1994). Psychology of learning for instruction. Boston, MA. Allyn & Bacon.

Schank, R. (1982) Dynamic Memory: A Theory of Reminding and Learning in Computers and People. Cambridge University Press.

--

Learning - the Zeigarnik effect

Here's an unusual but potentially very useful instructional strategy, one that takes advantage of peoples' need for resolution or for closure. The Zeigarnik effect states that people remember uncompleted or interrupted tasks better than completed ones.

Russian psychologies Bluma Zeigarnik first studied the phenomenon in the late 1920's after her professor, Gestalt psychologist Kurt Lewin, noticed that waiters remembered orders only as long as the order was in the process of being served.

Research indicates that:

1. the Zeigarnik effect is likely to appear if the subject is "ego-involved" in the task to some extent (such as in a learning activity)
2. the effect is more likely to appear if the task doesn't seem to be part of the training plan
3. the effect is most likely to appear if the subject has set a genuine level of aspiration in the interrupted task

Musicians intuitively use this effect when writing musical 'hooks,' or for concluding pieces of music. By using a "suspended" chord, where the third has been displaced by either of its dissonant neighbouring notes, forming intervals of a major second or (more commonly) a perfect fourth with the root. The resulting unexpected dissonance is then satisfyingly resolved by introducing the displaced note.

This can be heard to great effect in the main musical theme of Pinball Wizard by The Who (which transitions from a F#sus4 to a F#maj), or more sophisticatedly, as a plagal or "Amen" cadence in the final phrase of The Beatles' She's Leaving Home, or the coda (with typically Baroque flourishes) of Bach's Canon alla Decima, Contrapuncto alla Terza.

In Gestalt psychology, the Zeigarnik effect has been used to demonstrate the general presence of Gestalt phenomena: not just appearing as perceptual effects, but also present in cognition.

The implication of this is that learners who wish to remember material better should take breaks in unrelated fields or games before completing exercises.

... so the next time you see me playing Civilization 4 when I'm supposed to be writing that white paper, I'm learning.

--

A theoretical approach to e-learning content delivery in an Open Environment. E-Learning Course Development: Project Lifecycle 3

Theoretical approach – content delivery

Today's post discusses an approach to the delivery of multimedia content in an open (or indeed any) e-learning environment.

As Richard Mayer observes:

Multimedia messages that are designed in light of how the human mind works are more likely to lead to meaningful learning that those that are not. A cognitive theory of multimedia learning assumes that the human information processing system includes dual channels for visual/pictorial and auditory/verbal processing, that each channel has limited capacity for processing, and that active learning entails carrying out a coordinated set of cognitive processes during learning.

(2003, p.41)

Mayer’s Cognitive Theory of Multimedia Learning is based on three basic assumptions:

1. Dual Channel Assumption: Humans have separate information processing channels for visually- and auditorily represented material. Information processing occurs in three stages (see Figure 1).
   Information enters our information processing system via either the visual or auditory processing channel. This is the input stage. The information is then processed separately but concurrently in working memory. Working memory can be thought of as RAM, where relevant sounds and pictures are selected and organized. Eventually the information from both channels are integrated and connected to other information already held in long term memory.
2. Limited Capacity Assumption: People are limited in the amount of information that can be processed by each channel at any given time.
   Learners participating in any presentation can only hold a few images and a few sounds in working memory at one time. Psychologists have researched this concept of cognitive load extensively: while allowing for individual variability, memory span tests have demonstrated that on average, working memory typically allows the processing of from 5 to 7 chunks of information at a given time. Because of this restricted cognitive processing capability, people continually make decisions about the allocation of sensory information processing based upon the available stimuli.
3. Active Processing Assumption: People actively engage in cognitive processing to construct coherent mental representations of their experiences.
   Rather than being passive 'information collectors, people are constantly selecting, organising and integrating information with past knowledge. Active learning occurs when we apply cognitive processes to the incoming material. The result of this processing is the creation of a mental model of the information presented. The three processes that are essential for active learning are: selecting relevant material, organizing the selected material and then integrating that material into existing knowledge structures. These processes take place within our fairly limited working memory.

Figure 1 Cognitive Theory of Multimedia
[Click to enlarge image]

From these, Mayer extrapolates a cognitive theory of multimedia learning. Mayer’s theory asserts that in multimedia environments, learners are engaged in five cognitive processes:

1. Selecting relevant words for processing in verbal working memory
2. Selecting relevant images for processing in visual working memory
3. Organising selected words into a verbal mental model
4. Organising selected images into a visual mental model
5. Integrating verbal and visual models and connecting them to prior knowledge

As such, I would assert that a substantial proportion of the the initial design stage of the instructional development process is devoted to ensuring that the Split-Attention Principle (Mayer, Moreno 1998) – that students learn better when the instructional material does not require them to split their attention between multiple sources of mutually referring information, the Modality Principle - students learn better when the verbal information is presented auditorily as narration rather than visually as on-screen text, both for concurrent and sequential presentations and the Redundancy Principle - students learn better from animation and narration than from animation, narration, and text if the visual information is presented simultaneously to the verbal information were adhered to.

---

Mayer, R.E. (2003). Multimedia Learning New York, Cambridge University Press.

Mayer, R.E. Moreno, R. (1998). A Cognitive Theory of Multimedia Learning: Implications for Design Principles University of California, Santa Barbara [Internet] Available from: http://www.unm.edu/~moreno/PDFS/chi.pdf Accessed 12th April 2008

--

An Open Environment for E-Learning Course Development: Project Lifecycle 2

Phase One: Using a Constructivist Theoretical Approach

Bruner’s 1966 text Toward a Theory of Instruction described the key principles of constructivism (p.225):

Table 1 Principles of constructivism

Principle		Definition
Readiness		Instruction must be concerned with the experiences and contexts that make the student willing and able to learn
Spiral organisation	Structure.	The content must be structured so that it can be grasped by the learner.
	Sequence.	Material must be presented in the most effective sequences.
Generation		“Going beyond the information given” - Instruction should be designed to facilitate extrapolation and or fill in the gaps

Extending from these basic constructivist principles as well as from the work of other key figures in the constructivist school, Driscoll (1994) outlines five conditions for learning (p.382-3). Very much like the multiple approaches and interpretations that exist in constructivism, a number of conditions must be met for the approach to be implemented. It is useful to elaborate briefly on these conditions, as they are relevant to the learning approach discussed in the rest of this part of the blog.

1. Providing complex learning environments that incorporate authentic activity. Constructivists argue that learners should learn to solve the types of complex problem they will face in real life. Learning how to do this is difficult unless complex and authentic learning environments are available to the learners.
2. Providing for social negotiation as an integral part of learning. Bruner (1986) explains that learning is a cultural interchange between group members. Collaboration creates an opportunity for learners to share their understandings with others and to have others do the same with them. This provides multiple perspectives to each learner, and this negotiation process between peers should lead to enhanced understanding.
3. Support multiple perspectives and the use of multiple modes of representation. Because learning skills, behaviours and knowledge can be diverse and complex, constructivists believe that to achieve complete understanding the learner must examine the material from multiple perspectives. If they are not supported in this endeavour, the learner will achieve only a partial understanding of the material. Multiple modes of representation allow the learner to view the same content through different sensory modes.
4. Nurture reflexivity. Duffy and Cunningham (1996) characterise reflexivity as “the ability of students to be aware of their own role in the knowledge construction process.” (p.172). It could also be described as the learner taking ownership of their own thinking and learning processes. Driscoll (1994) assets that reflexivity and by extension critical thinking are central attributes in the constructivist methodology, as it enables learners to understand how and why cognition creates meaning. This enables learners to attain goals such as reasoning, understanding multiple perspectives, and expressing and defending their own beliefs.
5. The last condition Driscoll describes is to “emphasise student-centred instruction.” Bruner (1966) calls this “discovery learning”. By obtaining knowledge by themselves, learners select and transform information, construct knowledge, and make decisions in the context of a cognitive structure that provides meaning and organisation to experiences and allows the individual to “go beyond the information given”. Students are actively engaged in determining what and how they will study or gain understanding.

These principles and conditions position the constructivist approach to learning as an appropriate orientation for learning sans frontiers; using technologies like the Internet, websites and virtual learning environments, applying collaborative learning, problem-based learning and goal-based mechanisms, making Open Source Software and Course- and Content Management Systems accessible to learners, and using e-learning applications like online conferencing and collaboration tools could be the foundation for these multiple constructivist conditions for learning. (Duffy & Jonassen 1992, Driscoll 1994; Schank 1994)

These characteristics provide an appropriate framework for knowledge workers to learn (and for the learning intervention), given that their ongoing development is based in the context of already-established cognitive schemata (from the learners’ perspective), the knowledge and skills are applied to solve real-world problems, and their expertise (behaviours) are typically used in collaboration with their peers to enhance the performance of organisations.

___________________

References:

Bruner, J. S. (1966) Toward a Theory of Instruction. Cambridge, MA: Harvard University Press.

Bruner, J. S. (1986). Actual minds, possible worlds. Cambridge, Mass: Harvard University Press.

Driscoll, M. P. (1994). Psychology of learning for instruction. Boston, MA. Allyn & Bacon.

Duffy, T. M. & Cunningham, D. J. (1996) Constructivism: Implications for the design and delivery of instruction. IN: Jonassen D. H. (Ed) Handbook of Research for Educational Communications and Technology (pp.170- 198). New York: Simon & Shuster Macmillan.

Schank, R. (1994) Active Learning Through Multimedia, IEEE Multimedia, 1(1), pp.69-78.

Typology of non-formal learning

Similar to Rogers’ learning continuum (2004), Eraut establishes a matrix to identify varying types of non-formal learning, based on the timing of the stimulus (past, current, future) and the extent to which such learning is implicit, reactive or deliberative. A key component of the matrix is the intention of the learner’s activity. Eraut makes a distinction between implicit learning, which has a metacognitive component, deliberative learning (where the worker schedules time to learn) and reactive learning (where learning is explicit but almost takes place spontaneously, in response to recent, current or imminent situations but without any time being set aside for it). I would contend that these categories align closely with Kolb’s 4-Stage Experiential Learning Cycle. Similarly, a link can be made between the synchronous and asynchronous delivery mechanisms discussed in an earlier blog entry and Eraut’s Timing of Stimulus category.

Table 1 Michael Eraut’s typology of non-formal learning (2000, p.129)

References:

Eraut, M. (2000) Non-formal learning, implicit learning and tacit knowledge, in F. Coffield (Ed) The Necessity of Informal Learning: Policy Press. Bristol

Rogers, A. (2004) Looking again at non-formal and informal education - towards a new paradigm [Internet] Available from: http://www.infed.org/biblio/non_formal_paradigm.htm [Accessed 30th January 2008]

--

Non-formal Learning in the Workplace

Looking specifically at learning in the workplace, Michael Eraut in Non-formal learning, implicit learning and tacit knowledge (2000) makes a clear distinction between his interpretation of the term ‘non-formal learning’ and what others including Scribner (1988), Conner (2002) and Cross (2003) would call ‘informal learning’ – what Eraut terms ‘incidental learning’ or ‘the acquisition of knowledge independently of conscious attempts to learn and the absence of explicit knowledge about what was learned’ (Reber, 1993, quoted by Eraut 2000, p.12) This is, he argues, because most workplace learning takes place outside formal learning contexts, and informal learning carries with it connotations of “so many other features of a situation, such as dress, discourse, behaviour, diminution of social differences – that its colloquial application as a descriptor of learning contexts may have little to do with learning per se” (p.12). Not only does the term carry unwanted and confusing implications, but it is too vague to be of any real utility. For Eraut, an analysis of learning must focus on activities and the outcomes that that contribute to significant changes in capability or understanding. In a sense, Eraut does not define non-formal learning; rather, he defines the characteristics of formal learning (p.12) as:

1. A prescribed learning framework
2. An organised learning event or package
3. The presence of a designated teacher or trainer
4. The award of a qualification or credit
5. The external specification of outcomes.

The implication of this categorisation is that any learning that does not exhibit all of these characteristics should be classed as non-formal. Some reviewers (Colley, Hodkinson & Malcolm, 2002) make the point that Eraut does not make clear what the status is of learning in situations that meet some, but not all, of his ‘formal’ criteria. My interpretation of his characterisation is that the very nature of a formal activity - “following or according with established form, custom, or rule” (Merriam-Webster Online, 2007) validates Eraut’s description.

Tomorrow: Michael Eraut’s typology of non-formal learning

References:

Colley, Hodkinson, Malcolm (2002) non-formal learning: mapping the conceptual terrain. a consultation report [Internet] Available from: http://www.infed.org/archives/e-texts/colley_informal_learning.htm [Accessed 28th January 2008]

Eraut, M. (2000) Non-formal learning, implicit learning and tacit knowledge, in F. Coffield (Ed) The Necessity of Informal Learning: Policy Press. Bristol

--

More on Non-formal Learning

As discussed in a previous blog entry, one of the central components of the impact of learning (and specifically the development of knowledge workers’ expertise in organisations) is the context within which the learning takes place. A central pillar of this discussion is the type or format of the learning taking place. In the literature, it is apparent that a dichotomy exists between the paradigms of formal, goal-directed training programmes and informal – “learning at the watercooler” (Grebow, 2002) or what Michael Eraut (2000) describes as incidental learning that takes place almost as a side effect of work: “it is difficult to make a clear distinction between formal and informal learning as there is often a crossover between the two” (McGivney, 1999, p.1). Another complexity in the discussion is where is non-formal learning located in relation to the diametric opposites? For much of the 40 years since the term ‘non-formal learning’ was first coined (Coombs, 1968, p.1.) there has been a great deal of debate in the literature as to the nature of formal, informal and non-formal learning; the components of each of the paradigms, their boundaries and their overlaps. The locus of this debate is centred on arguments for “the inherent superiority of one or the other” (Colley, Hodkinson & Malcolm, 2002, p.2).

I support Alan Rogers’ (2004) view that a “new paradigm” for learning exists, in which “most programmes will be partly formal and partly informal” going from formal to informal and from informal to formal in both directions along a continuum (see Figure 1) . “Both forms of education are important elements in the total learning experience” (looking again at non-formal and informal education - towards a new paradigm, 2004).

Figure 1 the Learning Continuum

Similarly, Hodkinson and Hodkinson argue that focusing on the extent to which learning is planned and intentional may be a way of by-passing the distinction between formal, non-formal and informal altogether.” (Colley, Hodkinson & Malcolm, 2002).

References:

Colley, Hodkinson, Malcolm (2002) non-formal learning: mapping the conceptual terrain. a consultation report [Internet] Available from: http://www.infed.org/archives/e-texts/colley_informal_learning.htm [Accessed 28th January 2008]

Coombs, P. (1968) The World Educational Crisis, New York, Oxford University Press.

Eraut, M. (2000) Non-formal learning, implicit learning and tacit knowledge, in F. Coffield (Ed) The Necessity of Informal Learning: Policy Press. Bristol

Grebow, D. (2002) At the Water Cooler of Learning [Internet] Available from: http://agelesslearner.com/articles/watercooler_dgrebow_tc600.html [Accessed 30th January 2008]

McGivney, V. (1999) Informal learning in the community: a trigger for change and development NIACE. Leicester.

Rogers, A. (2004) Looking again at non-formal and informal education - towards a new paradigm [Internet] Available from: http://www.infed.org/biblio/non_formal_paradigm.htm [Accessed 30th January 2008]

--