Thoughts on Evaluating Non-Formal Learning

In previous posts, I've suggested that knowledge workers under time-pressure and with high demands on their skills are motivated to continually educate themselves on new topics. In small- to mid-sized enterprises, for example, workers are required to learn new skills, behaviours and attitudes in the context of their functional tasks. As they do not necessarily have the time to take formal training courses, they take advantage of the range of knowledge resources at their disposal, ranging from searching the Web, corporate knowledge bases, and other information repositories, as well as discovering information incidentally through social interaction with credible colleagues to assist their ability to “frame the problem” (Argyris & Schön, 1974).

As organisations are at root, cognitive enterprises, and the sum of the knowledge of the organisation is expressed collectively through the behaviour, skills and attitudes of its employees, the organisation is itself a cognitive entity. One of the ways that such an entity continually constructs itself is through the learning processes its members engage in.

Situated, experiential learning, delivered:

1. synchronously in a social setting where workers can engage in a dialogue with More Knowledgeable Others, subject matter experts and their peers,
   
   or,
   
   
2. asynchronously accessed when the individual learner has a requirement to acquire information and knowledge relevant to a task at hand

in relevant, context-specific formats, whether in a classroom, live online, on-demand over the web, or via an easily accessible format such as a DVD-ROM to suit a range of learning styles and environments, creates the conditions for “meaningful learning” (Jonassen, 2001) to occur.

Providing workers with a knowledge-centric learning and performance architecture, and structured, goal-oriented content, gives them the motivation or intention to learn, enables them to process “raw” information into actionable knowledge on an ongoing basis, and regularly inhabit a scaffolded environment which encourages the learner’s development. Digitally-based technologies are the optimal mediator for this learning process, which is called ‘non-formal’ in recognition of its flexibility and accessibility for learners, and also because of its pedagogical structure and outcomes-focus.

Over the next week or so, I'm going to look at ways to quantify the learning effect of non-formal learning its target audience beginning with an overview of that stalwart of evaluation, Donald Kirkpatrick.

References:

Argyris, C. and Schön, D. (1974). Theory in practice: Increasing professional effectiveness, San Francisco: Jossey-Bass.

Jonassen, D. H. (1996) Computers in the Classroom: Mindtools for Critical Thinking. Upper Saddle River, NJ. Prentice Hall Inc.

--

What is learning?

I guess it's now time to move on to some of the "big picture" stuff and begin to outline how I characterise terms like "learning" and "e-learning."

---

In the field of organisational development, the terms ‘training’ and ‘learning’ are often used interchangeably: both broadly refer to the acquisition of new or enhanced knowledge, skills, attitudes or behaviour through the medium of instruction whether delivered by a source of information such as a text or a computer application, a teacher, or a mentor. In my opinion, the key difference between the two terms is in the reflexive nature of learning, resulting in the integration of the knowledge, skill, attitude or behaviour into the individual’s cognitive constructs - a process Lev Vygotsky called “internalisation,” which he elegantly described as “an internal reconstruction of an external operation (1978, p.56). The Information Sessions non-formal learning initiative (which I'll be discussing in future blog entries). If you've read my recent posts, you'll understand that the theoretical and methodological framework for this initiative was developed within the Constructivist pedagogical model using techniques derived from the Social- and Cognitive schools of thought, with a focus on experiential learning as defined by Kolb, Vygotsky’s social development theory as it is applied in instructional design and Donald Schön’s and Chris Argyris’s views on professional knowledge and social learning systems.

References:

Vygotsky, L. S. (1978) Mind in society. Edited by Cole, M. John-Steiner, V. Scribner, Souberman, E. Cambridge, MA, Harvard University Press

Constructivism Pt.13: More Organizational Learning

It could be argued that components of Argyris’s and Schön’s position do not conform to the constructionist tradition, and it is possible to discern a positivist aspect to their thesis, particularly in the exposition of their notion of theory-in-use, which in my opinion exhibits characteristics of behaviourist patterns – for example B.F. Skinner’s ideas on operant conditioning in the strategies employed in single-loop learning. Similarly, it has been argued elsewhere (Easterby-Smith & Araujo, 1999) that Argyris’s and Schön’s work on Model I and Model II organisations relies too heavily on assumptions of what “good learning” (p.13) consists of, but that discussion is beyond the scope of this paper. Nonetheless, as Finger and Asún point out in Adult Education at the Crossroads. Learning our way out:

Unlike …Kolb’s learning cycle, where one had, so to speak, to make a mistake and reflect upon it – that is, learn by trial and error – it is now possible thanks to Argyris and Schön’s conceptualization, to learn by simply reflecting critically upon the theory-in-action. In other words, it is no longer necessary to go through the entire learning circle in order to develop the theory further. It is sufficient to readjust the theory through double-loop learning.

(pp.45-46)

References:

Argyris, C. and Schön, D. (1974). Theory in practice: Increasing professional effectiveness, San Francisco: Jossey-Bass.

Asún, M. and Finger, M. (2000) Adult Education at the Crossroads. Learning our way out, London: Zed Books.

Easterby-Smith, M. & Araujo, L. (1999) Current debates and opportunities. IN: Easterby-Smith, M. Araujo, L. & Burgoyne, J. (eds.) Organizational Learning and the Learning Organization: Developments in Theory and Practice, London: Sage

Constructivism Pt.12: Organizational Learning cont'd

In Organizational learning: A theory of action perspective, Chris Argyris and Donald Schön suggest that each member of an organisation constructs their own representation of the actual, tacit, applied organisational behaviours, also called its “theory-in-use” (1978, p.16). Argyris and Schön modelled theory-in-use to investigate its three components:

Table 1 Components of theory-in-use (after Liane Anderson, 1997)

Component	Definition
Governing variables:	Those dimensions that people are trying to keep within acceptable limits. Any action is likely to impact upon a number of such variables – thus any situation can trigger a trade-off among governing variables.
Action strategies:	The moves and plans used by people to keep their governing values within the acceptable range.
Consequences:	What occurs as a result of an action. These can be both intended - those actor believe will result - and unintended. In addition those consequences can be for the self, and/or for others.

They developed a simple three-phase model (see Figure 1) to illustrate how each of the components of theory-in-use interacts:

Figure 1 Model explaining the process of developing theories-in-use

This cognitive structure has implications for knowledge workers’ cognitive processes. Where the outcomes of the strategy align with what the individual wanted, then the theory-in-use is confirmed as there is a match between intention and outcome. However, a mismatch between intention and outcome may occur with unintended consequences. Outcomes may also not match, or even counter the person’s governing values. Argyris and Schön suggest two responses to this mismatch, and these are can be seen in the notion of single and double-loop learning. For Argyris and Schön (1978, p.2) learning involves the detection and correction of error. In single-loop learning (see Figure 2), when outcomes do not match expectation the initial reaction of knowledge workers is to employ a different strategy from their repertoire that will bring about the required consequences within the governing variable. The change is in the action only, not in the governing variable itself.

Figure 2 Single- and double-loop learning

References:

Anderson, L. (1994). Espoused theories and theories-in-use: Bridging the gap (Breaking through defensive routines with organisation development consultants). Unpublished Master of Organisational Psychology thesis, University of Qld.

Argyris, C. and Schön, D. (1974). Theory in practice: Increasing professional effectiveness, San Francisco: Jossey-Bass.

Constructivism Pt.11: Organizational Learning

In their 1974 work Theory in practice: Increasing professional effectiveness, Chris Argyris and Donald Schön support the constructivist argument that individuals (and in the context of this paper particularly knowledge workers) have schemata or mental maps both for their skill assets and they contend, with regard to how to act in situations. This directs the way they plan their activities, execute their tasks, and how they reflect upon their actions once tasks (or components of larger-scale tasks) are complete. They assert that it is these mental maps that guide people’s actions rather than the theories they explicitly espouse. Further, knowledge workers (unlike academics, for example) are typically required to apply their skill- and experience assets in real-world situations which exhibit degrees of uncertainty about both the situation itself and the desired outcomes. Much of the real-world job of the knowledge worker is more concerned with problem setting then problem solving. To move from a problematic situation to an actual problem, the practitioner must “frame the problem: …determine the features to which they will attend, the order they will attempt to impose on the situation, the directions in which they will try to change it. In this process, they identify both the ends to be sought and the means to be employed” (p.165). This process is what Schön describes as reflective practise. He divides reflection practise into two subcategories, notions of reflection-in-action, and reflection-on-action. Reflection-in-action has been colloquially described as ‘thinking on your feet’ and involves building new understandings based on previous experiences to predicate actions in the situation that is unfolding at present:

The practitioner allows himself to experience surprise, puzzlement, or confusion in a situation which he finds uncertain or unique. He reflects on the phenomenon before him, and on the prior understandings which have been implicit in his behaviour. He carries out an experiment which serves to generate both a new understanding of the phenomenon and a change in the situation.

(p.68)

Thus, when investigating a situation individuals are consider, and use, strategies based upon their repertoire, the situation’s frame of reference, what has gone before and potential outcomes. Michael Eraut in Developing Professional Knowledge and Competence (1994) - who we will encounter later in this chapter when we discuss non- and informal learning, and as such is worth discussing here - negatively criticises Schön’s evaluation, considering reflection-in-action and reflection-on-action as a iterative process rather than as discrete actions:

When time is extremely short, decisions have to be rapid and the scope for reflection is extremely limited. In these circumstances, reflection is best seen as a metacognitive process in which the practitioner is alerted to a problem, rapidly reads the situation, decides what to do and proceeds in a state of continuing alertness.

(p.144)

References:

Argyris, C. and Schön, D. (1974). Theory in practice: Increasing professional effectiveness, San Francisco: Jossey-Bass.

Eraut, M. (1994). Developing Professional Knowledge and Competence. London: Falmer Press.