I consider a case study methodology to be the most effective strategy to use when evaluating non-formal learning because this approach provides the means to develop a rich description of the
This assists in providing a context, and helps define the NFL initiative's value to the learners and to the organisation. An added benefit of the case study approach is that it enables the learning practitioner to employ a number of different data collection methodologies: participant observation, archive data collection, surveys, and interviews to answer the questions outlined in Table 1. The main purpose of using a number of data collection methodologies was to capture a range of quantitative and qualitative data, and to establish a means of triangulating that data to enhance validity and “trustworthiness” (Russ-Eft & Preskill, 2001, p.153).
Table 1 Questions the case study approach attempts to answer
| Quantitative Data | Qualitative Data |
| Who attends live “in situ” the learning event? (dept., role, skill level, location) | Description and chronology of a the learning event |
| Who views synchronous live stream of the learning event? (dept., role, skill level, location) | Transfer of new knowledge to the job (Kirkpatrick - Level 3) |
| Who views asynchronous on-demand version of the learning event? (dept., role, skill level, location) | Does the pedagogical approach used in |
| Who requests asynchronous DVD version of the learning event? (dept., role, skill level, location) | |
| Participants satisfaction with | What is the Business Impact of the |
| What do attendees know that they didn’t know before? (Kirkpatrick - Level 2) | |
Russ-Eft, D. & Preskill H. (2001) Evaluation in Organizations: A Systematic Approach to Enhancing Learning, Performance and Change.
--
2 comments:
some people don't like case study research (though I think its great). A good rebuttal of the doubters is given in Five MisunderStandins of Case Study Research: http://flyvbjerg.plan.aau.dk/MSFiveMis9.0SageASPUBL.pdf
Thanks for the comment and the link Eamon.
My own opinion is, in research it's a case of "horses for courses"; most "hard science" advocates I know would argue that if you can't generalise from a hypothesis, or reproduce the experiment, then the research has no validity and as such is a failure. QED
However, those of us involved in training / learning know that it's not so simple: no two learners are exactly the same, and the circumstances surrounding every learning intervention mitigate against purely empirical research.
One of the reasons I'm posting in such (tedious?) detail about the validity of qualitative approaches to evaluating learning, is that it seems to me that nobody is really standing up for the case study approach.
As in the case of Donald Kirkpatrick's Four Level Model, about once a month you'll see a polemic about how Kirkpatrick is irrelevant, outmoded or even (my favourite) "bollocks." The authors of such pieces tend not to provide a better solution to the task of evaluating learning though. Similarly, case studies are dismissed out of hand, which is fine, if you can show me an effective replacement for the case study methodology.
Cognitive processes and learning are almost by definition qualitative in nature; in my opinion it is appropriate to attempt to understand them using all the research instruments available.
Post a Comment