February 12, 2011
learning 2.0

Dear Kirkpatrick’s: You Still Don’t Get It

In the February edition of CLO Magazine, I had the opportunity to write an editorial entitled “Time’s Up – Learning Will Forever Be Part Formal, Part Informal and Part Social”.

Suffice to say I’ve caused a bit of a stir in the Kirkpatrick camp.

Specifically, I received an email from Wendy Kirkpatrick, Manager of Kirkpatrick Partners. It’s evident she took umbrage with the editorial.

You compared the model to a cockroach. I found this both unprofessional and offensive. 

Comparing the Kirkpatrick Four Levels model to the cockroach was actually a compliment. Why is it that the cockroach can’t shake its bad reputation? Surviving 350 million years should be thought of as positive, and for the Kirkpatrick’s, the model has filled similar boots. It wasn’t unprofessional & it wasn’t offensive; it was a comparison. Haven’t you seen Wall-E?

You did not take the time to check to see if the Kirkpatrick Four Levels have actually been updated in the last 52 years.

Actually, I did. In an editorial with limited word space, I did not have the print real estate to delve into your so-called updates. An update would have been doing away with the four levels itself and starting from scratch, defined for 2011 and beyond. An update to the model wouldn’t instruct your legion of followers that “any successful initiative starts with clear definition of the desired outcomes” and that “it can be accomplished by using the Kirkpatrick Four Levels model in reverse.” Reverse? Seriously? Let’s obfuscate an already overly complicated process by using the model in reverse during the planning stage, and then using it again (predictably in the regular order) to achieve said measurement. That’s not an update Wendy, that’s a conundrum.

Furthermore, in the ‘updated’ model there is noble mention of methods to increase confidence and commitment. Excellent, however, you further dig a hole by equating training and events to learning as the sole way in which we learn. See the bold and italicized words from your updated model:

  • Include the higher-level purpose and goal of learning in communications before and in discussions during the event. Make sure everyone involved knows the importance of applying what they learn when they are back on the job.
  • Allow enough time for skills practice during training.
  • Near the end of a session, facilitate a discussion about what it will be like to apply the new skills on the job. If anticipated barriers are mentioned, work collaboratively on solutions.
  • Include a question about confidence in the after-course evaluation form. For example, “To what degree are you confident that you will be able to apply what you learned?”

All of the work that my husband Jim and I do stresses the importance of what happens after the training event.

Thank you once again for articulating exactly why the Kirkpatrick Four Levels model is in need of a divergent path from the cockroach. Let me be clear – training is not an event; learning is a connected, collaborative and continuous process. It can and does occur in formal, informal and social ways every day in and out of your job. In your email, with the statement “what happens after the training event”, you have cemented (again) the root cause of the Kirkpatrick model. The ‘event’ is not solely how learning occurs. Whether in the original model, or the weakly updated model, the single largest flaw with the Kirkpatrick Four Levels model is the fact its basic premise is that learning starts with an event. Once you ultimately get past this stumbling block, the Kirkpatrick Four Levels model will potentially become relevant again, should it be suitably updated again.

Finally, I would like to say that you may want to consider when critiquing the work of others that there are real people behind the work.

I am well aware there are real people behind the Kirkpatrick Partners organization but I have no gripe with the people, your husband, Donald, whomever. My bane is with the model itself. It seems rather odd that an organization like Kirkpatrick Partners, with its history, clout and following needs to resort to playing the husband and people card. Or, perhaps, you are implying I’m somehow personally attacking the people of Kirkpatrick Partners in the editorial? If so, I’d suggest some thicker skin, and quite honestly to focus your efforts on a concept like RPE (Return on Performance and Engagement) itself.

In conclusion, I believe Kirkpatrick Partners have an ethical responsibility to the learning profession to wake up, throw out the existing (sorry, the updated) model, and demonstrate continued albeit new leadership. Kirkpatrick Partners are an entrenched entity within not only the learning profession, but within the corporate board rooms of our planet. You are no doubt aware that extracting the model from the profession is a near impossibility due to the gravity like grip it holds. Perhaps, however, you might consider these words and set a new course for Kirkpatrick Partners.

And you might want to read a few books if you haven’t done so already, such as The New Social Learning, Enterprise 2.0, The Future of Work and New Culture of Learning.

We never thought there would be water on Mars, but were proven wrong on this as well.

14 Replies to “Dear Kirkpatrick’s: You Still Don’t Get It”

  1. Absolutely brilliant. I think this is a classic example of someone who is ignoring Colin Powell’s maxim, “Never let your ego get so close to your position, so that when your position falls, your ego goes with it.” Sadly, until we get the point across that learning isn’t just an “event”, we’ll be stuck with this outdated thinking.

  2. Here, here! Excellent post, Dan.

    One book that you should add to that list is ‘The Working Smarter Fieldbook’, ~ Internet Time Alliance.

    For me, this sums it up…

    “Let me be clear – training is not an event; learning is a connected, collaborative and continuous process. It can and does occur in formal, informal and social ways every day in and out of your job.”

    I’m currently researching the Finnish public school system as an example of best practice in the Education sector. One of the HUGE differences with their model is…

    “The only point at which all Finnish students take standardized exams is as high school seniors if they wish to go to university.” Finland leads the way in Education, perhaps worth a visit to find out why that is and how it can be applied.

    By looking at current best practices one can develop next practices that create a real shift, but one must want to change and evolve. The consequence of not reacting will destroy all that you worked hard to achieve. Disruption is knocking loudly on your door, I wouldn’t wait too long.

  3. Dan, I could not agree more. I would go a step further and add Jack Phillips and Level 5 ROI into the mix. That concept has gotten so deeply entrenched into boardroom and in particular CFO mentality that we find ourselves chasing ROEDT…return on every damn thing. ROI may work for a piece of hardware that enhances productvity and clearly shown a return without the subjectivity of “confidence” and “probability” tossed in to compensate for the inherent voodoo of the process.

    I wrote a blog post last week on a concept called “EOSC…evidence of sustained capability”. This concept goes beyond just targeting outcomes, for they [outcomes] carry no value unless they are sustainable. Blame it on my healthcare experiences, but nothing is accepted or trusted without “evidence”.

    We need a new model. If training were the silver bullet, then four levels may be enough. Training is enough if all you want to track is training. But…the learning ecosystem is exponentially bigger than any cluster [take that word as you wish] of training courses. Training may always be a part of the mix, but it is diminishing rapidly as the velocity of business demands causes a convergence of learning moments with the need for flawless performance.

    I just spoke at Training 2011 and sat in a a bunch of sessions and we are all saying that informal and social learning are upon us and it’s time to leverage Web 2.0 technology and yada yada… It is time to re-invent what we call training and expande the paradigm to move down-stream, post-training [events] and into the work context where the workforce is engaged in the generation [or destruction] of tangible business value. No sales reps cloes business in the classroom event. No customer service reps resolve a complaint while taking an on-line learning course. They do those things under attributes of urgency and business risk. The game has changed. We much to measure and 95% of it has nothing to do with training. We swhould be seeking evidence that our efforts of the aggregate of formal, informal, social, celestial [next, etc. learning are generating the capacity in our workforce to sustain capability.

    Whew…nearly became a rant…not that there’s anything wrong with that.

    Take good care!


  4. Great stuff Dan!

    I attended a Kirkpatrick seminar last May in London and found it to be a particularly unedifying experience. It was heavily Powerpoint based and at one stage, Jim Kirkpatrick pointed to a slide and stated ‘Only 5% of learning is retained, the rest is quickly forgotten’.

    I raised my hand and suggested that this might be a result of the confusing and inappropriate Four Levels model. Jim was momentarily speechless and then branded me as a disruptive trouble maker, rather than welcoming my observation as a potential learning opportunity.

    The main purpose of the Four Levels model in the 21st century is not to bestow insight and awareness but to provide a revenue stream for Kirkpatricks (various). Jim more or less admitted this when he was try to sell us his latest book at the end of the seminar, a book whose principles he stated would soon be out of date, just in time for the publication of his next dreary tome.

  5. Excellent, Dan!! We really do have to get away from the association of events, training, courses etc with effective learning. All of those things may lead to learning. None of them are learning in themselves. As soon as we understand that learning becomes evident when someone does something differently (hopefully better) as a result of a stimulus we will be on the way to understanding its crucial place in the lives of individuals and organisations. Clinging to an old paradigm is ultimately doomed to the discard pile in a world where everything now tends to the meta and in which our understanding and insight is developing rapidly

  6. All together now…
    ______________ is the best teacher.

    (this is the easy one- everyone gets this)

    Now- follow up question:
    ________________ model accounts for this fact in this manner:

    (couldn’t do it? neither could I!)

    Simplest way to illustrate why there are severe limitations to the current eval models (not just Kirkpatrick). If metalearning and metacognition have taught us nothing else, it is clear that current models are not tooled to recognize the larger, connected ecosystem that is the reality of workplace learning today.

    I think there are two things that need to be said here:

    1. Thank you, Kirkpatrick. Don, you opened many eyes. It is because of your work that we are able to have this discussion today. So, a sincere thanks for laying some groundwork; we wouldn’t be where we are today without you setting it in motion.

    2. Now, we invite you to join us on the ride to discover what is needed for today’s workplace. Everyone is welcome, but a respected leader would be great to have on the team.

    It is clearly time to evolve, but we can’t discover new worlds if we aren’t willing to pull the anchor.

  7. Hey there Dan:
    The positions both you and Wendy have are very representative of competing positions in the learning profession. I’m not sure the polarization is serving us well. There are truths on both sides of the fence and the artificial dichotomy between formal and informal learning are broadening the gap. Some thought’s on that on my blog here:

  8. Thanks to all for the feedback thus far.

    Let me be clear – I am more than happy to laud, recommend and point back to Kirkpatrick Partners if they took their years of experience and helped properly reshape things.

    I am not trying to create an artificial dichotomy; I am yearning to see a model that is truly representative of all things formal, informal and social. One that equates equally your oontributions and your consumption. One that makes the correlation between your network, your skills, your learning … to engagement, performance and business results.

    That’s what I yearn for. That’s what I’d love to see Kirkpatrick Partners take on.

    I’m merely calling out the ‘white elephant’. No hate mail as of yet though.

  9. Great post, Dan, but I don’t think anyone in the Kirkpatrick camp has ever said “learning is an event”…they only reference “training as an event” (which is true). Your post implies that “training” and “learning” are one in the same. The word “train” is defined as follows: to make proficient by instruction and practice. The word “learn” is defined differently: acquire knowledge of or skill in by study, instruction, or experience. In my opinion, learning has been the continous, never-ending, social process by with we grow and develop. Training is simple study, instruction and experience (much more limiting). My gut tells me the Kirkpatrick folks may view them as they were originally defined (all a matter of semantics). With that said, I love your post, Dan…all insightful and good points. I just happen to love the great work Kirkpatrick Partners have done over the years a bit more.

  10. There are many evaluations systems that have surfaced only to loose the marketing war against the Kirkpatricks. Research should be scoured, results should be collected, and a symposium conducted so learning and performance development practitioners everywhere are given better choices. I can think of a half a dozen models from MarComm, cog psychology, finance, and other fields that are far more adaptive to how people process and use information today.

  11. I think that the Kirkpatrick model did do a lot to get us thinking about the quality and impact of formal “Training” sessions. It is very important to measure what we are doing and adapt/adjust to the results. That’s the true value of their model, yet they, ironically have not adapted/adjusted to the new needs and climate.
    I design and deliver formal training sessions for a living and I hate when people call it “Training”. It’s Learning! When people start thinking of it as a process, condition, environment we will be a lot further along.

  12. Dan, Never seen such a piece written in this way that allows a clear personal vendetta to come through in you choice and use of language.

    Keep it professional please!

  13. Dan,

    Well written! Entrenchment; institutional resistance; inertia-all commonly used terms in our profession to outline the false security in those elements of learning and allow doubt to enter into our minds at any hint of change to the status quo. I often refer to this as a fear of being innovative. Another good read on the topic can be found in What Got You Here Won’t Get You There by Marshall Goldsmith and Mark Reiter. Hopefully Kirkpatrick Partners will take the advice and join the march for innovation vice status quo!

Leave a Reply

Now Available!

Dan’s 4th book is now available. Click to order LEAD. CARE. WIN.

Buy Dan's Books
Listen to Dan's Podcast
Read Dan's Column On