Training ROI, a Lie or Just More Effort?

Workplace training is (or should be) a business function, yet management often does not treat it as credible – for good reason in an overwhelming number of cases. For trainers, instructional designers and Learning and Development (L & D) folks housed in HR, it is imperative that they start with how to align training results and organizational results. While no one likes a poorly organized, demotivating lecture that does not sustain the motivation to learn (Level one evaluation), it is important to move far beyond the learner satisfaction level of evaluation. If the L & D folks cannot demonstrate skill/knowledge learning, retention and application over the long term for the benefit of the organization, then it will impossible to have a shred of credibility that prevents training from (rightly) being seen as merely a cost-centre. Meeting strategic business objectives are central in business, and therefore tantamount to where the rubber meets the road in L & D

While the Return on Investment (ROI) Methodology, developed by Dr. Jack Philips, takes some effort and time to seek the data and calculate, ROI should be something to strive for, especially when training investments and stakes are high. Only roughly 5% of organizations actually conduct ROI on their training programs, and there should be a lot more efforts to tie efforts to business results (i.e. organizational level performance and the bottom-line). But to throw up one’s hands and say ROI or even Level 3 Evaluation (application of learning) is too difficult or can’t be done, is a “cop out.” Jim Kirkpatrick’s new twist on the Four Levels of Evaluation, developed by his father (Donald Kirkpatrick), is a clever retooling of the 50+ year old model. I enjoyed reading Training on Trial, Jim’s recent book on the new direction and energy for capturing training program results and tying them to valued business results.

The Four Levels now wisely begins with business results focus (Level 4), yet highlights “ROE” or Return on Expectations (*not to be confused with the C-Suite term “Return on Equity”). Of course, meeting stakeholder and particularly sponsor expectations, should be a given in any engagement -[internal or external. This new take on “ROE” will confound senior managers such as CFOs, who know it as Return on Equity. Trying to pass the new ROE term off as legitimate for strategic learning partners seems foolish. It amounts to little more than a return to the much maligned smiley sheet or Level 1 measure of participant satisfaction or reaction). A colleague of mine is presenting on what he sees as The Lie of Training ROI. To me, throwing out Return on Investment (i.e. unofficial Level 5) in the name of convenience in Kirkpatrick’s new “ROE”, is absurd.

“Return on Expectations” is noble in principle and yet a banal client or sponsor expectation for all internal or external projects. To ignore the numbers (i.e. ROI calculation) by ignoring them in favor of this new take on “ROE”, will not sit well with executives or help  legitimize L & D practitioners as strategic business partners who deserve a seat at the table. Trainers, business analysts, consultants and others should continue to strive for higher levels of evaluation in their work – ideally Levels 3 through 5 (application through to ROI). The thought of shaking off the responsibility to evaluate the efforts of instructional designers and trainers alike, in the name of a vague and less measurable concept (ROE) is nonsensical at best, harmful at worst. Jack Phillips wrote an article stating his position on the ROI vs. ROE debate.

I recently met several high qualified, experienced evaluators and performance improvement practitioners at the International Society for Performance Improvement conference, who also hold membership in evaluation societies (American Evaluation Association and Canadian Evaluation Society respectively). These evaluation specialists support evidence-based practice, agreed that the new Return on Expectations concept is not so new after all, and will never replace tangible metrics for training’s value-add to the organization. Personally, I side with the competence of Jack and Patti Pulliam Phillips, founders of ROI Institute, who developed the ROI Methodology, which they have written about in 35 books, and translated into over 30 languages.

Beginnings of “Engineering” Staff Performance

“…engineering is always a system of manipulating things.”-Thomas Gilbert, 1978

Recently I began working with a construction client, and met with one of their civil engineers. His card reads P. Eng behind his name, indicating his professional engineering license. His firm has sought my help to improve work in one area of their operation. When I looked at his business card I told him that I too was an engineer, only for human performance. Like bridge or road construction engineers, staff performance engineers must consider context, function and inputs or what I call “The 3Ws” – Workplace, Work and Workers.

I chuckle as I recall getting an engineering scholarship on a technicality since my program of study – Instructional and Performance Technology – was actually in the College of Engineering! Discussions around how it came to be that this cross-disciplinary field was housed in the Engineering faculty linked to the use of evidence-based, customized and systematic design of training and non-training solutions to solve workplace performance problems.

Dr. Thomas F. Gilbert, a Psychologist, wrote the seminal book which helped launched the field of Human Performance Technology (HPT). Gilbert is often referred to as the “Father of HPT.” Gilbert’s book, Human Competence: Engineering Worthy Performance (1978) stem from his work revealing learning programs frequently did not result in knowledge or behavioral change, and that other techniques (often in combination) were more fruitful. A Behaviorist, Gilbert focused on overt, measurable changes and specialized in testing and statistics. His main equation: (P = B x E) express that performance is a product of Individual and Environmental factors, depicted in his Behavior Engineering Model (BEM).

The BEM aids systematic analysis of root causes for a performance challenges at work, uncovering barriers and enables to improving results. The BEM is reminiscent of Gilbert’s Skinnerian background centering on stimulus-response study. Gilbert’s six-celled BEM has three Environmental factors – Data/feedback, Resources, and Incentives, and three Individual factors – Knowledge/Skills, Capacity and Motivation. Gilbert advocated analyzing Environmental factors first in that order (D, R, I) followed by the Individual ones (K/S, C, M). That would leverage the cheaper and more effective solutions first and gradually move towards the pricier, more difficult areas to address. Gilbert recognized these six variables as critical to manipulating for performance improvement. Today an award from HPT’s flagship organization – the International Society for Performance Improvement - bears Gilbert’s name, to honour his contributions to the field.

In working with my construction client I will analyze and address their group’s performance challenge by adopting a systems-level focus with “The 3Ws”, and drilling down for a closer examination of the third – Workers – with the aid of Gilbert’s BEM.  Happy performance engineering!

What generation are you from?

Poll takers wanted. Curious to see the variation of generations in the workforce.

Generations at Work, or Stereotyping?

Most people have heard of “The Generations at work” discussion theme centered on how individuals’ age differences determine how people act in work and learning. I’ve seen many seminars/articles on this topic (since 2005) and haven’t seen a positive result of the “Generations at work” seminars, articles and discussions. To me it’s tantamount to stereotyping. It neatly slots people into convenient categories with (presumed) behaviors. There is not always a good fit. For example, my mother is a “Traditionalist” and a computer savvy 71 year old who works full-time. She does not fit the mould for her age group. I have also met a few young people who shunned computers (rare as it is). While there are common themes often seen with certain age groups related to learning and work behaviors, it’s best to avoid generalizations.

Of greater value would be a session on this topic that opened up the dialogue about effective communication methods (e.g. pairing younger supervisors with much older supervisors for reverse mentoring on social media in exchange for role mentoring). The bottom line seems to be work expectation clarity. The same holds true for the training room – avoid painting a certain type of learner with the same brush, or you could end up unknowingly facilitating a self-fulfilling prophecy (i.e. people act like you expect them to).

Challenges with age and ability in work and learning settings have always existed (or co-existed) in classrooms and workplaces. Only in recent years have we have legislated fairness, and have various differences among staff have become more prevalent. This concept of “the adaptive dimension” (accommodating differences) takes me back to Teacher’s college (1994). All trainers/teachers need to adjust their methods or materials accordingly to respect and best meeting individualized learners’ unique needs as best they can in a group setting. In this light, it’s hard to see “The Generations” as a new, ground-breaking or value-added concept.

Canada’s long history of accepting and integrating cross-cultural, linguistic and other differences into a “Multicultural Mosaic” (as opposed to a “Melting Pot”) has become one of the distinct features of the Great White North. The acceptance and celebration of diversity – in the workplace, the classroom, and society in general – will continue to pave the way to economic success through respectful appreciation, cooperation and collaboration in years to come.

Appreciating cultural and other differences at work or in learning settings ironically helps us see how we are each unique. If you “Think different” rather than encouraging homogeneity, you open the doors to organizational innovation that’s critical to surviving and thriving in a dynamic and rapidly changing world. Learning to appreciate different perspectives, experience levels and skills of all staff including those of different age groups, aids group cohesion. McDonald’s has benefited from standardization of its processes and practices for a uniform product, no matter which of its locations you go to. With people, appreciating difference leads to understanding it. Stephen Covey, author of “Seven Habits of Highly Effective People” seems to agree – “Seek first to understand, then to be understood.” Supervisors would be well advised to do the same and not presume that a “Millennial” employee is being rude or evasive by texting his supervisor a message instead of calling. Providing clear expectations and making an effort to find a middle ground at work enables respectful communications for working towards mutual goals. Focusing on the ties that bind is more beneficial than concentrating on “Generational Differences”, which may not be valid and can do a disservice by painting members of an age group with the same brush.

Motivation and Learning: Ties that Bind

“Organic” adult learning grows out of love. When someone has a burning desire to learn something they typically find the teacher (or source). There is a strong link between motivation and the desire to acquire new knowledge and skills. The desire or need to learn grows out of relevance, prior learning and social or professional needs (e.g. work, parenting). In the Information Age where data is everywhere, we now have the freedom to learn nearly anything at any time. No longer are certain skills or knowledge reserved to those in the ivory tower or corner office.

At Work

For employers investing in work-related learning and development or training, there isn’t always the natural love of learning to inspire the effort. Still, being learner-centered and therefore tapping into what each person finds motivating, is key. The exception of course is “compliance-based training.” Getting trainees fired up for a long round of mandatory safety training videos or slides depicting information they already know, may require an endless supply of free cookies, paid training time and a good humoured facilitator (bribery and cajoling perhaps?).

Research has repeatedly shown that money is a short-term incentive at best and does not lead to long-term or meaningful engagement in task completion. While workers may jump ship for higher pay or end up poached like eggs, employers waving the almighty dollar hoping to capture the heartfelt desire to add value and 110% of an employee’s effort, will find this quick-fix woefully inadequate over the long-term. Daniel Pink’s recent book, “Drive: The Surprising Truth about What Motivates Us” reinforces that point. Pink focuses on the three intrinsic motivations that workplaces should concern themselves with: Mastery – getting better at what matters, Autonomy – being able to direct their own lives/work, and Purpose – being part of something larger than one self.

Too often employers, parents, teachers and others reduce motivation to “carrots and sticks.” Pink would argue that high performance and satisfaction require going beyond those basics. In fact, elementary Behaviorism may work better with small children and animals! Humans have the deep-seated need to direct their own lives, be creative and learn to make our lives and community a better place. Clearly what social scientists know and what workplaces (or schools) do is not always aligned to support motivation, particularly in learning.

Learning and Change

There are many authors, instructors and professional speakers whose intent is to inform or persuade, while others hope to inspire action. Ideally, speakers and educators should strive to do both. When learning is captivating and compelling (i.e. motivating) we are more likely to learn and retain better. Motivation to learn is linked to motivation to change – a competitive advantage for an individual, group, organizational or society.  Indeed to quote French author, Francois Rochefoucauld, “The only thing constant in life is change”. As a result the best we can do is learn how to learn effectively and efficiently to keep pace with (if not exceed) the rate of change, with the aid of our main competitive advantage – learning.