In my first job, I was part of a team tasked with rethinking the organization’s professional development calendar to help increase workshop and conference attendance. The professional development director had received anecdotal feedback that people were not attending the events because of their timing—most took place during the workweek. When the team conducted a survey to determine the best time to hold events, most respondents said they preferred weekend-based activities that didn’t conflict with their work schedules.
Based on the feedback, we moved most of the workshops and conferences to the weekend—and, as you probably guessed, attendance dropped precipitously. (I still cringe when I think about it.)
Most seasoned researchers would point out that our research design was flawed and that we were making the rookie mistake of using research as a blunt instrument. When asking people about their behavior, it is difficult to get reliable answers. Researchers suggest many reasons for this:
- Respondents want you to think well of them—in this case, that they are so involved in their work that they can’t afford to give up even a day.
- Respondents will usually give what they perceive is the professionally desirable answer.
- Respondents want to give the answer they believe will be helpful to the person asking. For example, respondents may have felt that the professional development programs were lackluster but used the excuse of timing for nonattendance, not wanting to offend the organization.
Understanding the Root Cause
It is difficult to collect reliable data about the three B’s: behavior, beliefs, and belonging. To do so, you need to probe much deeper and get to the “why,” or the root cause. Too often, we use simplistic research methods to understand complex situations, which can lead to faulty conclusions. In her book Weapons of Math Destruction, Cathy O’Neil explores case after case of flawed research methodologies that lead to flawed conclusions, with often tragic consequences. Following is one vivid example:Employers are increasingly using credit scores to evaluate potential hires. Those who pay their bills promptly, the thinking goes, are more likely to show up for work on time and follow the rules. In fact, there are plenty of responsible people and good workers who suffer misfortune and see their credit scores fall. But the belief that bad credit correlates with bad job performance leaves those with low scores less likely to find work. Joblessness pushes them toward poverty, which further worsens their score, making it even harder for them to find a job. It’s a downward spiral. And employers never learn how many good employees they’ve missed out on by focusing on credit scores.
Despite these challenges, research is essential because it informs strategy. In today’s complex education market, schools need to make research-informed decisions, which can make the difference between thriving or struggling. It can inform why some students are succeeding and others are failing, helping school leaders design effective intervention strategies.
Uncovering Causality
Most independent schools today embrace the importance of research in driving strategic actions, but few have a dedicated team of researchers on staff. And hiring outside researchers can be an extremely expensive proposition. Thus, what is a viable route for schools to obtain reliable research? I have a suggestion that I think could be helpful to schools: Jobs to Be Done (JTBD).Created by by the Detroit-based Re-Wired Group and put forth as a theory by innovation expert Clayton Christensen, JTBD is an interviewing methodology that seeks to uncover causality. The underlying theory is that customers make choices to help them achieve progress on something they are struggling with in their lives. Christensen outlines the science behind JTBD and shares case studies of organizations using it to drive transformation in his 2016 book Competing Against Luck: The Story of Innovation and Customer Choice, co-authored by Taddy Hall, Karen Dillon, and David S. Duncan.
In the book, Christensen describes how JTBD is different from many methodologies that rely on correlations. He illustrates by sharing a personal story:
I’m 6-feet-8-inches tall. My shoe size is 16. My wife and I have sent all our children off to college. I live in a suburb of Boston and drive a Honda minivan to work. I have a lot of other characteristics and attributes. But these characteristics have not yet caused me to go out and buy The New York Times today. There might be a correlation between some of these characteristics and the propensity of customers to purchase the Times. But those attributes don’t cause me to buy that paper—or any product. If The New York Times doesn’t understand why I might choose to “hire” its product in certain circumstances and why I might choose something else in others, its data about me or people like me is unlikely to help it create any new innovations for me. Correlation does not reveal the one thing that matters most in innovation—the causality behind why I might purchase a particular solution. That answer, I believe, is found in the job I’m hiring a product or service to do.
NAIS is testing this framework to explore some of the major questons facing the industry and to focus our resources to meet members’ needs and help them make progress on the issues they struggle with.
NAIS has conducted a lot of research over the years to understand members’ use of and satisfaction with the organization. That research has focused on what members like and how they use our products and services. But understanding the context of their situation and why they made the choice to join NAIS brings new clarity. Previous research had given us disparate pieces of the puzzle, but understanding an individual’s struggle through personal stories brings everything together. The experience is like that moment in the eye test when you finally choose the lens through which you can see perfectly. Although JTBD is not the solution to every research problem, it is extremely useful in understanding why people decide to buy a product or service.
Currently, we are applying the JTBD framework to the parent market. We hope to get more insight into why parents of young children make the choices they do in educating their children. That work will take into account various demographic filters so that the resulting research will be useful to schools of all types in all geographic regions.
In this time of education transformation, school leaders need to understand how to translate their school’s mission and vision into the context of student and family struggles. JTBD can provide that insight. W. Edwards Deming, father of the quality movement, once said, “If you do not know how to ask the right question you discover nothing.” It’s time for us to begin asking the right questions.
