The DNA of Work
The DNA of Work
Management - evidence vs intuition
In an increasingly complex world, management and decision making are ever more challenging. Using a variety of sources of evidence is a way to avoid our inbuilt, unconscious biases, enabling us to make better, more informed decisions. Eric Barends, MD of CEBMa shares the drivers behind the evidence-based movement and how to gather different sources of evidence to contribute to critical thinking and support decision making.
Eric also offers advice on how to judge the reliability and trustworthiness of research that we read about on a daily basis.
Also, AWA’s Céline Chaverot shares what she’s seeing as French companies and multinationals in France get to grips with the “back to the office” debate. While many have been waiting to see what would happen and what others would do, it seems employees have made their own minds up about what they want for the future.
AWA Host: Karen Plum
Featured guests:
· Eric Barends, Managing Director of the Centre for Evidence Based Management
· Céline Chaverot, Senior Associate, AWA (France)
Author mentioned by Eric: Daniel Kahneman – “Thinking Fast and Slow”
AWA Guest details: https://www.advanced-workplace.com/awa/about-awa/the-team/
CONTACTS & WEBSITE details:
Centre for Evidence Based Management (CEBMa) https://cebma.org/
AWA contact: Andrew Mawson amawson@advanced-workplace.com
Advanced Workplace Associates: https://www.advanced-workplace.com/
AWI contact: David Smalley dsmalley@advanced-workplace.com
Advanced Workplace Institute: https://www.advanced-workplace.com/awa/services/advanced-workplace-institute/
Music: courtesy of bensounds.com
Want to know more about AWA?
Thanks for listening to the DNA of work podcast
00:00:02 Karen Plum
Hello everyone, in this episode we find out why we shouldn't rely on our gut instinct as managers, why it's better to use different sources of evidence in decision making and how to decide if that piece of research you've just read about in the press is worth your time. Later we'll find out what's going on in France, in the great back to the office debate. Let's get started.
INTRO: Welcome to the Changing the World of Work Podcast, where we provide insightful, practical content to untangle and demystify workplace change.
I'm Karen Plum, director at Advanced Workplace Associates, where we combine science with 30 years’ experience helping organisations change the way they work, for the better.
00:00:51 Karen Plum
Welcome to this episode of the podcast where my guest is Managing director of the Center for Evidence Based Management. Eric Barends. Hi, Eric!
00:01:00 Eric Barends
Hi Karen.
00:01:01 Karen Plum
So just to kick us off, can you tell us a bit about CEBMa?
00:01:05 Eric Barends
The Center for Evidence Based Management was set up about 10-12 years ago and it started with a group of academics, scholars, researchers. First, in the field of management research or IO psychology that felt that a lot of research was being done that could be very useful for practice in organizations or leadership or management decisions, but was actually ignored or people were not aware of the research.
So these researchers were kind of frustrated, like why don't you use this research? It could be very helpful and can help you make better decisions on all these relevant topics. So they learned about evidence-based medicine from their colleagues in medicine. And they thought maybe we should start a similar movement where you know, we point out that there is a lot of research on topics that are very relevant to management and decision making in organizations, and maybe you should have a look at it.
00:02:10 Karen Plum
Absolutely, and I think that's where we first came to know about the Center. Probably seven or eight years ago. We were looking to try to make use of academic research in the work that we are doing.
00:02:21 Eric Barends
Yeah, and maybe I should point out that that's how it started. And of course from an academic research perspective that makes sense. However, and this is important from an evidence-based perspective, research findings, scientific research is only one source of evidence and the perspective of evidence based practice is that you take multiple sources of evidence into account, because often the research is not always applicable or it's not always clear.
So we started with an emphasis on scientific research, but later we also pointed out - have a look at your organizational data. Take into account the experience of your practitioners and also have a look at your stakeholders - the people that are affected by your decision. And talk to them and ask them how they look at this decision, how they see this problem. Do they feel this solution is going to solve the problem, etcetera, etcetera.
00:03:26 Karen Plum
And I think what we see in decision making very often is people relying on their gut instinct or purely their personal experience or their personal views. So I guess from that perspective looking for different sources of evidence is again something that we want to be encouraging people to do.
00:03:45 Eric Barends
Yeah, absolutely. And of course you can't blame people because your experience is what is in your head. So it's the easiest accessible type of evidence. It's there. And of course, if you've been a manager for 15 years, I mean what is there to learn? What is there to understand that - I mean, you know how things work, let's get on with it and get some results.
However, we know now you know, based on a lot of research on human decision making, that professional experience is not always the best source of evidence and is not always reliable and trustworthy.
00:04:26 Karen Plum
So how does bias come into this? I'm making decisions based purely on my view of the world and my experience. We know that there are maybe hundreds of biases that we're not even conscious of. So how do we guard against those biases coming into play?
00:04:43 Eric Barends
Well, that is actually the reason for taking an average based approach. If you rely solely on your personal experience, even when you're a manager for 20 years you are affected by what we refer to as cognitive biases and this is not my personal opinion or your personal opinion. There are three Nobel laureates who did extensive research in the field of human judgment and human decision making.
Daniel Kahneman wrote this beautiful book “Thinking Fast and Slow”, but also Herb Simon. There is so much research that unfortunately points out that our brain is wired in a certain way and that makes that we are very prone to cognitive biases, So I am very affected even as an evidence based manager by what we call confirmation bias. I will pay attention to evidence, information, experiences that confirms my prior beliefs, not because I'm an ignorant, arrogant person (well, little bit!), that's how my brain works. All our brains work that way and there's an evolutionary reason why your brain works that way - that helped you in the past.
00:06:09 Karen Plum
Yes, there absolutely is - if I can just jump in? I understand the brain evolved to keep us safe from threat and it learned to predict the future by remembering what happened in the past so we don't have to figure things out every time we come across a situation. This process kind of saves the brain energy, but some of those shortcuts we now recognize as biases, don't we? Some positive and some not so positive.
00:06:34 Eric Barends
Actually, it can impede and really flaw decision making. And there are lots of cognitive biases that we may be aware of, but as Daniel Kahneman says - listen, I've been studying this stuff for more than 15 years. I know all biases, I've studied them. Overconfidence bias, confirmation bias, groupthink, conformity bias. You name it, but I fall prone to all these biases every day.
00:07:06 Karen Plum
Yeah, I love it. I love him for saying that.
00:07:09 Eric Barends
Yeah, so being aware of these biases and knowing that the human brain works that way is not enough. Therefore we need to take an evidence-based approach, right? First of all, bring in multiple sources of evidence, not just only one. My own personal experience or judgment. I also need to confront that and compare this with findings from research, organizational data, talk with other practitioners - what is their experience and also the stakeholders. So to cut a long story short, evidence-based practice, evidence based decision making is just a safeguard to prevent you from falling prone to your own biases.
00:07:57 Karen Plum
I think it's really interesting that over the last year we've all been looking at this working from home experiment that nobody planned, but everybody’s been doing it, and everybody’s been writing on the subject. And various people have been conducting research on people’s experiences and every day I'm confronted with a new piece of research or a new opinion piece on the subject.
And as you were saying, it's very easy to go find the stuff that confirms your view of the world. And I do that as well. I want to make a point - find a piece of research, I don't even look too closely at how robustly that research has been carried out, but I'll quote it. I won't necessarily go looking for the piece of research that gives the counter argument.
So how can we guide people? How can we help them to think about the goodness of the research that they're seeing every day in the newspapers?
00:08:53 Eric Barends
That's hard because it's already, I would say an improvement that people are open to research findings and as you say you read a paper and say, well, new research has demonstrated (as they always frame it) and you will say oh, it's interesting, yeah well, let's think about this.
It's OK, that's good. And you're right, I will be able to find a piece of research that confirms my prior beliefs. If I absolutely am convinced that the earth is flat, I will probably find a scholarly paper that, you know, sort of suggests that way. That's how the Internet works. There's so much research, so yes, that's another step in evidence based practice.
You need to use multiple sources of evidence, but the second step is you also have to critically evaluate the trustworthiness, the quality of the evidence that's on the table, regardless whether it's the experience of your most senior consultant or manager, or it's this stellar research by this Nobel laureate that was published in this AAA journal. Whatsoever we don't care, that's all authority bias. We need to have a look at this piece of research in itself, see is the study a good study to answer this particular question? And by the way, is this finding confirmed by other studies?
OK, now I understand that if you Karen just read something in the newspaper that you may say oh how interesting and you share it with your colleagues. However, when the decision you need to make is actually important, involves time, money, organizational resources, people, etc. then that is not enough. You should absolutely dig a little bit deeper and see if you can find the original research and figure out what kind of research was it.
That's what we do. We train people to be able to do that, to have the skills to figure out what kind of research is it. Was there a control group? Was there a pre-measure? They found that A works better than B. But how much better does A work better than B? I mean, is it worthwhile the investment?
Compare it to the medical research - we can find that a new sleeping pill A works better than sleeping pill B. However, if better means one or two minutes of extra sleep at night, then we say technically, you're right. It works better, but hey, this pill is 6 times more expensive and has a lot of side effects, so it's not worth it. And this is the same as with research. We don't want to know whether something works, because it always works because of the placebo effect or whatsoever.
We want to know whether something works better than what we normally do and how much better does it work, and is it cheaper or more efficient, faster? So yes, I mean it helps if you're skilled, trained when you have a general understanding of what type research is actually a good way to answer this type of research questions and what do we know about the findings in terms of impact and effect size?
00:12:33 Karen Plum
Yeah, so it's really getting below the headlines really, isn't it? I mean, we're very drawn to the headlines and the conclusions that they draw from the particular piece of research that they've done. But what strikes me a lot of the time is that this research is usually a survey.
And if you're evaluating the goodness of research, where would you put a survey? A self-report? I mean, people have reported their feelings or their subjective views. Where would you put that in in terms of quality of evidence?
00:13:04 Eric Barends
You can't say anything about any research design as you just mention. A survey, we also refer to that as a cross-sectional study or a randomized control trial or a cohort study or a qualitative study that uses focus group. You can't say anything upfront about quality. It depends on the question it's trying to answer. So how many employees are more frequently absent from work more than 6 times a year? Then you get the number. Now a survey or a cross sectional study is absolutely a great method, I mean, provided that the sample size is OK and it's representative or reliable of course, but then a cross sectional survey is actually a really good type of research.
However, if the question is - we introduce agile working, has agile working improved the financial performance of the organization? Then a survey is not a good study design to answer that question. Because a survey asks all people at the same time, how do you feel about agile working? Do you feel it improves? So it's usually perception, it's the opinion of people which may be helpful, but it does not demonstrate whether it was effective.
You need to know if something was effective, how was the situation before the intervention before we introduce agile working or autonomous teams or open office designs - you need a baseline before and you need a measurement afterwards. That's what we call a before and after measurement. But also you need to compare it to a place where that intervention was not done so where they did not introduce open office designs or agile working or autonomous teams. So you have a fair comparison.
So the term fair comparison is like OK could we compare it with the group where this intervention was not done? And can we compare it with the situation before the intervention? That would constitute a fair comparison and a cross-sectional study does not include a fair comparison. For that, you need a controlled before and after study, so if the question is does A have an effect on B, and there is a cross sectional study, a survey, then the answer is that its actually not a very trustworthy study to answer this particular question.
If the question is how many people in our organization or how many times or, is there a difference, then a cross sectional study can be a very appropriate study.
00:16:06 Karen Plum
Finally, Eric, most people don't have the time or know-how to critically evaluate different sources of research - so going through all of the steps that you do at CEBMa to critically evaluate the different studies that have been done. But are there a couple of top tips that would help people identify whether a piece of research that they're reading about is worth paying attention to and which to put in the bin?
00:16:34 Eric Barends
Yes. The most important question you should ask yourself when you read something in a newspaper is how did they measure that? And is that a reliable way to measure? So I read in the newspaper that oh, people are more productive when they can use their lunch at their desk. OK, uh, very nice. How would you measure such a thing?
So ask yourself how did they measure that? And is that a reliable way to measure is actually the first barrier or criteria to figure out whether the research is any good. And when the answer is well, they asked people, or they sent the survey to, I don't know 600 change managers and all 600 change managers, well actually 500 of them suggested that communication is the most important element of organizational change - you would think OK, so this is actually the aggregated opinion of people, yeah – may be helpful, interesting, but think about how is it measured?
That already helps a lot. Of course self-report opinions or taking a sample of people is less reliable than measuring all the people or the situation and using hard outcome measures, numbers, numbers of mistakes or items produced or sold is of course way more reliable performance outcome measure than for instance, how do you feel you performed in the couple of last weeks? Do you feel it helps you to be more productive? Relevant, but maybe less reliable.
00:18:29 Karen Plum
Yes, and I think that when you ask people for their opinions you have to recognize that the opinion you get is the person reinforcing the thing that they want to do, or the thing that they don't want to do. So there's self interest in all of those responses. Anyway, I think that's a really good tip and I guess the takeaway really is to be curious when you're reading this sort of research. Ask yourself those questions and if the detail isn't there, maybe go looking. See if you can find that detail and just be a bit skeptical until you can satisfy yourself about that source of evidence.
Eric, thanks for sharing your thoughts. It's been great talking with you.
00:19:10 Eric Barends
It was my pleasure. Thank you for having me.
00:19:12 Karen Plum
Now we're going to have a quick break, and then we'll be taking a look at what's going on in France as part of our Focus series. See you soon.
ADVERT: Are you changing the way your organization works? Maybe you're trying to see if hybrid working is right for your business or needing to train people to work or manage in the virtual world? Perhaps you're trying to work out how much space you need once you adopt different ways of working, and what types of environments will work best. At Advanced Workplace Associates, we work with companies around the world helping people figure out the answers to these challenges based on what they want to achieve for their business.
Our friendly team of consultants blend workplace science with creativity to help you create the best work experiences for your people. We gather the evidence needed to decide on future ways of working through a range of studies and provide change management support when you're implementing new strategies.
If you'd like to talk to us, there are details in our show notes. We look forward to hearing from you.
00:20:09 Karen Plum
Welcome back. Now we're going to talk about what's happening in France and I'm delighted to welcome our Senior Associate in France Céline Chaverot. Hi Céline!
00:20:19 Céline Chaverot
Hi Karen, thanks for your invitation to share insights from France.
00:20:23 Karen Plum
So what do you think are the main trends that you're seeing in France at the moment? Are companies just waiting to get everybody back to the old way of working or are some looking to make changes?
00:20:34 Céline Chaverot
Well, actually we have both, but still many companies are waiting to see what will happen next and waiting to see what other companies will do as far as new ways of working and hybrid working are concerned. But we have to take into consideration the employees.
And for sure, according to the latest survey run by a professional body, 74% of French employees want to keep on working from home at least two days a week. Well, I would say that a major change compared to before the COVID crisis is that before this crisis companies were not taking into consideration employees view regarding remote working.
But now they have to because the 15-month experience related to working from home has been very successful. And even though before the crisis French companies could say no, we don't think that home working is a good thing, it won't work, anyway, we have been used to work that way and we'll keep it that way. But now you know everybody has been experiencing work from home and it has been quite a success.
Yes, and now employees can make a difference between the working life before the crisis and how it could be post COVID.
00:22:03 Karen Plum
Yes they have 14 months of experience of what it can really be like. You said that companies are waiting to see what other companies are going to do. Is there a difference between French companies versus global multinationals that also have an operation in France? Are you seeing any difference between those two types?
00:22:24 Céline Chaverot
Yeah, for sure, and I can also make a difference between French large companies because, for instance, some examples in pharmaceuticals and insurance - some large French companies having deployed working from home before COVID crisis, two days a week and now they are willing to extend these two days to three days a week. Ah, there's a strong willingness from these large companies to keep on with this trend.
Compared to smaller typical French companies operating at the national level, they are more suspicious about it, but it's very different of course, from one company to another, but I would say that compared to what I've been observing abroad, I could see that in the UK, in the States and elsewhere, companies were willing to say it will never be the same again, so let's go for it. Let's see what we can do as far as new ways of working and hybrid working are concerned.
While in France you know we were told that they will think about it from September 2020 and then in September, well, we need more time, you know to reconsider the questions. Wait for the end of the year and 15 months later we are still at the same level you know.
00:23:53 Karen Plum
Still thinking about it!
00:23:53 Céline Chaverot
Yeah, still thinking about it.
But I was listening to the radio this morning to HR leaders from a professional body and they were saying there is definitely a strong willingness now to move on from September and to have appropriate policies agreed between companies, employees and unions in order to make things clear in everybody’s mind. Let's see what happens next in September, but I'm optimistic that at last, things will move.
00:24:34 Karen Plum
Yes, it sounds like they're starting to formalize things, and I guess for many French companies that have a unionized workforce, that's really necessary. There has to be that certainty about what the New Deal is, right?
00:24:37 Céline Chaverot
Absolutely, especially if you are considering that some employees are willing to keep on working from home, but others are willing to go back to the office. And where is the limit in between? Because you know, the French Government is saying working from home should be the rule now on.
But what about employees willing to go back to the office? What companies should do about it? You know, formalizing, working from home with specific policy, specific guidelines will help everybody to understand what to do.
00:25:25 Karen Plum
Yes, and that statistic that you quoted, the 74% of people wanting to spend, probably more than half of the week working remotely, got to be a big driver.
00:25:37 Céline Chaverot
Exactly and another figure. One employee in three is ready to quit a company if its making return to the office compulsory, you know.
00:25:51 Karen Plum
Big percentages of people who really feel empowered to go and get another job. And I'm really interested by that survey - I was talking with Eric earlier about the use of different sources of evidence and obviously, surveys and studies are one of those sources, you know when we're making these really important decisions about the future.
Do you see that French companies will take that sort of evidence on board? Or are they really still just prioritizing their own personal experiences, or do you think things are changing?
00:26:25 Céline Chaverot
I think things are changing. You know, of course, evidence-based management is not very much in leadership practices, even though with the COVID crisis they have been running multiple surveys in order to get a better understanding of what is going on with their employees, how they feel about working from home, how it has been during this 15 months.
Could we consider them as a good source of it, and I've understood that Eric from CEBMa was saying that actually, yes, it could be a good source of evidence, and I think it's the type of data French companies will focus on when they will have to take the right decisions.
00:27:10 Karen Plum
Yes, I think it's all about what question you're trying to answer through those surveys, isn't it? That's really what makes the difference in terms of whether you can really rely on the results for the decision that you're trying to make.
So just to wrap up Céline, when you're advising French companies that are thinking about changing the way they work, what are you emphasizing?
00:27:32 Céline Chaverot
What I'm emphasizing is that one size does not fit all definitely, and I would recommend to go for a personal approach because every company is so different from one another and every employee is unique, so you have to get a very good understanding of who your employees are, what are the expectations and their needs first of all, to make them part of this transformation journey.
And second of all, to make sure that the output of this transformation process will match their expectation as well as it will match leaders’ vision, which is a very important point in this kind of transformation process, you know. Make sure that leaders in one way do know what they want to do and on the other hand, make sure that employees get a full understanding of this vision in order to have everybody aligned with this goal and to make sure the target is rich.
Another thing I want to highlight is the challenge related to management, 'cause one lesson learned from the COVID crisis - managing from a distance is totally different from managing on-site. And factors such as trust, social cohesion, information sharing, feedback, culture are very important in order to have teams involved, engaged and part of one single company whose culture have a perfect understanding.
00:29:23 Karen Plum
Absolutely, and all of those factors, as you say, come from the research that we've done on what makes teams work really productively together. And it just becomes more challenging when people aren't together and so companies that are looking to embrace those ways of working really have to understand the importance of those things so that they don't get overlooked.
I think that about wraps it up for today Céline, thank you so much for sharing your insights. It's been great talking with you.
00:29:52 Céline Chaverot
Thanks Karen for having me.
00:29:54 Karen Plum
And that's it for this episode. See you again soon.
CLOSE: Thank you for listening to this episode of the Changing the World of Work Podcast. Please subscribe or like the show so you don't miss any of our content. You can find more information on this episode in our show notes, including a link to the AWA website if you'd like to know more about us. Hope to see you next time. Goodbye.