#152: Fostering a Positive Data Culture

[music]

0:00:04 Speaker 1: Welcome to the Digital Analytics Power Hour. Michael, Moe, Tim and the occasional guest discussing analytics issues of the day and periodically using explicit language while doing so. Find them on the web at analyticshour.io and on Twitter @AnalyticsHour and now the Digital Analytics Power Hour.

0:00:27 Michael Helbling: Hi everyone. Welcome to the Digital Analytics Power Hour. This is episode 152.

0:00:37 MH: Now why can’t we have a great data culture? You probably listen to this show and at your company, you probably have this amazing experience where your work is aligned with the strategy of the company, leaders of the business and they enlist your support to solve complex problems, all the while investing in your growth so you can continue to increase your impact over your tenure at the company but for some reason, Tim, Moe and I, we keep finding organizations that don’t quite live up to that ideal.

0:01:08 MH: I don’t know what we’re doing wrong but on this episode, we’re gonna compare notes and see if we can come up with some reasons how we can improve the data culture at some of these places. Hey Moe! How are you feeling about data culture these days?

0:01:21 Moe Kiss: Well I’m just gonna say I don’t endorse that intro at all from a company perspective.

[laughter]

0:01:27 MH: I know. I know. I would say all of us have been part of great data cultures but we have also seen and heard about a lot of bad ones too.

0:01:33 MK: There’s always room for improvement. I would put it that way.

0:01:36 MH: It was sort of meant tongue in cheek because I think probably a lot of our listeners wouldn’t endorse that intro for themselves either.

[chuckle]

0:01:47 MH: Tim, could you just endorse my intro real quick?

0:01:50 Tim Wilson: Sure, this all feels touchy-feely so I’m gonna be maybe so uncomfortable a bit. I had no idea.

0:01:56 MH: Well listen. Tonight Tim, I’m just gonna need you to man up and do your fucking job and talk about data culture.

[laughter]

0:02:03 TW: Perfect.

0:02:03 MH: Sound fair?

[laughter]

0:02:08 TW: Why couldn’t you have acted that way when you managed me? You always wanted to know how I felt and how I was making other people feel?

0:02:17 MH: I don’t know. The hindsight is 2020 on that, right? So I’m finally learning. Anyways… And I’m Michael Helbling. So obviously, this is something that I think each of us has spent some time thinking through in different ways but Moe, I wanna throw it to you to frame up some of this because I think this was… Well actually, fun fact, Episode 20 of this podcast, we actually did talk about data and culture so we are due to do another go at it but Moe, you kind of brought the topic up so I wanna kinda kick it to you, kinda frame up how you wanna guide this conversation.

[laughter]

0:02:57 TW: Now the Podcast culture where the moderator completely ducks his responsibility and with no warning, throws it to one of his co-hosts.

0:03:05 MH: It literally says “Throw this to Moe for a framing.” In the notes. I’m trying to do what it tells me the notes.

0:03:18 MK: Well actually, Tim raised an interesting point as we were prepping and getting our head around this topic, which is that evolving data culture at a company is very similar to any kind of change management process and I had this amazing experience working with an executive at a former company and I kind of was floundering a bit. We had decided we were gonna shift what the company’s number one KPI was and we were gonna move it to a different metric and I was kind of floundering a bit like “Cool! We’re gonna move in to this metric.” and she was like “Yeah Moe. We need a strategy to do that. We can’t just decide we’re gonna do it and assume everyone’s gonna get on board.” and so I worked really closely with her on how…

0:04:03 MK: We basically had a three-prong approach, which was like how do you get executives on board? Your tradition top-down approach. Kind of like through the middle, how do you get your PMs and your engineers and your colleagues involved and get them to adopt this change and then bottom up, which is like, what are the things that your own team can do to help support this message or this change and I just literally went through that experience learning so much about how you have any kind of change in an organization, whether it’s a KPI or whether it’s data culture and so when I’m tackling this issue now, I guess I kind of think of it in that top down, through the middle and bottom-up approach of what are the different tactics that you can employ at each level so that overall cohesively the organization will shift in a particular direction.

0:04:51 TW: And there’s no way that it just actually happens overnight or doesn’t have bumps in the road and trying to figure out the timing and the shifting ’cause I feel like I’ve seen… I’ve been at one of the nine of the longest months of my career was where a hotshot senior analytics guy was brought in from a completely different industry and he was just like “Here’s how we’re gonna do it.” and I was like yeah, dropping one person in who hasn’t built relationships, doesn’t understand the organization, doesn’t fully understand the business or even the industry. Sharp guy but kind of all hell broke loose. It was chaos.

0:05:37 TW: He’d come in before I did, he lasted maybe two and a half years and then moved on because that was the idea. If somebody said “We’ve gotta be… We’re gonna be data-driven so let’s go hire a big name and he’ll kinda move his O Chart around and he’ll institute processes.” But I would say one thing he did not do was think about it from the, who is this impacting? For every single person it’s impacting, where is it that the benefit is going to exceed the pain? That’s what the co-worker I’ve got who has a degree in organizational psychology or something along those lines and so he loves to do that sort of research and it’s like you literally need to go role by role and try to figure out what is the… Not what is the lofty? Just imagine if we all had perfectly clean data, now that… You all need to change your day-to-day operations on how you log your time or how you… Interactions with customers but just imagine the future where it’s all great and then we as a company will see these glorious high level metrics rolled up perfectly and that tends to be a disaster.

0:06:55 TW: That’s what the executive wants. You’ve shown them, they don’t feel the pain and they get the benefit but you’ve gotta figure out how do you tell the story for… That’s where I like the… That’s where I poorly gave Michael direction on throwing it to Moe that thinking through the different… Everybody who’s impacted and everybody who is going to need to change, you kinda have to think through what is the change and what’s in it for them and what can you do to grease the skids for them.

0:07:27 MK: But I think one of the things that… To call out from that particular example you had is like you’re talking about one person owning this and I think any process is gonna fail if you have one person. Then it becomes completely dependent on the relationships of that person and how good they are at stakeholder management. I know Ruth Bader Ginsburg passed away a few weeks ago, which I’m still completely distraught about but my favorite quote that I was reflecting on from her was like “Fight for the things you care about but do it in a way that will lead others to join you.”

0:07:58 MK: And when it comes to change in an organization, you need to do it in a way that other people get brought into the process and other people wanna be involved and whether that’s your teammates, you always need to have a lead on something like this but you really need it to be a collective effort and I know my team have started by just all acknowledging that we want this to improve because if we’re all united on “This is a problem and we wanna fix it.” That’s the first step ’cause then you’ve got a team who are trying to I guess, infiltrate the psyche of the organization versus one person coming out saying like “This is how we’re gonna do it.”

0:08:34 TW: When you say it’s this ’cause I guess that’s… We’ve sort of called this fostering a positive data culture and improving a data culture, which is pretty… That’s kind of a squishy and lofty thing so…

0:08:46 MH: We need to define some things?

0:08:47 TW: Well versus the… I guess even Moe, as you’re thinking through that example, how much of it is “We collectively wanna make things better.” Or is it a little board narrowly defined that we wanna do X and Y and Z ’cause the specifics… I much more throughout my career has been focused much more on the bottom up and much more on the “I can think through a better way to do this and I’m going to start doing this and that will… There will be a halo and there will be other people who come on board.” That is a very long and slow way to effect change. You get to some critical mass and then maybe it bubbles up and all… Everything comes together but I went for a long time saying “I don’t need executive sponsorship. I’m the one doing the damn work.”

0:09:41 MK: You totally do. You do.

0:09:43 MH: Yeah.

0:09:44 TW: I could make this happen and I don’t know that it’s a binary. I don’t think it is a yes or a no. ‘Cause you can have misguided executive sponsors. People, executives who they’re sponsoring it and they’re not really doing anything.

0:09:55 MK: Well, then you picked a shit executive sponsor.

0:09:57 MH: Well or they don’t know any better. Tim I think earlier in my career, I experienced more of the bottom up too ’cause that’s part of the org I was working in and as I’ve reflected over the years, I had an experience where part of my career in the company was really positive from a data culture perspective and part of it was actually not that positive from a data culture perspective and over the years, I’ve really thought about what changed and what changed was leadership. Here’s the crazy thing, we actually had less analytics talent in the positive culture then we theoretically did in the negative culture and that’s just something that’s been sort of been standing out to me a lot as I’ve been talking about this and it just so happens, I actually gave a talk about how leaders can drive a data-driven culture with… Data-driven is sort of like even a word that probably none of us really even like very much but it’s something that people use a lot which is, that’s another big part of how do you have a good culture when you can’t agree on terms and terminology at all?

0:11:04 MH: So that’s part of it and so I’ve become convinced over the years that this has to be… If you want any kind of lasting change, it has to be done from the top and then down, although what I would encourage everyone to encompass is a commitment to it from no matter where you are. There’s people… I’ve heard it said and I like the saying “Leaders lead.” And this is very much a leadership challenge. Creating space for people to do analytics well or incorporate analytics in mastering the data, applying the data to context or integrating it with like the decision-making process of the business is all about creating opportunities for people to execute and leaders are the ones that are gonna go create that cushion and make space for people to be…

0:12:04 MH: I don’t know what the right word is for it but it’s sort of free, I guess in a way. The word I’ve been using is sort of creating the psychological safety to actually do analytics. Embrace the uncertainty of analytics. ‘Cause a lot of our cultures are risk-averse. We can’t do things because, unless we know how to do them already, we can’t try something out because if we’re wrong, well then it’s our butt.

0:12:33 MK: But I don’t think it’s just about I guess, giving people permission to make mistakes and learn from them. I suppose when I think of data culture, I also think about people having what they need at their fingertips to answer simple questions. For me, that is a big part of data culture. It’s like the worst example of data culture is when people are really frustrated because they feel like they don’t have the data they need to do their job. That doesn’t mean they should be able to answer really complex questions, that’s why we have the analysts but they should be able to self-service simple questions like “How many users did we have last month?” Things like that but it’s also… And you touched on a Michael, it’s really about using data to inform their decision-making.

0:13:16 MK: Are they turning to data when they have important decisions to make? And from a company perspective, how is data being used to create and shape goals for the company? So those are the things that I really think about and then the last one that you mentioned is people being able to use data as learnings not as pointing out failures so people getting up and sharing the experiments about like “Hey, we try this experiment. It didn’t work, this is what we learned from it and now we’re gonna pivot this way.” That to me is a sign of a really healthy data culture.

0:13:49 MH: Yeah. Well it turns out it’s a sign of a healthy culture period. Something Evan LaPointe says a lot that I really like is a lot of times companies focus on culture around the work but don’t really think about culture in the work. So we try to do things like “Oh. We’ve got snacks in the kitchen and a foosball table.” and it’s like we’ve got this culture but it’s sort of like, how do we treat each other in the nitty-gritty of the deadlines coming up and the meeting and those kinds of things. I’m like “What’s that culture like?” And I think that’s what’s missing a lot of times is sort of people demonstrating that it’s gonna be okay. In a certain sense, I think people feel at risk so then they push other people to feel risky. I don’t know the way to say that.

0:14:40 TW: I’ve got to call… I will… I think there is an overwhelming number of people who tell themselves that there will be penalties or ramifications or negative repercussions if they use data and fail when I don’t think most organizations actually operate that way, that is such an easy thing for me. When you say “Let’s set a target for your KPI.” And people freak out, it’s because they’re imagining something that nine times out of 10 wouldn’t happen.

0:15:12 MH: And the thing is, it’s the fear itself that’s the problem, not the fact that something bad would probably happen to them. I don’t think you’re wrong Tim. I think people could do a lot more than they give themselves credit for, they are afraid to approach it ’cause they don’t see it lived out or demonstrated in any of the other environments that they’re part of in the side of that business or corporation.

0:15:33 TW: But that’s the thing, they’re not seeing somebody using data and getting punished for it, they’re not seeing it being lived out and therefore they extrapolate out to “I will get punished if I don’t have a positive outcome.”

0:15:45 MH: Right and so from the bottom up or from the middle, then maybe the lesson is take bigger risks, take bigger swings, believe in yourself a little bit and maybe that’s… If you’re not the CEO or the VP of Analytics setting the table for all of this, maybe what it is is to sort of challenge yourself so kind of to your point Tim, is sort of be data-driven anyway and hope that your company doesn’t come stomp on your head.

0:16:12 MK: Okay. Just to tip in to Tim’s camp for the moment, kinda back yourself and be confident is very airy fairy and not super tangible for someone listening. I think there is a lot of stuff in… Tim’s work is probably a testament to I guess the kind of approaches that you can tackle at your own level. One of the things that our team did, which has had a huge impact from that real bottom of angle is, I took the executives in our team aside and said “I’m on a mission to improve the data culture in this group, I really want us… This is what data culture means, I really wanna improve it. One of the tangible things that I would like to do is that every single time… ”

0:16:54 MK: So we do group show and tells, we do companies stand ups, all that sort of stuff, every single one of those starts with a data update and I would like your support to do that and they do and I started doing the updates and now I share it so everyone in the team does them. It also means that everyone gets to know the analysts. I even did one session on like what does each analyst do?

0:17:15 MK: We did one session on we’re all about dashboards and what are some of the common questions we could answer? If someone’s done an interesting piece of analysis, like someone just walked through what causal impact means and we do this at a very high level, like it’s not super technical but it’s the ideal place where you have the audience of everyone in the company and it’s something little that you can do and to be honest, if some executive comes back to you and goes “That’s not a priority for our company stand up.” Then you’ve got way bigger issues because the majority of execs will be like “Actually, you’re right. That’s a great idea, let’s do that.”

0:17:49 TW: But there’s, just to break that apart a little bit, that’s great. I think that’s a great example. There has to be a forum where it makes sense to do that and there’s also two big risks. One is if not done well, it can be the, I’m making the [0:18:07] ____ making it about me, right?

0:18:08 MH: Yeah. Yeah.

0:18:09 TW: Let me tell you all of the stuff that I do and then there’s the risk of that not being communicated effectively. On the one hand, there’s… That’s the sort of risk you should be taking say “Look… ”

0:18:19 MK: You should.

0:18:20 TW: “You’re gonna be in front of the whole company and you need to make this compelling and interesting and that means you better do a good job.” I’m thinking of some of the… There’s this bigger beef I have where there are analysts who piss and moan about “I’m just a report monkey. I don’t get to do any analysis, I just… ” This, that and the other and then you look at what they actually produce, I’m like “Well, your work product is kind of shit so if you wanna improve, if you drop one of those analysts into that scenario, they would basically move the data culture in the wrong direction.” If you drop a BI team that says “Oh. We have all these cool dashboards we’ve built but we built it in a fucking vacuum.” So they’re not actually answering any of the questions anybody cares about so…

0:19:04 MK: But Tim, that’s your job as a leader, right? For example, when I had junior people that I was putting in this situation, I set aside time with them to go through their slides with them and to do a practice run before they presented. Every single time someone from my team presents, I’d watch it and I give them immediate feedback afterwards like “Hey, this slide was too busy.” or like “You communicated this point really well, let’s use that as a jumping off point for the next time you present.” It’s your job to make sure that your team are ready to do that task and just support them.

0:19:36 TW: Well, you’re conflating things again though, right? That it goes back to it. Multiple pieces have to come into play, like me working with other organizations that don’t have a positive data culture and I look at it and say “Well yeah. You know what? You’ve got some pretty lousy analysts who, they’re kind of hopeless.” The organization has some potential. To me that’s seeing the system and figuring out what needs to happen where. We gotta have some small wins to get some basic capabilities in place. I think that’s… You have to try and I think you’re right. If it blows up, then you know what, you move on somewhere else but I’ve worked in multiple cases where… Well yeah, go ahead Michael.

0:20:25 MH: If the analysts are all bad, it means the leadership doesn’t actually understand the problems that those analysts are supposed to be solving.

0:20:38 MK: I agree.

0:20:39 MH: So they just haven’t hired the right talent so it’s still… And the thing is, is the buck stop somewhere and at the same time, I think your encouragement to the analyst of like “Hey, really focus on your skills.” And again, that is why Tim Wilson, everyone listening is the quintessential analyst…

0:20:57 TW: Okay. Well so I’m gonna jump in and just talk over you. [chuckle]

0:21:00 MH: Because… No, I mean you are the Batman of analytics that you’ve got a utility belt with something with everything in there so…

0:21:10 TW: I think that’s…

0:21:11 MH: I’m just gonna keep coming up with analogies. [chuckle]

0:21:14 TW: I was at a conference a few weeks ago that was virtual and so there was the chat and there was, somebody was… It was a fine presentation and then in the chat, somebody made a note and it was basically about building a data culture and it was an okay presentation, a few little… Some good nuggets and this one person said “Well, my boss and I are the only people at my company who get data.” and so then people were like “Oh man that sucks. Oh that’s terrible.” and I’m like “Dude, you…

0:21:43 MK: You sound really arrogant.

0:21:44 TW: You sound… Yeah and so I went and looked him up and I was like “Okay. You know what you need? You need yourself an analytics translator.” [laughter] But I guess… But that’s another little piece of that from the bottom up, you building some relationships building… Like Moe, if you had walked in on your… A week into the role and told the executives, I wanna improve the data culture, you had to build up credibility and some wins and some respect…

0:22:20 MK: Yeah. Of cause.

0:22:20 MH: And trust.

0:22:20 TW: And trust and all of those pieces and that is kind of a bottom-up thing. It’s just like when somebody says, if they ask me for some stupid metric but it’s the first time I’ve interacted with them, I’m not gonna not give them. It’s the how do I say yes and? How do I make this… How do I not make it sound like I’m a blocker but start to do a little bit of education and then try to turn that person into an advocate.

0:22:44 MH: What also though… What Moe is done with that credibility though is she’s turned it around and created that top-down cultural space for other people to excel and so in your example Moe like, okay I set up the meeting and I’m gonna have analysts come in and they’re gonna share something about their work with the whole team, they’re gonna give data perspectives and data information to the company and then you’re gonna go back and give them feedback and tell them how they can improve. You’re literally de-risking the space for them so that they can be part of the process of which they were not part of before and by the way, you’re gonna help them understand how they can continue to elevate in that process and so that’s a huge deal because it’s like okay…

0:23:29 MH: Again, I always take it back to impact and meaning. In terms of everybody’s work, you want impact and you want meaning and so by doing that, you’re helping them understand more and more the meaning of their work and as they talk and see people understand them and those kinds of things, they also understand how the work is impacting other people ’cause it’s sort of like I live over here as an analyst and do this analysis, I pull this data, I run this report, whatever it is. Ideally we’re all getting to see cool things but what you just described is what so many people are missing is any kind of connectivity of relationship going up through the organization and so I really love your example that you shared because without really saying it, you just demonstrated how you’re actually building credibility upward but also then creating space downward.

0:24:21 MH: So like Moe, you might be the quintessential analytics leader. I don’t know.

[laughter]

0:24:27 MK: Jesus. Oh God!

0:24:30 MH: I don’t know. Actually just a little pause there. That’s where we could use your help, everybody listening, if you could help us come up with a cool catch phrase for Moe, that would be so great.

[chuckle]

0:24:40 MK: Oh dear! Oh dear!

0:24:44 TW: So when I think about an organization where there is a strong analyst team, small, very very small and not actually doing… Focused more around getting the enterprise data wired up and really, really pretty lean but really is really sharp but an organization that hasn’t had that culture and they were frustrated by it and that one has worked and it’s slow but it winds up being a lot of making the steps forward, getting the data integrated or available.

0:25:15 TW: Michael, you said something somewhere in your presentation about the… We’re good with the getting the data or the instrumentation is where we’re ahead of the curve in general.

0:25:26 MH: Well, if we’re good at anything, we’re good at capturing a lot of data…

0:25:31 TW: We are most competent with obtaining data…

0:25:34 MH: Yeah.

0:25:35 TW: Yeah. That’s kind of a default generic over-simplification but it’s a good one and then it became a… Helping them figure out how to engage with the business who was looking at data. That’s the thing. The people who say “Oh I’m really all about the data.” But then you’re realizing they’re not actually… They’re all about the data and that they show charts a lot but they actually don’t connect that to action and working with them to say “Hey, we’re giving you the data, which is our responsibility but we’re also gonna now engage with you in your world at the business level and now we’re gonna help you figure out how to think about data in a different way.” and that is fundamentally shifting a different organization… It’s a multi-year path but it does go back to starting with a core group of really, really talented, multidisciplinary analysts, to start with.

0:26:31 MH: Yeah. Well and so there’s a company called New Vantage Partners, they do this survey of executives. I think maybe who is the Competing on Analytics guy.

0:26:45 TW: Tom Davenport?

0:26:47 MH: Tom Davenport. I think maybe he’s part of that thing, maybe. I don’t know but anyways, they do this survey every year, in 2020 they had this section where they asked all these leaders, what is the biggest challenge or what are the challenges to you actually being, in their words, a data driven organization? And 91% of the responses had to do with culture, people and process and so it’s like the tools really aren’t our problem in becoming data-driven. Even great data visualization, while I think all of us believe very, very much in becoming good at that, it’s the actual context that you operate in is actually holding it back and I got a chance to speak to a group of business leaders recently talk about this and I was like… “Do you find it encouraging at all that 91% of your effort to solve this problem has nothing to do with you potentially becoming more analytical yourself?” you simply just need to be better leaders to your teams and people.

0:27:53 MK: But that’s where I think in terms of top-down tactics, that leaders can do a lot. So almost every company has goals of some description or objectives; Key Results, OKRs or some process and we’ve just gone through this with our last lot of planning, where every single goal that we have, there is a link to a dashboard that tracks the relevant metric and that is something, like I said to the exec team, I’m basically like “Hey, just letting you know, we’re gonna… ” and my boss also has done an amazing job of this of like, we need to make sure that there is a dashboard or a look or a graph somewhere that every single metric we talk about, someone can click and see how we’re performing against that particular KPI and that doesn’t involve a leader suddenly becoming way more analytical, it just is about interjecting some data thinking into our processes.

0:28:51 TW: But there’s the challenge of the leaders who say “I’m on board with this. I wanna have dashboards that we get insights regularly from the data.” or they spout out the “You can’t manage what you can’t measure.” They drop platitudes right and left. I think there’s that challenge. I’ve thought about when interviewing, if I was gonna go to a different job, how would I actually really suss out if the mindset of the team that I would be supporting was such that I have felt like I could mold it.

0:29:34 TW: In hindsight, I went to companies where they were pretty excited to talk to me ’cause I could string two words together about data and they were like, we’re so far behind on analytics and we need you to come in but with their expectation for like how you use data and it’s fine if they say, we know we need to but we don’t really know how and we’re bringing you in to help us is positive. I read the Fortune or Forbes or HBR talking about machine learning and we gotta really be digging it. We gotta be bringing these actionable insights or these… The buzzwords that kinda turn my stomach.

0:30:20 TW: I’ve got a company I was talking to that I somehow got myself roped into speaking to them just through an old contact, which will be fine and I’ll talk about data informed decision making, the Tim Wilson perspective but then it was like “Oh, you gotta talk to this VP about this.” and that guy was dropping some of the cliches and the platitudes. I’m like “Well, I think you would be frustrating to interact with on an ongoing basis. I’ll entertain your team. I may help them shift the way they operate but man, I would not take a job here unless you and I had a real heart to heart and I pointed out that these things you said are just misguided. You’re dropping the buzz phrases that get scattered throughout the industry.” So I don’t know, figuring out the self-awareness of the executives is key.

0:31:12 MH: Well, it turns out leadership has more to do with what you do versus what you say and I think that’s what you’re touching on.

0:31:21 MK: I think both matter…

0:31:23 MH: No, they absolutely do but one matters way more than the other and that’s what Tim is talking about. You can say you wanna be data-informed and we wanna use KPIs but then you go shit on anybody who comes to you with a good idea or tries to innovate in their space or whatever, like negative behavior, negative behavior, native behavior ’cause think about Moe, even… I hate to keep picking on your example but it was a great example. You did a bunch of things and then you said things to people but it was primarily, you went and did a bunch of things that then created the space for that and then I’ll share another example from a manager I had which I’ve talked about on the show before, Kerbie Wegner, thank you so much. You’re the best.

[chuckle]

0:32:10 MH: But she had no background in digital analytics when she took over our team and she sat down with us and she said, Okay “Here’s what we’re going after, in other words, here’s our goals, what do you need for me to be able to accomplish them?” And then she let us create the whole plan and then supported us in executing it and it was such an amazing lesson because what she did was not go talk a good game about how analytics was gonna be flying out of our department and we’re gonna transform this organization. She just said, “Here’s our objective. You need to tell me what you need to make this work and then I will steadfastly block for you in the organization and not let other things slide into your work or all those things so that we can actually go achieve the objective.” and she’s the VP of something pretty important at our company today because she’s that awesome.

0:33:04 MK: But that’s also… That’s exactly what I mean about pulling the team in, like when we started talking about data strategy, everyone in the team acknowledged that we wanted to improve and we wanted it to get better and we basically opened up a doc and said to all the analysts at the company like “Hey, what are your ideas that at each of these different levels, what are the strategies we can employ and so you’re actually collating everyone’s idea is not just that one person of like “Here is how we’re going to do it, I’m gonna write a strategy doc and tell you all how we’re doing it.”

0:33:33 MK: Including the team is the best way to get buy-in and I mean, it’s the same for your execs, right? That muppet that you talked about Tim, I feel like there are some lost causes but I guess I see that any job I take, part of my role is always going to be to mold the data culture. That is a core part of our job is to help improve that in any company you work at and if you have a really shitty exec, then I probably wouldn’t take a job there, I know that’s a really snobby thing to say ’cause not everyone’s in that position.

0:34:05 TW: I feel like I could be getting hired with that expectation and I really suck at doing it so…

0:34:09 MH: Well and actually not gonna lie. A big reason why I did what I do now and didn’t go take a job in some other company or other places, probably because I don’t totally know exactly how I’d figured that out and I don’t wanna deal with leaders who don’t get it ’cause here’s the other thing. Actually, this is good for us to talk about ’cause I think one of the big things people run into is kind of a buzz saw is that there’s some stuff already happening and actually to build a good culture, you sometimes have to tear that down and then rebuild and I feel like sometimes it’s a hard hard thing for companies to be like “But we already have like this, this and this and why would we need to get rid of it?” It’s because it’s built on really crappy things and I’m just trying to come up with an example of what I’m talking about.

0:34:58 MK: I feel like Adam Greco when he was on, actually did a really good example of explaining that where basically like, “Hey, here’s a reason we’re gonna pull out this tool and start from scratch.” and really walk us through how you can make that turn into a really successful change process.

0:35:16 MH: Yeah but it could also be the dashboards we have or the key performance indicators we have. It’s like “Okay we’re trapped in this and we have to grow from here.” and it’s like “Well actually, if we wanna grow at all, we gotta go back down and climb this way.”

0:35:28 TW: Well Moe you had brought up, I think before the show and then in some of our prep, the whole idea of self-service and a positive data culture, having the right level of self-service available and so I would have loved for you to… Like their self-service gone awry where there’s not enough support or the data is too complex or the tools suck or people are just using the… They’re just weaponized the data and all sorts of stuff flying around that’s not accurate and then there’s the other extreme which is “Nope. Nobody can self-serve, they have to go through an analyst.” So we’re…

0:36:08 MK: Just my worst nightmare. Yeah.

0:36:10 TW: Yeah. So where does that… ‘Cause there’s always a desire, I think, for self-service but then sometimes in our organization it’s like well, there’s a desire but there’s maybe an expectation that I should be able to self-serve without any education or training or work. They’re like “I wanna just pull this.” And it’s like well, sometimes the world you’re working in is a little too complicated to think that it’s just log in and it’s a Google Analytics, simple half dozen dimension. So how do you think about self-serve and enabling it safely but supporting it and expanding it. What have you done on that front?

0:36:52 MK: I think that… I feel like training is integral when you talk about that like going through the middle part. Training is, you have to suck up that you’re gonna have to invest some time in training.

0:37:03 TW: And that’ll be ongoing, you can try to record it. That’s the other, people do like… They’re like “Oh we did the training.” I’m like “Yeah but five more people started a week later.” so…

0:37:11 MK: No. No. You have to be doing data training. We wanna start building it in… We do already do a data introduction in our new onboarding process but we want to have more regular things so things are on data visualization and all that sort of stuff but we did also realize our tooling was a problem so previously, we used a tool where you had to know SQL to basically query any data, that’s a huge blocker and so we made a decision as a team like…

0:37:38 TW: A little bit of training, come on. Select where…

0:37:41 MK: We’d actually do SQL training but the point is not… And some people in the company are like “Every PM and marketers should know SQL.” and I’m like “Well, that’s not reasonable.” I don’t think that’s a reasonable expectation and so we actually made a decision to change tools and the team were amazing. They did the hard work. We’ve been through a year of changing our data warehouse, building out data modeling that is actually correct and works properly for our dashboards, building a whole bunch of dashboards, which let’s be honest, analyst hate doing but everyone sucked it up and was like “These are the blocks we have to build.” and even this morning, I had one of my favorite marketers message me just being like “Hey Moe, I looked at this dashboard… ”

0:38:22 MK: So we have specific dashboards that the analyst team have built, they have been QAd by another analyst, all the numbers have been checked, we have a QA process for all of our dashboards and then they get an Analyst Certified little badge on them which means that any stakeholder knows that they can use that dashboard confidently and so she went and looked at the dashboard and said “Hey, I wanted to pull monthly active users from my particular channel for these dates, I just wanna check like have I done this all correct? Does this mean that we’ve had this much growth? Am I interpreting this metric correct?”

0:38:52 MK: And I said to her, I was like “You are my rock star example of an amazing stakeholder because you went and used the tooling that we’ve given you, you checked and then you’re just coming back to do a like “Hey, this is the assumptions I’ve made, is this all correct? I’m gonna be sharing it with a pretty big audience, is this all cool?” And we were able to go back and be like “Yep, totally nailed it. Yes, you have grown your channel that much, isn’t that amazing?” and so much of that is because of the really ground foundational work that the team had done and hats off to the team because building dashboards sucks but it has to be part of self-service as just training, which can be really boring.

0:39:36 MH: But I was gonna say, what I noticed in your story is what you’ve done is you’ve actually created data literacy across your stakeholders by working them through those dashboards and helping them understand, which gives them the confidence to do some of the steps and data literacy, which again, I’m gonna go back and still the one that Debbie Berebichez gave from the MIT Media Lab, which I really like and I’m gonna keep using, which is to be able to read, work with, analyze and argue with the data and in a certain sense, what you just described was their ability to go work with that data and just to go acquire the piece of data that they needed and to be able to have confidence that they did it correctly so that they could go on to the analysis and arguing stage of it so that I can go present my narrative around why this data means the thing that I’m doing from the business side.

0:40:27 MH: So I love that story and actually, I have a similar experience in that we built basically these dashboards for, this was a client and to help the different groups that we’re gonna be looking at them, we built sort of a companion to the dashboard that literally was sort of like a worksheet that they could walk through and it would actually show them like “Here’s what this metric means, here’s for our business, here’s a good… Maybe this number is bad in this case and good in this case or here’s the numbers we’re targeting, what do you think?” And I would ask some questions.

0:41:03 MH: It never is as good as you want it to be ’cause as a consultant, you don’t get to stand on everybody’s shoulder and make sure they do it but it was a great effort to kinda take one more step and instead of showing just “Here’s a dashboard, I assume you know what to do with.” it’s more like “Hey, come sit down with this dashboard and then use our little guide to kinda walk you through it.” which I thought was really cool.

0:41:25 TW: It’s funny ’cause it goes back to one of the organizations working with now that I’m actually… We’re getting a lot of… Having a lot of success. One of the things we’ve done is we’ve actually built… And it’s in a spreadsheet for now, which everybody, we have our little core group of marketers who are kind of our pilot group. They were kinda selected and then we meet with them every two weeks and we work through them with their campaigns but we’re using a spreadsheet. We have sort of their process for planning and then monitoring and then ultimately closing out and having a post-mortem on campaigns and that goes through these are the questions, these are the business questions you need to have good answers to and some of it’s the easy stuff, who’s on the project, how much is the budget, what channels is it gonna be but then it walks them through a little bit of the framing of the KPIs but it gets them to…

0:42:22 TW: There’s a tab that says “Okay, then when you’re doing your check-ins on the campaign, if it’s a small campaign, these are the four things you look at.” ‘Cause we’ve got the supporting BI platform where all that data is available. If it’s a bigger campaign, there’s actually, in that spreadsheet, a branch that says “This gets a custom dashboard.”

0:42:42 TW: So there’s a planning sheet and at this point, the analyst team will work with you to get that fleshed out to make sure they’re building a dashboard that really supports the campaign and then you follow a different process. So it was building a whole bunch of collateral around the planning and the managing so it felt they knew exactly kinda where and how they should use the tools. It even has a hypothesis backlog sheet that says “Every time you’re thinking through, if something’s not going well or you have an idea, let’s go ahead and capture that.” So we had the training, we’ve built the process pieces and really, the technology of the data across all these different data sources that’s been in place for a while and this was now kneading to the training, the process, the process included building some other kind of tools for them to work with.

0:43:30 TW: And that again, is very much on a path, it comes from the middle but Moe, as you were talking about that the analyst team did all this stuff for the dashboards, the marketers, I guess they clearly were like “I’m not gonna learn SQL.” so what was their pain? What did they have to do to go along for the ride to actually say “We’ll show up at the training, we’ll understand these dashboards, we would prefer to be self-served rather than just being able to Slack you a message.”

0:44:00 MK: Well, you don’t wanna block them. I always pose it is as like “We’re a blocker to you. If you put it in our backlog, you’re gonna get this in a week. If you figure out how to do it yourself, you’re gonna get it in a day.” But I also think that it comes down to real diligence from the analyst because when someone asks for something and this is something we’re really going through. When someone asks for something, you can’t just give it to them and lots of people in the team are doing that because it’s faster and you have to actually push back and say “No, I’m gonna put aside 45 minutes, I’m gonna jump on a Zoom call with you and I’m gonna show you how to do it.” and I love the idea of having guides so that the next time you can be like “Hey, we’ve gone through this, why don’t you try with the guide first and if you get stuck, let me know.”

0:44:40 MK: But you have to really push back.” and we have one stakeholder actually, who when one of the analysts tried to say like “Hey, let me show you how to do this.” the stakeholder was like “No, I just want you to pull the numbers. Can you do that?” And so that’s kind of like one of those tricky situations.

0:44:57 TW: Unless we say “Well, I could but if you just want the backlog, you’ll get it in a week.” [chuckle]

0:45:03 MK: Well, it does depend on who the person is. When the founder asked me for numbers, I don’t say “Let me show you how to do that.” I just pull the numbers ’cause it’s the founder but one of the things that you try and do with people like that, those people that are just really resistant, you’re not gonna be able to convince them so I actually feel like you wanna go sideways. So there’s a couple of things like one, you teach everyone else around them how to do it because then they look like a muppet because suddenly everyone else can do something that they can’t but the other thing is, you also pull them into a public channel because what they’ll do is they’ll message an analyst in a side bar or a direct message and you get them to come back and be like “Actually, do you mind popping this in the public channel just so that we can make sure that the first analyst gets to it as quick as possible.” and then suddenly, everyone can openly say to this person is asking for really stupid shit very regularly and so it’s like a really manipulative way of adding their shitty behaviour.

0:46:01 MH: No!

0:46:01 TW: You’re slack shaming them.

0:46:03 MH: It’s actually… No! You’re using the culture to have people kind of conform to the norms that you’ve established. It’s positive peer pressure.

0:46:14 TW: But that gets to… And this, I’ve definitely employed this as well, you’re gonna have the… Say you’ve got five different people and some you’re like “Okay, this is gonna be a really long road to hoe.” these other ones, they’re kind of about there and they’re ready so I will invest a little bit more to make the ones who are ahead more successful faster so that I can then use that as a… Can turn them into champions and then also use them as the examples in the case studies. Like Moe, the example you just had, you’re gonna repeat that. Yeah, you’re repeating it to our legions of fans and listeners but you’re gonna use that internally. You’re gonna point to as it’s purely positive that “Hey, this was great. She self-served, she did check-in ’cause she recognized part of data literacy is to understand the stakes with the data that you’re pushing out.”

0:47:07 TW: If you had pulled that, you might have checked with somebody else to say, give a second set of eyes. There are like 27 different positive stories you can spin out of that one example to make her kind of a hero and it’s kind of a model and it’s not from… It’s from her perspective.

0:47:25 MK: And that’s part of data culture, right? It’s like that stakeholder who did say like “I went in and self-serve this.” like “I went back and immediately gave her a really good feedback.” but we do have a Kudos channel and I’ll probably go shout out later today and be like “Oh my God! I was really excited to see the stakeholder who had an amazing time self-serving. Awesome work, you pulled this stuff yourself and we’re able to unblock yourself or get the data you need it immediately.” and then you’re also shouting that positive behavior to the rest of the company as like “This is the example we wanna see.”

0:47:54 TW: Now kick over to the other public Slack channel and see our response to this other person for…

[laughter]

0:47:58 MH: Yeah, in the not Kudos channel. So two quick things; as a leader, the other thing to do is train yourself not to ask for data but ask for help solving your business problem and so basically, the analyst is there… So a great analyst will do that step of sort of being like “Let me investigate this idea a little bit more and pull out from you what you’re trying to do.” ’cause it’s about getting the masters of the two domains to get together in the middle to get the right solution but treat it as an equal thing and what I like about that is that if you’re good at business and you do that, you actually then train, help guide and mentor the analyst in their understanding of how the business works, which is a huge benefit to them and their problem solving and analytical capability will advance as a result of that too.

0:48:49 MH: So as a leader, it’s a huge payback for you to spend the time to explain your problem as opposed to just ask for data.

0:48:57 TW: And you’re setting the expectation for the analyst that you’re here to understand the business and creatively on how to solve this because there are an enormous number of analysts who because the society has conditioned them to be, they ask for data, we produce the data. They ask for a dashboard, we produce the dashboard. So a leader who says “Yeah, that would be easy for you. I know you’re the data expert but instead, let me pull you in to brainstorm about this.” and that is one where, although, I am now getting to really see that you do not need to have a ton of real world work experience to be able to do really solid, critical thinking in a business context and all you need is to be encouraged to say “Ask the question you think is a stupid question, don’t go off and agree to do an analysis until you actually understand the business question.” and that’s another part of that creating that space.

0:49:53 TW: I was just talking to somebody today, where the analysts who think that they’ve been told through different implicit and explicit messages that they need to have the answers and so therefore, they’ve been conditioned to not want to say “Wait a minute, what is a qualified lead? How do you think of that? What do you mean when you say a booking?” Getting that level of comfort to say, as an analyst, you may understand… You may know where to get the data, you might not actually understand what the data is and the only way you’re gonna understand that is if you really get clear on what the business side is and guess what, you will learn very quickly that most of your business counterparts are happy to sit down, draw on a whiteboard, explain what they’re actually thinking but you better be ready to engage with them and bring thoughts to the table, not just… You can’t shut that down and say “Yeah, yeah but what data do you want?” Again, that’s the killer of the relationship.

0:49:56 MH: Okay and then the other thing I was gonna say Moe, I was working with one of my clients a couple of weeks ago and we were doing an analysis and the data had a missing piece so we had to re-pull some of the data and as I was waiting for that to happen, I heard Moe’s voice in the back of my head saying “If you know how to pull, write some SQL, you would be able to do this yourself.” And I was like “Oh! I’m stuck!” Okay, we gotta start to wrap up this. I think what we ended up with is that we’re all three fans of a great data culture though. So I don’t know, I don’t wanna speak out of turn but I think it sounds like we all agree, that would be a good thing.

0:51:40 TW: I’ve gone ahead and logged it for episode 243. We’ll do another touch in update on probably how we’re doing.

0:51:47 MH: Good. Yeah, yes in 100 or so episodes, 130 episodes. Alright so great conversation and really great insights. So one thing we do is you both are aware, we do a last call, something that we think might be of interest. Moe, you wanna kick us off? You got a last call you wanna share?

0:52:09 MK: Well, mine’s about data translations. So I don’t know if Tim does want me to share.

0:52:17 MH: I desperately want you to share.

[laughter]

0:52:17 MK: I came across these books, actually it was a guest that recommended them, they’re by Chris Ferrie and they’re really hysterical ’cause it’s like Bayesian Probability for Babies and Quantum Physics for Babies, there’s a whole collection but anyway, I brought this one to work to show my team and we ended up reading it together and decided actually, it was the best way to explain Bayesian probability to our stakeholders, which might seem like a little bit of a put down but literally being able to explain concepts to a child is one of the core skills that we need to nail as analysts and if you can get… It doesn’t matter what the resources, if you can find a way to explain something to someone that makes sense to them, this particular book on Bayesian probability uses cookies and whether or not it has candy on it, which I feel is very relatable but it’s… I like the idea of taking really complex things and making them really understandable for your stakeholders, plus they make great gifts for friends so it’s a double win.

0:53:24 TW: I’m thinking of that as one of your analysts saying “Okay, for the next show and tell, when I stand up, I’m gonna do a dramatic reading of Bayesian Probabilities for Babies.”

0:53:33 MK: Well we were thinking of maybe not telling the stakeholders that it was from a baby book.

0:53:38 MH: I sort of envisioned Moe in the team cafeteria doing a read-aloud time and showing everybody the pictures and like reading…

0:53:45 MK: I didn’t do the read-aloud. Someone else in the team did it. Yeah, it was very dramatic.

0:53:46 MH: Oh, okay. That’s awesome.

0:53:52 TW: I’m gonna kick right now ’cause I have not been able to do one of these like explain some concept through simulation or visuals and I’m convinced the next one I do is that a fun and simple and clear explanation of just dependent variable for oriented towards business users ’cause I’ve realized how much as a… When working on our data science projects, that’s the… One of the questions is, what’s a dependent variable, what are your independent variables? And that’s kind of simplistic, Moe’s flipping through the very thick cardboard pages in Bayesian Probabilities for Babies to… They’ve already covered it. So I’m there ’cause I think that explaining, this is the parent distribution of cookies, yeah.

0:54:42 MK: Yes and this is the posterior distribution of cookies.

0:54:47 TW: Wait, can you hold that up one more… Wait, hold on. Let me say those cookies again? Hold on, just a little bit longer. A little bit longer. Okay, perfect. Now, there you go.

0:54:57 MK: Oh you totally a photo.

0:55:00 MH: Yeah, you did.

[laughter]

0:55:00 TW: How long can you… So we’ll see the show notes.

0:55:04 MH: Oh that’s awesome. Okay. Alright Tim, what about you? Last call to share?

0:55:09 TW: Sure. Somebody has done something that I have claimed could not be done and he has done it really, really well and I don’t know how long it will last but the question that comes up every so often is comparing Adobe Analytics to Google Analytics and that is an exercise that is fraught with risk and really hard to do for years and years and years and I’ve continued to maintain this. Jenn Kunz was one of my gold stars because she really had a deep understanding of both and would think about them both objectively but Nikolay Gradinharov, do you know him, at QA2L?

0:55:45 MH: Oh, yeah.

0:55:47 TW: Yeah great guy and the fact is, he could have added a column for Webtrends as well ’cause I think that’s where he started but he has done a multi-part blog post that is comparing Adobe Analytics to Google Analytics and it is really thorough and really good and he’s even coming a little bit of it from an organizational perspective and even he and I were chatting a few weeks ago and he was like “Yeah and now there’s App + Web.” there could be a third column but I was amazed at the depth and thoroughness of it and so anyone who is in that boat of saying “I’m using one, how does this actually compare to another? I’m taking a job where it’s switching, my company is just considering switching.” It’s six parts but it’s amazing and my hat is off to him for actually pulling it off and pulling it off well.

0:56:46 MH: Nice.

0:56:48 TW: What about you, Michael?

0:56:50 MH: Well as luck would have it, I also have a last call. So recently I ran across, I think it’s sort of a podcast but I saw it in video form on YouTube and it was a play list of these really short videos by, I think it’s a guy who’s like a venture capitalist named Keith Rabois. I don’t really know much about him, except he’s been involved with a lot of big companies but he did this whole thing, a series about how to build an iconic company, not The Iconic but an iconic company and I just, after listening to a number of those videos again and again, I just found myself nodding my head and being like “Yes, that’s right, that’s exactly right.” So I was sort of like “Oh! No, this is actually a really good thing to talk about.”

0:57:38 MH: Anyway so I just highly recommend the whole series but probably my favorites were probably the one he did on insisting on focus and measuring inputs and not outputs in terms of sort of planning and things like that which Tim, you actually might like ’cause he kinda goes in on the OKR process, maybe not the best one. Okay so that’s my last call.

0:58:03 MH: Okay, you’ve probably been listening and like the intro of the show sort of postulated, you’re like “Yeah, our data culture is so wrapped up and totally cool.” Well, we would love to hear from you or even if it’s not, we’d love to hear from you too. Ideas, thoughts, things like that. Ideas for a great nickname for Moe as the quintessential data leader of tomorrow. I don’t know what the right way to say, just workshop it with me people, it’ll be great.

0:58:29 MH: Anyway, the best way to do that is either on the Measure Slack, which we are all part of or our Twitter or on our LinkedIn group and we would love to hear from you. From time to time, people even reach out and suggest topics for our show and we’re happy to hear those as well. Alright, none of this would really work very well lately without the amazing contributions of our producer, Josh Crowhurst and we continue to benefit from his association with the podcast and by you, listening right now without knowing it, you’re also benefiting and so we appreciate him very much.

0:59:07 MH: Now, if we’re gonna have an amazing data culture, you’ve gotta stop listing to this podcast and you’ve got to get to work and I know that as you go, do what you’re gonna do to build a better data culture for your company. I know that both of my co-hosts, Moe and Tim agree, you should definitely keep analyzing.

0:59:30 S1: Thanks for listening and don’t forget to join the conversation on Twitter or in the Measure Slack. We welcome your comments and questions. Visit us on the web at analyticshour.io or on Twitter @AnalyticsHour.

[music]

0:59:43 S5: So smart guys want to fit in so they’ve made up a term called analytics. Analytics don’t work.

0:59:52 S6: Analytics? Oh my God! What the fuck does that even mean?

1:00:01 S7: But honest statistics need to be defended because while it’s easy to lie with statistics, it’s even easier to lie without them.

1:00:12 TW: And are winning more favors now that we have somebody spontaneously a month later pining for the multi touch [1:00:17] ____ I was like…

1:00:19 MH: I know.

1:00:21 TW: Those were an okay idea that could’ve been amazing if we had but we’re not and…

1:00:31 MH: Well we stopped doing it so that we could focus our efforts on our social media which also…

[overlapping conversation]
[laughter]

1:00:41 MH: It doesn’t give me any joy to report that back but that is technically what we decided.

1:00:49 MH: You know my favorite quote of all time around this is from Stu and its, he’s like “You can either have data driven decision making or you can have decision driven data making.”

[laughter]

1:01:00 MH: I just think about that all the time. I think it’s such a great quote.

1:01:04 TW: I had to delete another email about taking my EQ training.

1:01:12 MH: Oh man! Mr. Cranky Pants is right.

1:01:20 TW: Rock Flag and Data Culture for babies.

1:01:22 MH: Ooh! That’s a good one.

Podcast: Download | Embed

Subscribe: RSS