[music]

0:00:05.9 Announcer: Welcome to the Analytics Power Hour, analytics topics covered conversationally and sometimes with explicit language. Here are your hosts, Moe, Michael and Tim.

[music]

0:00:22.2 Michael Helbling: Hello everybody, it’s the Analytics Power Hour and this is Episode 203. From time to time, it’s good to check your base assumptions. A great analyst will always try to come at a problem from every direction, after all, you can catch a fish from a boat, or you could dive beneath the waves and catch a fish under water if you have the right skills and tools. All that said as a preamble to what I think would be a very interesting discussion on this episode, thinking through the complexity of what we do in data and analytics both with tools and methods, and whether or not we’re missing some beauty and elegance in solving it with broader solutions. Alright, Tim, what do you think? Are you ready to dive in? [laughter]

0:01:09.8 Tim Wilson: Let’s go deep.

0:01:11.2 MH: Let’s go deep.

0:01:11.7 TW: Deep in complicated thoughts.

0:01:14.2 MH: I’m the one that’s the furthest from the water usually in the data and analytics space, so I’m that guy way back on the dock. And unfortunately Moe isn’t able to be with us today, but I know she would like this discussion. However, we did wanna add a guest, and frankly, we’re very excited to talk to him. Frederik Werner is a senior specialist of digital analytics at DHL, you may also know him from his excellent blog posts covering Adobe Analytics products at fullstackanalyst.io and he’s been named a two-time Adobe Analytics champion. He has a background in psychology, and we couldn’t be more excited to have him as our guest. Welcome to the show, Frederik.

0:01:55.7 Frederik Werner: Hey, I was super excited to be here, thanks for the invitation, guys.

0:02:00.1 MH: Yeah, we’re sort of on, it’s all a mutual admiration society, it seems like, once we finally started talking. So we’re really excited to have you on the show.

0:02:09.9 TW: It’s kind of a pseudo-second appearance since he was name-checked…

0:02:13.8 MH: That’s true.

0:02:14.1 TW: Multiple times in our live.

0:02:17.3 MH: And from the Vegas show.

0:02:19.4 TW: From the Vegas show, yeah.

0:02:20.6 FW: Yeah, I know how to use Twitter, yeah.

0:02:22.0 MH: Yeah, it was during that time where we turned to each other and it was like, “Why hasn’t we brought him on the show?” And that was the beginning of this, so perfect. [laughter] So Frederik, we started with a topic, we kinda changed the topic and then we expanded the topic to what we’re gonna talk about today, which is…

0:02:39.5 TW: It’s almost like we made the whole planning of the show too complicated.

0:02:42.6 MH: Yeah, maybe. [laughter] And so just to get us kicked off here, first off, maybe what I’d love to hear from you is what are you observing in the analytics industry that has you concerned about whether or not we’re making things too complex? And what solutions you think? And we’ll dive into solutions but we’ll start with, what are you seeing in the industry that…

0:03:06.5 TW: And what’s the answer in three sentences? Yeah.

0:03:09.1 MH: Yeah, this could be a real short episode. [laughter]

0:03:12.0 FW: It’s gonna be super simple yeah, just do this. They were like, yes, all of us have been in the industry for over a decade by now, and one thing that I’ve always observed, no matter in which industry I was, no matter in which size of company I was, was always that people have preconceptions about what analysts do, and that goes both for the business side and also for the analyst side. And also some preconceptions on what you need to know and what you need to learn to be, I don’t know, qualified or allowed to do analytics and all that.

0:03:45.8 FW: And I’m a very big fan of democratization and getting data in the hands of people, and whenever I try to do that, I’ve been confronted with people saying, “No, this is super complicated stuff, I don’t know analytics, I don’t know this tool, I don’t know how statistics work and how I’m going to make sense of the data.” And I’m always like, “But, are we making this too difficult for people?” In the industry when we’re talking about modern data stack and all the different tools we use and using SQL and joining big tables and big query and all those different things, I’m like, “Is this what we should do? Is this what we should represent to the business?” Because as you said in the intro, I am also a big fan of Adobe Analytics, not because I like red tools or anything, but just because its the best I found. So like I said that now in the beginning, so yeah, Adobe Analytics, cool tool, I like it. And yeah, that’s why I was always a little bit introspective and thought, “How can I reduce complexity to my stakeholders, can reduce complexity towards the people who should actually be using data?” And yeah, that’s how this topic came along.

0:04:58.3 TW: So is there… It’s funny, I think I can make it complicated just trying to think about the different buckets in which we might wanna make things complicated or simple. This is kind of a… The big question there was years ago, and it still rears it’s head up periodically in the digital analytics world of, is analytics hard or not? And then you get various strongly opinionated people who are trying to give a binary answer to the question of analytics is hard or analytics is simple, which honestly, I can’t tell you who argued which side and I don’t really care. To me it’s, there are very complex aspects of analytics, there’s when you’re trying to do things with building a model, you can make it really simple and have it draw really wrong conclusions, then there are things that are simple but really hard. I guess the spectrum of, is it simple to complex? Is it easy to hard?

0:06:05.9 TW: I think asking, really thinking about your business and really articulating what success looks like, simple conceptually, but can be really, really hard to do. If you can do that, then the data may be actually easy. So I don’t know, I may have already overcomplicated things ’cause I’m convinced this could only… This discussion has to happen in three dimensions or maybe even four. So I am biased towards… [laughter] I wind up in the complex needlessly so.

0:06:37.4 FW: Yeah. And that’s a very good question in terms of what complex actually means, because when you think about analytics as just analytics and what we do then of course we always find ways to make it more complex and to always take it apart because we are analysts. We are very good at taking things apart. That’s what the word means. [chuckle] And what I then trying to find is analogies to other areas. And for example, think of a car driver versus a plane pilot, and to the car driver, flying a plane sounds really scary and really something that they wouldn’t feel comfortable, I hope. [chuckle] I wouldn’t feel comfortable. [chuckle]

0:07:18.0 FW: But to the plane pilot, it may not be that difficult. It’s just become second nature. And they went through a lot of training and it has some relevance to them. And this relevancy aspect is very important in there because if we take data as something that we… That’s another point I’m very opposed to. If we take data and analytics and all that as something that people should do, but don’t want to do based on their own interest, then that always feels very complex to them. And that always creates barriers and they don’t want to do that kind of thing because we’re always like, “Hey, you need to be data driven. You need to look into all the data because we see the value in that,” and as people don’t see that, or are not held accountable for their situation awareness that they need to create. And it can be something that is very complex to them. Don’t know if I’m making sense right now, but…

0:08:19.4 TW: I think of what you’re saying, you’re reacting to when there’s a dictate from on high that we must be data driven. And there are people who are like, “Well, I’m gonna get my hand slapped if I don’t have some charts in my presentation.” The reluctant, if they haven’t fully internalized where data can help them with their job or where they’re actually enthused about it, they may be delivering and accessing data, but they’re not really gonna be trying to use the data. And that just injects a bunch of noise and unpleasantness into the system. I may have just… Is that where you were… You said something and I was, I kind of got at the trying to… You said if they don’t see the relevance or if they don’t see why they need to use it, it’s gonna be a problem? Or did I just head off on a tangent?

0:09:20.5 MH: It’s sort of complexity isn’t the horizon of applicability or interest. So it’s sort of, to the extent that I have to solve this problem, I’m willing to go in and do those things and it doesn’t then appear to me to be as complex. Whereas if it’s not applicable as a problem, it’s like, “Okay, well, if I go,” I know that data warehouses exist, but I’ve never written SQL as a marketer, let’s say, or a marketing analyst. That just seems complex until I need to actually write a query that does a thing that I need to know about. And then I can google it and figure out how to get that query written. And now suddenly I’m writing SQL and it’s not as complex anymore, because it’s been brought into my view, so to speak. Is that kind of what you’re saying?

0:10:11.3 FW: Yeah, both of those.

0:10:12.9 MH: Okay. [laughter]

0:10:17.3 FW: Definitely both of those are very valid, but yeah, it’s… We had in the industry multiple iterations of people should be data driven. And then we are like, actually, they shouldn’t be driven by data, whatever that’s supposed to mean, but they should be data informed. And we went through multiple iterations of phrasing that kind of thing. From a psychology perspective, I would always say, people should be aware of their situation, be aware of their surroundings. They should have situational awareness and going to a different area of, for example, retail analytics. If you have a small store in some small city, as the shop owner, you can just stand in the middle of the store and just see what people are doing. You can install all those in store analytics systems, like Bluetooth, beacons and all those different things, but you can just stand in the middle and see how people are feeling and if they have any problems. So should that store owner then be data driven or data informed or whatever?

0:11:24.4 FW: They need to be aware of any problems in their business and they need to find opportunity for their business. And now going to, sorry, then going to digital, that’s obviously harder to do because you can’t watch people directly sitting in front of their computers. So from that kind of angle, I’m always thinking of how can we bring that situation awareness in terms of show meaningful things to them and inform them how their customers are doing to allow them to change things if they want to also keep things up that work well.

0:12:00.0 TW: But I think that is part of the, to me, the crux of education and relevance, is that if you kind of taking the store example and extrapolating a little bit to a… If it is digital, and if it is… Every year we run a big advertisement in Times Square during back to school. And every year we get our best sales of the year come during back to school. And I would claim that there are a non-trivial number of marketers out there who would say, “We see a relationship. We see that running this advertisement in Times Square is causing… ”

0:12:51.9 TW: Is benefiting our back-to-school, the classic… And unless they have an understanding, it’s not… Is it simple or is it complex to say, “Wait a minute, you can’t just say correlation is not causation,” you need to have a little bit more focus to realize that if you did something and something else happened, there may or may not have been causality. You can logically say, “Of course there’s causality, because I’m a thinker and I’m situationally aware, and this just makes sense,” which is fine. May be fine. Or you could say, “Well, wait a minute, something else could be going on. How can I use data, how can I do an experiment, how can I do a more sophisticated analysis, how can I engage with an analyst or a data scientist to figure that out?” I think that’s… To me, that’s the leap, is that there is a tendency, for whatever reason, people who are not drawn to… I don’t think this is about learning SQL or learning R, or even learning advanced statistics and machine learning, I think it’s… There is a desire, “Hey, we ran the campaign. Tell me how it did.” Well, that’s a simple question, but that doesn’t remotely have a simple answer.

0:14:13.4 TW: It may have a simple answer with a million caveats, or it has a complex answer with a thousand caveats. And that is kind of the tension that to me, there is this expectation that we have all of this data, so therefore, be it a no-code tool or be it a whiz-bang data scientist or be it some dashboard, I, the business user should be able to just get a simple answer to a simple question, and that’s not always there. So, maybe in my roundabout way I’m getting to, we have this confusion between when we say simple. I’ve said that used to be when I was in an office and somebody would come by and say, “I have a quick question,” and I got to where I would just say, “Let me stop you there. To the question, no correlation to the complexity or the involvement of the response.” And they’d kind of chuckle, but I’m like, “I said it for a reason. Trust me, your simple question, what was the ROI of that campaign? Okay, yeah, but we can’t do your word count and equate that to what’s gonna be the effort involved.” So, I think even framing this whole topic is more complex than we thought it was maybe heading in. [laughter]

0:15:33.1 FW: That’s fun, right? We like complexity.

0:15:38.2 TW: Without going anywhere down any sensitive information, you’re at one of the largest organizations on the planet, I think, and…

0:15:46.9 FW: Indeed. Half-a-million people.

0:15:48.7 TW: Interestingly, way on the outside, a ton of just logistics and running the business, that’s the core of what the business does. So, what do you see with the users of the data? Are they asking questions that you’re like, “Well, I know you think that should be easy to answer. But alas, come into this room and let me draw on the whiteboard for you, it’s more complicated.” Or what do you run into in the reality of a large enterprise with a super complex… Just the digital data is complicated, and that’s arguably not the core of the business by a long shot.

0:16:29.9 FW: Yeah. It’s complex, yeah. [chuckle] I think that’s part of why we love it. So, that’s without a doubt the preconceptions that a lot of people have going into that, that also from historical questions that I’ve been asking, that there’s no really simple questions to what’s the… No reasonable answers to what’s the question of what they should be doing. And that’s always when I’m talking to them, when I’m trying to refine their questions, when I’m trying to enable them to be self-service, and all those fun things we try to do. What I’m always trying to help them is to reduce the complexity of the question that they are having, because they think of so many potential influencing factors like why is this marketing campaign performing the way that it is? Because it could be seasonality, it could be other marketing campaigns, it could be like a big branding campaign that nobody knew of, and all those fun things that happen in that enterprises. And I’m always trying to reduce that towards a point where they feel like they are empowered to take action, if that makes sense.

0:17:44.8 FW: To reduce the complexity of what they’re dealing with towards the point where they can, in an ideal word, launch an A/B test and experiment. Because that’s, I think what it comes down to in a lot of situations, to get people out of that feeling of being overwhelmed, and seeing all the data and seeing all the complexities, and all the potential confounding factors, and all those different things, and offering them a way to just say, “Well, I’m going to try this out.” Trying to sit down with them, trying to see like, “What are your options here? Looking at this user journey, looking at this conversion funnel, what can you actually do? What levers do you have at your disposal?” And then trying to allow them to try to change the levers a little bit, and try to see what impact that has. Because at the end of the day, they always try to have the analyst or the data tell them what to do, that usually doesn’t work. So, it’s a hard thing to accomplish, unless you guys will tell me that it’s actually easy and there’s a [laughter] cut-in-stone answer for that. But yeah, that’s what we’re always trying to do. Take away complexity, don’t add detail to an already complex situation and trying to unstuck them, unlock them. Does that make sense?

0:19:12.0 MH: It does make sense. The fatal flaw is probably differences because probably in Germany is not this way, but in the United States my experience has been that you’re making an assumption there’s competency on both sides, and that isn’t always the case. [chuckle]

0:19:28.0 TW: Both sides of the Atlantic?

0:19:30.2 MH: No, no both sides of the…

0:19:30.3 TW: There’s competency in Germany, and competency in… No, yes. [laughter]

0:19:31.9 MH: Of the conversation, the conversation that you just described. There’s competent people everywhere, but on both sides of that equation a lot of times in that discussion the marketer is not even aware of what levers they should be pulling necessarily sometimes, and hence the framing of the question. Or the question is framed the wrong way, and then sometimes the analyst does not really understand the role or the job of the marketer very well to be able to say, “Hey, these are the levers.” And so, it’s almost, to me, that what you just described is a problem that has to be solved at a more fundamental level of each of us knowing a bit about the domain that we are serving in that conversation, right? Or accessing.

0:20:23.7 FW: That to me is such a crucial point, because coming from that… Again, I’m always coming back to that psychology and sitting in front of a person that you have known like an hour ago and who’s telling you how complicated their job and their life is, and you need to get on a level with them, like try to help them to cope with whatever situation they are in without knowing anything about their management position or anything like that. So instead of saying like, “I’m the analyst, and I need to sit down with a marketing person, and I need to tell them what they can do,” like, I need to figure it out together with them. And I need to coach them, and maybe just say like, “We’re going to ask your agency,” or whoever you have to coach you on that, and like, “We’re going to get through this together,” instead of saying that the analyst knows or needs to know a lot about the domain, because that to me is a… Like, I’m always having trouble to wrap my head around that, to say the analyst should have domain knowledge to a point where they can give recommendations to the business, because then they should be on the business side, right? The analyst who knows more about marketing than the marketers should be a marketer. Why are they working in analytics then? Yeah, that kind of…

0:21:47.2 MH: Because they’re sick and tired of the politics in marketing, is really the answer. [laughter]

0:21:51.8 TW: But I will say the extreme… Like, to the competency, the indication of kind of shared incompetency is when a business user comes and says, “Can you give me this data?”, and the analyst’s response is, “Well, what are you gonna do with it if I do?” To me, that’s like… That’s when you’ve gone to the extreme of broken, because you’ve got the user not articulating what decision they’re trying to make or what problem they’re trying to solve, and they’re being overly prescriptive and probably don’t realize that the data’s more complex or that’s not gonna get them what they need, and you have the analyst not engaging on the business discussion, but instead throwing up a barrier and saying, “You have to tell me what you’re gonna do with it.”

0:22:38.5 TW: I think the other extreme, and I’ll mount the soapbox, I do think the analyst needs to strive to understand as much about the business as possible. There are times where analysts will be timid about asking, ’cause they somehow feel like they’re supposed to already understand. I like the framing that you just had, Frederik, where you’re like, you sit down with some business person who starts just spouting out all of the complexity, and the analyst… I’ve watched the analyst kinda seize up and be like, “I guess I’m supposed to know all of this. I better run off and come back and give them the answer,” as opposed to saying like, “Hold on, help me out here. I don’t understand your world. Can you explain to me… Okay, I’m sorry. I can go do some research, but when you say connected TV, what do you actually mean? And are you trying to decide whether you should spend more? You have extra budget, and you’re trying to put that into Facebook or connected TV? Can we talk about why you’re trying to make that decision?”

0:23:43.7 TW: And I do think it goes to the analyst putting on a… Put themselves in the business… In their stakeholders’ shoes and have that business discussion. It does seem like, yeah, when the analyst defaults to “Well, let me explain to you that Facebook’s kind of a walled garden, and we can’t get this, and we can’t tie that together,” the eyes are gonna glaze over. It sounds like, “Oh, the analyst is trying to make things complicated.” It’s like, no, things are complicated, and the analyst is there to help try to get value out of it anyway. And they may have fancy tools, or they may not, but the analyst damn well better be able to answer quickly, like, what decision are they trying to make? What are they trying to do? And what levers do they have to pull? And he’s like, “Well, I’m not really sure they even know what levers they have to pull.” Well, then you know what? As an analyst, you need to help them articulate what levers they might be able to pull, ’cause there’s no amount of data that’s gonna help give them an answer. It’s like, nothing… No amount of data’s gonna help them make a decision if you haven’t even articulated what the decision is that could be made. So, is that the solution to the complexity, is to actually say, “Nope, back to communication and soft skills,” and Michael runs a victory lap, and…

0:25:04.4 MH: Yeah, I’m already feeling it. We’re headed… The train is headed into the station, and I love it. [laughter]

0:25:12.9 TW: I mean, I did watch with… Recently, I was out, and so I watched this… One of those cases where a needlessly complex, kind of ill-conceived project got started. I was completely out of the office for two months; I came back. Guess what? It wasn’t done yet, but it was wrapping up, and it was like, “We have just done Markov chains and pulled a bunch of data and run all of this stuff,” and congratulations. This literally, analytically demonstrates the thing that the company already, absolutely knows, like, definitively. Congratulations. Now you’ve given them a complex answer to the same question that they already had the answer to. In some cases, maybe they thought they had the answer, and this was gonna completely rock their world. This was not surprising at all, but by golly, we’ve had multiple data scientists involved with it. And the client insisted that we do it. I’m not faulting our team for doing it, but I’m like… And now we’ve invested, and now we gotta go talk about… We gotta get excited about how complex it was, and maybe that’s the other… That’s the kind of fetishizing of complexity to say, “Ah, we said Markov chains. Ooh, we have a matrix, we have a heat map. Look at this. Like that… We’ve done complicated things, but what does it really tell us? Oh, not a damn thing.” But congratulations.

0:26:37.0 FW: That’s also something that I’ve always observed with some level of amusement. Because people always like to immunize whatever they are trying to do, and data and like throwing the data scientist on something, and spending a lot on Markov chains and heat maps, what not you can tolerate that problem, just to get their point across and just to be able to run with whatever, and like whatever conclusion they may want. They want to get their way and they are willing to use data for that, and they are going to use statistical and ask us to say, “Well, the statistics says we should do this thing,” which statistics never do, but actually that’s what they are going to say. [laughter]

0:27:26.5 FW: And that’s a point where I’m always like… I’ve had that point with a previous employer where marketing agency was actually doing A/B test on a campaign, and they were really trying to have the result of the test, which was really cool, like 30% conversion uplift, like a really good campaign test, and something they really did well. But when they presented their results, they were like, “Hey, I quoted the statistical analysis and all that, and I could showing like four point something percent of significance and all that, so it’s a really good result and highly significant and all that stuff. And I was like, I was sitting in the audience because the marketer just took me with him to that meeting and I was like, “Did he know before you run that test which version you assume was going to win?” And they were like, “No, it was completely random, we didn’t know which one was going to win.” And then I was like, “Cool, so you did a two-sided test, right? So that significance interval that you just mentioned is not going to apply to that because you need to cut that in half.”

0:28:35.0 FW: And you’ve seen the end of this two sitting on the marketing side like flushing and like, “Oh shit, I’ve been caught.” And I was then saying, “Let’s put that all aside. Is the result significant in terms of business value?” And it was. So, I was then saying why are we even discussing this point, if it’s meaningful to the business and significant to the business, why do we need to put all those immunization layers on top of that just to not be questioned, and just to be believed? And like, yeah, that’s something I’ve observed where I was like, “Cool, but if people don’t trust you just based on your statement and just on you saying ‘This is what we should do,’ then maybe there’s a different problem behind it at the core.’”

0:29:24.7 MH: So you just illustrated my deep, deep angst to it, anytime I put anything, anything remotely statistical that it’s… There’s gonna be…

0:29:35.7 TW: Sorry.

0:29:36.5 MH: I’m a hundred percent sure there is somebody smarter in the room who is gonna call me on it and I will absolutely freeze up, and…

0:29:43.2 FW: Dude, I just delivered that one example you needed to never do that again. [laughter] Now you’re never going to do that.

0:29:49.7 TW: And you’re forgetting that that was me.

0:29:50.7 MH: That’s right.

0:29:52.0 TW: You’re forgetting that that was me that you just completely annihilated.

0:29:57.9 MH: No, I like this framing, and I think there is definitely an impulse to make what we do very complex sometimes, because it’s this mistaken assumption that will make us seem more valuable or make us seem more necessary. And that is interesting ’cause I’m big on simplification, mostly because I’m not super smart. So, I like to make things as simple as possible. And I’ve been in situations where people have been like, “Yeah, all these layers,” and you ask that one question, it’s basically like, “But is it like this?” And they’re like, “Oh, yeah,” and it’s like, “Oh, then why did you build this house around this one little thing? Just say the thing.”

0:30:44.0 TW: But the flip side is, and I think this goes back down that slippery slope of, “You know what, we just took these two metrics and we plotted them over time, and look, they’re moving in tandem.” And we say, “Well, you know what, they look like they’re kind of moving in tandem, but let’s put them on a scatter plot and see if they really do.” And oh, look at the R-squared, then we say, “Wow, this has like an R-squared of 0.92. People understand that, we can show the scatter plot and we show this tight relationship, and there is a level of complexity.

0:31:20.0 TW: I will fall back to non-stationary data and first differences, which I’m rattling off stuff that sounds complex. It does take a minute to explain it, it’s not that hard to get some intuition about how that is a super dangerous thing, and watching. And so, I think as an analyst at times we wanna say, “Wait a minute, something feels like maybe I need to dig a little deeper,” and I dig a little deeper and I’m like, “Well, maybe I should do this other kind of analysis. Oh, that completely blows it up. Thank God I didn’t actually present that to the client.” Or I did a deeper analysis, it actually further holds. That’s the challenge, ’cause now you say, “Well, I want to. In case there’s someone more sophisticated in the audience, I want to kind of make sure they know that I dug a little bit deeper, but ah crap, now the whole audience has to hear me talk about first differences or P-values, or whatever it is, in finding the line, the right line of complexity to balance between… Yeah, we did due diligence, we dug a little bit deeper, but the digging deeper introduces complexity.

0:32:33.0 TW: The line where that’s further evidence of the amount of uncertainty that’s been reduced, versus now you’re just making people’s eyes glaze over, versus now you’re into a place where somebody may start debating whether you should have taken a Bayesian approach, and that was Frequentist, and now you’ve got somebody else just trying to show how big their brain is. And it’s tough, it is kind of a minefield you’re always walking through. If you make it too simple, it may be wrong, if you make it more complex to present it simply, you may have it be misinterpreted or may not have credibility, because somebody would say, “Well, you just did the most simplistic analysis.” Like, “No, I did a deeper one, I just didn’t show it to you.” “This is terrible. I give up, I quit. I’m gonna go become a flight attendant.”

0:33:27.5 FW: Partially, I would actually like to be part of that room where that deep of a discussion can happen, because I would be looking back at my career like, “Where would that actually have happened?” But that’s… Coming back to a few episodes ago, where I was actually name dropped with my tweet in terms of like if you put people in a position where in some sort of meeting they’re going to have that question, potentially the preparation of that meeting wasn’t all that good. Because if that level of discussion happens in a meeting, either it’s a work in progress meeting and you invite people for their opinions and you discuss openly, or it’s a…

0:34:12.5 TW: Well, your example was all said and then they’re like, “Hey, Frederik, why don’t you come along with me?” And the agency walked in and was like, “Oh, crap, we prep totally and then they brought in the wringer.”

0:34:21.6 FW: With…

0:34:22.5 TW: Oh, sorry.

0:34:23.8 FW: If that agency wouldn’t have shown me that results, the first time in that meeting, I would happily have given them that feedback earlier, but they didn’t. So that’s when, and that’s when heading to because, if that type of discussion is happening for the first time in a meeting, where you’re actually like, “What’s the purpose of that meeting then? Are you trying to get a decision from someone? Are you trying to just get buy in for what you’ve already done? Do you want to get some more budget? Or what’s the goal of that meeting?” And then, if that discussion derailed up to a point where you discuss statistical methods and all that, there must be something else going on. That’s always the psychological side of me saying like, “What’s behind it? What’s actually at play here? Because, surely it isn’t that you… That we now want to discuss the statistical approach that we’re taking, there must be something.” And that’s always why I’m trying to get to, to just say, “Hey, what are we discussing right now?” What’s the matter here? And that’s not analytical, that’s not statistics, or data science or anything.” So yeah, that’s a…

0:35:38.8 TW: Yeah. It’s organizational dynamics, right?

0:35:42.5 FW: Indeed. It’s just psychology at the end of the day. And I’m…

0:35:45.6 TW: Damn it, Michael. Michael, you point another…

0:35:48.8 MH: Well, like I said, we’re trains in headed into the station, and it’s on time, just like… No. [chuckle] Well, I wanna surface another aspect of this, which I actually think will be interesting to you, Frederik and our listeners, which is part of the confounding aspect of this, is the fact that vendors in our space, and I think generally speaking, are incentivized to make their products or tools seem like they solve the problems that people are having automatically or magically, which is never usually the case. And what’s been your experience? Or could you talk to that from your perspective of sort of like, “Yeah, how do we layer that in?” Because basically, to solve a problem a marketer will be like, “I’d like to buy this tool.” And it’s like, “Okay, well, that sounds like a great tool but timeout, before we add that on, have we thought about doing this with what we have, which may be more than sufficient for the task?” But that is often not discussed or whatever and then the vendors are not really… It’s to their benefit to make us believe that the solution is automatic.

0:37:01.3 TW: This is unfair. I mean, you’re asking Frederik who said that he is an Adobe champion and Adobe has never…

0:37:06.9 MH: Well, no, I…

0:37:07.5 TW: Never, in the history of Adobe, never promised that their solution was going to…

0:37:12.6 MH: I wasn’t bringing us…

0:37:13.9 TW: They’ve never sold, they’ve certainly never bought nice steak dinners for customers.

0:37:16.8 MH: Hey Tim, I was asking Frederik’s opinion on this.

0:37:19.7 TW: Okay. [laughter] I’m just saying, it’s unfair.

0:37:22.5 MH: And if he gives a two… A by-the-book Adobe answer, we’ll both pounce on him, okay. But I don’t think he’s gonna. [laughter]

0:37:31.5 FW: There’s a solution to all of your problems and there’s this workspace like everyone knows, I don’t need to say it.

0:37:36.0 MH: No, I’m so glad you brought up customer journey analytics. No, I’m sorry.

0:37:39.9 TW: Analytics.

0:37:40.4 MH: Yeah, no. Go ahead.

0:37:41.9 FW: Have you heard about query service and experience platform? It’s awesome. No. I’ve seen it happen too. There’s so many, especially a few smaller analytics solutions out there on the market to focus a lot of their marketing on that, I’m hesitant to say, but gimmick. Like that one feature where they say…

0:38:03.8 MH: Yeah. Or the one problem that they solve. Yeah.

0:38:07.3 FW: Exactly. Like this one very complex problem where some marketer of the tool or some product manager can say, “Hey, this very specific problem that either everyone has or everyone thinks they should have, that is going to be super simple on surface level with that tool.” And then people are like, “Hey, we need to have this yet another solution on the website, and we’re going to funnel our data into yet another tool.” And then they usually start using that tool and they discover that, “Yeah, that feature is working. Can we customise it? Nope.”

0:38:45.5 MH: Yeah, exactly. [laughter]

0:38:46.4 FW: “Can we do anything else with it? Nope. Can we do our basic level marketing, All Pages report? Nope. Wait, so now we have two tools and two tools to believe. And now we’re going to ask questions in terms of like, ‘Hey, which tool is actually right? And we’re going to add some more complexities and GDPR in server side and whatnot to explain all those small differences.” And yeah, that’s a fun journey.

0:39:14.8 MH: Yeah, but it definitely is a confounding factor to being effective because you run into these situations and it’s like, Okay, well, we’ve gotta learn Google Analytics because we must… And we also need to run… We were running this and then half the time you’re spending answering questions of why these two don’t match each other, and it’s like those are the kinds of things analytics people get pulled into. And even when you are, at least what I found, even when you’re trying to be very systematic in your approach and careful in your approach, you’ll run into frontiers that your tools are not great at, and you find yourself being like, “Man, did we pick the wrong tool or is there a better tool for this.” So there’s layering that I think happens naturally too, as you progress through an analytics journey, like I have a client who we are working through kind of our third iteration of an ETL tool because they’re just… It’s progression as well as capability, and it’s sort of like, okay, well, there’s stuff I wish we would have known, but now we do, and so we’re making that thing happen, but the goal was to end up on one, right now we’re sort of spread across three, which is not the best.

0:40:32.4 FW: That’s just what I’m observing with all those, especially start-ups, who think, Hey, Netflix is using this very, very low level event-driven approach and they have all those custom net data pipelines through Kafka and they are throwing it in some form of visualization tool for that one use case and another visualization tool for that second use case, and Tim is going to roll in and say, “That tool can’t do 3D pie charts, I need another tool.” [laughter] And at the end of the day, you’re going to end up with so many tools just to solve that one more edge case, right?

0:41:07.3 FW: And you’re going to take focus on… Instead of looking for that, again I’m going to be biased now again, but instead of looking at that one solution that can do the majority of things very well and then the rest at least well enough or offer some way to do that, then they’re going to be focusing on those edge cases and say,” For that edge case, we need to get another tool.” And then you’re going to end up with 20 different visualization tools, and for retention use cases, you’re going to use probably Aid and you’re going to use Tableau for product use cases and the data never matches up and you actually… You create many silos within your company and you’re actually preventing the marketing team from collaborating with the product team because all the data is in different tools and never going to be able to share anything.

0:42:03.8 TW: But I guess that’s the having… I wish organizations could get better about keeping the Edge Case Solutions in a bucket. The problem is the Edge Case Solutions are usually trying to move up market or down market, which means they’re competing. If I could roll out… Adobe Analytics is a digital analytics platform, it doesn’t have the heat maps. It’s never really tried to and it would be a…

0:42:34.2 FW: Well, let me tell you about Activity Map! [laughter]

0:42:36.2 TW: Yeah. They would say it’s like… ’cause it’s not really the nature of how they’re collecting data, it’s just not gonna be great, so I’m like, it totally makes sense to say, supplement that, throw on, but use this strictly as a heat mapping tool. Now the problem is, if you supplement it with quantum metric, well, they’re out there saying, draw the Venn diagram, and they’ll say, well, okay, there’s a pretty decent chunk of overlap and who owns what. And I almost want to say,” Let’s get the most dinky little point solution to give you just a heat map, so you’re less likely to be tempted to start comparing.” But the other piece is that the complexity or the data governance programs will come in and say,” We’re gonna pick our system of record.” And it’s like, okay, that sounds great, you have your system of record for orders. And if you’re counting total orders, this is the system of record. The problem is, when you wanna slice orders by original source, that’s not in the system of record, so now you’ve gotta go to another system, and when you slice it, it’s not gonna add up.

0:43:42.7 TW: And they were back down the world of complexity. I do think though, back to one of your original points, Frederick, if the person who’s trying to use it cares about it and you get the five or 10 minutes to say, “Understand this disconnect.” Hopefully they are willing enough to not see it as, well, you’re just being mean and difficult and you’re not… You’re just not smart enough or you’re not willing enough to make this work. And they recognize that, okay, to slice it, the sums aren’t gonna add up, but they’re gonna be good enough and I’m gonna move on and now let me use that slicing. But it’s just there’s this long continuum where it starts with simple, and then step two is, Oh crap, it’s more complicated than we wish it would. You know what we need to have, we need to have some system to manage the definitions of all of our data, that would be simple. Then you wind up with, all of a sudden the modern data stack is complicated because it’s all adding layers to try to get to simplicity and clarity. And that just logically is really tough to pull off. Let’s add more things to make it simpler.

0:44:56.3 MH: That one never works.

0:44:58.2 TW: It never works.

0:45:00.3 MH: Never works.

0:45:01.1 FW: I would really like to have Moe in this discussion though because I was thinking, how are you doing this? How are you managing all that? And how do you allow your marketing and your product folks to collaborate? Because that’s also a point where I’m so much where you… In businesses and especially large businesses, just comes from collaboration and people talking to each other. There’s this increase in efficiency and effectiveness through doing deeper analysis of your purely marketing problems or your purely product-driven problems and all that. But so much more efficiencies in business could come from just, “Hey, let’s throw all our marketing and product folks into one room, and because they know how to use their analytics tools, they can just show each other what they do.”

0:45:51.1 FW: And because your solution works well enough, they can just try to adapt their things a little bit. So the marketing people can also take product metrics into account and vice versa, and especially with that modern data stack in all those fragmented solutions, I feel that’s even discouraged. I think it would be really hard to allow that collaboration to happen and a decent mind… A simple mind, a lot of business where can just come from that simple discussion in terms of, “Hey, I’m running my marketing campaign, maybe I should add some product metrics to that. Let me talk to the product folk and get their opinion on what they are using”. And that’s always what I’m trying to allow my business to do. And the discussion that I’m trying to motivate them to have, because they don’t just need to talk to me.

0:46:43.6 MH: Would it shock you to believe that that same thing is true even in small organizations?

0:46:48.3 FW: What?

0:46:48.6 MH: Like communication and discussion across actually improves efficiency and things like that. [laughter]

0:46:53.5 FW: No way. [chuckle]

0:46:53.8 MH: Yeah, I found it to be true, yeah, 100%. Part of this is also, to me, sounds like decision quality is very important to this whole thing. You know what I mean? There’s some centralized or top-level decision quality that either makes this structure that we’re kind of envisioning and discussing work well, like choosing the right setups, choosing the right tools, choosing the right way to approach the tools, those are all things that I think are a piece of this recipe. I don’t know. Reactions?

0:47:31.7 TW: The decision on how you approach… The actual decisions you’re making in the business or the decision of what your text… Your data stack looks like?

0:47:39.0 MH: Decisions around your stack, decisions around your org structure, decisions around how you integrate, those all seem to be components of how this will actually work out. There’s many organizations where the data organization is just simply a red-headed stepchild, to use an Americanism, to the marketing org. And they just do whatever they… They have no…

0:48:05.3 TW: Agency?

0:48:07.4 MH: Agency, right, I was thinking the Seinfeld term, they have no hand, but that’s not really represented well across all of our listeners. [chuckle] But anyways, go watch Seinfeld, it’s hilarious. Okay.

0:48:18.0 TW: Or even your co-host, but we’ll look that up later.

0:48:21.8 MH: Yeah, you look it up later. So yeah, that’s right, Tim and pop culture references. Yeah, no agency in the process, they just sort of stuck with whatever they get told. And honestly, it’s like… Those are the people I would just tell, “Go find a different job, go work on a better team”. But it is true, those things matter a lot to, I think, this discussion.

0:48:40.3 TW: I’ve been… Frederik, you brought up… Earlier, you were mentioning product analytics and marketing analytics and I definitely… That’s happening in kind of the digital analytics world now where I feel like it’s a case where the vendors are drawing this hard line, which is now teaching businesses to say like, “We’re doing educational presentations about product analytics versus marketing analytics”. And it winds up like creeping into this like, “Well, which tool should I buy?” And I’m like, “Something feels really wrong that we’ve lost the thread of ‘What’s our current tech stack?’

0:49:22.9 TW: ‘What are the main decisions and questions that we’re trying to address?” And yet we’ve got… Clearly have the industry, the product vendors are actually out there successfully framing the discussion. There’s a consultancy, like we’re getting sucked into it. And I’m like, “Wait, wait, wait, hold on, time out, what decision… Are we making a decision between two products? Are we actually really clear on what it is we’re trying to do and what the best path forward to do that is?” But I will say no more without offending anyone who gets this deep into the episode and recognizes themselves.

0:50:08.3 FW: Yeah. That’s definitely a challenge, because circling back to the topic, ’cause we’ve been so much in love with our complexities, now we’ve actually allowed vendors to go to a point where they say, “We have the solution to your… All your nerdy analysts telling you how complex things are. We are going to make it simple again for you. And we are going to give you the thing that they like spent weeks and months on building their retention analysis in SQL, or building their models for all that stuff. We’re going to give you that with one click because we are know going to make it simpler for you. That definitely can lead to a more siloed situation in your business. And back to your question, Michael, also, what we need to put in place…

0:51:00.1 FW: And I think what people tend to forget is that we are talking about tools, and tools have a purpose. They are no like reason within themselves and they are no… [chuckle] They don’t have any values without serving a purpose. And for a business, I think it’s very important to, before you even go into what you should do or how you should set up your stack or whatever, to define some values, how you want to run your business to say, “We want to actually encourage and engage in department spending discussions. We want to have the product people talk to the marketing people and collaborate on their data.” And then whatever tool you choose needs to be compatible with that approach and with that mindset, because otherwise, if you’re going to stay in your silo, then every silo is going to have their one or three tools that they’re going to use, and you’re going to end up in this situation where your vendors are pitching against each other and the marketing people want to go with the one tool they like and the product people want to have another tool. And that’s bad. That’s a, I think, a result of not setting the stage correctly and not saying, “Hey, [chuckle] we want to actually make the business better, not just within our departments, but also across them”. And yeah, we need to have processes and tools and who are close in mindsets that actually allows for that.

0:52:27.2 MH: Sounds like a place I’d wanna work. All right. [laughter] We have got to start to wrap up. This has been very interesting and a very cool discussion. And I like the way this… It feels a little different from other shows, ’cause I think it’s just sort of like a… This is like a true lobby bar… So thank you, Frederik, thank you so much. This is fun. All right. One thing we like to do is go around the horn and share a last call, something we think might be of interest to our listeners. Frederik, you’re our guest, would you like to share our last call?

0:53:00.2 FW: Sure. So we’ve talked about collaboration and crossing the boundaries of what we do quite a bit today. Like one thing that I always like to just engulf myself into is all things, product management, because I think there’s a lot to learn also for analytics people in terms of how you actually want to deliver business. How do you want to find that one part of what you do that actually drives business value? And for that I want to give a little shout out, not that he needs it, but like to John Cutler, both on Twitter and on his Substack was like my… Like I just love the guy and whatever he writes and whatever he tweets, because it’s so much free wisdom and it’s so cool and super awesome.

0:53:44.9 FW: I think everyone should follow him because it’s good analytics advice, good product advice, good life advice, it’s so cool. So that’s my first shout out. And the second thing is a little bit more to the psychology side again, because what I’ve learned also across the years in the industry is there’s a lot of people in our industry that suffer from things or like not suffer, but at least qualify for things like ADD or ADHD, even. So those people have a tendency to also try out a lot of different productivity tools and tools that help them to structure their day and all that. And I found this neat YouTube video that I think we’re going to link, that actually tries to… Like that put those people like front and center of those tools and actually shows some approaches that might help them and I thought, “Huh, that that’s pretty neat.” And that’s something to try because yeah, people always try to, or like to try new things. So why not try this one? Which I found really cool, and that video to avoid toxic productivity, which I find a very great… [chuckle] Like that’s also going to be my last shout.

0:54:55.4 TW: That’s awesome.

0:54:57.1 MH: Nice, thank you. All right, Tim, what about you? What’s your last call?

0:55:00.9 TW: I am gonna go with the interactive data visualization from engaging data, which I think is a gentleman named Chris Webb, but it’s one of those sites where it takes a little digging to figure out who’s actually behind it, but he did a… Interesting I think he’s in the UK, but this is a splitting the US by population where you basically pick like a pattern north to south or most to least, or concentric rings, and you pick how many ways you wanna subdivide the country, and just based on county population. So it’s actually, I thought, well, this is gonna be this kind of clever idea, but I’m not gonna actually try to think about it too much. And then I found myself doing things like subdividing the US from high to low population density into two divisions and realizing just what… How things vary in the US. So it was a very cool visualization worth five minutes of poking around on engaging justdata.com. And that’s my last call. What’s yours, Michael?

0:56:06.3 MH: Well, I’m glad you asked. So I actually found a blog post recently that I thought was just useful. A lot of companies advertise with Facebook and a lot of companies work with Facebook to send data from their website and things like that to Facebook, but don’t really necessarily understand how those things match up. So this blog post by Elad Levy kind of walks through how the event match quality and the structure of how you… How Facebook’s data is constructed and I found it kind of interesting just because it’s one of those areas where there’s a lot of touch points and a lot of people and it’s not always something that people think all the way through. So anyways, it’s good. It’s a little bit technical, but I think it’s a pretty good article. It just sort of walks through how they’re doing matching and both deterministic and probabilistic kinds of things. So a good read if you are interested in, A, privacy, and B, your company’s advertising on Facebook. These are all things that should matter to you.

0:57:11.9 MH: All right. Well, this has been a lot of fun and it’s been… We also miss Moe as well, so she will be we back with us next time, hopefully. And Frederik, thank you so much. It’s been such a pleasure to have you on the show, I would say, finally, because I think there’s just sort of been like… For years, I’m pretty sure Tim and I have been reading your blog posts and things like that and being like, “Wow, this guy is really something.” And then it’s been… You have not diminished the legend by coming on the show at all, and in fact, maybe increased it a little. So thank you so much for coming.

0:57:50.9 FW: Happy to hear. It was a pleasure. Thanks for having me.

0:57:54.8 MH: And as always, we’d love to hear from our listeners. So please feel free to reach out to us. The best way to do that is the Measure Slack group, it’s a great community, and you would be well served. You find all of us there as well as on our Twitter and our LinkedIn. So we’d love to hear from you. We ask questions and things like that. And Frederik, you’re also on Twitter as well. Can you share your Twitter handle with people?

0:58:21.2 FW: Yeah, it’s very complex, it’s my name, like first name, last name, that’s it, FrederikWerner. One word.

0:58:26.8 MH: Well, that’s pretty organized of you. Well done. [laughter]

0:58:29.4 FW: I’m a very, very creative guy, yeah.

0:58:32.2 MH: But well worth a follow on Twitter. I think I’ve followed you there for years, I think for many years. And obviously no show would be complete without a shout out to our excellent producer, Josh Crowhurst, thank you, Josh, for everything you do to make the podcast happen. And I know I speak for both of my co-hosts, even though Moe’s not here and Tim, but I say no matter the complexity, just remember, keep analyzing.

[music]

0:59:05.0 Announcer: Thanks for listening. Let’s keep the conversation going with your comments, suggestions and questions on Twitter at @analyticshour, on the web @analyticshour.io, our LinkedIn group, and the Measure Chat Slack group. Music for the podcast by Josh Crowhurst.

[music]

0:59:23.1 Charles Barkley: So smart guys want to fit in. So they made up a term called analytics. Analytics don’t work.

[music]

0:59:29.7 Tom Hammerschmidt: Analytics. Oh my God, what the fuck does that even mean?

[music]

0:59:36.3 MH: Off the edge. So, I’ve been thinking about this intro actually for quite a while actually, Tim, mostly because I was like, how do we make this not one where we end up in a big argument, both on the show and with the rest of the industry? ‘Cause this one is definitely gotta have some hot stuff in it.

[music]

0:59:57.3 FW: I’m sorry for that.

1:00:00.7 MH: No, that’s okay. It’s why we do what we do. What are they gonna tell? They’re gonna tell Tim Wilson that he’s wrong? I don’t think so.

[music]

1:00:09.8 MH: Rock flag and miss you, Moe.

Podcast: Download | Embed

Subscribe: RSS