[music]

0:00:05.7 Announcer: Welcome to the Analytics Power Hour. Analytics topics covered, conversationally and sometimes with explicit language. Here are your hosts, Moe, Michael and Tim.

[music]

0:00:22.5 Michael Helbling: Hi everyone, it’s the Analytics Power Hour, and this is episode 182. This is the penultimate episode for 2021, and what a year it’s been for the podcast, and we’ll do a deeper review next episode. This one has something pretty special about it, but first, let’s do a very proper introduction of our co-hosts. Tim Wilson, Senior Director of Analytics at Search Discovery…

0:00:48.8 Tim Wilson: No, I’m actually a senior data scientist now.

0:00:51.9 MH: Oh, you are, or you changed your job title… We’ll get into that in a minute.

0:00:56.0 TW: Not at all.

0:00:56.8 MH: We’ll get into that in just a minute.

0:00:58.8 TW: I am not at all.

0:01:00.6 MH: And also the quintessential analyst.

0:01:03.3 TW: Definitely not that. I am as much that as I am a senior data scientist.

0:01:06.7 MH: You can’t take it back now there are thousands of people carrying a sticker with your picture saying that all across their laptops across the world, that we sent to them, that I had made. Moe Kiss, you are the marketing and engagement data lead at Canva. Welcome.

0:01:25.6 Moe Kiss: Thanks.

0:01:27.3 MH: Yeah. See? No ambiguity about your title. That’s great, see? And I am Michael Helbling, I’m the managing partner of Stacked Analytics. So our first show this year when we came back from hiatus was a discussion about many of the articles and content that we’ve consumed over the years by Cassie Kozyrkov and… Well, to put a nice end to this year, we’re gonna discuss more of her work again, but this time with Cassie herself. It’s a bit complex to compress her bio into podcast format, but here’s what you need to know, she has multiple degrees in Economics, Statistics and neuroscience, she’s the chief decision scientist at Google, is one of the first to spearhead the brand new field of decision intelligence, she’s spoken at innumerable conferences, including the Web Summit in Lisbon, which is the biggest technology conference in the world. LinkedIn has recognised her as the top voice in many… The last three years, I think, in data science and analytics, but most importantly, today, she is our guest. Welcome to the show, Cassie.

0:02:34.7 Cassie Kozyrkov: It is a great pleasure to be here, and I don’t know if you’re telling the future, but so far it’s two years running. It would be pretty cool if it’s three years, but… Yes.

0:02:45.6 MH: We’re almost there, 2022. Here we come. LinkedIn, do the right thing. Alright, let’s start with a question that I think is probably front of mind for most of our listeners, which is as I dug in and did some research, and I found this really old Google blog post about when you started out, and in that you talked about how one of your past times was playing MOBAs or Multiplayer Online Battle Arena games, which is something I have a huge fun playing, and I was curious, which MOBA did you like? And what role did you play?

0:03:18.6 CK: I do play, I play League of Legends. I tend to play ADC.

0:03:24.8 MH: Perfect. Okay, I play either top lane or support usually. So there you go. Very nice.

0:03:28.9 TW: So this is the first time that Michael has actually been more prepared for a discussion than Moe or I have.

0:03:34.1 MK: I know. I’m totally shocked.

0:03:36.2 TW: Just Googled MOBA. Okay, realized that’s general class. Okay, good.

0:03:41.4 CK: No, I’m not good at it. I’m not good at it, but I like it, but my reaction speed is like a sloth that’s been taking drugs, so that’s okay.

0:03:50.5 MH: Same. Same. That’s why I play ones that don’t require fast action, I don’t know how you’d handle ADC, that’s like… The reaction times are stupid, anyway, we’re not gonna…

0:04:00.7 CK: I just click… I just click. I don’t worry about the… [laughter]

0:04:05.3 MH: Sure. Anyway, in any case, there’s actually something that I think maybe we could get you to tell Tim what you think of it, ’cause I think there were some reactions to our last episode that you had, and I think that’s a great place to start this conversation. So why don’t we kick it off with just a little bit of that?

0:04:22.5 CK: There was so much aggression in that other episode. Wow. I wanted to tell Moe, I don’t know if I want this guy to be part of this, I wanna talk to her, she seems fun, but what the hell?

0:04:34.4 TW: I was gonna ask if… I just wanna be… I wanna be an unpaid intern who is just your executive assistant, I wanna know how I can get that job title, that’s the one I want is, Cassie’s unpaid intern, executive assistant.

0:04:48.1 CK: You can give yourself that job title, you don’t have to do anything and score. I just won’t write your references, but nothing actually stops you, just like nothing stops people from calling themselves statisticians when they don’t know their hypothesis from their elbow, so… Go right ahead.

0:05:06.0 MH: Tim, what were you saying?

0:05:07.7 MK: Well, I was gonna just add. Okay, so I told the team at Canva that Cassie was coming on the show. And normally the team are like, Yeah, cool, that’s nice. You have a podcast, whatever, eye roll. And this time, the amount of reactions to Cassie being on the show was out of control, everyone was losing their mind, they’re like, I’ve literally watched everything she’s ever done on YouTube, and my boss kinda lost her shit as well, which was kinda nice.

0:05:31.9 TW: That’s not possible, it is not possible to watch everything.

0:05:34.9 MK: Anyway… But the funniest thing was that one of our machine learning engineers, I was like, Well, if you can ask Cassie anything, this is the question he had, which was, Why decision scientist? And should the industry adopt this? And I’m like, “Really? Of all the things you can ask Cassie, that’s what you wanna ask?”

0:05:54.0 CK: You know what, I’ll take that one, I’ll take that one because if we’re gonna clear the air… Tim, I’m coming for you soon, but… Let’s clear some other air first. Okay, to anyone out there who has any question related to, “How can I copy your career, Cassie? Is it by studying what you studied, is it by having the same title? Whatever it is.” I have some unfortunate news for you, the fundamental thing that is the common thread in my career is that what I wanted to do is be useful, and I didn’t care what it was called. And so what was really useful at the time that I was doing the things I was doing and I was studying the things I was studying is not going to be the same as what is useful as you, whoever you are listening out there is rising in your career. So if you want me as a role model…

0:06:50.7 CK: I mean, heaven help you, but if that’s what you’re going for, then what you’ll do is you will look around and you will ask, What is the best way that I can make myself useful and am I worried what the job title is? No, because what I’m gonna believe is that I’m gonna be useful and excellent, and at the end, there will be a career out of it, and if I’m useful enough and excellent enough, maybe I’ll get to name whatever it is and you can be Chief flying squirrel overlord of whatever thing, and you get to make that up, which is great. So if there’s something, be useful. Now, why Decision Science? Because what I’ve believed my… Well, let me start. Someone asked me recently, and why did you move from data science? From having a data science title to a decision science title? And why did you move from data science to decision science?

0:07:39.4 CK: Those are two very different questions. They’re not the same question. I never moved from data science to decision science, that… Well, I never moved during my professional career. You know when I did that move? When I was a teenager, because when I was really little, I liked playing with data ’cause it was pretty and ’cause my brain is weird, but as I was growing up I thought,” Well, I wanna do something important, data are pretty, but it’s our decisions that are important. If a data point falls in a forest, or an opinion falls in a forest, and no one’s there to see it, it doesn’t turn into any action, it may as well never have happened.”

0:08:12.8 CK: So everything that we do, all the data we analyze, all the information we look at, all the opinions we hold, they are only important in the way in which they lead to actions, and so you may as well just analyze everything from how it’s going to contribute to your actions and your decisions, your opinion doesn’t matter, except in the way that it tends you to act. So start there. So that was part of growing up for me, moving from thinking only about data to thinking about decision-making, but of course you can’t abandon data if you’re gonna be serious about decision-making, ’cause what’s decision-making, it’s turning information into better actions, hopefully better actions and so if you care about that then you gotta care about information, that’s data.

0:08:53.1 CK: You gotta keep up with the tools and technologies that allow you to do the best thing that you can with the information available to you, so if you’re not gonna learn those tools how can you really say that you’re a decision maker mastering their craft? So it’s gotta be both. It always was both. Now, why the title? Because there are a lot of interest in the data stuff. Great. I love it. There’s not enough interest in the decision stuff, someone’s gotta champion that, we need a decision perspective, more of a decision perspective on data, and that is why.

0:09:27.6 TW: So I have two… So one on the… I feel like I need to… Well, I’m gonna try to not just get defensive, I won’t get defensive. I will get defensive.

0:09:36.3 CK: Get defensive Tim, go on.

0:09:38.4 TW: I would be happy to… If I get to add to my CV that I was eviscerated by Cassie Kozyrkov, I’ll take it, that might be my career pinnacle, but it’s funny ’cause if you go back in the… Way back into the ’50s and ’60s, or my understanding, not my personal recollection was that we started with decision support systems, DSS was what then kind of evolved into BI and became a very… It was a technology kind of an IT-centric thing, and I don’t even know why… That’s a very… Decades outdated term. So it almost does seem like it kind of started in the right, this is using technology and data to support decisions, and I don’t know why it drifted away from it, I definitely feel like is there is a pull to believe that just if we collect enough data, then decisions happen or insights emerge, and it is attractive, there are lots of people who want to just learn how to plug in the mechanics and the wires.

0:10:45.6 CK: This is not at all what we’re gonna have an evisceration over so lets just… In case people are wondering what’s coming, it’s something else, but first off, decision support, that term, it both thrills me and it rubs me the wrong way, because on the one hand, yes, what you truly want is a support for a decision process, so you want to be able to handle the information that is required to enable the decision maker to make a decision, fantastic. That kind of decision support? Yes, please. [chuckle] Supporting a decision that you’ve made already and then pretending that the information is where it comes from, that kind of decision support, I am not a fan of at all. Except if when I like some theater… ‘Cause that is theater… It’s quite unfortunate when large things are at stake and people get the two confused.

0:11:39.2 TW: That’s decision rationalization.

0:11:41.2 MK: Yeah. But how do you coach your stakeholders? If they’re doing that… I feel like I spend my career trying to coach stakeholders to make better decisions or to not just go down the path, and I’m still at this place of like… Are there some people that you just kind of wash your hands of and give up and be like, No, there’s nothing I can do here. I have done everything.

0:12:06.4 CK: Well, okay, so this is a complicated one. When I’m in my best mood, then I say I have hope for all humanity, everyone can become a little bit that better… When I’m in my worst moods and I think, “Oh my goodness, how did anyone put this person into any kind of position of adult responsibility?” This is terrifying because some things need grown-ups, yes, and some people are not… I don’t know if we’re gonna cut that or keep that, whatever you decide, but… My point here is that, two things, one, society does not treat decision making as a skill, it is, but it’s also an art and a science and it takes some talent.

0:12:56.0 CK: So if you fundamentally have zero talent for it, I hope again, in my sunniest of moods, that we can take you from zero to a little bit better than zero, but can we take you to the levels of virtuosity? It would be really cool if we ever had training programs that could do that. Right now we certainly don’t. So nurturing talent is what this must be about, at least now. That said, society tends to treat decision making as, I don’t know, it’s not even a skill, why should we train it? We’re doing it all the time, aren’t we all good at it? No, unfortunately we’re not. It is an art, of course, it’s also a science, it’s also a skill that needs to be trained, needs to be developed, and we should try to get ourselves better at it, if we want to have better, more effective lives.

0:13:49.9 CK: As fans of decision analysis and those kinds of disciplines like to say, your whole life, everything about how your life turns out boils down to two things, the quality of your decisions and luck. You only have control over one of them, so improve that one to the extent that you can. So that’s what they’re saying there, what I’m saying is, if you’re gonna start playing with tools like AI systems and other kinds of massive data driven automations, what you’re doing is you are extending and enlarging that single individual or small group of individuals who are responsible for that system for saying what it meant for it to work, what it was for, how do we know it was working, what is good enough, what problems is it solving, how should it work? Those people, it is their wishes, their decisions that this system will echo forever, and when it is a huge system, it’s taking this one person or a few people, what used to be small entities who would have limited influence on their peers, and now you’re giving them global influence on everybody.

0:15:00.4 CK: And so as you enlarge yourself with technology, you become bigger, it becomes easier to step on the people around you, that means if you wanna be a good person, if you care at all about this world, you should understand that with… What is that corny thing? With great power comes great responsibility. And you have the responsibility to make yourself a better decision maker, to make yourself the adult that is worthy of that kind of enlargement. Technology is a lever and the people behind it, those are the people that it’s enlarging. So this is a really important skill to develop and society doesn’t see it as a skill.

[music]

0:15:39.0 MH: Alright. Let’s step aside for a brief word about our sponsor, ObservePoint, a platform that helps keep your data clean, complete and reliable, so you can more confidently do the sorts of things we’ve been discussing during this episode.

[music]

0:15:55.8 MK: That’s where it helps. If you feed garbage into the machine, what kind of decisions are you gonna make? ObservePoint audits the tagging and data collection across your entire website, finding missing tags and errors and alerting you immediately if something goes wrong.

0:16:09.6 TW: And beyond simply checking individual pages for proper data collection, ObservePoint can be configured to check the specific functionality of your most important pages and user paths. The platform keeps track of the auto results over time, so you can keep an eye on historical trends to make sure your organization can analyze the data with confidence. ObservePoint also now has the ability to check for privacy compliance on the site by identifying all of the tech that is collecting data to ensure compliance to digital standards and government regulations for customer data.

0:16:39.8 MH: And that leaves you looking cool. So if you, dear listener, want to learn more about ObservePoint’s data governance capabilities, go over and request a demo at ObservePoint.com/analyticspowerhour. Now, let’s get back to the show.

[music]

0:16:58.4 TW: Well and does it… Because you say it really eloquently, and I think I’ve heard Mac Ershof, who’s kind of friend of the show say it as well, and even Annie Duke talks about decision making under conditions of uncertainty and you’re trying to make better decisions… Kind of echoing I think where Moe was, where there is this idea that with enough data in a sophisticated enough machine, I get told the right decision as opposed to… Is that what you’re saying? When you’re saying making decisions, understanding that you’re making a decision under uncertainty, and that the goal was not to remove all uncertainty before you make a decision, but to… I don’t know, uncertainties like this lightbulb that went on for me two or three years ago when I was in my pursuit of trying to learn about data science, and to me, I feel like that’s a miss with… The business world thinks that enough data is gonna remove uncertainty.

0:17:54.3 CK: Well, so I would start a little earlier with an uncertainty. So where I often observe this really stark evidence of poor decision skills, very classic thing, is when someone runs into the spotlight saying, “We need to use data to decide between A or B.” And then you poke at them a little bit and you find out that they have never even asked themselves whether A and B is the question. They haven’t even thought whether there might be B, C, D, and all the other options out there. And is it even in this domain, or are there completely other things that you might want to spend your time optimizing, caring about, and making decisions about? How broadly have you thought? Super narrow-mindedness is one of the hallmarks of really weak decision makers, one of the things that you wanna teach as part of your basic decision skills is to think broadly, to think about as many options as possible and to do that quickly with less effort, that’s one of those core skills. As part of this narrow-minded focus on an A or B, and let’s just get enough data to make sure that we pick A or B correctly. What we forget is that the generation of those questions in the first place, there is no right selection of A or B.

0:19:19.7 TW: Okay. So I think what you’re saying, if it’s not even the right question, what good is certainty? Because you can never be sure you’ve actually got the right questions. So in some sense uncertainty is always there.

0:19:31.6 CK: Absolutely, yeah, because if you think even you’ve got a situation where your business is about to go under and will A or B save it? Hang on! Do you even want to save it? Is this even the job for you? Right? There’s always a broader way to think about it. There always might be a different answer that a different decision maker might prefer. And so what is the right question? And what is the right answer will vary from decision-maker to decision-maker. It will vary from society to society. So what we want is decision makers who are thoughtful. More thoughtfulness is always better. But will two different decision-makers frame their decision in the same way? Will they ask the right questions? Will they focus on the same thing?

0:20:13.7 CK: No, absolutely not. Which is why it’s so important to choose people who have both the skills and the intentions and the benevolence and who have the wisdom to focus on, I won’t say the right things, ’cause that will kind of invalidate my whole point, but the right things. [laughter] Who will focus attention into decent directions that you’d be glad to have them in charge. Now, are they doing it optimally? No, because again, it becomes deeply subjective. So some objectivity can be injected later. First, there’s always a subjective bit, and so you hope you’ve got grown-ups in the room doing that subjective bit. But even one society to another doesn’t agree on what is good in life. Socrates, right? [chuckle]

0:21:09.5 CK: We don’t agree on a lot of very important questions. And so from society to society that will echo down into smaller things like Should we profit maximize? Should we invest in tomorrow? Should we care more about today? There’re all kinds of stuff deeply subjective. So you want good people in the room. Now, coming around a little to Moe’s question and back to, are they redeemable if they are a garbage fire of decision skills. So and even we can kind of put it into this fight that Tim and I wanna have. So I’m gonna try to do all the threads. It’s gonna be incredible. My head will explode. Let me drink more coffee, but…

0:21:51.8 TW: We can leave off the fight that I’m gonna lose.

0:21:55.3 CK: Oh, no, no. We gotta… I’m getting there. Will they think I am a pain in their butt or not? That is Moe’s question. So I come here and I wanna be like an annoying wrench in the works just to stop them from doing anything stupid. Well, I’m not actually gonna do that because that is A, arrogance on my part, and if I am not the decision maker to whom this was delegated, I might be a usurper. That’s the whole scenario. If the king is mad, should we take over? That is a whole nother can of worms. But let’s say that the people in charge are the people in charge and they are responsible for the decision, and it is on us to help them out. Let’s say that that’s the scenario. If their decision isn’t important, then they may be making it sub-optimally, but any amount of pedantry on our part is wasting everybody’s time.

0:22:46.8 CK: We should just chill out, seriously. So if they’re doing something and there’s no consequences to it, whatever, come on, we got better things to do. Let’s sit on the beach with a cocktail. But if it actually is important, and it’s important to them as decision-makers, and they care. Well, I’m not going to say that the problem is that they’re doing it wrong. I’m going to say, “Huh, there’s a lot on the line for you, isn’t there? And you know you would like maybe not to lose a few million dollars on this. Is that right? Am I right? Now, you know, in scenario one, you lose a few million dollars. In scenario two, you don’t. Now how do you feel about that?” Right now, in that kind of discussion, they might be a bit more receptive if I’m coming at it, “I want to help you with something that’s actually important to you, and here is what it takes to get the thing that you claim is important to you.” Well, now we’re speaking the same language.

0:23:42.4 CK: Now, a lot of the time, what’s important is not what on the surface people say is important. Like they may say that what they want is a machine learning system that’s really good at classifying cat photographs or whatever it is, but what might actually be important to them is that what they want is to say that they have machine learning capabilities in the company. And no matter how shoddy this system is, it can be the worst thing ever, just as long as it’s kind of machine learning, so they can say they have one. That’s enough? Well, then, if I’m coming at this explaining to them how much their performance sucks, I’m speaking the wrong language. I’m not addressing what they actually care about. And if I rush in there like a prime nerd being like, “You are doing it wrong.”

0:24:29.1 CK: Well, first, what arrogance. Second, I need to really… I need to be so confident that I have understood them, the subjective parts of what they care about. If I’m not confident about that, how can we have this discussion? We can’t. There’s a lot of listening. And I’ll say a tip for anyone who’s interested in consulting. It may change later in your career, but early in your career, here’s what I suggest, you tell your client for the first half-hour, you’re gonna suggest nothing, maybe even for the first hour, for the first session, you are gonna make no suggestions, you’re just gonna listen and you’re just gonna ask questions. So they expect that you’re not gonna give them any advice. Now, that’s a good skill. Listen, make sure that you’ve heard what they care about, then go home and think about it. And make sure that your solution isn’t just pushing whatever technology you’re in love with right now, but it’s solving their fundamental problem.

0:25:19.7 CK: So, Moe, if these people have an important problem that they care about, and you can stop them from burning themselves, and you can describe this in the language that connects with it. Well, it seems like there’s a good match there, if you say, “Hey, you know there’s a cliff edge, and if you run off it, you’re gonna die. Maybe that it’s good to slow everybody down in that moment. But if I’m just like, “Hey, don’t run, I don’t like running, running is bad. I learned in a class that running isn’t the right way to do philosophy.” Well, no one’s gonna wanna work with you. Of course, of course. Now, coming back, here we go. Tim…

[laughter]

0:25:55.0 TW: Got it. Coffee kicked in.

0:26:00.5 CK: Right. And also back earlier to some notes that we said… Okay, there was a lot of angst in your other episode over stuff I’ve written because I said analysts are these and statisticians are that, and if you are the quintessential analyst as in that sticker, Tim, it’s some offensive stuff against analysts, so… Right, let me clear the air. I have said earlier that I actually don’t care about job titles, and I would be super disappointed if the reason that people are listening to me is that I have a cool sounding title. That would be really sad. So call me the head of MOBA games or whatever for playing badly. If I still have something useful to say, then listen. That that’s what I’m hoping we’ll do.

0:26:43.2 CK: So I don’t care, not for me, not particularly for other people. What I wanna know when I encounter someone is, what can you contribute? What are the skills, what are the tasks we can match you to? And so I know I play a little fast and loose in my blogs because I will talk about analytics and then I will shorthand it. Instead of saying people who have the skill of analytics, I will shorthand that as analysts. And you will think that I’m referring specifically to people who hold the job title of analyst. Job titles are a mayfly, economics is a monster, because as soon as a job title gets sexy, too many people who don’t have the skills run into it, and everyone who’s legit will go find a different job title. So it doesn’t stick around in any case. So honestly, I don’t care, and if someone picks up my blog post in 50 years time and they’re reading it and they’re like “Oh, this is not what analysts do,” great! Humanity has progressed. I am about it.

0:27:37.4 CK: However, the thing that keep defining to be analytics is a particular approach or set of skills, because my main concern is when we are dealing with something, what skill sets need to be represented in the room. Now again, also, I don’t care if they’re represented in one body. That’s more economics. If you have a large company and a large team, it may be more efficient to hire skill sets separate rather than look for the one perfect unicorn who does everything. If you are a start-up and you get one chip and one person, then you’ll say, “Oh, this person might have the title of Chief Analytics Officer for my start-up of three people, but I kind of hope they can also do statistics.”

0:28:16.7 CK: And it would be nice if they also have some decision-making skills, and so all that gets lumped in there. So when I put these hard line distinctions between things, there is no point at which I am saying, “An analyst should not do statistics, does not need to know statistics,” because that will depend on what that person’s actual job is, and there may be additional fun skills, like maybe they work for… I don’t know. We were talking about League of Legends. Maybe they work for League of Legends. And I don’t know if this is true today, but I believe a few years ago, every employee had to play the game, so you gotta have that skill too. Right, is that being an analyst? Well, in that job it is. So I wanna be careful about job titles. But the skills I’m gonna stick with, there’s a lot of confusion from not identifying what everyone is supposed to do and what it takes to get something done.

0:29:10.6 CK: And so that is why I put these distinctions up so that people who might not have so much experience or savvy can look at what I’ve written and they can ask themselves, is there somebody in the room who can do this for us? Have we assembled the right team? So that’s what I’m going for, and so now coming back to some of the other stuff where you’ve got a room full of really data savvy people, no one’s got decision skills, and I don’t mean decision responsibility, ’cause we’ve already seen… We can have children in adult responsibility roles, it doesn’t go well. But is there somebody with the skills to do this? And if there isn’t, let’s get those skills in there, otherwise the whole thing is a disaster, unless it’s not important, in which case, party time. But when it’s important, that’s when we have to worry. That’s when we have to ask, “Is there somebody here who can actually do it?”

0:29:56.8 CK: And society does have this huge problem of not recognizing that it’s a skill and using decision responsibility as a reward for other achievements like, “Hey, you’re really great at leadership or people management or back stabbing or whatever it is.” I’m kidding, I’m kidding. But also not. But anyway. And then, as a reward for all this, you tend to get a large organization of people because let’s see what a powerful gorilla you are, and then we can all be at a cocktail party and be like, “How big are the decisions that you influenced?” Let’s take a deep breath. Do what you’re good at. [chuckle]

0:30:33.6 TW: Well, so, one, just to defend myself, having talked about… I’ve never been focused on job titles. And I’ve had discussion with people of like, exactly the same thing. Do what’s useful and figure it out. It’s similar with the shorthand. I went through for a hot man and I said, “Oh, I’m gonna become a data scientist. Not ’cause I wanted the title,” because I was working in digital analytics tools in Excel, and I was like, “I think I wanna learn R and I understand if I learn R, I’ll understand statistics, and that feels like it’s growing a skill set that I should have,” and I went through a comical that has been public in a very small niche, and because you kind of did come back to the… So roles or skills…

0:31:17.4 TW: Let me ask if this is me starting to kinda grasp, because I think where I got a little confused and I think it’s getting clear, is that it is fine to say that five years ago, I could not run a linear regression. Now, I could take an entire set of data and run a linear aggression and say, “Oh, I can kind of interpret this,” but I ran it on my whole set of data. So this is really just me kind of exploring and trying to find some things that might make us think. That exact same linear regression, and this is maybe the last six months, and it’s largely to your credit… If I said I’m gonna take that same data set and I’m gonna split it up into test and training, and now I’m gonna run the regression on my training, and then I’m gonna test it… One, I’m still doing the same thing, it’s got more rigor and I’m avoiding some over-fitting, probably still kind of in the doing analysis.

0:32:19.4 TW: If I say I’m gonna split it up into training, and I would take my training and split it into training and validate, and I’m really gonna build something, ’cause I wanna push out something that is predictive into production… That feels like it’s kinda moving beyond analysis, but they’re all saying linear regression, and how far can I run with what I get out of those? Even understanding that there’s a distinction between those and what I’m heading off, what I’m avoiding, what’s faster, what’s easier, how far I can run with it, feels very, very useful to me and I was not there a year ago. Is that fair?

0:32:56.6 CK: Yeah, that is fair. And I wanna add to this that what you described is either a very happy thing or a very miserable thing, and what it [laughter] depends on is whether behind the scenes of all these there is a decision maker, a skilled decision maker. That maybe you, again, that maybe the person themselves, the analyst, the data scientist or it may be somebody else. But the person who has said, “Here is what needs to be done, and why. Not even what needs to be done. Here is the final vision. A thing we’re trying to achieve, and not the means by which we’re gonna do it.” Somebody who actually does not care in their heart of hearts whether it is a linear regression or not, because if our starting point is, let’s do a linear regression, it means that there is somebody earlier in the process who’s out to lunch.

0:33:50.7 CK: So it’s somebody else has to say, “This is what we want to achieve,” and then Tim, you go, “You know what? The thing that is gonna do this, is a linear aggression, and here’s how… ” And maybe you were the one and you said, “Oh, what we need to do, what we need to achieve here is such and such and here’s why it’s important,” and then I look through my tools and I go, “Aha, if I use a linear regression, then I can… ” So that’s the… It’s really beautiful.

0:34:20.0 TW: Yeah and I don’t think… I’ve never had anyone asking for a linear regression. I feel like even from working with data science, like learning the language of just in my own head trying to listen for what a stakeholder is doing and thinking, what is their dependent variable? I don’t wanna ask them, “What is your dependent variable?” But it’s a useful framing to figure out what is the thing that you ultimately care about, and a lot of times, I feel like the decision makers, even that’s not clear, it gives me the language to at not, what is your dependent variable? But until I can say, “Ah, this is the dependent variable they care about,” it gives me the licence to keep probing to try to understand what it is they’re trying to do, and that’s kind of a simplified version. But, yeah.

0:35:10.0 CK: And I also say that if you start with not how it’s going to be done, or which tools, which is linear regression or not, you’ll see that your one tool that might be… Let’s say you have a needle, just because you have a needle, it doesn’t mean you need to sew with it. You might also use it for a medical application. You might use it to pin a chart to the wall, who knows… Once you have this tool, if you knew it really well, you can see all the different ways that it could be adapted, and when it’s something as simple as a needle, any fool can grasp that there are a few things we can do with this needle. When it is something a little more complicated like a linear aggression, and you’ve only seen it used one way, and you’ve never really thought of… Stepped back and been like, what is this thing really? What are all its possibilities? Then you might miss opportunities to use it because it’s not classically associated with some kind of application.

0:36:11.8 CK: So the same linear regression, as you said, you can use it for analytics to help explore your space, because by understanding that when a linear regression does some kind of behavior, you can learn about correlation, which means that you can quickly, instead of reading your spreadsheet, which is not never what you wanna be doing, you can kind of see that this one is moving and this one is more positive, that one is more positive, that tool gives you that kind of exploration in summary, maybe instead you’re doing some statistical inference and you have some decision that’s based on if I have enough correlation to convince me in a new data set. Or, oh, I have a tool that can help me if I just understand and make some distributional assumptions and all that. It’s a very different way of using the tool by the way, you don’t care in the other setting.

0:36:56.5 CK: Then I can get at this thing that I’m trying to achieve, or maybe I’m trying to automate and just… Whenever this one is high, output me a high one over there. There, that seems like a machine learning-ish application. And when you’re doing end-to-end machine learning, well, then you’re gonna need machine learning with statistics and analytics also because you need an analytics approach to quickly figure out what seems to be working, where, what kind of patterns might I exploit, otherwise you’re groping around in the dark and your data set better be super simple, otherwise you’re gonna take forever. Analytics is what stops machine learning from taking forever. And then you think it works… Does it actually work? Hello, statistics. And all of those might be linear regressions, but used completely differently, and so starting with the kind of attitude of, “I’ve just graduated from college from a class and I am armed now with a linear regression and I will arrive at work and I will use it. Then you really hope that you’ve got a decision maker who is doing the thinking for you, because in that scenario, you’re not thinking. You’re just… You’re thinking, you wanna just apply the first thing you got.

0:38:05.0 TW: But I guess, making the case to analyst, and this is coming from right at about 20 years where I say I was a full-time analyst and I spent a big… I felt like I was… In hindsight, I was missing a lot, and literally just trying to learn some of the basics of statistics in the… Any time we start talking about confidence or you start talking about anything that’s an interval or uncertainties or working with time series data and some other things, using different diff and diff to try to make stuff more stationary, and I know those are all very, very tactical techniques, but kind of not having gone through formal training in it, I feel like every analyst, everybody who’s calling themselves an analyst, it behooves them to go through that to actually have a bit of a broader understanding and maybe it’s the curse of knowledge.

0:39:02.8 TW: There are, I know people who’ve been doing… Have been analysts for years, and they do a bar chart and they draw conclusions. I was at a conference a while back where the analytical malpractice was insane, and I was like, “This is an agency and this stuff is being told to people and it is so egregiously wrong.” And I was like, “Well, this is somebody who got a little bit of knowledge and didn’t understand it,” so I guess that’s… I feel like there’s this… All those techniques, just like learning to understand a linear aggression model helps you understand how data works, what it can and can’t.

0:39:40.5 CK: So let’s say something that is a little, a little gritty in this career. You have to watch carefully for that little bit of… There’s two kinds of privileges that allow you to have small skillsets. Privilege number one is you get to be paid for unimportant work. Amazing! I mean, I don’t want it, but there is kind of a relief psychologically maybe, because you’re not gonna set anything on fire by making a mistake. And so in that case, you don’t know any better, but whatever the world goes on, because you’re not touching anything important, it’s okay. That’s privilege number one. Privilege number two is you are part of a large and very organized team that has people with many different skill sets to help one another out. That is an incredible privilege as well.

0:40:33.7 CK: So that’s… You might be the best person at just pure exploration, pure exploratory data analysis, you do nothing else, but you… If given a bar chart would start concluding things. But that’s okay, nobody allows you to make the conclusion because they go “Mm-hmm. It’s their decision. Come over here!” And that stops any kind of problem in its tracks right there. So, those are two positions of… Let’s call it, career privilege. You can get away with stuff. And so, yes, I recognize my own privilege working for Google, with a lot of very smart and specialized people, I can only reach out to somebody who has skills that I don’t have. Chances are we’ve hired them. We even got doctors working at the company so… [chuckle]

0:41:19.3 CK: So that’s incredible now, of course, in other jobs, especially in start-up, where one person has to do more because no one is gonna be their safety net, well, then it’s very difficult to do anything that’s downstream, if everyone in the parts before doesn’t know how to do their job. So now you are forced unfortunately to start picking up these little skills to guarantee that what you make is useful, right? If you don’t have anyone with good decision skills pointing you in the right direction, then you have a very sad set of choices. Either you have to work really hard to become that person also. Probably no one’s even gonna recognize it because the problem was there, and they hired you who’s downstream before that upstream person, so clearly they don’t value it enough to fix it upfront, so when you go and become it and fix it yourself.

0:42:15.0 CK: No one’s gonna pat you on the head. They don’t even know they have that problem, but you know that if you don’t fix it, here’s your uncomfortable choice number two, everything you do will be useless. So, it’s ugly, it’s hard, however, it has its own rewards. Those kinds of scenarios where one person can be a little more flexible and can be more of a unicorn, and isn’t a cookie cutter version of everyone else… Those are thrilling careers too. So what we don’t recognize enough in data science generally, the data science is my umbrella term for the skill sets of analytics, statistics and machine learning. [chuckle] As defined by the number of decisions under uncertainty we know we wanna make before we begin analytics and non-statistics, a few, and machine learning, many. Okay. Anyway, we got that out the way.

0:43:05.6 CK: But… That umbrella thing is what we gotta realize about data science, the umbrella, is, it is downstream of other things. It is downstream of three very… Four very important things. It is downstream of domain expertise. It is downstream of leadership and decision making. It is downstream of data collection; and it is downstream of data engineering. And if any of these is required to any kind of level of expertise and it is missing, your job is hell. Welcome. We don’t talk about this enough that it… As a downstream thing, you have to be very careful about signing up for that job and checking that the people that you rely on can do their jobs also, because otherwise it’s, “Here, let’s do a trust fall. Let’s see, I’ll catch you, I’ll catch you.” [laughter] There’s no one standing there behind you and your data science career is terrifying.

0:44:00.8 MH: That’s big.

0:44:02.0 CK: We need to understand this about this career.

0:44:03.5 MH: Alright. It’s that time again for the quizzical conundrum that gets the guests and the questions and everybody just totally confused. It’s the conductance quiz. Alright, before we get into the quiz, Moe and Tim, let me just tell you a little bit about our sponsor Conductrics. AB testing vendors, a lot of times, make it sound so easy, as if there’s some sort of silver bullet for experimentation. Running an experimentation program is just hard work, and you need a technology partner that’s gonna be honest with you about how hard that is and also innovative to go and solve those challenges and that’s exactly what Conductrics does for their customers.

0:44:46.6 MH: They’ve partnered over the last decade with some of the world’s largest companies, and they’ve delivered effective customer experiences along with best-in-class AB testing, contextual bandits, predictive targeting. They always provide honest feedback and go above and beyond to help clients achieve their testing and experimentation goals. Go check them out at conductrics.com. Okay, let’s go do this quiz. Alright, so this is why I’m so excited about this quiz. Well, A, topically, it’s something that’s right up my alley. Secondarily, Moe, I think I did the right thing to keep peace amongst your broader family, but you will be the judge. So are you ready to find out who you’re representing?

0:45:30.0 MK: I’m terrified.

[laughter]

0:45:34.6 MH: Alright. So yeah, we pulled these at random, and I’m delighted. So Moe, you are representing Shane Arnet, our listener…

0:45:41.0 TW: In Columbus.

0:45:42.7 MH: Who is of the show.

0:45:43.4 MK: Howdy Shane.

0:45:46.6 MH: And Tim, you are representing Michele Kiss.

0:45:49.8 TW: Oh dear. [laughter]

0:45:50.8 MH: So, I decided… You can swap you, now, if you want represent Michele, Moe.

0:45:54.7 MK: No, no, no, no.

0:45:57.8 MH: Okay. “Cause I felt that holidays would get awkward if for some reason something would have happen, having Tim as the fall guy, think is just a better plan all around. [laughter] So that’s sort of what I went with so… Alright, let’s get into the question. Here we go. Now, anybody in data and analytics and anybody with any class at all has definitely had the opportunity and the delight to read the book, The Hitchhiker’s Guide to the Galaxy by Douglas Adams. And in that book, there was a running, if not terribly amusing joke of Hadley referring to 42 when something was seemingly unclear, overly complex. I love how this is actually fitting into our topic. [chuckle]

0:46:37.8 MH: In the book, a pan-dimensional hyper-intelligent species of beings created a computer, Deep Thought, to answer the ultimate question of life, the universe and everything, and after millions of years of calculation, Deep Thought provides the answer 42 to the two descendants of the original programmers, when the two complain about the seeming meaningless of the answer, Deep Thought responses, “Well, I think the problem, to be quite honest with you, is that you’ve never actually known what the question is.” Since coming up with the right question is much harder than arriving at the answers, the task is beyond even Deep Thought’s capabilities, and it proceeds to design a much more powerful computer that can complete that job. The meaning of 42 is that the hard part is first knowing and defining what problem needs to be solved before picking a solution. This is especially true for causal inference. Now, what was the name of the two descendants who received Deep Thought’s answer? One of the names, what is one of the two descendants? So is it A, Fenchurch; B… No Googling. B, Gargravarr; C, Garpit; D, Loonquawl; E, Wedge Antilles.

0:48:01.8 MK: Do you know what the funniest thing is?

0:48:03.1 MH: Yes.

0:48:04.6 MK: I ran an event called Compliance Month. Yes, it does sound as dry as that it was actually compliance, and this was the theme that we used and the whole team before compliance month got together, ’cause the whole theme was The Hitchhiker’s Guide to the Galaxy and every event was around that, and we watched this movie and read the book. Do you think I remember any of this shit? Certainly not.

[laughter]

0:48:35.8 MH: I’m not sure the people in the movie get named, but definitely in the book they do.

0:48:43.3 TW: Yeah.

0:48:44.6 MH: Tim, do you have a recollection. Who could forget the two descendants of… I forget the names of the two people who actually did the question…

0:48:53.7 TW: Yeah, I mean, yeah, I’m drawn to a specific answer, so I feel like I’m gonna go for eliminating some other ones.

0:49:03.7 MH: Okay.

0:49:04.3 TW: So are two of these actually correct?

0:49:07.2 MH: Well, I think we want the name of one of the two descendants, so I think we’re just looking for one answer here.

0:49:11.2 TW: You’re looking for, right, you’re looking…

0:49:12.6 MH: Yeah.

0:49:13.7 TW: They’re two possible… Whatever.

0:49:15.7 MH: Now, if it helps at all, the original programmers were Lunkwill and Fook.

0:49:20.7 MK: Oh that’s super helpful. Thanks.

0:49:22.5 MH: Yeah, no, no problem. Happy to help.

0:49:24.4 TW: So. I’m gonna go for eliminating E.

0:49:29.2 MH: You’re gonna eliminate Wedge Antilles.

0:49:30.8 MK: Interesting.

0:49:33.3 MH: Interesting.

0:49:35.0 MK: ‘Cause that was what I was gonna guess. But anyway, carry on.

0:49:37.2 MH: Well we’re gonna go ahead and eliminate it and guess what, Moe, you can no longer guess it because Deep Thought didn’t take very long to think about that, and it’s like that’s correct. That is not one of the two descendants who received Deep Thought’s answer.

0:49:48.8 MK: Alright.

0:49:50.6 MH: So we have A, Fenchurch; B, Gargravarr; C, Garpit; D, Loonquawl.

0:49:56.9 MK: I’m gonna try and eliminate A.

0:50:00.2 MH: Fenchurch. Okay, let me think about that for a minute. [laughter] Yes, we can eliminate A. I tell you, these eliminations make it much more exciting ’cause I don’t know what…

0:50:16.7 TW: Well I still don’t know which one I think it is, so I could go for what I think it is or I could…

0:50:19.3 MH: Well you now have a one-in three chance, but you’re also playing for some serious family bragging rights.

0:50:28.1 TW: I’m gonna go… I’m gonna think that it is Gargravarr, B.

0:50:31.1 MH: B. Gargravarr. Moe, do you wanna place a counter on that?

0:50:37.4 MK: Sure, I’ll guess D.

0:50:40.9 MH: D, Loonquawl?

0:50:42.7 MK: Yeah.

0:50:44.7 MH: Okay, well, I have good news for one of you. [laughter] And that is that one of the descendants who receive Deep Thought’s answer was D, Loonquawl. Well…

0:50:57.8 TW: Sorry Michelle.

0:51:00.2 MH: Well done, Moe, you did it.

0:51:00.6 MK: Well…

0:51:01.3 MH: Awesome.

0:51:02.9 MK: Now I wish I kind of had played for Michelle, so she got something. [chuckle]

0:51:06.7 MH: Well, actually, we kind of see, we probably do give something to everybody, so it’s sort of not a big deal, it’s more about the bragging rights, but anyway, thank you both for playing, thank you to Conductrics for the great quiz. And thank you to listeners. Let’s get back to the show. Alright, Moe, I know you’re dying to ask a question and we’re so close to having to wrap up, but like…

0:51:30.0 MK: Okay.

0:51:31.5 MH: You’re gonna probably fly to the screen, so go.

0:51:34.2 MK: Yeah. So Cassie, one thing that I kind of… So I’ve watched the training that you did on machine learning for Google staff, and it’s phenomenal. You’re a fantastic trainer, but one of the things that I kind of keep coming back to is this approach of, let’s start with the problem and then figure out what the right method is to deal with that problem. And in some cases, it might be machine learning, in other case it might be analytics or whatever it is, but the bit that I’m trying to wrap my head around is… As an organization, you do need to have some kind of an overarching strategy so for example hot topic of the day, a machine learning strategy.

0:52:20.6 MK: How does that reconcile with the approach that you advocate for because the approach is kind of like, “Let’s hire a really diverse group of people. Let’s put them at problems and get the right people in the room,” and then you use the right approach but then it’s like we have to have this overarching strategy and I don’t have that fits because when you’re writing the strategy, people are like, “Well, I want use cases. I want you to explain in this strategy how machine learning is gonna be used in the business,” and it’s kind of like, “Well… ” I don’t know, Tim looks confused so maybe I’m not doing a very good job of explaining this.

0:52:57.9 TW: No, well, this seems like it’s, I think a… I’m wondering if in my… The little thought bubble in my head is gonna align with how Cassie responds.

0:53:05.7 CK: Why don’t you have a guess? Why don’t you have a guess as to what Cassie is gonna say?

0:53:10.3 TW: Well I… Oh dear, great, so now there’s the opportunity to like, “You’re wrong.” [laughter]

0:53:16.1 CK: I get my revenge. [laughter]

0:53:20.7 TW: Well, this seems like the… Like if somebody came and asked… Having a machine learning strategy, if that’s what they need, then fine, but that’s defining the tool instead of actually defining what you’re trying to solve with machine learning, is one option, but if that’s what the executive says, is we need to be doing machine learning that may not seem like it’s the smartest for the company’s bottom line but if that’s what they want, then it’s machine learning. [laughter]

0:53:47.2 CK: I wanna be fair. I’m being vulgarly pragmatic here. I like real things and real value, and I am not very good at… Very patient with the pantomime as theater, I see you’ve probably seen from some of my responses. However, as the grown-up, I have to appreciate unfortunate realities, and those realities do mean that what a company actually produces is not where it’s value comes from, perception also plays a huge part. And if by whatever mess we’ve gotten ourselves here, that people think that the buzzword machine learning means something special that adds some kind of value, then we would be fools not to recognize that reality, and maybe then… Whether or not machine learning is helping our business deliver real value, maybe in a way we need some kind of machine learning around just in case.

0:54:52.4 CK: And then if that’s where it’s going and we’ve said we’re gonna have some, well, if we’re gonna do that, we’re gonna hire the people anyway, then let’s think about where we can make the most use, where we can get back as much value as we can, and then you’re gonna need that machine learning strategy. So that’s thing one. Thing two is if you kind of know what you might want machine learning for already, and you know it’s gonna be an intense task and we’re gonna take data, it’s gonna take sessions, analysts, machine learning engineers, UX, and all that.

0:55:22.6 CK: It’s a large number of people that you’re gonna have to coordinate, you’re gonna have to do the budgeting, you’re gonna have to do the hiring, and so now we’re gonna have to have some conversations about, “Oh, what’s it gonna cost? What’s gonna get us? Where do we prioritize?” And that again, becomes machine learning strategy. Now, what I would hope at least is that someone, someone in this process has been very honest with themselves as to what they think they’re investing in when they do this. If it is optics but at least you know this and you’re not lying to yourself then fine, more power to you, this might just be the reality.

0:56:00.3 CK: If it is to solve a particular kind of problem or to give yourself the capability to potentially solve some kind of problem in the future, or to hire all the people so that your competitors don’t hire them so that one day no one solves this problem but at least your competitors don’t, whatever it is, at least I hope that there is someone senior who’s not lying to themselves, because, yes, it’s bad to… It’s unethical to lie to other people but it’s just plain stupid to lie to yourself. So it makes great sense when there’s a good reason, there’s a whole host of good reasons to have a machine learning strategy then because it is a symphony of coordination, of different job roles, of things you need to buy, it’s a gift that keeps giving so you’re gonna maintain this thing forever. You need to think about some stuff, there’s your machine learning strategy but I hope that you know the reasons that you’re embarking on this journey.

0:56:53.0 MH: We have to wrap up but oh, my gosh.

0:56:55.3 MK: I know, I’m like we could talk for another whole day and I still would feel like I would have questions.

0:57:00.9 MH: Cassie, thank you so much. I think the thing… It’s hard to enunciate but I honestly think you may have changed the trajectory of my career in this conversation a little bit. And what’s interesting is there’s very few people that do this but you are one of these people for me, is somehow you’re able to enunciate with such clarity the things that weigh in my head that I can’t quite get out and there was a couple of those moments in this episode that I’m gonna go back and listen to and I’m so excited about ’cause they’ve just sort of spotlighted a couple of things so I’m very excited for our listeners to get that chance. Thank you so much for coming on the show and just sharing with us. I love how you think about this stuff and it’s pretty cool, pretty much the highlight of the year, I think for us, we’re like, “We like Cassie so much, we’re just gonna talk about her articles whether she’s on our the show or not.” But for us to get the chance to kinda regroup and after Tim was so terrible, initially, that was super gracious of you. Anyway…

0:58:10.4 CK: Sorry, Tim for teasing you so much. It was a great pleasure to be here and you lot are… You’re fun so… Yeah, I mean…

0:58:19.3 MH: We do our best.

0:58:20.7 CK: Hey, I’m sorry I gave you a hard time Tim.

0:58:21.1 MH: No, don’t apologize.

0:58:22.8 CK: But it all boiled down to it was just some…

0:58:23.6 MH: Don’t apologize…

0:58:24.3 CK: Some innocent… No, I will. I will. I will. Because I did have excessive fun, I’m sorry Tim, but it’s pretty…

0:58:29.5 MH: From my point of view, that was an excellent decision on your part.

0:58:33.7 MK: I agree.

0:58:34.3 MH: All right. Let’s get into last calls. That’s something we like to do where we just go around the horn, share something of interest that we like that maybe our listeners might like, too. Cassie, you’re our guest. Do you have a last call you’d like to share?

0:58:47.4 CK: Yeah, so I’ve been releasing a recording of Google’s internal machine learning course. This is called Making Friends With Machine Learning, and it is for everyone. In fact, the challenge, the prompt that I received when I was making it, is make a course that a researcher in machine learning, a professional researcher, will like, but also that a high schooler with no mathematics will like and understand. And they thought they were being funny and I said, “Okay, I’ll make you this course, but I will make it a little bit leadership oriented.” Because, again, I want us to focus back on the why. Why do we do things? And so that is the course. It is… It doesn’t require equations. It doesn’t require mathematical knowledge. It takes you through it with analogies, but it goes quite deep in what everything means, and now I’m rolling around to my point as I keep rolling around to these points and in closing brackets, but my point is, part four of this course has been a long time coming, and it has very recently been released on YouTube, so anyone who was is waiting for part four, you can now find all that on my channel on YouTube.

0:59:56.9 TW: Awesome.

0:59:57.4 CK: And the link, I’m sure you all will put it below but that’s…

1:00:01.2 TW: It will be in the show notes. Absolutely.

1:00:02.9 CK: Yeah. It’s bit.ly/machinefriend. So yeah, come make friends with machine learning, whoever you are.

1:00:08.6 TW: I can say the first three parts I found so useful. I’ve I learned that the goal is to get a model that over-fits that that’s kind of when you…

1:00:17.0 CK: Oh no, I’m dying.

[laughter]

1:00:20.9 MH: Stop it, Tim. Stop it. Alright, Moe, what about you? What’s your last call?

1:00:27.1 MK: Mine is not related to data at all, but it’s just delightful. So I just read a book, which is a novel. It’s just for fun, but it anyway, it’s called the Dictionary of Lost Words, and it’s a story basically about the team that first created the dictionary. And it just… The reason I’ve loved it so much is because we’ve actually had someone join my team recently who studied linguistics, and I kind of was like, “What does that even mean?” And suddenly, by having read this book, we were just able to have the most incredible conversation about language, and meaning, and the power of words, and it just… Anyway, I really enjoyed it. Like I said, it’s got nothing to do with data, but it was just a really nice read. So anyway, if anyone’s looking for a good book. I I’d give it a strong nine out of 10, which would… Definitely would recommend on the MPS scale.

1:01:25.1 MH: That is right. Excellent, thank you, Moe. Okay, Tim. What about you? What’s your last call?

1:01:28.1 TW: So I do have to admit, I have a spreadsheet, Cassie, of various posts when you kind of sent out the list of things where I was reading, so I’m not gonna go through… I mean, that I could have done like, “Oh, and this and this.” I do think p-values with puppies, which I’m sure people constantly come back and tell you, and that is wonderful. And then the birthday paradox, and when I intuit at that I will and just don’t have to keep going back and reading it… I will have arrived, but I’m not gonna do that. I’m gonna do one that’ll make Michael proud because it’s like pop culture. So… My podcast well ran a little dry until I wound up going and listening to all of the endless thread. They did a whole series that were on memes, so they actually talked to Disaster Girl, they talked to Rick Astley, they talked to Scumbag Steve, they talked to… They researched Kilroy, where that came from. So it was like, I don’t know, seven or eight episodes that were just on kind of memes, as broadly defined as possible. And I listened to them all like in a couple of days. And so that’s just me trying to be semi-pop culture literate. I only had the lady, screaming at screaming lady, screaming at cat. So most of the memes I actually recognized without having to Google what they were, but it’s a good listen.

1:02:47.1 MH: Proud of you, Tim.

1:02:49.4 TW: Okay, what’s your last call, Michael?

1:02:50.6 MH: Well, I have actually a few I’m gonna run through quickly because as the New Year is getting ready to start, there are a bunch of events coming up that our listeners should be aware of. And some of you are, but I want to catch you up. First, and sort of… Maybe most excitedly is super week is coming back at the end of January. Tim, I think you’re gonna be there, correct?

1:03:15.3 TW: I’ll be there. You’ll be there as well, right?

1:03:18.3 MH: I also… I can’t believe it. I’m… Provided my passport comes back in time, I will be there too, so…

1:03:24.9 TW: Likewise.

1:03:24.9 MK: I’m really pissed that I will not be there.

1:03:25.7 MH: And I am pissed too, Moe, because… But hey, life happens, but actually the next month in February, MeasureCamp North America. It’s gonna be a virtual MeasureCamp again. MeasureCamp is phenomenal. I know, Moe and Tim, you’ve both been involved with organizing those events in your various cities, so it’s just a great non-conference conference. It’s so easy to be part of, so I highly encourage that one. And then lastly, the Marketing Analytics Summit is doing a virtual online summit March 1st and 2nd, so that’s also available virtually, but come to super week if you’ve never been, especially if you’re a listener in Europe. Come on. You gotta come. And you heard Cassie talk about broadening your perspective and becoming a better decision maker and thinker. What better way to do that, if you’re a US citizen, than going over to Budapest and spending a week with all the great analytics minds in Europe, and broadening your horizons perspectives on how to think differently about analytics in the world?

1:04:26.9 MH: Okay, that’s my last call. Listen, I know you’ve been listening to this episode. You’ve been having the exact same reaction I had, which is, “Oh my gosh. I can’t believe it.” And guess what, it doesn’t have to stop there. We’d love to hear from you. You can get to us easily. You can reach us on the Measure Slack group, or on Twitter, or on LinkedIn. And I think it goes without saying, but I’m gonna say it anyways: Go follow Cassie on her Twitter. Her Twitter profile is at… I don’t know how to pronounce it, but it’s Q-U-A-E-S-I-T-A…

1:05:00.3 TW: That was the… What is the history of that? Is there a story behind that? I I’ve looked for it.

1:05:04.2 CK: Yeah. The data is Latin for, “That which is given.” And…

1:05:09.6 TW: Oh.

1:05:09.6 CK: That word is, “That which is sort.” It is the opposite of data. So data is what we give. And that word which I don’t know how to pronounce either, but let’s do it in American and quaesita is fine.

1:05:23.3 MH: Quaesita.

1:05:23.5 CK: That’s what we’re looking for. It’s the word from quest. But it’s the opposite of data. And so it comes back to that theme of, the important thing is not what we’re given… The answer’s lying around, we’re gonna use that for something, for what though? What are we seeking?

1:05:37.3 MH: See, at some point, does it become perturbing how smart you are [laughter] for others like, ’cause like, damn, no, but that makes so much sense. That was awesome, awesome. Anyways, my point, follow her on Twitter, also, I don’t know anybody on Twitter, but… Why isn’t, why isn’t she verified on Twitter? Come on, blue check, let’s do this. [laughter]

1:06:00.0 MH: Okay, let’s wrap this up. Obviously, no show would be complete without a mention of our excellent producer, Josh Crowhurst. He does a wonderful job making sure this show gets out to you every other Tuesday. And we thank him for that. Okay, Cassie, once again, I just can’t… I believe I’ve been expressing, but I’m gonna express again. [laughter] It’s been such an honor. Thank you so much for coming. It’s been so great.

1:06:27.1 CK: This was great fun. Thank you all for making it awesome.

1:06:29.5 MH: And I know that regardless of what role or tools you’re using to solve the problems of data and analytics out there or what title you have to do it, the one thing I know with confidence that I can say for from my two co-hosts, Tim and Moe, is keep analyzing.

1:06:51.2 Announcer: Thanks for listening. Let’s keep the conversation going with your comments, suggestions and questions on Twitter at @AnalyticsHour on the web, at analyticshour.io, our LinkedIn group and the measure chat Slack group. Music for the podcast by Josh Crowhurst.

1:07:09.1 Charles Barkley: So smart guys want to fit in, so they made up a term called analytics. Analytics don’t work.

1:07:14.0 Thom Hammerschmidt: Analytics. Oh my god, what the fuck does that even mean?

1:07:22.8 MH: Okay, other logistics. We are more nervous than we usually are because you’re kind of a big deal and…

1:07:35.6 MK: Yeah.

1:07:35.6 CK: And I have some things to say from their that previous episodes too.

1:07:43.4 MH: Uh oh. Clear the air. Yeah. [laughter]

1:07:43.7 MK: Shit. We’re in trouble.

1:07:44.1 MH: It’s time for…

1:07:45.0 TW: I was like, I saw you commenting on that one, I’m like, “No!”

1:07:53.7 CK: Okay, Michael, you’ve got to introduce the this shit to someone. Do it. ‘Cause…

1:07:55.2 MH: No. It as to be them… This is good. This is very good.

1:08:00.3 CK: No survivors. No survivors. Only one. There can be only one.

1:08:03.5 MH: I am so excited, so excited for what we’re about to accomplish here. Alright, let’s go in five.

1:08:14.2 CK: So there are so many different things and that I’m gonna have to try to remember all of the things. ‘Cause I am tangling myself up in all the, we’re opening brackets and we’re not closing them and opening any more brackets. This is great. This is how you lose a bracket, but okay. So…

1:08:29.8 TW: You’ve been looking at my code.

1:08:33.3 CK: Yeah, I know right. Finding, losing, you’ve been looking at mine too. It’s all good. [laughter]

1:08:40.7 TW: Rock flag, and I was wrong.

Podcast: Download | Embed

Subscribe: RSS