Subscribe: Google Podcasts | RSS
Subscribe: Google Podcasts | RSS
We try not to navel gaze too much on this show, but our 200th episode felt like just enough of a milestone that we could do a mid-year “look back, look forward” show with a 7-year range. And we tracked down our original Commonwealth representative to join us for that discussion. Did we (first) party (cookie) like it was 1999? Maybe not, but that’s the sort of reference you get with Jim Cain, the founder of Napkyn Analytics, and a co-founder of this very podcast!
Photo by angela pham on Unsplash
0:00:05.9 Announcer: Welcome to the Analytics Power Hour. Analytics topics covered conversationally and sometimes with explicit language. Here are your hosts, Moe, Michael and Tim.
0:00:22.9 Michael Helbling: Hello everyone, this is the Analytics Power Hour, and this is Episode 200. Can you believe it? It’s been 200 episodes? So much has changed since 2015, when we first put on our headphones and started cussing about analytics in podcast form. Back then, we were the Digital Analytics Power Hour, and we argued about whether it was still okay to use Excel and how technical an analyst needed to be. Yeah, but one thing was always true. We showed up and brought our informal conversational approach to whatever was going on in our industry. Hey, Moe, how are you doing?
0:01:01.0 Moe Kiss: I’m doing great.
0:01:02.7 MH: Great. It’s great to have you.
0:01:04.2 Tim Wilson: How are you going, Right?
0:01:06.1 MH: I know, but I switch it up, I switch it up, Tim.
0:01:07.5 MH: It’s so confusing.
0:01:08.7 MH: Hey, Tim, how you going?
0:01:10.7 TW: What are you talking about?
0:01:11.9 MH: See, I switch it up. See, it’s not just always the same. 200 episodes in and I gotta keep you on your toes. Anyways, those are my two co-hosts, Tim Wilson and Moe Kiss, and I’m Michael Helbling. And for this episode, we wanted to bring back an OG of the podcast. That’s right, Jim Cain is the Founder and Chief Innovation Officer at Napkyn. One of the original trio of the Digital Analytics Power Hour, and today he’s back for our special 200th episode to catch up on what has been going on since that fateful day in May 2016. A lot has happened in our industry and in life. Welcome back, Jim.
0:01:52.8 Jim Cain: Hello friends. And pleased to be here and nice to work with Moe to finally make the Power Hour 50% commonwealth.
0:02:02.6 MH: That’s right. In theory, some guests have probably made that true as well, like Tim Harford, and I think he’s OBE too, so does that make it even more common wealth?
0:02:16.5 JC: Yes.
0:02:17.7 MH: Okay, so Anyways, but there’s many other things that you certainly were around for the first of Jim and… Well, I think maybe a good way to kick off is maybe just talk like what’s been going on for the last… What greatness did the podcast launch you onward to?
0:02:34.1 S1: Yeah that’s right.
0:02:34.5 MK: So I can prepare myself.
0:02:34.6 MH: The podcast is a springboard.
0:02:38.1 TW: What’s Moe have to look forward.
0:02:41.1 MH: No, what have you been up to? That’s probably the first thing people would wanna know.
0:02:44.4 JC: Most people know me from my semi-professional hand modelling, and…
0:02:48.5 TW: That’s right.
0:02:49.9 JC: That’s a big part of who I am, and I’m glad that it’s talked about. The leaving of the podcast was correlating to… You guys are going through to 70-hour work weeks, right? And 2016, I remember just sitting back, probably even with you guys and going. “We’re not gonna be this busy forever, it could not possibly keep getting this much more busy right?” But since then, it just kept getting busier and busier, Napkyn grew and grew. Our relationship with Google became kinda more and more pronounced, and so we eventually became I think, one of the biggest resellers of Google Analytics 360 in the Americas, along with all the associated consulting and stuff, and we actually joined a larger consortium of companies in December, when we became part of the Kepler Group, which is a really amazing world-class media company out of New York, and we joined up with them, so we’re kind of on a new phase of Napkyn, and I’ve moved from Chief Executive Officer, where a lot of run the business things to do into the chief innovation officer, or as I pronounce it, the Chino, where I’m a lot more focused on strategy and clients, and where is the market going? I think the things that I had the most fun with when we were recording six, seven years ago, is now my full-time job, so I’m really excited about it.
0:04:08.4 TW: Well it makes sense, you used to not record with pants, and now you’re wearing chinos, so it makes…
0:04:12.9 MH: Yeah, real progress.
0:04:14.7 TW: Yeah, it’s progress.
0:04:15.9 JC: And I think it’s nice that you assume I’m wearing pants, that shows a lot of respect.
0:04:21.7 MK: But, okay, so what does a chief innovation officer do? What does your day look like?
0:04:28.3 JC: Well, from 8:15 to 10:45, it’s mostly hand modelling you know what I mean? It’s hard… It’s hard to multitask but I’m willing to do it for my fans. A lot of it is honestly the kind of stuff that you guys do on this show, where have we been, where are we going? How can we help our customers and our team? So help our team prepare the skills and the programs and help our customers prepare to be ahead of market trends, I have made a number of wrong calls in my career, I’m not gonna name any names Attribution 360, but I’ve done some things Attribution 360 in the past that maybe I regretted. But the more you spend time in the enterprise level of what worked, what didn’t work. You’re able to help your customers go “In two years I think you need to be here and not there.” And so that’s what I spend a lot of my time doing right now. And it’s also freed up more time to do speaking and things like this and getting out to events, so it’s a nice change, but it’s a bit of a trip.
0:05:28.3 TW: Yeah, well, as someone who’ve always has really deeply appreciated the concept of low-hanging fruit, I’m sure… No, I’m just kidding of course.
0:05:37.3 JC: I’m really leaned in on that, actually.
0:05:40.2 MH: Oh really? Oh Geez.
0:05:41.6 TW: Wow, you have changed.
0:05:44.3 JC: I give them the kindest of regards. I do.
0:05:47.2 TW: The kindest of regards [laughter] Okay, well, some people will get some of these jokes probably very few.
0:05:58.9 MH: The one percent.
0:06:01.9 TW: There’s listeners who’ve gone back and listened to the whole catalogue. Which… Quiz time, which launched first, GA 360 or this podcast?
0:06:09.5 TW: I think it was still Analytics Premium.
0:06:11.5 JC: I think this podcast.
0:06:13.3 TW: You are correct. It was this podcast launched first, March 2016.
0:06:18.8 MH: Really.
0:06:20.5 JC: It was GA360.
0:06:22.1 MH: That’s crazy to me. I did not know that. And here we are on the cusp of GA4.
0:06:25.8 TW: I got a whole list I could sprinkle these throughout.
0:06:29.1 MH: Yeah, some things just seem like they’ve been around forever, but maybe that’s one of the things that has been around forever, is the podcast. But I don’t know, like there have been a ton of major industry changes that we could talk about, and I don’t know. What’s a big change? From your perspective, Jim the…
0:06:48.3 MK: Is it like fucked up that I wanna start with Attribution 360?
0:06:51.8 MH: No, actually, let’s dive…
0:06:54.3 MK: I know but just I feel like I keep coming back to being like. “Can we talk about Attribution?” and now we’ve all changed our mind, and I feel like everyone listening is gonna be like, “I’m so sick of this Moe. Move on.”
0:07:04.4 TW: You know this was the beginning of the end for Jim as a co-host, but you know what? You go ahead, blaze that trail, Moe.
0:07:12.3 JC: We’re either gonna make a noose or a hammock here’s some rope. I could look… I mean, we could certainly talk about Attribution since 2016. I mean, Attribution 360 is just me kinda making fun of a tool that went bang. There were some companies that built some really interesting tools, I think the easiest way to describe why Attribution hasn’t historically worked is because it wasn’t properly implemented. It was possible in cookies all worked kinda surveillance state level AdTech and MarTech ecosystems to build a really well-functioning Attribution machine but it required effort from multiple departments. It required agency buy-in, so forget about it, there were just too many moving people pieces to make Attribution work, but I think the tech was really sound. I think now it’s impossible to do Attribution to the individual conversion level, and Attribution needs to have a dictionary level change in terms of what it means, ’cause I think there’s still some exciting things that can be done, but not to a person that’s… We’re back to audiences. It’s 1999 again, you know.
0:08:17.8 MK: And what do you tell your clients that are still hung up on it? Do you just have this conversation very frankly?
0:08:26.6 JC: Mm-hmm, yeah.
0:08:26.8 TW: Yeah I say, the measurement of addressable channels is dead, long live addressable channels. And then what options do you have instead? Which basically, it’s Media mix Modeling plus experimentation or just to make hold out testing, I guess.
0:08:43.8 MH: Not to beat the… I mean yeah, this is a drum I’ve beaten a bit. But when you said the definitional change, to me, it comes back to the… Attribution is the assignment of value, which is not… It gets treated as though you’re measuring value, which gets assumed to be that you’re measuring incremental lift, right? To me, that’s this definitional shift that needs to happen that you can… You’re attributing value like it is right there in the word, you are choosing how to attribute and assign value, which is very different from the truth of measuring, and I don’t think it… We can do that quiz… Did Google buy Adometry before or after the podcast launched?
0:09:25.5 JC: After.
0:09:27.3 MH: Anyone?
0:09:28.3 TW: I think it’s after, right?
0:09:30.8 MH: It was actually before.
0:09:30.9 JC: Ohh.
0:09:30.8 MK: Ohh, I was gonna say that but everyone was so confident. I was like, I just don’t know this one.
0:09:37.0 MH: No well, I was wrong last time, I was wrong last time, and I was like, “Well, I’ll just skew the other direction”
0:09:42.3 TW: Geez.
0:09:44.3 MH: Sorry, that was May, 2014 so…
0:09:45.6 JC: But there’s…
0:09:48.0 MH: Okay, wow.
0:09:49.5 JC: Like on the whole lies and damned sales guys or whatever it is I used to say.
0:09:53.5 MH: Hype, lies, and the shitty sales guys, I looked that one up too.
0:09:56.0 JC: There it is, that. [chuckle]
0:09:58.3 MH: Yeah, nice. It’s a classic.
0:10:00.5 JC: But Attribution was sold that way. And one thing I’ve also noticed over the last six or seven years is that all the people that were really good at selling tag managers back in the tag manager boom, all went to selling Attribution tools for a few years, and then they all sold CDPs, and now they’re all selling probably NFTs or something, but the most kind of…
0:10:22.0 MH: Well, now they’re all selling server-side versions of their CDPs, and then NFTs I don’t know.
0:10:29.0 JC: You mean like web trends… Oh no, it’s not like web trends… Yes it’s exactly like web trends. Yeah.
0:10:33.0 MH: No.
0:10:35.5 MH: Yeah, it is interesting ’cause our industry does sort of have this forgetful nature about where some of the stuff actually started or whatever. But they chased the technology solution, right? You go to Adometry, ClearSale, and Convertr, all of those and I don’t know, what do you… Jim what do you… If you wear your chino hat, making the link to CDP is actually is pretty interesting ’cause like where are we on the hype cycle with CDPs? It seems like it’s being sold as “This is your solution to cookies going away, this is your solution to privacy constraints and regulations.” How much are you getting hit up with CDPs and where are you landing on it these days?
0:11:24.5 JC: I maybe have an unfair bias against CDPs, ’cause I just see so much hype cycle around them, and I see so many large… So I jokingly, for several years, have called the CDP the enterprise fondue pot, ’cause every company owns one and no one’s ever cooked with it, sometimes they own two or three of them, but no one knows how to make food. And I don’t know a single organization that has gone past one use case, so I know a lot of enterprises that bought a CDP for media suppression, and that was like, it would justify the cost of the product, but they never implemented a platform. I know very few organizations have implemented a platform, and when we think cookie-less world, I think one of the fundamental things that makes a CDP work is the fact that it’s ingesting all the data and owning all the data, and it’s got its own kind of black box like everything that’s been sold for the last 15 years. And I think directionally where we have to go is for organizations to repatriate their own data, so I actually see the CDP market being in pretty deep shit because those individual use cases like media suppression or individual personalization use cases, those can be built on top of a pre-existing data store, and I think businesses are starting to realize they need to own their own data.
0:12:38.0 TW: Yeah, I concur.
0:13:47.4 JC: Oh yeah.
0:13:47.8 MH: And that kind of torched themselves and CDP seems like it’s over promising, and we’re gonna be looking back in five years, 10 years and yeah.
0:14:54.5 MK: No.
0:14:55.6 JC: His truck started and his wife came back.
0:14:58.9 MH: Truck started. Yeah.
0:15:00.3 MK: I definitely have not heard that one, I’m feeling completely puzzled.
0:15:03.6 MH: That’s oh, man. That’s no enterprise fondue pot, but…
0:15:11.2 JC: That’s It’s a dinger.
0:15:13.5 MH: I’d say that’s more of a classic. I feel like the Enterprise fondue pot is a Jim Cain original.
0:15:13.9 JC: I stole the other one. That’s not me, that’s it’s my dad who got it from somewhere else, I’m sure.
0:15:21.7 MH: Yeah.
0:15:23.1 JC: But honestly, there’s a whole new, marketers are getting much more sophisticated and a lot more bought in when it comes to data, but I still find a lot of businesses with all the change in the COVID and everyone’s like any sales person that goes, just sign here and this problem will go away. And that’s how people have been selling in digital for like we were just saying, 10, 15 years, and I don’t know what the next category of that is. I think it’s on the AdTech side, but something’s gonna come out and screw everything up for everybody, and it’s probably an AdTech tool.
0:15:52.7 MK: Okay, in my head, I’m like my first instinctual question is, who does it well, but maybe that’s not the right question, maybe it’s more about how do you balance this need to solve that problem and someone attempting to serve it on the silver platter versus the pain of trying to fix it in-house.
0:16:15.0 JC: I mean, I think part of it is just anyone with a quota is gonna try and hit their quota, and that has nothing to do even with our category, right, it’s just these are very sophisticated products that sometimes the person you’re selling to can be tricked into it. And I’m not throwing sales people under the bus, I am one. But there just tend to be hot button categories where a lot of businesses maybe get oversold, and I think CDPs are one of those, personally.
0:16:44.2 TW: Is it fair? ‘Cause I look back to when we started and now, and to me, one of the evolutions, ’cause this goes back really just a couple of years kinda in the rise of the analytics engineer, which we had Claire Carroll from DBT on, is one of the kind of helping to spread that is this between an analyst and a data engineer, so I wonder if the technology is where it’s, We’re selling technologies to enable people who are solving hard problems, not technologies will solve the hard problems, the technologies that are the helper platforms and tools, because they recognize everything is specific to the company, you’re gonna have to come up with a solution, and it’s not gonna be some buy this off the shelf thing, punch the go button, and you’ve removed the need for people, that’s what the sales people are pushing, there does seem to be a whole layer of tools, and I feel like we wind up talking about them, talking to Prukalpa from Atlan, your metadata management, active metadata management, the metrics layer, DBT, a whole role where we’re trying to still have people who are doing hard work and have to have skills developed, but they’re the ones who are being enabled with the technology as opposed to somebody getting wined and dined and being told buy our platform, deploy it and all your problems go away. It’s that two too hand wavy, in and…
0:18:53.5 TW: Yeah, that’s definitely one big change.
0:19:44.2 TW: Yeah. So, GDPR went into effect before or after the podcast launched?
0:19:49.2 JC: After.
0:19:49.5 TW: Oh, come on way after. Okay, that was… You know what? That was some low hanging fruit.
0:19:56.0 MH: That was… Yeah.
0:19:56.3 JC: But when was it approved, ’cause it was approved years earlier, they gave everybody a long window.
0:20:01.1 TW: Only a couple of years, it was only… I think even that was like way after.
0:20:06.0 MH: I thought it was 2018, wasn’t it?
0:20:08.3 TW: Well, it went into effect in 2018. But I think it got…
0:20:09.9 MH: Yeah, oh okay. It went into effect.
0:20:10.2 TW: It was like 2016, was when it was passed and agreed to, and then it was like a two-year runway.
0:20:16.2 MH: Got it.
0:20:16.7 TW: I think. Some European will… Can call me on it, if that’s off, but good job.
0:20:23.5 MH: Anyway, but it is interesting because, Jim, something you’ve kind of talked to people in the industry, and certainly as we look at sort of the tech stack that companies are using for data, it does seem like back in the TMS/CDP heyday, if you will, those were all selling into marketing organizations to…
0:20:45.8 TW: Wait. TMS/ CDP heyday?
0:20:48.0 MH: Yeah, like, Tag Management into CDP.
0:20:51.0 TW: Oh, TMS into… Okay.
0:20:54.0 MH: Yeah, yeah. So, that was all sort of like, “Hey, we’re gonna help you manage all this data and do it really fast, or your IT department doesn’t have to get in your way.” Now, I think I’m seeing… I’m curious what you’re seeing, it seems like IT departments are starting to become back in charge of these data stacks and their workings.
0:21:12.4 MK: It’s ’cause the marketers don’t want responsibility for it, so they’re giving it back to us.
0:21:18.0 MH: Well okay. No, I mean, so that’s a good… So I wanted your opinion on that, Jim, ’cause I definitely see a shift. And it’s very intriguing ’cause analytics with web trends and log file analysis started in IT, and then marketers got a hold of it and started driving most of what was happening in IT, and now I feel like it’s kind of moving back out of marketing again.
0:21:40.6 JC: I don’t think that’s unfair and I don’t even think that’s necessarily bad, as long as it’s not a 180 degree shift.
0:21:50.0 MH: Yeah, it’s gradual.
0:21:51.5 JC: Well, but it shouldn’t be. So here’s an example, we work with a lot of Fortune 1000 organizations, all of whom bought Adobe Analytics 20 years ago, because it was the 800-pound gorilla, it’s the tool of record, and it was purchased by IT through proper channels, and it is maintained through proper channels, and literally nobody fucking uses it. Everyone is using the rogue deployment of Google Analytics that was snuck in by someone in the paid search team, 10 years ago, and… But IT has got their arms around Adobe, and these organizations are coming to us now buying Analytics 360 because of its integration into the media buying products and Google and it actually does make sense for that reason. But the IT organization is going, “Why are you buying a car? We already have a car. We’re invested heavily in the car.” Marketing saying, “No one wants to drive your car. Your car sucks.” That hasn’t changed at all. And so there’s a missing gap.
0:22:45.8 JC: I gave a series of talks last year that I call, “The Four C’s”. But about four different types of people inside of business, which are really business users, executive stakeholders, the IT organization and the legacy data organization and they all have unbelievably competing priorities in terms of what winning means for the business. And unless you can actually get a functional group of ideally those four people, but like IT provides capacity and marketing provides requirements and feedback and it becomes a virtuous cycle, it doesn’t matter who gets to carry the ball, you’re not gonna win.
0:23:25.0 TW: I feel like it’s worth just noting like, the Adobe Analytics world, their play is more of the… They’ve got the CMS and the e-commerce and they have their platforms. It’s a Venn diagram of where, Google’s got the analytics and they’re aligned with the marketing stuff, which I don’t think… IT doesn’t really care… They don’t care about your CPC or your conversion optimization. And Adobe’s kind of doubled down on… You’re gonna provide a personalized experience basically through your owned media, not that they don’t have Audience Manager, but like, you would think even a fourth grader would be able to tell that Google’s got a clear competitive advantage there, when it comes to hooking into the marketing ecosystem. And then, you have people saying, “You can’t compare Adobe Analytics and Google Analytics”, which that just seems laughable, ’cause you absolutely can. But…
0:24:23.7 MK: But then you also have a whole range of businesses that just don’t use either.
0:24:29.1 TW: Yeah, and that’s… I would say another one of those emerging trends, somewhat, the ones that have said… To me, those are more… Those tend to be younger organizations that have said from the ground up, “We have the infrastructure to… ” whether it’s Snowplow or a homegrown, totally homegrown, “We can capture all of this and manage it.” But yeah, that’s a… But even those organizations still wind up often with, at least Google analytics for the marketing. Like as Jim, as you were describing that, the number of companies where, “Oh, we don’t have Google Analytics on the site for… We’re not using it for analytics, but we need to have it for marketing.”
0:25:11.6 MH: Like audience targeting and stuff like that, yeah.
0:25:14.8 TW: And then they start using it, so…
0:25:15.9 JC: Yeah. And I actually think that’s okay. It’s okay to have an, as well as, analytics tool as long as it’s fit for purpose. Right? So, if the people… Like, if finance is using Adobe? Great. If marketing is using Google, in that same organization, great, because Google Analytics is actually a critical spark plug in a post-cookie marketing engine. It has to exist. It just will. The problem is when Google Analytics says, “You had a 1000 conversions” and Adobe says, “You had 17.” And that’s a data engineering issue. As long as you’ve done proper data foundations and tag management, Google and Adobe will be very close and then it doesn’t really matter what your front end reporting tool is, I don’t think.
0:26:00.7 TW: Well actually, now that Google is modelling conversions… That might not be as close by channel anyway.
0:26:08.3 MK: So, yeah, look, we have more than one tool, and one of those tools, which shall remain nameless, but is something akin to what we’ve been talking about, I just say to people like, Sure, if you wanna use that for an understanding directionally of what’s going on, that is totally fine, but when it comes to reporting to the business, this is what we use. So if you wanna have a play around because you’re comfortable and you wanna do a bit of digging… Go right ahead. It is directionally totally fine to use, but just when it comes to reporting, that’s where I draw my line, I’m like, If you’re sharing numbers with the business, you use… Need to use the main tool on record, but it sounds like Jim has a different view.
0:26:49.5 JC: Well, if the tool that you’re talking about rhymes with Frugal manalytics, then the problem with people using it for directional reporting is that they’re going to take those insights and deliberately use them for media buying. That’s the big thing about this post next July, GA4 in the New World, that first party data store that’s inside GA or in GCP potentially, is going to be directly turned around and used for buying and managing media campaigns, so if that instance of… I’m sure it’s not GA in your instance, but if it was… If this reason you’re saying use it for directional reporting is ’cause you don’t trust its accuracy, then it would really be important to get it to align with your record of truth, ’cause someone’s gonna spend tens of millions of dollars of budget through it, maybe…
0:27:41.4 MK: To be honest, it’s actually more like SEO and content people using it. And like people looked at this page or whatever, and this was their engagement and yes bounce rate, and I’m like, You knock yourself out there, I’m not getting involved, but it’s not the performance marketing team.
0:27:57.6 JC: Okay.
0:28:00.4 TW: But even if it is, if you’re using it for an operational purpose of saying, we’re gonna optimize our media… Just know that at the end of the day, this is how things are being credited, it seems like that… That’s okay too. If at the end of the day, if you said, I need to get to the end of the street and I’m gonna navigate with a compass and somebody else says, use a GPS and we’re gonna measure you on whether or not you got to the end of the street, or not, and we’re gonna do that with somebody standing there with a stop-watch, do I care whether you’re using a compass… Now, if my compass I’m sitting it on top of a magnet and I’m walking all over the place and I show up three hours later, then yeah, maybe you shouldn’t be using that, but I think that to me, that makes some sense, use it. Google is gonna be, or whatever rhymes with Frugal, Moogle, Frugal manalytics, whatever the tool is, there is gonna be a need to say, We have to make operational decisions. I still think there are cases where… And we’re not talking about canva ’cause you just said not so much performance marketing, I think there are definitely cases where there are performance marketers who have learned how to fill their day by looking inside Google or inside a tool and tweak and tune stuff, and then they simultaneously tell you how the magical machine is optimizing stuff.
0:29:24.7 TW: And I’m like, What the fuck are you doing with your day? Either you’re turning on the machine and letting it run, so don’t tell me you’re doing anything, or you’re sitting there tweaking shit all the time, in which case, do you need to… Is that really valuable or you’re just looking at bad data and just like playing around with the levers because you can… And it gives you a little bit of a sugar rush, and now you have a story to tell somebody, I’ve watched clients show spreadsheets, go in, they invalidate how all of the data is really no good, but they look at it every day, and they make decisions and adjustments, and I’m like, “What are you doing?” I can see how you got to this point. But the emperor has no clothes. In that case.
0:30:08.3 JC: That gets exponentially harder when the business is also using a separate tool as a tool of record, right?
0:30:17.6 TW: Somewhat, I guess if the tool of record is saying who’s getting credit for it, but I think that’s where there’s a combination of… Is the business doing well? Yeah, I don’t know. No, yeah.
0:30:32.9 TW: Well, if my Chino moves are strong, and I am correct in saying that in a couple of years, people are going to be owning their own data and kinda curating it and governing it, and then the reporting layer goes on top, then it doesn’t matter how many reporting tools you have, ’cause they’re all cooking with the same ingredients. At that point, it’s moot.
0:30:51.7 MK: To be honest, though, I’ve been thinking this for years, but I think like Tim was saying it’s more like the result of the fact that I’ve worked at younger start-upy companies where there is a desire to own your own data anyway, and there’s been this real… Well, there’s been like the move from the data team to want to build in-house because you get to do cool shit, you get to work on cool tech, you own the stack, like you expand your skills, you can contribute to open sources, all these reasons, but I think from my experience in the historian tech industry, people already own their data, there is a lot of reluctance about where you share data and why, and we have really strict rules now about when we can share and why, and it’s definitely not acceptable anymore to just be like, We’re gonna pump all of our data into this other company and it’s gonna live there, like there’s a lot of resistance to that.
0:31:49.4 TW: But what about when it comes to the… When it comes on the marketing side to the buying audiences, which is just getting horrendously exposed as being bot-ridden, crappy, targeting… Programmatic scam, and it’s slowly getting more and more exposed that… What was the study that was like, even nailing down gender, was worse than the flip of a coin, does that also start to go in-house where it’s like, “Yeah, you know what you’re gonna do?” You’re gonna say, “These are the websites that I want to advertise on.” And I am going to take that back in-house and say, “I’m gonna go negotiate to have my ads running on these sites.” I’m gonna go back to contextual advertising where I know I am running an ad on sites that I care about. ‘Cause like that’s the other not owned data is, “Oh, you’re just part of the Google ecosystem. And with Google magic or Facebook magic, we can find you the women, 24 to 35 who have one dog and two cats.”
0:32:56.6 MH: Hey, everyone, it’s that time for the quizzical query, the Conductrics Quiz, the conundrum that brings laughter and delight and prizes to our listeners, as Tim and Mo compete on your behalf to answer tough questions, proposed to us by Matt Gershoff of Conductrics and the Analytics Power Hour and the Conductrics Quiz are sponsored by Conductrics. They help build industry-leading experimentation software for AB testing, adaptive optimization and predictive targeting. Find out more at conductrics.com. Alright, are you ready Tim, to know who you’re competing for?
0:33:38.4 TW: Ready.
0:33:38.7 MH: This one has super weak implications, big time. It is… Yehoshua Coren…
0:33:46.2 TW: Oh geez…
0:33:47.9 MK: No pressure Tim.
0:33:49.2 MH: Yeah that’s right. And Mo, you are competing on behalf of Antonio Garcia Chellis, so also a listener. Thank you both for listening, so here is the question, and this one without further ado. I’m not running into a room. I’m not upset. I’m just asking what is the main reason we bother using random selection in our experiments is it A, to calculate P values, B, to mitigate confounding, C, to compare confidence intervals, D, to minimize type 1 error, or E, to estimate posterior distributions.
0:34:30.5 TW: I totally know the answer on this one.
0:34:33.6 MH: And bandwagoning is allowed on this one, we’ve gotten that directly from our sponsors, so if you feel like you know the answer, Mo, and you wanna bandwagon, that’s totally allowed here.
0:34:44.1 TW: You wanna pick an answer first Mo or you want me to go first?
0:34:46.8 MH: Mo probably knows the answer, too yeah.
0:34:49.2 MK: Good to know. Tim, you go first.
0:34:52.6 MH: I’m gonna go with B to mitigate confounding.
0:34:56.1 MK: Oh, you’re waiting for me. Yeah, I was gonna say that too, so…
0:35:00.2 MH: Okay good. I also felt like that was the correct answer, it’s sort of weird for me to say it in retrospect now, but I think we’re all thinking it’s answer is B to mitigate confounding, and that is correct. So Yehoshua and Antonio, you’re both a winner. So congratulations. So confounding.
0:35:19.6 TW: And Matt actually wrote a great Post about this a couple of months ago…
0:35:23.6 MH: Yeah, actually we’re gonna link that in the show notes to tie into this but… So go to our website and look at our show notes, we’ll have a link to that post, which is an excellent discussion around some of this. So “confounding” is when unobserved factors that can affect our results are mixed in with the treatment that we wish to test. A classic example of potential confounding is the effects of education on future earnings. While people who have more years of education tend to have higher earnings, a question economists have asked is if extra education drives earnings or if natural ability, which is unobserved causes both the years of education and the amount of earnings people receive. If we’re able to randomize which subjects are assigned each treatment, we can break or block the effect of unobserved confounders and we can make causal statements about the treatment on the outcome of interest.
0:36:15.0 MH: So there you go. Present, per mitigate, confounding. Alright, so thank you both, Mo and Tim, for your thoughts on that Conductrics quiz. Remember Conductrics is there for you if you need help with software to do AB testing, adaptive optimization and predictive targeting. And let’s get back to the show.
0:36:37.1 MK: My understanding, and you guys can correct me if I’m wrong, because I definitely don’t know as much about this space as you guys, was that those big platforms like Google and Facebook are moving more to the black-box. They’re not like, we’re gonna find this perfect audience based on demographics, it’s like, You tell us what to optimize for, you give us the best signal you possibly can, and we will build a model to optimize towards that. We don’t care if it’s a woman or a man, if they’re 22 or 36. We just want to know what the best signal is for our model, and then you put it in the black-box and are expected to trust it.
0:37:13.2 TW: Well that’s one thing if it’s a… Well, it’s still a little problematic. If there’s a conversion where you can pass right back to them in near real time that they… If it’s a low consideration conversion, still problematic, they still might be going and grabbing people who were going to convert anyway, like that’s still problematic. There’s still an enormous amount of awareness advertising for the CPG brands that I think are still saying, “We’re gonna advertise to the audience that you want to target”, which I… I mean it needs to happen. Yeah, I wanna advertise to people who might buy from me.
0:37:52.3 MK: But then isn’t that why you do experimentation? To make sure that…
0:37:58.2 TW: I hope so.
0:38:00.7 MK: Whatever you’re passing back is actually adding incremental value and not just capturing users that were gonna come anyway? Oh my God I feel like we’re going…
0:38:05.2 TW: Yes.
0:38:06.8 MK: Down a massive… We’re going off-road.
0:38:08.1 TW: Well, but all of that, I think there’s getting more… There is more skepticism now than there was when the podcast started, and yet at the same time, there is more hype and promise around machine learning and the black boxes. So it’s like… It’s just an escalating battle, I think.
0:38:23.2 JC: Well all we’re really doing is re-stating how people did digital marketing in 2008, ’cause that’s what we’re going back to. Right? Like contextual marketing, that works, but no one did it because targeting an individual was more effective, right? Going after audiences of people similar-ish to people that our first party have bought, that’s what they… Like we’re… I’m trying to figure out a way to work in, we’re gonna first party like it’s 1999, ’cause I love Prince, but I haven’t been able to make it land yet. But anyways, it’s just…
0:38:58.8 TW: You keep workshopping that. Now, let’s change our attention to one part of the industry. We’ve been talking a lot about what things have been challenging. What about something that’s going 100% right? And that is talent and how people are working. I’m just kidding. That’s fraught with problems too. [laughter] But analytics talent has not gotten less expensive in the intervening years, not by a lot. In fact, maybe for about three months, during the middle of the pandemic, March, April and May of 2020, you might have been able to get a analyst on sale, but after that, I think it snapped right back to full price and the amount of demand for analytics people has grown so much. I don’t know. I’m curious of all of your thoughts.
0:39:52.6 JC: Sure. I think some of it is COVID and this whole concept of the great resignation a little bit. And you see it… My wife’s a massage therapist. They’ve gone through nine receptionists just because they keep getting offered more money ’cause there’s jobs for days in the service sector and your job is sometimes the only thing you can control when you can’t leave the house and you’re stuck with your kids for 18 months. And so some of it was that. [laughter] Not that I’m dwelling on it. But some of it’s that. And I think in our particular corner of the world with all the changes to cookies and people trying to in-house data and things like that, we’ve had a tremendous amount of turnover at Napkyn and it was all wonderfully talented people who have gone on to do other cool things. Frankly, it’s salaries we couldn’t keep up with. And every single competitor of ours, like Tim, you work at one of them, I’m sure you’ve seen a lot of turnover over there as well. All the other major GA resellers who are analytics shops are talking crazy numbers. We’re all careful to tell our recruiters not to call each other’s staff.
0:41:00.5 JC: That’s where we’re at in the world right now. We have, I think, three recruiters on retainer, which is just absolutely bonkers. And it’s a great time to be an analyst, right? I’m seeing people with three years of experience being offered 150 grand a year.
0:41:18.1 MK: Are you fucking kidding me?
0:41:19.7 JC: No, ma-am.
0:41:21.0 MK: Geez.
0:41:22.8 MH: Yeah, I mean they have to have worked at those three years at Napkyn, right? So that comes with a certain prestige. Well but what’s concerning to me, and this is where a little bit of a… From the digital analytics to the broader analytics, I think there’s a challenge in that there are a lot of people who, they learn a little bit of how to implement a tool or they learn enough of the lingo of some platform, and then they hop. They think they’ve learned analytics. Whereas to me, the people who are saying, “Wow, this is hard. It’s challenging. I’ve got so much to learn and I’m getting challenged to do it on the analysis side,” are… They can hop as well. Companies don’t seem to be doing a good job of even looking for those people. They wind up saying, “Can you help me implement this tool?” But I do wanna know if, Jim, you still think hiring bartenders is the secret, because there has been a dip in the food service industry and in episode one, you went on at great length about how bartenders made great analysts. And do you still contend that is the case? And therefore, is that a secret place to go find some great analysts?
0:42:39.0 JC: 1000%, yes.
0:42:40.6 MH: And this is why I have a drinking problem today.
0:42:44.3 JC: So interestingly… [laughter]
0:42:45.8 MH: That’s right.
0:42:47.2 TW: ‘Cause you’re recruiting?
0:42:47.3 MH: Yeah, I’m going to all the bars. I’m like, “Hey, let me talk to you a little bit about analytics.”
0:42:52.8 JC: Michael, why did you fall asleep in the basement again? I was recruiting.
0:42:56.3 MH: That’s right. That’s a recruiting expense.
0:43:00.2 JC: We have a team member who has been with us just shy of a year. He’s actually from the Columbus area. Shoutout to Columbus. And he was a bartender before we got his resume and he’d done his GA cert. He had some experience in marketing and stuff, but did bartender leap off the page? Yeah, it absolutely did. I think it’s an unbelievably great set of foundational skills for dealing with people. And he’s actually on our implementation team, not on our analyst team, and he’s great with clients. I think it’s great. One thing I wanted to add to the point that you made too, Tim, is I think that there’s gonna be a real reckoning as the economy contracts over the next 18 months and it kind of breaks my heart ’cause all these talented analysts who, like if someone’s gonna offer you six figures a few years out of school to take a manager job, you take the money.
0:43:46.4 JC: I don’t think anyone’s making… You take care of yourself and your family and your career. But a lot of these new analyst jobs that were created were for organizations that don’t have a culture of measurement. And I think the amount of pressure that’s being put on very young analysts to perform at a very high level is very high. And I think when HR budget starts to drop as we see the economy starts to get weird in the next year or so, you’re gonna see a lot of really talented people that grabbed that ring, and unfortunately, maybe shouldn’t have. I don’t know if that’s nuts, but…
0:44:16.8 MH: Yeah, I think there’ll be contraction all over, but I do… I mean, the one thing that’s always been true about analytics is it works counter-cyclically too. It doesn’t mean that we’re not gonna see some of that, like you just said, Jim, because the expectations are super high, and at the same time, the organizations that are making those expectations don’t have a culture that’s gonna support execution. So I like that point quite a bit. And it’s worth thinking about quite a bit, if you’re an analytics person and looking at the job market, I always try to tell people these days, think about the team you’re joining, not the company you’re joining. What’s the team like that you’re gonna be on, who you’re gonna be working with? ‘Cause that’s gonna be more indicative of somewhere you’re gonna enjoy versus somewhere that’s just gonna be trying to use you for your talent, and then throw you aside.
0:45:05.4 MK: That is like the opposite advice I give people, like the polar opposite.
0:45:11.5 MH: Really?
0:45:12.4 MK: Yeah. And I used to be a bartender, so that’s clearly why I was meant to be an analyst. But…
0:45:17.0 JC: Yes.
0:45:17.3 MK: Yeah, no, I actually say like, “I don’t look for the job, I look for the company.” I’ve worked out that if I pick a company where I’m aligned with the values and through the interview process, I feel like they like live those values, I don’t really give a fuck what the job is as long as it’s in data, and then I find a way to make the role what I want my job to be over time. And every time I’ve used that method, I’ve been super fricking happy. Because I think the problem is teams change, but if you pick a company that’s really good and that culture stays on, regardless of turnover in the team or moves within the business, there’s always gonna be little pockets of the business that are shit no matter how amazing the company culture is, but…
0:46:00.2 MH: Well, to be fair, Moe, I’m currently attempting to pick both for myself.
0:46:05.7 MK: Aww…
0:46:06.3 MH: So… [laughter] Anyway, okay, this has been great. It’s been a lot of fun. And we do need to head to a wrap. But Jim, always insightful. As always, it makes me misty-eyed, not really, but sort of, like reminiscing, and you obviously have a very clear view of the industry and where it’s headed, some things never change and everything always changes. And so that’s kind of like a kind of a unique recipe for analytics, I guess. But anyway, one thing that also never changes, although I don’t know if we did this… I’m trying to remember. Tim, you probably didn’t look this up in your, “When did this come out?”
0:46:49.7 TW: I did not.
0:46:52.1 MH: When we started doing last calls on the podcast, but anyway, we do a last call, just something our listeners might find interesting, Jim, you’re an original co-host and our guest for this 200th episode, do you have a last call you’d like to share?
0:47:05.5 JC: Other than how much absolute joy it brought me for you guys to invite me to this, Moe, you’re such a better addition than I am to this triumvirate and it’s been so amazing to listen to you guys for the last couple of years. Very, very cool. And thank you guys so much for letting me kinda join the milestone episode.
0:47:21.5 MK: I’m blushing, but it’s not true, but I’m blushing. [laughter] Also, maybe this is why I have such a like, not phobia, what’s my weird thing about how I think I’m not funny because now that I’ve hung out with Jim, I’m like, “Oh, now this is why I have a issue with the fact that I’m not funny because I had big shoes to fill”?
0:47:43.1 MH: Oh well. Anyway, okay, what about you, Tim? What’s your last call? [laughter]
0:47:52.0 TW: I don’t know what to say. You are funny, Moe. You are very funny.
0:47:56.5 JC: We should all get together somewhere, maybe some time in the next month or so where there’s a lot of bourbon and we’ll just hash this out in person.
0:48:06.2 MH: I like it. We will have hashed this out in person by the time people hear this.
0:48:11.8 TW: Yeah, we will have done the coaching, so we’re gonna re-do that Moe and say, “Thank you Jim, for teaching me.” I was gonna say something about, “Well, Moe, with Jim, we laugh with him, with you, we laugh at you,” but that was… That’s just like…
0:48:25.7 MH: That doesn’t sound…
0:48:25.8 TW: Too mean and it’s not true.
0:48:26.2 MK: But it’s kind of true.
0:48:29.6 TW: We’re gonna get angry emails about that now.
0:48:31.2 MH: We have not gotten an angry email about me being a dick in like two weeks. So I just wanted to make it easy for somebody. They can mail it in.
0:48:39.6 MK: Oh dear.
0:48:40.5 MH: What’s your last call, Tim?
0:48:44.2 TW: So I’ll go with a, pretty far off-topic, but I did wind up binge watching or binge watching, binge listening the first two seasons, it’s a podcast called, World’s Greatest Con. It doesn’t really have much to do with analytics, but season one was all about…
0:49:02.3 MH: Yet.
0:49:03.2 TW: Operation Mincemeat. That season came out before that was on Netflix, but it’s about a World War II kind of subterfuge, the Haversack ruse on Hitler. And then season two was all about game show cons. And at the end of season two, they did like listener questions and one of the questions was the obvious like, “Wouldn’t the world’s greatest con be one that they didn’t get caught?” which they acknowledged. So, the tie to this podcast is that I think the overall name of the entire podcast was not super well-thought-out and I think we have often felt that way about the digital, no longer digital Analytics Power Hour. But it’s a fun podcast to listen to if you’re looking for fun stuff to listen to.
0:49:52.6 JC: Season three of the Cons will be all about Adobe test and target.
0:49:57.8 MH: That’s season three. “CDPs, are they the world’s greatest con?” Alright, Moe, what about you? What’s your last call? And make it funny. [laughter] Sorry.
0:50:19.3 MK: So, I keep it… Somehow coming across all these different articles with data visualizations that move, and I’m not gonna lie, most of them have been about elections, but I find it really fascinating. And so here in Australia, we had an election a couple of months ago, and basically the argument is that the political landscape in Australia is changing significantly from having two major parties to now having this group of independent candidates that are called the teals, mainly women who are advocating for climate change, and we’ve seen this really big shift in the traditional groupings of politics. So there was…
0:50:58.5 TW: They’re advocating for climate change? We don’t need advocates for climate change. Climate change is happening just quite well on its own. Sorry, carry on.
0:51:04.6 MK: Advocating for doing something about climate change. I haven’t had coffee. Be kind.
0:51:09.6 TW: I’m sorry. [laughter]
0:51:14.5 MK: Anyway, there was this really incredible article by the ABC, which I’ll share, the ABC in Australia, which is our independent news broadcaster. And just… I was really interested in watching how they had some very complex graphics, but managed to walk the audience through them in a really smart way. And I thought it was a really great example for like how to do complex data visualizations, but still take the audience on the journey.
0:51:41.2 TW: Is it one of the scrolling ones where things change, or is it more… Or you’re just looking at…
0:51:46.7 MK: Like as you scroll, it shows you like the graph before in 2016, then ’19, and then now. And so, it shows you how different electorates have changed. But like I said, it is quite complex.
0:51:57.7 TW: It seems like those introduce the narrative component to the… I’m always amazed when I see those because they’re like, clearly they thought through, “This is what I’m trying to show and build,” and it does seem like a… Those done well are amazing. I’m looking forward to checking that one out. Long live Scott Morrison, or RIP or whatever. I don’t know. Fair thee well, Scott Morrison. What’s your last call, Michael? I’m just pissing people off right and left. So, jump right in.
0:52:32.2 MH: I thought you’d never ask. Well, no, so I was reading a blog, which is a thing I do from time to time. It’s weird because blogs… Talking about 2008, right, Jim? That used to be a big deal. Now we read Twitter, or at least I do. But I’ve just enjoyed a number of their blog posts, it’s from an agency called Common Thread Co., which I think they kind of focus on e-commerce a lot, but marketing stuff. But I’ve actually just really enjoyed their blogs and… Not everything, I can’t speak for all of it, but there’s been some really good ones just about sort of e-commerce tips and things like that and state of the industry. So we’ll link that in the show notes. You can take a look if you’re into reading blogs like I am. It’s called Coach’s Corner from Common Thread Co. Alright.
0:53:22.0 JC: Like the famous hockey coverage in Canadian history, Coach’s Corner? That’s what they’re calling it?
0:53:29.5 MH: I don’t know if they’re… Yes, that is exactly right. Calling back to the most famous hockey coach in Canadian history…
0:53:37.1 JC: Don Cherry. Not… Not a cool human being, but he did do Coach’s Corner.
0:53:41.1 MH: No. Okay.
0:53:45.1 TW: So, who will have a better NHL season, Ottawa Senators or the Columbus Blue Jackets, on this upcoming season?
0:53:52.5 JC: If the Blue Jackets’ plane crashes, I’m feeling pretty good about Ottawa next year.
0:53:54.2 MH: Okay. [laughter] Alright. Well, 200 episodes in and we’re still doing whatever we want on this show. But you know what we do like is we like hearing from you, so if you’ve got a comment, question, show idea, rating, review, anything you wanna tell us, we’d love to hear from you. Best way to do that is on the Measure Slack group, or you can also reach us on Twitter or on our LinkedIn page. Jim, are you active out on the socials much at all? Do you wanna share a Twitter or anything like that, people can follow you?
0:54:27.3 JC: @HandModel1978 on Instagram.
0:54:31.3 MH: Okay. I’m gonna look that up after the show to see if that’s actually real, but I don’t know where it’s gonna take me.
0:54:38.1 JC: Because I know how these platforms work, I don’t have any social medias other than the LinkedIn. So feel free to come say hi to me there.
0:54:43.8 MH: Alright. So go talk to him on LinkedIn, a great place as always. And of course, no show would be complete without a shoutout to our excellent producer, Josh Crowhurst, making the world go round. Another thing we didn’t have in those first couple of years, Jim, was a producer, which I’ll tell you what, it’s pretty luxurious. And anyway, but big shoutout to you, Josh, thank you for all you do. And I know that I speak for all my co-hosts, both present and past, Jim and Moe and Tim, no matter the state of our industry and what kind of stupid tools that are getting bought in to your company, remember this, you can’t hurt yourself by keeping on analyzing.
0:55:33.0 Announcer: Thanks for listening. Let’s keep the conversation going with your comments, suggestions and questions on Twitter at @AnalyticsHour, on the web at analyticshour.io, our LinkedIn group and the Measure chat Slack group. Music for the podcast by Josh Crowhurst.
0:55:50.6 Charles Barkley: So smart guys want to fit in, so they made up a term called analytics. Analytics don’t work.
0:55:55.6 Tom Hammerschmidt: Analytics, oh my God, what the fuck does that even mean?
0:56:06.2 MK: This is really fucking annoying because I have something to say, but like between the radio that is now playing at the front of my house and the intense hammering…
0:56:13.8 TW: None of us can hear the hammering Moe.
0:56:17.8 MK: Oh, it’s stopped. Give it about three seconds. As soon as I start to say some thought, it would be very, very loud. But you can hear the radio, I’m guessing.
0:56:24.0 JC: Do you know what intense hammering means in Canada? I’m very offended. [laughter] I don’t know what that means in Australia, but whoo!
0:56:35.6 TW: Are your parents visiting again, Moe? That’s why it was important that I hit the record button before you started relaying that.
0:56:43.7 JC: Yeah, so we can put that in the outtakes…
0:56:44.1 MK: Fuck you, Tim! [laughter] You seriously, Josh, you better not put that in the outtakes.
0:56:53.2 TW: Oh, but the “Fuck you, Tim” can go right in.
0:56:56.3 JC: Yeah, that’s perfect.
0:57:02.8 MK: So like, are you still drinking cran juice and… It looks like coffee now.
0:57:06.5 JC: I just poured it into my empty water glass so you could behold it in all of its Canadian…
0:57:11.2 TW: Yeah, that looks like a Clamato.
0:57:12.3 JC: An alcoholic ceviche.
0:57:16.5 TW: Alcoholic ceviche. [laughter]
0:57:19.2 JC: Which is also my drag name, which is convenient.
0:57:24.7 TW: That’s perfect. Ladies and gentleman, coming to the main stage, Ceviche!
0:57:33.5 MH: Okay.
0:57:34.2 TW: Do we know how we’re doing this?
0:57:37.1 MH: Yeah. We’re gonna kick it off and then we’re gonna start talking and in about 45 minutes, we’re gonna stop talking. [laughter]
0:57:47.4 TW: There we go.
0:57:48.3 MH: It’s easy. Jim and I have done pitches on less info than that, so we’re ready to go.
0:57:58.0 JC: Go work the front row of the audience.
0:58:00.2 MH: That’s right. We’re at the front row. None of us heard that, Tim. Yeah, you’re mute… Your mic is dead. [laughter] I don’t know what just happened.
0:58:11.8 MK: What a fucking debacle! What a fucking debacle!
0:58:15.3 MH: Why don’t you just type it… Why don’t you type it in chat and we’ll get Jim to sing it? [laughter]
0:58:23.0 TW: Rock flag and enterprise fondue pots…
Subscribe: Google Podcasts | RSS
This site uses Akismet to reduce spam. Learn how your comment data is processed.
https://media.blubrry.com/the_digital_analytics_power/traffic.libsyn.com/analyticshour/APH_-_Episode_220_-_Product_Management_for_Data_Products_and_Data_Platforms_with_Austin_Byrne.mp3Podcast: Download | EmbedSubscribe: Google Podcasts | RSSTweetShareShareEmail0 Shares
Subscribe: Google Podcasts | RSS