#116: Analytics Education at Scale with Justin Cutroni from Google

[music]

00:04 Announcer: Welcome to the Digital Analytics Power Hour. Tim, Michael, Moe, and the occasional guest, discussing digital analytics issues of the day. Find them on Facebook at Facebook.com/analyticshour. And their website analyticshour.io. And now, the Digital Analytics Power Hour.

00:27 Michael Helbling: Hi, everyone. Welcome to the Digital Analytics Power Hour, this is episode 116. You know, not many people can see it but there is this imperceptible line that you cross in a product’s life cycle where you have to make some decisions about how people are actually going to figure out how to use your product. You know when you’re a startup or you’re really small, you’re just trying to provide decent documentation at least, or hopefully, users figure things out on their own, but as you grow, you need to make sure people know how to use your product. It’s no different in the analytics space. I mean there’s a plethora of tools and technologies that we all use now, and knowing how to use them well can seem more than a little daunting. So that’s what we’re gonna talk about. Analytics education at scale. Hey Moe, you have a stand-out analytics education moment?

01:21 Moe Kiss: I do actually. It was so bad. It’s when I got my first job in digital analytics, and I had to pass the GAIQ exam and I thought that I spent like a day convincing myself I was gonna fail. And now I’m like, “Ugh, honestly.”

01:34 Tim Wilson: Wow, look at that. Kissing up to the guest. Well done.

01:38 MH: Okay, Tim. Hey, let me finish my intro. Tim.

01:41 MK: That was legit my first one.

01:43 MH: No. That was really good and I think Tim, you shouldn’t, you should treasure that experience.

[laughter]

01:52 MH: How did you learn your first analytics tool, Tim? Let’s turn the tables.

01:56 TW: I’m pretty sure it was four years before Urchin was acquired by Google Excel. How I learned SPSS NetGenesis, it was completely bumbling around with whatever, I think it was like a Windows help file documentation, was probably how I was learning it, by hook or by crook.

02:15 MH: Yeah, and I’m Michael and I learned Webtrends by documentation and the Webtrends forum. That was my main method. Okay, but we need an expert. Somebody who’s had some time to think about this problem of scaling education at scale, maybe in the some of the biggest ways ever. So Justin Cutroni is the Director of Education for Google’s Apps, Analytics, Display, and Video ads business. And he’s held a number of other roles inside of Google as well. Prior to that, he was with Cardinal Path, and before that EpicOne. But today he is our guest to share some of his insights in building educational capabilities for a little tool called Google Analytics. Welcome to the show, Justin.

02:57 Justin Cutroni: Hello, it’s good to talk to everybody.

03:00 MH: Well, it’s great to talk to you. And it’s exciting ’cause I think, probably Google Analytics is a tool that probably touches almost every single listener we have. So it’s pretty cool to get a chance to have that kind of reach, and it’s been around now since what? 2005, when it got re-released as GA. And so we’ve all had a little bit of time, but like the educational content has really been something that’s sort of, I would say, sort of set the standard in our industry. So maybe first, to get us started, just maybe a little background on those education programs. Where it came from, where it is now. Stuff like that.

03:38 JC: Sure. So, what’s interesting is there’s not one core Google education team, there’s some areas where we collaborate and we have teams that support multiple products, but for the most part, like the team that builds Google’s Cloud education is different than teams that work on our ads business. And so I got started in education, it was similar, it was like a silo-type thing and I was on the Analytics team. And we started out super basic, and we had a lot of product documentation, but coming from the consulting world, which I think I’m talking to a lot right here, the one thing that always frustrated me about educational stuff for measurement and analytics tools was it lacked the context of all of the business side of our world, right?

04:24 JC: Like there’s this whole rich universe of understanding business strategy, marketing activities, all of that stuff. And so we wanted to create a new way to provide more context around the education, and so we added to the documentation, we added online courses. And those courses were like videos, and they were much more interactive, and they talked a lot more about how when you’re dealing with analytics you really should have your business hat on to think about how all the questions you want to answer, and the data that you wanna have. So, we scaled in that direction and then we started to realize like, “Well, for Google Analytics, a lot of problems that people have are setting it up and getting the data.” And how the hell do you learn about this thing if you can’t even implement it? So we started to say, “Well, we need to build a demo account.”

05:17 JC: So then our education involved building of demos of the products. And then we started to think we’re like, “Wow, people are probably gonna need more than just learning online.” And so we brought our partner program into our educational programs because partners are amazing. They are out there on the ground, and they are working in different languages and different regions where we’re not. So partners in my opinion are a really important tool for our driving education. So we brought them into the fold, and then there’s a bunch of other stuff like in my world we fold developers in as well, we need to educate our developer partners, and how they work on the product. So really it was a very broad view of what education was, when we started working on it with Analytics it was thinking about scale, it was thinking about multi-modal like online versus in-person, it was thinking about getting the right information and the right people the way that they wanted it. So, yeah that’s how it evolved for Analytics and now similar programs exist for a bunch of other products as well.

06:25 MK: So, with the straight away thinking, we have to go to online tutorial videos, was there any discussion ever of in person, or do you think that’s just the nature of the business now, that you need to be able to scale, and people need to able to watch it whenever they want, tap in?

06:42 JC: Yeah, well, when I started, my team was one, two, three, four, people, and so it was scaled right off the bat, and not only was it small, it was also, we’re supporting multi-millions of people, and multiple locations. So, we never really thought about in person. It was straight away, like, let’s go online.

07:07 MK: And so, the Google Analytics team is one of the first to develop this type of training at Google. Was their business support straight off the bat, or did they have to be made a really good case for investing the resources into something like this?

07:22 JC: Yeah, that’s a great question. It was definitely supported by my boss. We didn’t add any new resources when we did this. It was just, I had one person that worked on video production, I had a couple of writers, and I had myself. So we used me as the subject matter expert, and the video team, and that was it, in order to get it done. Now, it’s way different, with lots of different roles and lots of different people and yeah, but it was super scrappy.

07:50 TW: In hindsight, whether it was a genius move or not, you had written so many definitive blog posts as Google Analytics was making its way. I probably personally was probably 25 visits to the site. I think it was your custom variables, when custom variables had rolled out, before custom dimensions and metrics. So, my sense is they either got very lucky, or were just very prescient in getting somebody who actually had spent multiple years writing, “These are the things that need to be taught.” ‘It’s interesting, when you say that, providing education beyond just the nuts and bolts of the tool, I feel like that is often the challenge when you wind up taking somebody who worked on developing the product, and say, now, teach it. And so, then literally their view is they see the world through the lens of this is the way the tool was designed. And I think that is something that happened, and it’s one of the reasons that the education was pretty successful was, I think, from the get-go, there was a… Now, we need to have one… We have to not be too into the product.

09:02 JC: Yeah, no, I think that’s a great call. And every product manager has their own view of things, and what’s important, and I think a lot of product managers sometimes don’t see the entire picture. So yeah, I think that was definitely the way that I approached it, was keeping my consultant hat on and not my, “I’m a Googler” hat. And we try to tow a line where we want the content to be very googly, and we wanted to live up to our brand, but we don’t want it to be marketing. And I think that’s another really important thing because the users, we just wanna be somewhere in between those two voices. We wanna be very open, and very forthright, and call a spade a spade when we have to, and I think that’s a little bit different than what marketing would do with gloss over stuff. We wanted to be super direct, super transparent and think about what they needed for us. I’ll be honest, when we built our first online stuff, we didn’t use any data, it was basically what I was thinking we needed to teach. Now we have so much data, and we’re doing so many things in terms of customization, and targeting of educational content based on people, based on business, based on roles, all sorts of stuff.

10:22 JC: So, it’s still I think good, but it’s really evolved from, “Hey I’m a practitioner. I know what people need”, to, “Hey, I know now that there’s 15 different types of practitioners.” And in certain Geos there is even more, and we need to customize it for all of these people.

10:39 MK: Yeah, that was actually something I was interested in touching on. I feel like content, when you’re online must be quite difficult because, I suppose, the assumption is that you start with the basics, but then there’s always gonna be this group of, I suppose, like power users that need that more advanced content, and how do you, I guess, get there fast enough that they stay interested in the product, and keep using it? Or do they just end up I guess, playing around and exceeding what the training can offer?

11:06 TW: Do the power users need more training or to the power users tinker around and figure it out?

11:11 JC: Yeah, I tend to think the latter. I think there’s definitely a 5% where they’re gonna go off and do stuff, and really, they’re gonna dive into the developer documentation, which is probably our most hard-core content and unravel things on their own. And the goal is to make it easy for them to assess that, “Hey, this thing that I’m looking at, it’s totally not for me. I belong over in this other sandbox playing with this other group of people.” And that can be done by self-selection like, “Hi, welcome to our education. Are you Mary Marketer? Or are you Janet the Coder?” And letting the person then dive into the stuff that’s more appropriate.

11:52 TW: That’s actually… Another just a compliment to Google is the documentation has always been, in my view, really pretty solid. So, it has seemed like when I wind up with some bit of minutia. And again, custom images and metrics are a case where kind of confusing, I assume there’s video tutorials on it, as well, but by the time you’re working in that world, then you really just wanna have a refresher. Like, now what happens with the visitor scope custom dimension? And if that documentation is that’s super useful. I will say I’ve worked with other products where the maturity of the training and the documentation, they potentially are out of sync. And where does, where you do you fit when it… I mean the documentation is a completely different group and that’s just they’re run like gangbusters with thoroughness as well, and so, therefore, they just do a line? That’s not by design, it’s just by kind of… I’m gonna sound like I’m really fawning here, but I really feel do like the Google documentation.

12:54 JC: So, on the measurements, on the analytics side, we have five technical writers, and on like, I also again, AdMob and all of the classic double-click products fall under the stuff that I do. So in total, we have about 20 or 25 technical writers that cover all of those products. And the strategy that we have is that those writers actually sit with the product managers, the engineers, and the UX folks that they support. So they’re there as the product is getting built and they work with the teams to define the vocabulary, and the vernacular of features that are getting built. So they’re there from the genesis and they can see how a feature will evolve. They see the research and the user feedback about why this thing is needed, and what it’s gonna be there for, and how to use it. And as the product gets built, the documentation gets built right alongside it. So we feel like that’s really important, because again, while they’re not a practitioner per se, they’re there seeing this thing evolve, and understanding like, “Why is Google building this thing? Why is this feature existing?” And we actually encourage them not just to be technical writers and do the documentation like you were talking about Tim. They create a lot of the stuff that’s in the product, like the tooltips, and product help, and stuff like that as well.

14:18 TW: That’s funny. In my, I mean early in my career, right before I got into web analytics, I was a technical writer, at a hardware and software company, and we were assigned, I mean I was ultimately in the motion control group. I was actually building installers but also playing with the tool and so I think that National Instruments made a good call. Same thing, I think it was fairly thorough. I wasn’t developing the product, but I was at a point where I’m like, “I need to be able to live and breathe this enough, that I can explain every potential detail of it.” Cause I think that is, from the documentation side, you do really want to have every sort of corner cased, slice, you want every dimension listed, you want every metric, you want every caveat listed. So it makes a lot of sense to have them paired up, cause little quirks are there for a reason, and it’s probably good to have them approximate to the people who are saying, “We have to build this little quirk in.”

15:14 JC: Yeah, and I’ve in my time at Google over the last seven years, like my college motto is “Lehr und Kunst” which is German for “Theory and Practice”, so I’ve always been a huge fan of explaining the underlying theories, the underlying way of things working because I feel like if people get that, then they really can master those etch cases, and those corner cases because they have a little insight into like, “Oh, this is that the data’s processed.” Or, “This is the way that the JavaScript gets constructed.”

15:45 MK: So hypothetically, let’s say tomorrow you go to a brand new company. It’s very small, there’s nothing there. It sounds like education is essential to make a product successful. If you were to do it again, where would you start? Would you just focus first on that really solid documentation? What would be your steps, do you think?

16:08 JC: So, I’m gonna put on my analytics consultant hat, and I’m gonna be like, well…

16:13 TW: Can you find it? Is it in the back of the closet?

16:15 JC: Well, like Joe McGranaghan, I went from practitioner to I’m the consumer now, but it’s… What are you trying to get to with the business? Now, there’s obviously I love the whole holistic idea of education is helping everyone be more successful, but there’s some pretty hard business reasons to have an effective education program. You can do two things that are really important to the business, you can help it make it a lot of money, and you can help it save a lot of money. And so I think a lot of small businesses start with the latter. If you’re developing some time of SAS, and you’re gonna probably start with trying to reduce your support costs. So you’re gonna build up things like help centers, frequently asked questions, you’re gonna aggregate community content so you don’t have to create more content. You’re gonna try to get some advocates on your side that can speak out in public in order to leverage that content cause content is super expensive to create and to maintain but I think the first place that I would go is, “What are our biggest support costs? Now, how can I reduce those by creating educational content?”

17:20 TW: It’s actually funny. So the podcast software that we use, that we’re recording with, this ZenCastr, it’s been interesting that very, very small operation and they have really gone in, I mean heavily on, they clearly are paying attention to, “What are the questions? What are the struggles? ‘Cause little hiccups, and they have very, very thorough knowledge-based articles so it’s, to me, they’re clearly responding to what question did I have to answer four times? I’ve now emailed somebody this, I know what it is, let me pop it into Zendesk.” Or whatever they use.

17:55 MH: Those are integrated now, Tim, where Zendesk will actually pop out your knowledge base based on all the tickets you get in there and stuff like that. And they’re not the only ones, I’m sure.

18:04 JC: Yeah. My team, even with the stuff that we do, my team aligns with our support teams, and our service teams and they sync on, “Alright, what are we seeing for volumes of support tickets on certain areas?” And we’ll, every quarter, we’ll prioritize some stuff to help drive that down.

18:21 TW: So, Moe asked about sort of the in-person, and you said, for the need for you guys to do it at scale, not really ever an option but there are definitely partners out there who are not only providing education in-person, they’re actually selling it. So they’re getting revenue by providing that for, I assume, well 99.9% sure you guys are totally, totally cool with that, but where do you see paid in-person training? Is it just, “Hey, some people need that, some people want that.” And that’s why we’re good with it or where do you see that fitting?

18:56 JC: Yeah, I think it goes back to the user and it’s how people learn, and that’s why you’ll see in a lot of our stuff and other companies’ stuff too lots of different formats of content. There’s video content, there’s documentation and you might have the same piece of information represented multiple times. And again, it goes back to people learn in different ways. I think that’s why there’ll always be a need for in-person training because there are learners that will favor sitting in a room having an expert in front of them where they can go back and forth and interact with that person in order to iterate through learning something. So it’s just a very, very difficult thing to scale, but it’s absolutely necessary. So I think it’s a great thing.

19:41 MH: On that topic though, it is interesting ’cause a lot of enterprise software companies choose to have that be an in-house function and do that training that way, and I think Google is a, I don’t know if you’re unique, but a little bit different than a lot of other companies in that you never really try to establish that model internally. Do you feel like that was a decision that was made as a function of, “Well, the tool is free, so why would we try to create a revenue stream around training?” Or was it just, “We see other people doing it, so no need for us to pop into this space.”

20:14 JC: Yeah, no, I definitely think the standard version of Google Analytics, obviously it’s a free product. So there’s gonna be different levels of investment in different types of service offerings. So there’s that for sure. But I also think it’s just got, even like Google marketing platform, it’s got such a massive footprint with GA 360 and like for us to have the size of service team that we would need, it’s really hard. That’s why we have an ecosystem of awesome partners, everything from Search Discovery to Accenture. Like we really believe that the ecosystem approach is a completely valid approach where if we have partners that we can vet their skills, we know they do great work, we’re super happy to go into deals with them and recommend their service to some of our large advertisers and publishers.

21:07 MK: Okay. So we just wanna rewind a little bit ’cause I’m a complete in-person learner and it’s actually something that I personally struggle with a little bit that I’ll… I try to do stuff online first, but ultimately I find I get the most out of being in person and it is because of that, the band to the ability to ask questions and go back over something in the break. So one of the things that I’ve personally been struggling with a little bit is that I keep going to big events by multiple different vendors thinking that the content is going to be helpful ’cause I’m like, “Why would you spend a whole day learning this thing unless you’re pretty advanced?” And then I head along, and I very quickly learned it’s more of a marketing thing and it’s, well it’s super basic, like here is how you create an account and then I’m like, “Oh, this wasn’t worth a day out of the office,” and then I get really disappointed. I guess as the person trying to figure out what you ordered in person…

22:05 TW: In personal learner but you’re perpetually disappointed by it, so maybe you’re not an in person learner.

22:10 MK: No, actually… Sorry, the best ever training session that I’ve gone to ever, ever, ever, ever was at the Google office for BigQuery. There was a lunch and learn session. It was only an hour and a half and it was super technical. They had like three super technical people there and all of a sudden like all these things just clicked for me, but I keep trying to figure out how do I know when I’m looking at an event that it’s, I guess, ’cause other people are like, “Moe, you should know better than this. This is definitely pitched to like the CMOs that they’re trying to sell tool to,” and I keep struggling with how do I know whether it’s the event targeted to the CMO or how do I know that it’s an event targeted to someone more technical? Do you have any advice on how I can figure out what I should go to ’cause I just think there’s so much on. And so how do you pick, how do you figure out which of the training in-person is actually the right in-person training?

23:02 JC: I think it’s interesting that you switched at the end of your…

23:07 MK: Diatribe?

[laughter]

23:08 JC: Diatribe there from the word event. You switched from the word event to training. And events aren’t training. Training is training. And I think if you wanna go to training, it’s a very different thing than an event. Events are in my opinion marketing.

23:24 MK: Okay.

23:24 JC: Almost all the time.

23:25 MK: Okay, that’s very helpful.

23:27 TW: I think you can also look at who’s organizing it. It’s just based on the Google, on the cloud side. Like I went to an event that was around, I don’t know, it was like TensorFlow and Colab, like it was an event where they were marketing it but they also were very clear they were having a third-party speaker come in and walk through soup the nuts how they were doing something. I think it was a Colab project that they were walking through. I mean it was mostly over my head. So it was one where they were… It was marketing, but they were marketing to the machine learning, the developer crowd, and there was legitimate takeaway stuff. So I think it’s even… I think there can be events that based on who they’re targeting at the event can still wind up giving you something you can walk away with.

24:13 JC: Well, I mean, do you remember, I don’t know, I may be dating myself, but I remember 15 years ago when every vendor in the analytics space, they would have that classic marketing tool where you’d get a one-hour webinar and the first 30 minutes was like a deep dive expert thing and then the second 30 minutes was a vendor marketing pitch, right?

24:34 MK: Yeah.

24:34 JC: You’d have like, “Oh, here’s Expert X,” and talking about something technical, and then it’s like, “Oh, here’s our marketing team for 30 minutes.”

24:42 MK: But…

24:42 TW: I remember them as being flipped that it was like we’re gonna get to what you actually want, but first, you’re gonna have to…

24:47 JC: Yeah, yeah. Start with the PowerPoint and then do the demo after.

24:51 MH: And maybe that was it.

24:52 MH: But actually, our industry has to include education and marketing because we have to educate people on what to buy. So that’s a little different than more well-established professions that have been around for 100 years versus 25 like that.

25:07 JC: Well, I mean, are you… So are you talking… Are you talking about just like educating people about the industry and why they need to care about education, or care about like analytics or…

25:17 MH: No, like educating why you should care about this differentiated feature of our product and how it will affect your business, those kinds of things. So like there’s a function of education that is also marketing.

25:28 JC: Well, so that’s really interesting, right? Because I think that’s marketing, and one of the ways that I…

25:33 MH: Yeah, I agree.

25:33 TW: I agree.

25:34 JC: Yeah, I mean the way that I… ‘Cause I sit right next to our marketing teams because our marketing teams tell the why story. My team and education tells the how story.

25:45 TW: That’s a good way to think of it.

25:46 MH: Yeah, and that educating on our new features, to me, it’s marketing and you’re getting such a… You’re getting completely one-sided view of what the feature and the value and the benefits. So I hope people are going to those with a high degree of skepticism, not being like, “I heard one side educating me on what this awesome functionality is. I must go buy that.” I don’t know.

26:14 JC: Yeah, to me, it’s… And that’s the way I’ve been trying to draw the line and the content that we… The line in the sand that the content that we create is like our content is about the how. Now, in our how story, if we have an hour, we might have three minutes of why at the beginning, which I think is really important to set the context, but I think marketing’s job is really the why. They’re there to differentiate the feature, extol the feature, and we’re there to tell people why to push the buttons or the best way to push the buttons.

26:47 MK: But if there is a use case ’cause as you guys are talking about this, I’m like, “Wait, is that marketing?” I obviously have a totally uninformed opinion, if everyone else is agreeing, except me, but I always think about the context of a use case. If someone is like, “Here’s something that you can do, a piece of analysis that you’re interested in, and this feature will let you do it this way with these parameters,” and yada, yada, yada, that’s where I get the most value. It’s like, “Oh, now, here’s something I can go and do with this.” Or it’s something I can go back to my CMO and say, “Hey, this is why we need this tool because it will allow me to do this.” And so, I feel like it can be a blurry line.

27:24 JC: But it doesn’t tell you how to do it. If it doesn’t dive deep into the details and say like, “All right, well, first, you need to create a data connector. And then once you create the data connector, you need to drag this widget into the workspace. Then you need to apply the filter.”

27:41 MK: Oh no, I totally… I apparently want that, too. I’m like, “Now, tell me how to do it so that I can actually go and do it.”

27:47 JC: And I think that’s where education comes in. It’s the totally telling you how to do it.

27:51 MH: Okay, so what role does the community play for you in that? Because there is a huge community, and they’re… Up until recently, there was a massive Google Plus GA group that would share knowledge. Google then mercilessly…

28:08 TW: What happened to that Google Plus group?

28:09 MH: Cut down it’s… No, this is…

28:11 JC: I don’t work in that part of the business.

28:13 MH: Pour a little bit out for it. Rest in peace. But there’s also the Measure Slack today, which is becoming a place. And back in the old, old days, which we actually just realized, that still exists, is the Yahoo Web Analytics Forums back when we were all kids. Yeah, exactly, throwing it back. And then there is sort of the individual gravity well that is Simo Ahava in terms of the content that he produces and educates people on. There’s a whole, almost community around just his content.

28:42 MK: There is literally community around his content.

28:44 MH: Oh, yeah. So talk about that a little bit, the context of how you approached that because I think that’s also interesting for people who wouldn’t know how to balance or look for new opportunity, even.

28:53 JC: Yeah. Well, I think that’s… And I think any part… I think with an educational effort, it’s an ecosystem play. And so, obviously, we said there’s some standard stuff you need to do in order to support some very definite business objectives like reducing service costs. But the ecosystem play is great because it can handle a lot of those edge cases that Tim might have mentioned. So if there is a gray area that’s very unique, it may not get as much attention from the stuff that we create, but there’s a thirst for that. And especially a product like Google Analytics, there’s always been a very passionate user base, and that’s where the ecosystem of experts comes in and should be encouraged. And I know that for us, we make a conscious effort to share that content through our social channels to promote that content because it is good, valuable content that fits the needs of a certain segment of users. And for us, it’s very intentional that we promote that stuff that people are making, like Simo and lots of other folks out there.

30:04 TW: Well, even, if you take Jeff Sauer’s, the Jeffalytics, that it’s in… He’s actually delivering content in the same format. But presumably you’re saying, “Look, if it’s meeting a need that you guys aren’t meeting, but it drives them to use the product, then good on you.”

30:21 JC: Totally. From our perspective, someone signing up for Jeff’s courses is probably gonna have a very different experience. I don’t know the details, but I’m sure there’s a lot more interaction with Jeff. I’m sure there’s some type of a little bit more opportunity to answer questions or whatnot. That’s something that doesn’t really happen with our stuff just because of the scale of it and our ability to provide that type of experience. But at the end of the day, our goal, whether it’s our content or the ecosystem creating the content, is for people to push the buttons better. We want them to be successful in their jobs. And again, Google is more successful when people use our products better. If it’s Analytics, if it’s AdMob, if it’s part of the GMP like DV 360, that’s what we want. We want more educated users that use the products better because they’re gonna help the other side of the equation. The education will help better product usage, which in theory, should drive growth of our business.

31:24 MK: Okay, so just one thing, though, you have all these, I guess, advocates out there that are talking about the platform that are filling this need. It’s also really tough, though. What if they suggest stuff that you disagree with from a, I guess, “This is the best way to use the product,” or best practice…

31:42 TW: I don’t think Simo has ever recommended hacking the tool and maybe using it in a way that maybe it wasn’t exactly. That’s a good point.

31:49 JC: Nobody’s ever done that before. I don’t know anything about people…

31:52 MH: I mean, there’s whole products built on top, right, so Stephane Hamel Da Vinci tools, which changes the way the product works.

32:00 JC: Yep. Yeah, I know. Absolutely. And I think, again, ultimately we want people to do their jobs better, and so if people are using the product in different and unique ways, as long as they’re not breaking terms of service. As long as they’re not doing things that are against our relationship with that user, then that’s totally fine. If you’re storing first name, last name, e-mail address, in Google analytics, then we’re gonna be a little bit bent out of shape on that. But if you’re figuring out some way to create primary keys and store them in systems, and then join those in BigQuery, like good on you.

32:41 MH: So, if we’re talking various of these discussions, this probably goes back to when you were talking about the different forms. Online versus in person, different needs, and this whole topic which obviously is at its core, we’ll wind up, we’re focusing on the Google analytics ecosystem scale, but that easily broadens to other broader analytics products. But it’s occurring to me that it actually is equally applicable to enterprise analytics programs, and a lot of the thoughts, like I’ve watched and worked with companies that are saying, “Hey we’re rolling out this new platform.” And they figure out how they’re gonna train all the users at launch with putting 0.2% thought into how do we handle the person who starts and needs to use the tool a week later? How are we giving people a refresher? What about the people who sitting and watching a video isn’t gonna work, they need some hand-holding? And a lot of what we’re talking about here to me seems like wow, if you actually are a small group who’s rolling out a platform in the organization and it’s again possible, I’m thinking about one [chuckle] specific scenario right now, which this is really hitting home with me.

33:57 MH: That idea that you know what, you do need to deliver the same information through multiple channels, in an on-demand format, in a documentation format, in an in-person format, in a consultative format. Those are all fine. Trying to design, we’re going to… And even to the partners, the super users. Who in the organization is really jumped on board, they’re in one group that is one user, but they have become really engaged with it. How do you take advantage of them, as Evangelists, as pushing the tools, providing content. So, that was just a… Did not have that thought coming in to this discussion, but it seems like a hundred percent it applies.

34:40 JC: Yeah. I think I followed you, and I think kept with it.

[laughter]

34:47 MH: Now you’ve had the true analytics power hour experience.

34:51 JC: I think I may have even convinced you of something, but yeah. I mean, there are some… Obviously it depends on the size of your company, right? And again, what are your objectives? And so, big global corporations, like their go to market teams will probably think a lot about education. Product…

35:10 TW: They should.

35:11 JC: They should. Like, they should be.

35:13 TW: Right I mean, a lot more, it’s like hey we bought the tool, we implemented it, and we’re already are over budget and a little extended on our timeline, so we’re done. Why are insights not emerging? Because they didn’t. But, that was the cynical Tim.

35:26 JC: Oh, really? Cynical Tim exists? I didn’t…

35:28 MH: Tim educates out of frustration usually.

[laughter]

35:37 JC: I know exactly what you mean. It’s like, I’ve had many of those moments.

35:42 MK: Yeah, but it’s funny actually. Because this has been coming up quite a bit. As we talk about, I guess, how do you immobilize people within an organization, get them to pick up a tool, whatever it is. As we see the transition from data warehouse to data lake, one of the conversations that I’ve also been privy to, which I’ve found pretty frustrating, is the, “Oh, we’re gonna put everything in a data lake, so therefore we don’t need to document.” The idea is that everyone in the business learns it inside-out, and back to front by playing around and getting stuck into it, and so documentation is a waste of time. And I recoiled with shock horror, because why would you have someone play around for three hours to figure out what a particular metric is?

36:26 JC: Some people don’t learn that way. Some people don’t learn by playing around. That’s not their modality. They learn by sitting down and reading a book.

36:34 MK: Yes. So okay, great. Now I feel justified. This is good. I’m still on the I do believe in documentation, but… Yeah. I’ve been part of some very heated discussions.

36:44 TW: You’re okay with putting comments in code as well? ‘Cause you had that brief tangent where you were saying that was the new thing too, was if your code’s written well enough.

36:51 MK: The new thing is to have no comments in code, because your code should be good enough that anyone reading it can figure out what you’re doing. I believe in comments in code. Well ’cause what if you’re the new person? How the hell are you meant to learn if you don’t know what that thing is yet? You’re gonna go, you’re gonna look it up?

37:06 MH: But that does bring up something, and I wanna change gears a little bit, to make this a little more closer to home. So, oftentimes, at least in my experience and other people, you are dealing with people who are brand new to our industry, and let’s paint that in a broad way. Not just digital analytics, but even some of the other areas that you even functionally cover within the world of Google, and things like that. Coming directly out of college, perhaps. Or in a couple of cases, even had people coming from different careers, and making the switch over. If you were saying to… Sitting next to a person like that today Justin, what advice would you give them about how to shape their educational path as it pertain to… You can keep it within the world of Google if you want, or you can go more broadly in terms of analytics today. Where would you… How would you assess, and then create a plan for them?

38:02 JC: Ah, man. Now I’m creating personal learning plans for people. I wasn’t expected to do that on a Tuesday night.

38:08 MK: Yeah, but the thing, I’ve already figured out my next event/training session, and what I should go to and what I should say no to, so this is super helpful.

38:16 TW: Justin, just tell them to reach out to you on Google+, and you will give them a personal learning plan.

38:21 JC: My hourly billing rate is under four figures, so…

38:23 TW: Subscribe to my blog on Google Reader, and you’ll see. And then connect with me on Google Plus and it’ll be free, it’ll be a free personal learning plan.

38:33 JC: So I guess, I think speaking about analytics specifically, if I was in a job, the first thing that… One of my favorite books is the first 90 days. I don’t know if anyone’s ever read that, The first 90 days. Good book to read if you ever get a new manager or if you ever start a new job. And it’s basically, “Make sure you know what your boss wants.” [chuckle] That’s the gist of it. But if I was in a new job, new in analytics, I would first of all get a clear understanding of what is expected of me from my boss like what do they expect me to do? And then I would start to figure out like, “Okay what do I need to know in order to do that?” And in my looking back through everything that I’ve done, I would always start on the business side of things. And I always start to think about like, “Alright, how does this expectation that he has of me… How does that fit into the business?” So if I’m a new analyst and I’m doing something measurement-related he’s gonna ask me for some data or some insights, and I’m gonna wanna have a super clear understanding of like, “How does that make him look good? How does that help him do his job?

39:38 MK: Or her.

39:39 JC: Or her. Sorry, thank you very much. And I would also understand the organizational layout. So, how does my boss, how does she interact with her peers in order to have the business be more successful? And then I would start to dig into, “Alright, how do I solve these problems? How do from an analytics perspective, if it’s a tool, how do I get this tool to work for me?” And I would look at lots of different sources of information. I think, again, everyone should know how they learn. I’m a tinkerer, I like to play with stuff. I get a very high level overview, and then I’m like, “Let me kill myself with this thing, let me shoot myself in the foot,” and I wanna play with stuff. So I would go down that path. Someone else might sign up for an online course, someone else might sign up for specific training. Adam’s training or something if they’re using one of Adobe’s tools. But I think I would tie everything back to what is expected of me, what do I have to deliver. And then more importantly, like how do I get that done?

40:44 MH: Oh I think that’s good, ’cause I think you said a couple of things that really stood out to me in that actually some of the highest priority things were actually more soft skill-focused than hard skill-focused in terms of how you might approach that. So I think that’s pretty interesting Tim that not just hard skills are important, yeah.

41:05 TW: Can we talk about emotional intelligence for a little bit?

41:08 MH: Oh.

41:09 MK: Okay, I do have one more tangent I wanna go down before we finish off.

41:14 MH: You are on thin ice Moe. You go, we’ve got maybe a few more minutes.

41:21 MK: Okay, so I was listening to a podcast the other day, and one of the things that they were talking about was open-source and how open source is incredibly difficult to maintain. One of the issues is documentation. I suppose, just as someone that works in this space what do you think is the path forward for Open Source? Because listening to it, it was like only people that are super new and motivated are really gonna be able to contribute to open source, and eventually it gets to a point where all of the experts are so busy with their high-paying super amazing jobs that they don’t have the time or the will. And the maintenance is really tough. And I suppose I’m just curious to hear your perspective about where you think that fits in because, obviously we’re talking about Google has heaps of resources, but for those open source tools there often isn’t any.

42:11 JC: Yeah, and I think… And I’m just riffing off the cuff here, but if I had to take a guess, I would say anyone getting involved with open source code is probably a developer or probably pretty technical, and I think that’s an area where…

42:29 TW: Or a mess because they were recovering analysts maybe?

42:34 JC: That might be it, right, yeah exactly. But I think that’s where… A spot where like… We were talking about developers, and developers going right to the really hardcore documentation. I think the open source community in general tends to be technically adept. And so, would it be great if there was really good documentation or other educational stuff? Sure but does the audience actually need it? Maybe not, and then maybe even go on one step further, like what’s the business objective of open source? It’s not really there to make a ton of money, it’s not really there… They’re not really worried about their support costs they’re worried about the project growing, getting adopted getting implemented, getting used. And again, the audience that’s gonna do that probably can figure it out.

43:28 MK: Yeah that’s fair, that’s fair. I just always wonder like how closely is adoption tied to some type of education, whether that is documentation or learning? Like to grow, is that necessary?

43:39 JC: Yeah, so that’s an awesome question. And that’s actually how we measure some of our education. One of the things that I’ve pushed the team to do over the last three years is literally create attribution models so that we can understand, “Okay if someone reads a piece of documentation watches a video or take a course here and then they enable a feature seven days, 10 days, 30 days later. We actually think about our success in education based on those types of attribution models and so, I absolutely think that feature adoption and configuration is something that you should measure as part of your education program. And it’s all on that side of driving usage and revenue in the business versus obviously the whole save the money part.

44:24 MH: All right, we do have to start to wrap up, but this has actually been really an awesome conversation. Maybe ’cause you’re an awesome guy. I don’t know, it’s weird.

44:34 JC: Oh, were you talking to Tim? You’re talking to Tim.

[laughter]

44:38 MH: But one thing we do like to do on the show is what we call Last Call. We go around the horn and talk about something we’ve found that’s interesting that might be interesting to others. So, Justin, you’re our guest, do you have a last call you wanna share?

44:51 JC: Yeah, so my journey… Like, I’m basically management now, which is fun, scary and all sorts of things. And, as a manager and a leader, one of the things that I’ve embraced a lot is vulnerability and the works of Dr. Brene Brown. She writes a lot about…

45:09 MK: That was just it. She’s on Netflix now. She’s famous.

45:13 JC: I was a fan boy way before Netflix, so…

45:16 TW: So are Moe and Michael, ’cause I was like, “She’s on WTF with Marc Maron.” I know that name. Who is she? I’m like, “Oh.”

45:22 JC: No, no. Way before, with the TED Talks, like way before, way before. So, I think, her latest book, Dare to Lead, is great. If you are management or if you’re in a position of leadership, if you have a team, or if you ever interact with other human beings, it’s a lot of good stuff in there.

45:41 MK: I actually just finished it. And one of the things that I most like about it is the fact… And I think maybe this is the mistake that people have about vulnerability, just being honest about everything is not being vulnerable. And I think that the way that she describes it is actually really valuable, especially for someone like me, who is really honest.

46:00 TW: So, my deepest exposure to her was listening to the recent episode of her on Marc Maron, who, he’s funny ’cause he is… It was not a typical guest of his, and I literally went through the whole thing, and she is delightful and hilarious, and I am not following this at all, which either means that I am deceiving myself, or I’m a sociopath.

46:21 JC: Well, so here’s one more thing. All of her stuff comes from data. She’s done years and years and years of research to understand this stuff. So, even if you’re like, “Oh that’s bull crud. I don’t buy vulnerability.”

46:39 MK: Yeah.

46:40 JC: There’s data.

46:41 MH: No, no, no, but…

46:42 JC: There’s data, Tim. Data, Tim.

46:44 MH: Data, Tim.

46:45 MK: Data.

46:49 TW: To be clear, I’m not saying whatever she’s saying is not right or wrong. I’m just like, huh?

46:55 MK: This makes me awkward.

46:57 TW: It’s like…

46:57 JC: It’s just that Tim does not seek to benefit in that way in his life.

47:01 MH: Or maybe… Maybe if we put it into some type of chart, maybe a pie chart for you, Tim.

47:06 TW: Jeez.

47:08 TW: This guy is full of great ideas.

47:09 JC: Actually, you know what, Tim? I’m sorry. I know pie charts are old. We’ll make it the donut.

[laughter]

47:15 TW: There you go.

47:15 MH: Wow.

47:16 JC: Because the donut is so much better.

47:18 TW: But as a tribute to Hans Rosling, can we make it a donut that then moves over time?

47:22 JC: Yes, as long as there are at least five, we’ll make it a 3D donut that also moves through space and time so we can have five…

47:29 TW: Do not disparage the memory of Hans Rosling by trying to bring in a pie chart into what the genius that was.

47:37 JC: Or Google tie-in. Motion charts.

47:39 TW: Anyway, may he rest in peace.

47:42 MH: Okay, well, bring it back around.

47:46 MK: Okay, I’ve got a solid one, team. So, I’ve started listening to another podcast called Not So Standard Deviation. It’s by Hillary Parker of Stitch Fix, and Robert Peng, who was a university professor and I can’t remember the university, but it’s very famous.

48:02 MH: He’s at John Hopkins, I think. I got my one hour Coursera course was Roger Peng, I believe. Yeah.

48:06 MK: Yeah, yeah. Same. Same. Anyway, so there’s two episodes that I’ve listened to that I recommend. One, back to statistics. They spent a whole episode on statistical significance, whole episode, good hour. I’m honestly impressed that you could talk about it for an hour, but also learn some stuff.

48:25 TW: I’m wondering when you say good hour. I wondered, which meaning of, that it was for an entire solid hour. Okay, it was good.

48:32 MK: And I was like, they were talking about that one idea for an hour, that’s what impressed me. And the level of detail, which was actually very good. The second episode, which was episode 75, which is called Data Science Light. I do preface this though, by skip to minute 19 and 35 seconds ’cause the first 19 minutes, they’re talking about milk, which is weird, but after that, after minute 19, they’re actually talking about maintaining open source stuff, which is what I alluded to before. And basically if you’ve contributed to R packages, how do they get maintained, what if there are dependencies or the packages, all that sort of stuff, which I also found super interesting. So, Not So Standard Deviation is my listen for the week.

49:15 MH: My last call is a booklet I recently read. It’s not really a book, it’s a booklet. So there’s this guy, a lot of people know who Peter Drucker is, but we’re now at a time where most have forgotten. And there’s this little booklet. I think it was a reprint of an article he wrote called “Managing Oneself” which is a really easy read. It’s probably something you could read over your lunch break. It’s really, really short.

49:41 MH: In it, he covers a bunch of different things that are great sort of starting points. So I highly recommend it. It will actually lead you probably to a number of other Peter Drucker type books. But that book was a nice easy way to kind of surface what I would call key topical areas that Drucker felt like were really important. And it really resonated with me because. Like you Justin, not in the exact same way, but having become a leader over the years, and having to try to grow myself, you’re always looking for ways to give yourselves paths to develop and grow. So that book was very helpful. So, I don’t know. Drucker is a little bit strange. He definitely has some older ideas. After I read through a book with a couple of co-workers where it was pretty obvious in that book. He didn’t think women would be in business, so it was sort of like, “This guy should do this,” and I was like… Or girl, like, “What are you talking about?” So anyways, different time. Good book, take it for what it is. Alright, so that’s our last calls. As you’ve been listening and waiting for us to get through our last calls, you’ve probably have been thinking about a number of things or questions or ideas or comments, or educational material that you wish Google would come out with, and this is your chance, so we’d love to hear from you.

51:03 MH: Probably the best way to get in touch with us is through the Measure Slack or on our LinkedIn page, or on Twitter. I don’t know to what extent, Justin, you’re our active on Twitter, or at all, but if you know his Twitter account, you could try to reach out to him that way.

51:18 JC: It’s just my name, I have a Twitter account, it’s active. Unless, I don’t think they’ve kicked me off yet.

51:23 MH: See, there you go. But probably the best way to talk to Justin is through the wonderful people that work across Google as part of a team. But we would love to hear from you, love to hear your thoughts, so please reach out to us. Justin, thank you so much for taking the time to come on the show. It’s been a pleasure to have you.

51:42 JC: Thanks everybody, This was super fun. It was really good to catch up and I really enjoyed our discussion.

51:47 MK: Ditto.

51:48 MH: Awesome. Well I know that I speak for both of my co-hosts, Tim and Moe, when I encourage all of you out there, no matter how you learn, that as you learn, you keep analyzing.

[music]

52:03 Announcer: Thanks for listening and don’t forget to join the conversation on Facebook, Twitter, or Measure Slack Group. We welcome your comments and questions. Visit us on the web at analyticshour.io, Facebook.com/analyticshour, or at Analytics Hour on Twitter.

52:23 TW: Smart guys want to fit in, so they’ve made up a term called analytics. Analytics don’t work.

52:30 MH: Analytics, oh my God, what the fuck does that even mean?

52:39 JC: If I still cared about that, that would be something I would use later [chuckle]

52:45 MH: So you don’t want me to talk about how Jeff Goldblum was driving across the desert with the school children trying to get away from Thanos?

52:53 MH: I will not sit here and have you malign our guests, Tim.

52:58 MK: I can’t believe that’s the thing kids do now. In my days, we were trying to smuggle booze and stuff.

53:06 MH: I have a very structured way of thinking, so I’ve really honed the way I think. And so any information that’s not necessary or mandatory, I completely move it right out.

53:16 MK: But I think knowing that I’m a Zennial is actually mandatory.

53:19 MH: Yeah, obviously it didn’t make the cut.

53:22 TW: Now that I work with them, though, I realize that he makes the decision very quickly and he gets a lot of type one and type two errors go through that filter.

53:31 MH: Oh yeah, a lot.

53:34 TW: We use this thing called Google Docs. I don’t know if you’re familiar with one of the nice benefits of it?

53:39 MH: Yeah, I really think it’s gonna catch on actually. A lot of companies will start using it someday.

53:45 MH: I think Jim gets annoyed with things like rainy days and everything [chuckle]

53:51 MK: Crackle, crackle, crackle, pop, pop, pop. Snap, snap. I was thinking. I was like, “There’s definitely a third one and I’m missing it.” Oh shit.

54:04 MH: I was so confused and probably our listeners are too. I have no idea, Tim.

54:10 TW: Okay.

54:10 MK: Oh, just throw out a guess.

54:14 TW: The whole episode I’ve been Googling. I thought it was Latin and it turns out it’s actually German.

54:18 MK: I knew it was German.

54:19 MH: I did say it was German.

54:20 MK: You should have been listening instead of Googling. My personal motto is “Chutzpah”.

54:25 TW: Is that Latin?

54:27 MK: I love that that’s where your mind went. Not like, “Oh, interesting.”

54:33 MH: I like that you can hold a conversation and search the internet at the same time, for German phrases that are also college mottos.

54:40 MK: He’s got mad skills, mad skills.

54:42 MH: I’m having trouble just sitting.

[music]

Podcast: Download | Embed

Subscribe: RSS