#097: Spotting and Assessing Analytics Talent with Simon Rumble

00:00 Michael Helbling: Hi, there, this is Michael Helbling for the Digital Analytics Power Hour. I’m excited because we are coming up on our 100th episode as a podcast. Moe, Tim, and I have been brainstorming like mad trying to figure out how to make this episode special. And we’ve come up with, I think a great plan. We wanna make it special by including what’s most special about the podcast and that’s you, our listeners. So we want you to submit your questions, we can answer on the podcast and here’s the great thing. If your question gets selected, you win a great prize that’s unique to the podcast 100th episode. So, submit your questions. Here’s how you do it. You send a 30-second audio clip introducing yourself and then your question in 30 seconds or less, to contact@analyticshour.io. Again that’s contact@analyticshour.io and your sound bite or question clip, just include it in your email, introducing yourself and asking your question, and then you’ll be in the running. Any question goes, if you get selected, you win a great prize. We’re gonna take submissions from the time you hear this, up until September 15th. So from now until September 15th is when the submission deadline is. So, submit your questions, we’re excited to hear from you, our audience, make you guys part of our special 100th episode. Alright, well, let’s get it going. Oh, bonus points for anyone who includes the words quintessential analyst when speaking about Tim Wilson.

[music]

01:40 Announcer: Welcome to the Digital Analytics Power Hour. Tim, Michael, Moe, and the occasional guest discussing digital analytics issues of the day. Find them on Facebook at Facebook.com/analyticshour and their website analyticshour.io. And now the Digital Analytics Power Hour.

[music]

02:03 MH: Hi, everyone. Welcome to the Digital Analytics Power Hour. This is episode 97. Have you ever thought to yourself, “Hey, I need to hire an analyst. And not just any old person, but someone who really fits the bill.” Or have you ever thought to yourself, “Hey, I need to get hired as an analyst and not just for any job, but one that really fits the bill.” Well, not to oversell it, but the next hour will potentially change your life. Provided you listen to this podcast and then immediately win the lottery. [chuckle] Now, let’s introduce our hosts.

02:44 Moe Kiss: Hi, I’m Moe Kiss from THE ICONIC.

02:48 MH: And I’m Michael Helbling.

02:49 TW: And I’m Tim Wilson from Search Discovery.

02:52 MH: Alright, and now we need somebody who could help discuss this topic, and probably a lot of people could, but we were keen to chat again with a former guest and a friend of ours, Simon Rumble. You will recall that he is the Co-Founder of Snowflake Analytics. It’s a specialized consultancy around Snowplow Analytics and prior to that, he led data analytics and CRM at Bauer Media. Welcome back to the show, Simon.

03:22 Simon Rumble: Thank you. Moe tells me I’m one of a very elite group of people who’ve been invited back, or you’ve run out of useful idiots or something.

[chuckle]

03:32 MH: Well, actually, we’ve invited most people back. You’re just one of the very few who’s accepted.

[chuckle]

03:44 TW: Yeah, no, this is rarefied air. I mean, I’ve been on this show many times, but you’re one of the few who’s come back as a double guest, so welcome.

03:53 MH: Yeah.

03:53 SR: Nice to be here.

03:54 MH: Well, alright. Let’s get this topic started. We’re talking about trying to identify, assess analytics talent across the board. So, let’s dive in.

04:07 MK: So, a few weeks ago, Simon and I were wandering down Pitt Street and I started to hit him up about finding good talent, ’cause he seems to have this weird knack for spotting the gems in the myriad of applications. And I started asking him, “What actually helps you inform your decision?” So, Simon, there’s a few people in your team who have all gone on to have pretty incredible careers who I have a lot of respect for. What helped you identify those people in the first place? What do you think made them stand out?

04:41 SR: Yeah, I’ve had a bit of a patent I’ve used for a while hiring people. So, I start with hiring mostly juniors. So that’s people who don’t really have the skills I need. Mainly that’s through necessity, ’cause the people I need don’t really exist or are too expensive or are already taken. So you have to kinda create them. So the kind of approach I’ve often taken in the past, although that’s changing a little bit as the kind of talent has changed, is to hire junior web devs. So that’s the ad looks for junior web dev skills, but tells them that it’s actually an analytics role. Most of my work tends to be fairly technical focused anyway, so it’s less the analyst side of things, a lot more on the implementation side, so web devs are what you’re after. And then you’ve got to meet a lot people. And you read a hell of a lot of terrible CVs. And one thing that I can say that’s universal about juniors is they write terrible CVs. You’ll sit there for an hour talking to them and you’ll learn all this amazing stuff about the person and then you say, “So, just a little bit of advice. All this stuff we’ve talked about, put it in the CV,” ’cause they don’t tell you what they’ve done. They tell you that they’ve stocked shelves at Kmart and which subjects they studied in primary school and stuff like that. Nothing useful.

06:00 SR: So, meeting lots and lots of people and having them wash out, reading lots and lots of CVs, and then finally, I always do a technical written test. So, yeah, the written test is a great way to kind of confirm, or not, the things that you’ve decided in the interview.

06:17 MH: And are those… But you said you’re hiring people with web dev, but telling them it’s analytics, is the written test around the dev… Like JavaScript type stuff or are you actually giving them a written test? I guess one is the written test administered on site or is it one that they can take with them? And then the second question is, what’s the nature of those questions?

06:38 SR: So I started using a test, I actually inherited it from my predecessor at an agency called Datalicious that Moe has also been through that particular mill [chuckle] and gone through it. Look, we had a good time and we learnt a lot. It was all fun and games, but we did learn a lot. But at that, I inherited a technical test for hiring juniors from there and it covered… So this is going back some years, so it covered PHP, JavaScript, SQL, some general analytics questions and yeah, that’s kinda it. But the idea of the technical test was that you would say to the person being interviewed, “If you don’t know this stuff, it’s fine to skip it and just tell us that you don’t know it. If you feel like taking a crack at it, have a crack at it. If you’ve told us that you know something, either on your CV or in the interview, we kind of expect you to do that bit, but if you haven’t, that’s okay. And hey, if you’ve never sensed PHP before, but you kind of get the syntax ’cause it’s similar to most C derived languages, have a crack at it, that’s fine”.

07:41 SR: And so that written test, it generally takes about 45 minutes to an hour. We tend to do it on site, although we’ve done it remotely for people who’ve interviewed remotely as well. And I’ve kind of evolved it and changed it fairly dramatically subsequently, but it’s probably got the same kind of structure still, just ’cause it kinda works. And I’ve used it for junior highs and senior highs, and it really does… It kinda points out the bullshitters, but also points out the amazing talent that you might not have realized was amazing. So I have an anecdote about that. So I’ve just hired Jethro or actually a year ago hired Jethro for the third time.

08:19 MK: For those who don’t know him, he’s an absolute super genius. I would be super lucky to work with him one day in my career.

08:25 MH: Well, we’re having him on the next episode to talk about what makes you leave a manager and yet come back [chuckle] again, and again, and again.

08:32 SR: So, when I first hired him, his CV, he hadn’t finished high school. He did a tech college course in web development where he did VB and really unpromising CV. I brought him in anyway ’cause he showed the right kind of interest. We had an amazing conversation. He had that spark of creativity and curiosity and wanting to know how stuff works. Then we set him lose on the test. He nailed absolutely every question and he found an error in the test itself. [chuckle] So, we hired him and it turned out, sure enough, like I’ve hired him three times. I think I’ve finally convinced him, the third time around that it’s not just me who thinks he’s awesome, so that’s good. [chuckle]

09:30 MH: So, has his performance on the test improved each time?

[chuckle]

09:32 SR: Yeah, I think the test is probably 70 or 80% written by him by now. So yeah, retesting him probably wouldn’t be great. [chuckle]

09:41 MK: So hiring juniors is a somewhat risky proposition. There’s a lot more coaching that needs to go on, there’s a lot more impact on the business in terms of they can or can’t hit the ground running. Is it… Have you had any horror stories or like any times it’s gone completely wrong that you could kinda share with the listeners what you’ve learned?

10:07 SR: Sure. Look, you’ve got to be willing to cut your losses really quickly with the juniors that you have. So they’ve gotta be… You’ve just gotta be like within, in some cases, awake, you know that they’re not gonna work out and you just have to cut them loose and go, “Look, this isn’t gonna work.” Give them some advice and send them on their way. Try to be very generous with severance in those circumstances. When they’re on a probationary period under Australian employment law, we really don’t have to give them much notice, but I think that’s a bit of a dick thing to do. So we tend to give them some runway. But you’ve just gotta be willing to cut really quickly.

10:41 SR: I haven’t had any horror stories that I’d say, because I think, the written test really shows up what you’ve got. Probably the main one, I have hired people who’ve not had the right attitude. So they’ve had the right technical aptitude, but not the right attitude, and I think that’s my… My radar for that has got better. I really do want voracious curiosity and interest and passion out of anyone, ’cause especially… We’re a consultancy. If you work for a consultancy, you’re gonna work really hard. You’re gonna be struggling a whole chunk of the time because you’re not gonna know what you’re doing and you’re gonna need to learn a huge amount of stuff. The upside of that is, you’re gonna get exposed to a whole range of different problems, a whole range of different approaches, a whole range of different technologies, and within a year or two years of working, you’re gonna be an expert. You’ll really be ahead of the game. So, we need to find the people who wanna get to that point.

11:39 TW: Michael, you’re a hirer of juniors. You’re kinda building both on the… I feel like we should make sure we get to the non-technical analyst as well as the technical analyst, but I know we’ve had that discussion in the past, but that’s kind of your bread and butter as well, right?

11:54 MH: Yeah, we definitely had to develop kind of a rationale ourselves and so, I think what’s interesting is we also came down on this intellectual curiosity being a core component, and trying to identify that amongst all other things. To your point, you don’t get much in a resume that tells you this person is like, likes to figure out why things tick, but you can figure it out in talking to someone.

12:20 TW: How? How? How?

12:24 MH: Well, one of my strategies is, start a podcast with them so that you get a chance to interact over a number of years.

[chuckle]

12:32 MH: And you can really determine what kind of analyst are they really.

[chuckle]

12:38 MH: And then… That’s right, Moe. I’m coming for ya.

[chuckle]

12:43 MK: We’re not good enough friends yet, Helbs.

12:47 TW: I feel like I say, that I avoid, I feel like I’m always bumbling, but sometimes I walk out of an interview and I’m super excited or I’m not. I don’t really trust my intuition and it’s like I really want that intellectual curiosity, but I don’t know how to actually find it.

13:03 MH: No, it’s just, it’s in there.

13:05 SR: I often spend some time talking to them about the team and the project and what we do and how we work. And so I’ve done this both client side and agency or consultancy side. You talk about what you do, what your ambitions are, and you can tell if they get excited about that. They’ll ask interesting questions, they’ll follow up, they’ll get involved in the conversation. That’s the kind of person I want, is someone who gets enthusiastic about the project, what we’re working on.

13:40 TW: Okay.

13:40 MH: Yeah, and it’s also just applying to a question. Like, “Well, let me think about that.” And then you can see them working on it. So, we interview for three main categories for people we recruit directly say, out of university. Which if we go down the person with experience, we do a different recruiting process or interview process. But, if we’re recruiting out of university, that person’s not gonna have necessarily a deep background in analytics or those kinds of things. So, we’re looking for the precursors. And so, that intellectual curiosity, problem solving and sort of an estimating or thinking through the problem kind of capability. And so those are the three main components that I think make up a really good person who can do analysis. On the technical side, I think there’s just, you have to do a technical test. We just have to see where somebody’s at. You can train people, but you gotta see. And I think it’s actually interesting, sometimes people who don’t do particularly well on a technical test or actually the people that you’re more interested in because of how they went after it.

14:56 MH: And I’ve been in the opposite situation where someone comes in and their resume or CV says, “Oh, I’ve got JavaScript or jQuery or whatever.” And so I’ll say, “Well, how would you rate yourself like from one to 10?” And they’d be like, “Oh, I’m like an eight.” And so I’ll be like, “Oh, well, could you write this down or this?” And they’re like, “Uh, ooh, ah.” And I’m like, “Yeah, see, I’m like a one or a two and I can do that.” [chuckle] And the conversation gets really awkward. [chuckle] But it’s important ’cause you gotta figure out, well how do people evaluate? And sometimes people are really down on their skills and you gotta learn how to help them elevate them. And then some people overthink their skill set and you gotta bring them back down to… And it’s a moving target. And it’s difficult for everybody. I remember a friend of mine, Akil, here in Atlanta, is trying to hire a developer for his team, an analytics team, and is really trying to help not only find the right talent, but educate the market a little bit, too, for what they’re looking for. And I thought that was really helpful. It’s like there’s a difference between a developer and a programmer, and I don’t remember which one was the one he thought you should be but, be the good one.

16:15 TW: Michael, I wanna both call you on something and I’m gonna make a point in the process. So…

16:21 MH: Oh, boy.

16:22 TW: I avoid interviewing ’cause I hate it, I dread it, I don’t feel like I’m that good at it, but my go to method, and I will prep for them, is to use behavioral interviewing. And I was taught that that is, ask a question, “Have you ever had a time of X?”, ’cause I feel like people say behavioral interviewing, they don’t really understand what it is. I was taught a very formal method of, “Tell me about a time when you did X.” And then they say… And then you’re listening for them to tell you the situation, tell you their behavior, and then tell you the outcome. And they will struggle. They will talk around, some will talk around the situation and that gives you a very clear, gives me a very clear focus to say, “Okay, but what did you do?” And even if the outcome was unsuccessful, that’s okay, that’s not a knock. It’s just you walk through that and, to me, that is the one thing that I kinda hang my hat on. So, therefore, given that, you’re looking for problem solving. What the fuck do you do to find problem solving? What is the question you ask? Or do you go off of their resume and key off of that? So I’m gonna ask you a behavioral question of how do you actually look for problem solving?

17:27 MH: So I just ask a random question…

17:30 TW: Or, no, no, no, I want you to tell me about a time where you needed to find… That was my bad, my bad of asking the question.

17:35 MH: Oh, you’re interviewing me to…

17:38 TW: Tell me about a time when you actually needed to assess somebody’s problem solving capability and how you went about that?

17:44 MH: Sure.

17:44 TW: And I’d like you to explain the situation, what you actually did or said and then what they responded and what that told you.

17:50 TW: Tim, that’s called the STAR method. Situation, task, action, response. And it’s actually government, might be old and bad in lots of ways, but government have been interviewing in this way for at least 15 years.

18:01 MH: It’s called the STAR Method, Tim.

18:04 MK: STAR method.

18:05 TW: Okay. But could it be…

18:07 MK: Back to Helbs.

18:07 TW: Trying to think you’re… Yeah. Moe, You were in grade school when I learned this method, and I haven’t stayed up on the… So, yes, okay. Our method’s awesome too.

18:19 MH: So let me tell you about a time, Tim, when I was interviewing a candidate who is also now your co-worker.

[chuckle]

18:25 MH: I needed to assess their problem-solving capabilities. So in the course of our conversation, I simply pulled from current events to ask them a question of estimating how to solve a particular challenge, one that did not have an easy to resolve answer. And I’m looking in that conversation for them to apply assumptions, ask questions, build out a scenario, and basically, walk around the problem and try to look at it from many different angles. And as they do that, their ability to do that and engage in that way, is indicative to me of some other problem-solving capability.

19:11 MH: But you literally said… So climate change or…

19:16 MH: No.

19:16 MH: When you say current events, there was a…

19:17 MH: No, no, no.

19:18 MH: So the current event at that time happened to be a musical festival that was put on, or attempted to be put on, in the Bahamas, called the Fyre Festival.

[chuckle]

19:25 MH: Yeah.

19:26 MH: It was a couple years ago. And all these people came to this lonely beach in the Bahamas and there happened to be nothing there, except some outhouses and maybe some baloney sandwiches. So there were thousands of people stranded, and they’re not the kind of people who appreciate being stranded. I believe the guy who did that actually is doing some jail time right now. Anyway, back then it was just a funny news story, but what we worked through was how many people are stranded and how quickly can we get them back to the mainland? And that was the problem.

19:58 MH: Okay. So, literally, it wasn’t a digital marketing challenge, it wasn’t a digital analytics, it was literally…

20:06 MH: No. No, ’cause I don’t think that’s necessarily how you have to… If it was someone with experience, then I might come at them with a more specific digital analytics challenge, with context around it and those kinds of things. But I just wanna see them think about and have fun with solving kind of a hard to know problem. “So where are the Bahamas?” “Okay, how many people?” “How many people fit on a boat?” “How many people fit on a plane?” “How many pilots do we have?” “Is there a runway?” “How do we… ” So all these different things. And it’s the creativity of assessing the problem and then determining the result, that’s what’s gonna tell me the answer to the problem.

20:47 MH: That’s like the classic, how many pennies would fit in this room? I don’t really care if you’re right or wrong, I just wanna hear you work…

20:54 MH: Yes, exactly. It’s all about how you attack it.

20:57 MH: So Simon or Mel, have you guys ever gone that route?

21:00 MK: In my interview, I got asked whether or not The Iconic should open a warehouse in Melbourne. That was the question that was given to me. And…

21:08 MH: Great question.

21:08 MK: I actually walked out of the interview thinking I hadn’t gotten the job. But the interview, I thought it was so much fun, because it was me and one of the co-founders just bouncing off each other, talking about the problem like, “Okay, what assumptions should we make? How should be tackle these?” And it was actually that talking through how we would tackle it that got me the job, but at the time, I was like, “Oh shit, I didn’t give him an answer”. But the point is not the answer. The point is to talk through how would you actually go about tackling this if you went back to your desk and you had to figure it out.

21:38 MH: But the million dollar question is, do they have a warehouse in Melbourne now?

[chuckle]

21:43 MK: No. We don’t. We had one many years ago. Many, many years ago.

21:47 MH: Just checking.

21:48 MK: But we don’t now.

21:50 MH: So Simon, do you go that route? Do you do any of those types of questions? Or is because you’re mostly technical, you can stay closer to…

21:58 SR: Yeah. I don’t ask those kind of questions. I have a bit of an aversion to the, “How many M&M’s will fit in a two-liter Coke bottle?” kind of questions, ’cause I just think they’re… I don’t know, I found them a… Because they’re so contrived, they’re not necessarily that… I don’t know, when people ask me those questions, they kinda piss me off.

[chuckle]

22:20 SR: ‘Cause they’re just not… Ask me something real. I like your idea of… I like Moe’s one of, “Should this business open a warehouse in a new city?”, ’cause that would kind of do something. No, I don’t.

22:35 TW: But it serves that dual purpose, because if the interviewer was actually grappling with that, then it’s… He’s getting to talk through what he’s… It’s kinda like, Simon, where you started, saying, “I talk about what we’re doing and the challenges we’re facing”. So it kinda combined both of those. And I’ll say, I think, Michael, the Fyre Festival example, that’s got more tangents. This was a real problem that somebody faced, maybe not our business. Whereas, I guess, if you go one step farther to, “How many M&M’s fit in a two-liter bottle?” or “How many coins would fit in this room?”, now it is [23:11] ____ contrived.

[overlapping conversation]

23:11 TW: I like the builds more story into it. And maybe, to your point, Simon, I think I would be annoyed by, “How many M&M’s… “, ’cause it’s so constrained. Where’s my freedom to be creative and really explore angles, if I…

23:29 MH: Although if they take off and start quoting James Surowiecki and Wisdom of the Crowds because of the anecdote that’s early in that book, I’d probably just hire ’em on the spot.

[chuckle]

23:38 MH: Right? ‘Cause they start with counting… Well, sorry, maybe that’s a bit of a too obscure reference.

23:44 MH: Don’t worry, we’ll put it in the notes.

[chuckle]

23:46 SR: But look, honestly, I think it’s probably something I should do more of, because even when I’m hiring for a technical person, I still need analytical capability. Because everything that we produce is going to go in the hands of an analyst at some point. So often it’s I need instrumentation to serve as a query, some question that the business has come up with, the tech person has to go and do that instrumentation. It’s actually better for them to go and think about what questions might be asked and, or might be possibly asked, and do the instrumentation upfront anyway. So it’s, it is worth having that inquiring mind. I guess, I kind of get around that by conversation, I suppose, is how I’ve done that, but I do like the slightly more formal way of doing it, so I think I’ll try that out.

24:44 MH: Yeah, well, I am, if anything, I am informal, so I tend to ask very informal questions in the interviews, and act informally, but I’m always judging. [chuckle]

24:56 MH: Certain people think, “I thought I was just having… Getting a cup of coffee in a Starbucks and chatting with this guy, turned out, I was interviewing”.

25:04 MK: Yeah, so I wanted to go back to something Michael said a little while ago about using a scale to get people to rate their skills because it’s something I’ve been giving so much thought to of late, and that’s the issue that lots of women, when it comes especially to technical skills, and there is a one to 10 rank your SQL skills, a lot of women, and I include myself in that, like, I would put myself very low on that scale, even though my skills might be higher. And the… Yeah, I’ve been thinking about it in the context of like, “Okay, is a test always better then, because then you’re not asking people to self-assess, you actually are saying, ‘Use your skills and then we’re gonna figure out where you sit on that spectrum’.” But, I mean, is that the workaround? I…

25:49 MH: It’s also important, probably, to point out what the purpose of that question is. It’s not actually to place someone on the scale. It’s actually to figure out how far off the scale they actually are.

26:00 MK: Mmm.

26:03 TW: You mean how far… What do mean by how far off the scale?

26:05 MH: In other words, so all of us assess things in different ways, so how far apart are we in our own assessment of each? So actually, we even do this or have done this in the past with our performance management, where the person and their manager will fill out separate sheets and they’ll place a random dot on a chart that basically says, “This is my level of performance in this category and in this category, place a dot. And then basically, it’s not important about where you place your dot, what’s important is how far apart or close together you are when you combine them. And that’s why that, why you might ask that question.

26:45 MH: Certainly… Because if I asked someone who then undervalues their skills, I’ll know. Like, I gotta ask this question in an interview like, “How would you rate yourself in Excel?” And I’m not as good as Tim, but I’m pretty good at Excel, more than I used to be.

27:04 TW: On a scale of one to 10, I could rate myself about a three so…

27:07 MH: Yeah, exactly. And that’s exactly how I responded. I was like, “I’m probably in this range, but I’ll tell you, I feel like the more you learn, the more there is to learn,” and that’s what kind of excites me about Excel, generally. So that’s, it’s not a, it’s not a question where you gotta get someone to give you a number, and then you stick ’em there because they said a wrong number.

27:28 MK: Here is the next thing that I’ve been struggling with ’cause as we all know, this show is my outlet for figuring out my future career and how I’m gonna navigate it.

[chuckle]

27:38 MK: And so what happens with these tests? So, Simon, when I applied for my job at Datalicious, I got given a test. And they’re like, “Oh this should take you a couple of hours.” I’m pretty sure I spent two or three days on it, and I didn’t dare tell anyone. In fairness, I’d never worked in the industry, I didn’t know any of the stuff. I’d never opened Tableau, I had to go do all this stuff. I actually think I even got Tableau somehow… Like one of their sales rep to help me in just the data because I’d never used it and I didn’t know how and… Anyway, it was this whole thing. But I spent days on it. And ultimately, yes, I got the job. But is that something, that as a candidate, I should just not tell anyone ever about? If you… It takes you much longer than it should? Or is the fact that it takes you longer a sign that maybe you’re not the right person for that job?

28:30 SR: So, just in defense of that test, that wasn’t my test.

28:33 MK: I know.

[chuckle]

28:34 SR: So Datalicious has a tech side and an analyst side and the analyst test was different. If it’s the test that I did when I took, got the job I… ‘Cause I had to do the analyst test as well as the tech test. It was a crap test. [chuckle] It wasn’t, it really wasn’t very good. I think they had a habit of just throwing whatever client problem they had that week at candidates and saying, “Here, have a go at this”, without really framing them very well. So that’s the thing, I think you need a test. It’s actually, it’s gotta be well-crafted so… So what you got may well have been two day’s worth of work for someone with no domain knowledge or knowledge of the tool or anything like that. So it’s… Because I’d do a written test rather than a take home test and I actually want to talk about that… Is you can actually gauge how long someone takes. And so one of the worst hires that I ever made took two and a half hours on what should have been a one hour test. And from his performance after the fact, what we worked out is that he must’ve cheated. He either phoned someone or was looking up all the answers on Stack Overflow or something like that, ’cause he passed the test. He really shouldn’t have.

29:43 MK: Yeah, but hang on a minute. I think that… And we’ve talked about this, one of the key qualities that you’re looking for is curiosity and ability to self-solve and dig and find a solution. So often, every time I’ve had to do a test, it’s been “go home, you’ve got a couple of days to do it and you can use whatever you want resource-wise”. Is that the word… As long as I get the answer, does it matter?

30:11 SR: So, I mean there’s levels of… There’s degrees of needing help. That’s the thing. We do the test. I always do. They’re open. You’re welcome to look stuff up ’cause you’re not expected to remember the syntax of obscure programming language A or B. But if the entire test that you do is looking stuff up and it takes you so long that it’s obvious you had to look everything up, that’s a sign. And maybe the way to approach that is to do some follow up with the person who takes a long time. ‘Cause it might just be that they’re really nervous and they’re really thorough and they really came to get it right. It might just be massive over attention to detail, and that’s probably a positive trait. So maybe it needs follow up rather than straight washout. But take home tests, I’m not a fan of. There’s the simple cheating aspect of it. People could just get someone else to do it which means it’s not necessarily that predictive. And you might say that doesn’t happen, I say it does happen. It happens a lot, and I actually… I think that has a tendency to take up way more time of the candidate’s time than is fair. And candidates like Moe will go on spend two days on something when they’ve been told it should take two hours and you’ve just wasted two days of Moe’s time, and I don’t think that’s fair. She’s not getting paid for that.

31:33 MK: Yeah, but I don’t know. I see both perspectives. There’s a bit of also how much and in that case, when you’re changing industries, you don’t have any context. You do have to look up every single thing, but I guess it comes down to the… How much is what you’re looking for those qualities and how much of it is it like technical domain knowledge, and can one ever outbalance a lack of the other in the hopes that you are a good enough learner. You can pick that stuff up down the track.

32:04 SR: Yeah. So we always have a discussion with the candidate after the test to talk through how they approached it, and if there’s things where they went, “Oh look, I’ve never done whatever programming language it is but I kinda had a crack at it,” and we encourage them to do that. Then that kinda captures that discussion in “well they don’t know PHP but they’ve done other programming languages”. And you know what? Their approach was sound. The syntax was wrong. It’s not gonna compile, but that’s fine.

32:33 MH: It seems like… Not to shift to the strategy of the candidate, and I’ll say upfront. I don’t wanna have two episodes in a row where I wind up being the clueless dude because Moe, you did kinda slip in the, like as a woman, so now partly, maybe we’re being tested. I’m sort of a fan of just brutal honesty as a candidate or as an interviewer. Now I have, I know that at one point, wasn’t even interviewing but I managed to run a very good candidate off because I met up with the people who’d been interviewing, chatted with the guy in a bar and pretty much, apparently convinced him that he should run like hell from the agency. Now [chuckle] I think I might have done the right thing for him but the person who was trying to hire him wasn’t real happy with me. But it seems like if they said two hours, they may have been telling everybody two hours for two years.

33:13 MH: And everybody’s scrambling like crazy. It’s not taking anyone two hours but nobody’s calling ’em on it, and it’s just somebody thinking that it’s a two-hour test. But like why not be completely honest and say, “You said two hours. This took me way more than two hours.” Maybe you round down a little bit. And if they say, “Oh, well shit, it should’ve taken 45 minutes. We don’t want you.” Great. Like why not be incredibly honest ’cause what you’re trying to do on both fronts is find somebody who’s gonna be the right fit. And what I’m hearing from all of you guys is it’s trying to get them talking or doing in a way that you can get a little bit… Where you’re kinda forcing them to remove the mask of whatever they’re trying to present. And you’re trying to get one level down to who they truly are and what they truly know, and unfortunately, it’s just kind of challenging is to figure out what to ask or exactly what to ask them to do in order to get there, but nobody wins if somebody bluffs their way through, right?

34:31 TW: Right.

34:31 MK: But isn’t that the whole process? I don’t know. I understand from an interviewer perspective, you don’t want people to bluff but you kinda have to sell yourself too. You can’t be like, “Oh hey, I’m really shit. I’m awful. Please don’t hire me. This took me two days instead of two hours.”

34:50 MH: Well if you’re shit at that, right? Moe, like you’re…

34:52 MK: Part of the interview process is selling yourself.

34:55 TW: If I went to get hired for a finance position and if I manage to figure out how to get hired for a finance position, I’m gonna be miserable inside of a month, and they’re not gonna like my performance. So I would say, I mean okay, if you’re like I’m one paycheck away from being homeless then I gotta get any old job, yes, but in general, if you say, “Yeah, this was really tough.” I don’t know.

35:24 MH: That’s a really good question Moe, because there are tactics in the interview process, sort of forms that one follows, if you will.

35:34 MH: I’m gonna ask you to be more specific. If you could maybe use an example of a situation and a tactic, [chuckle] like a start.

35:41 MH: I’m done with… I’m so done. I’m not trying to prove myself. [chuckle]

35:46 TW: I’m sorry.

35:48 MH: No, but like I’m… Tim and I lead the charge with Impostor Syndrome right? And I’ve overcome it more than Tim has [chuckle] and now I’m really proud of myself. But no… So it’s hard to stand in there and be like, “Oh yeah, I’m a huge big time deal. You should totally hire me ’cause I’m big deal in analytics”, but when you walk in the room, you have to have a perspective and you wanna bring your best forward. And so a lot of times what I do and actually this goes to a point of when interviewing someone who’s an analyst, who’s an experienced hire; one of the most important things for me to get from that person is their grasp on narrative. Can they give me a really clear picture of how they drove something from within their organization they worked with? Whether big or small, how did… Really compelling picture. Rich with detail, with lots of follow through, and a mastery of what was happening? So it’s not like, “Oh yeah, I instructed my team to do this.” No, you did this, like, you got in there and you analyzed the data, you brought these things forward, just those kinda things. Sorry, move that aside back to sort of follow these forms, you have to present yourself a certain way, you have to be… Represent yourself a certain bit, but you don’t have to walk in and not be yourself.

37:21 MK: I don’t think I’m saying like don’t be yourself.

37:23 MH: Okay.

37:25 MK: I’m just saying that you do have to sell yourself. And you kind of can’t downplay your skills, because if you don’t even believe in your skills, how can you expect anyone else to? And even though you might not believe in your skills, part of impostor syndrome is being like, “Well, I’m just gonna fake it a bit until I make it.” Which does kind of work a bit. I just think if you cop out and just be like, “I’m gonna be totally blunt,” be like, “I don’t know all these things”, I don’t think you’re gonna get a job.

37:51 MH: I disagree, because I’ve done that and gotten the job…

[chuckle]

37:56 TW: It’s a seller’s market. People are desperate, what?

38:00 MH: Yeah. Wait, analytics? You’re hired!

[chuckle]

38:04 TW: Can you spell it?

38:08 SR: I mean this is a really important thing is… It’s really, it’s easy for those of us who have got a history and skills that are in demand and enough money in the bank to cover the mortgage for the next month or two, to kind of pick and choose our jobs and be ourselves, but when you’ve got the pressure of… “I need a job. I really need a job”. It’s a different situation. And what terrifies me about interviewing is missing someone because they don’t have the personality, my personality type. Someone who’s shy, not self-confident, and can’t talk well about their own skills and their own… What they do. Part of what I try to do in interviews is draw those people out, ’cause I mean, at the end of day, ’cause I’m hiring techy people, they are the classic introvert, not able to talk themselves up, kinda types. The best ones don’t believe in themselves. They really struggle to tell you why you should hire them. From the CV all the way through. So it’s about finding those people.

39:18 MH: So, I really like that that’s what we’re going to, is we wanna recognize those. But I’m also, as you’re talking, I feel like with interviewing, there are times where we fall in this trap of, we have to find the great candidates and we better not misfire, but it is a Type One, Type Two problem that all of a sudden I have playing in my head, all the stuff I’ve been getting thanks to Microsoft and others better about, that interviewing is inherently imperfect. And Simon, you said early, you’re gonna misfire and be prepared to say we’re not gonna know that until two or three months in. And eject them like a virus would be the… Thriving on Chaos, is that the book? Built to Last, but… But the interviewing is inherently, it’s not like you’re trying to get to a 100%, the best model would be, everybody has to work for you for three months. And then you’ll get to where maybe 80% of… Or 90% of your hires are good. And so that seems like that’s this other piece of it is actually being okay with, it’s not gonna be perfect, interviewing is inherently flawed, but recognizing what are your shortcomings? I’m gonna gravitate to people who are extroverted and informal. I better make sure that it doesn’t let me undervalue people who are not…

40:44 SR: Yeah, and there’s two different approaches to hiring, there’s… I would characterize the Google approach to hiring is that they never want to make a bad hire, but that means that their process is ridiculously onerous. And I’ve had plenty of people who washed out of Google’s hiring process and they’ve been awesome so it’s… They have the “never hire a bad egg” approach. I’m happy with occasionally messing up and like I said… Yeah, get rid of them and do it quickly.

41:14 TW: Yeah, my approach has been to mess up less and less over time ideally.

[chuckle]

41:20 SR: Sure. Of course, of course.

41:25 TW: But it’s to your point, you can’t get it right all the time from an interviewer’s perspective. I think from the interviewee’s perspective sometimes you don’t get it right either because you pick a job that actually you’re not gonna like. You don’t assess the company or the role for yourself. And I think that’s probably, like we haven’t really talked about it, but that’s actually equally important. What kind of job are you gonna love?

41:47 MK: I was watching something the other day, which seems so relevant right now. One of our engineers shared it. Her name’s Aubrey Blanche and she’s… I can’t remember her exact title now, but she works at Atalassian in the people and culture team, and she talks a lot about the hiring process and how you make sure you have a really diverse team. And one of the points that she makes that I absolutely love is we need to stop talking about cultural fit, because cultural fit means I’m gonna hire someone like me. And that’s not necessarily constructive. So what we need to talk about is, are they a good values fit? Here and five company values, does that person believe in those values? How are they gonna demonstrate them in the workplace? Because soon as you say cultural fit, you’re hiring an extroverted say, white dude, that is interested in taking out the cats next door with a water gun. [chuckle]

42:40 TW: My understanding is that… I thought cultural fit winds up, I thought, and when I’ve seen companies that have had that as part of their process, is they looked at the people who embody their values the most and then say… And that may be kind of flawed. I don’t think I’ve ever been asked to give a cultural fit interview as one piece…

43:01 MK: I have.

43:02 MH: No, no, no. I absolutely would expect that you would. I’m more judging how often I managed to embody the values of the organizations that I work for. [chuckle] I haven’t been tapped for that. Well, as a matter of fact, just this week I’m asked to interview someone, it is not for a cultural fit.

43:17 TW: To your point, Moe, actually in the very beginning of the book, “The Undoing Project”, which we talked about in the show before…

43:26 MH: What an asshole!

43:30 MK: You are dead to me. You are dead to me.

43:31 TW: Is that gonna be your last call?

[overlapping conversation]

43:34 MH: Erase this, just erase this. We’ll erase it.

43:37 TW: I just loved how the general manager of the Houston Rockets wouldn’t let anyone on his scouting team talk about this person is like this person when they’re talking about basketball players unless they could say it in the context of them being a different ethnicity. And I thought that was amazing, because it literally stopped that comparison in his tracks and I was like, that’s genius. And to your point, Moe…

44:03 MH: That’s in the Undoing Project?

44:04 TW: Yeah, in the forward…

44:07 MH: But, re-say that again. So, I haven’t read it.

44:10 MK: Basically what was happening was scouters were making comparisons between players, and because they looked similar, they were equating their player skills as being the same, because they were six foot, and they were both… Yeah.

44:27 MH: Okay, that’s like Moneyball with them saying that, “Oh, he has a baseball body.” And okay… Got it.

44:34 TW: And so, this is why so many teams missed on Jeremy Lin who had… And they made this point in the book, had some of the quickness that he had was unheard of, and as a player, it was off the charts, and yet nobody was seeing it because their comparisons were off. And so the way that the general manager of the Houston Rockets tried to rectify this, was by making it so that you were not allowed to make those comparisons unless you could be like, “This African American player, but I’m gonna compare him to Steve Nash,” or something like that, and then you could truly see it. Anyways, I know that’s not a values fit, but I really love the way in that scenario, in that world, that he tackled that little problem.

45:22 MK: But I think that’s the same problem, is like what Audrey’s point is, that when you talk about cultural fits, you often are making that same mistake, of, “This person is like that person, or like me, and therefore they’re gonna suit our organization”.

45:37 MH: Alright, this has been a great conversation, but we do have to wrap-up. Before we do that, let’s go around the horn and do a last call. It’s a thing we do, something we have seen recently, we think might be of interest. Simon, you’re our guest, what’s your last call?

45:53 SR: So this is kinda random, but it’s something that [45:55] ____ sent out a few weeks ago and it’s a YouTube channel called Primitive Technology, and it’s this guy with a science degree who runs a lawn mowing business in North Queensland. So this guy goes out into the bush and he’s got this property and tries to recreate technology from scratch. So he starts by… If he wants shelter, he starts by harvesting grasses, drawing them and weaving them so that he can make himself a thatch hut. And then he decides he wants to make iron so that he can start making the next level of tools. So he has to build a smelter, that smelts iron molecules out of bacteria to then melt down so that he can then finally turn those into tools and stuff. And it is really incredible stuff.

46:49 SR: So there’s two tips on this I’ll give you. One is read his WordPress site as well, and turn on closed captions in the YouTube channel so that you can see, ’cause he actually writes in notes of what he’s doing, but he never talks. It’s just really strangely meditative watching, and the story telling process is incredible. I’m a bit hooked.

47:11 MH: Alright, Tim. You wanna go next?

47:13 TW: Sure. So, I have done posts from this site before as last calls, but it’s the Pudding is the site, Pudding.cool and the post that I’m gonna use as my last call is life after death on Wiki, Life After Wiki death, something like that. But what they’ve done… And I love the Pudding, just because they do such a phenomenal job of both going deep in data and laying out kind of a narrative online, as you kind of scroll through it. But this is an analysis of the page view data available from Wikipedia. And they have gone through and looked at when celebrities die, there is a spike in traffic to that celebrity’s page, and looking at kind of how big that spike is, what the fall off is comparing various celebrities to each other.

48:00 TW: So it’s just kind of fun. It’s a little interesting diversion, and it’s using a page view data, which is data that we’re all very, very familiar with. So it’s kind of a fun little diversion.

48:08 MH: Alright, Moe I’m gonna let you go last, that way nobody will remember what we talked about before, so… My last call is the Digital Analytics Hub, which is coming up in October, so about over a month from now. So, it’s a conference I have attended many times, I really, really like it. I wanna tell you how much I like it, I like it so much, I am giving up my spot to go, to someone else on my team who I think will benefit from it because I really think it’s something else. So if you have the chance, go to the Digital Analytics Hub… Plus, it’s in Austin, Texas this year, I believe it’s mid October, like the October 17th-ish time frame, some really great workshops before, and great huddle based conversations for two days in Austin, Texas. Worth your time. All right, Moe, what is your last call?

49:10 MK: Well, my last call has already been ruined but I just wanted to say a big thanks to Fred Pike who recommended… I saw him at Super Week and then again at C-Excel, and he kept on my case about reading “The Undoing Project” which I was so resistant to read, because I don’t know why it’s just sat on my bookcase for six months, and I’m now set… I’m getting everyone to read it because I’m absolutely loving it. Just hearing the story of Traversky and Kahneman has been amazing, I’m obsessed. I was like asking people in my holiday, a month or so ago, about all the tests that they run with their students, and I kept asking people in holidays to test it out and see if that’s what people if actually do. So if you haven’t read The Undoing Project, I really, really recommend it.

49:56 MH: You neglected to mention that you actually received an email from Daniel Kahneman.

50:01 MK: No, but that was kind of exciting.

[chuckle]

50:02 MK: Although very, very short and sweet, but…

50:05 MH: Alright, well, this has been a really awesome conversation. I’m sure if you’ve been listening, you might have a couple points about how you interview, or how you’ve gone through interviews that help refine the process of identifying the best talent, and we’d love to hear from you. The best way to do that is through the Measure Slack, and actually Simon’s on there as well, and is active under the name shermozle, which I think is your Twitter handle too. So reach out to him there, I’m sure he’d be happy to connect with you. I would love to hear from you at Slack, Twitter or on our Facebook page, we’re happy to hear from you. Also remember, just a few more days, we’re still accepting entries, questions for our 100th episode extravaganza spectacular, where Tim answers a hundred questions from a hundred listeners. No, I’m just kidding, that’s not how many there are.

51:01 TW: But really we are excited about our 100th episode. So please send us your questions, we’re really looking forward to that. The deadline is again, September 15th, so just a few days from now. Alright, well once again, Simon, it’s been a pleasure speaking with you again, thanks for coming again, on the show and obviously for my two co-hosts Moe and Tim. I wanna say to all of you out there, no matter what your talent level, keep analyzing.

[music]

51:35 Announcer: Thanks for listening. And don’t forget to join the conversation on Facebook, Twitter, or Measure Slack Group. We welcome your comments and questions, visit us on the web at Analyticshour.io, Facebook.com/Analyticshour, or at Analyticshour on Twitter.

[music]

51:55 Speaker 6: So smart guys who want to fit in, so they made up a term called analytics. Analytics don’t work.

52:04 MK: Tim’s just getting a heart attack over there.

52:06 MH: Tim and I’s work styles are very different.

52:09 TW: Yes, they are. [chuckle]

52:12 MH: And we keep finding people who write us up as being like a top five or top ten, and one really had like… It basically called us out as just being people who like each other and have a good time like…

[chuckle]

52:24 TW: We should send them a cease and desist letter.

52:28 MH: It says my void connection to Moe has failed again, and Moe, you’re not muted. Nothing bad is going to happen from here.

52:38 TW: Alright, it’s light, witty and deeply personnel, so I feel like “Woooh”, I’m not sure we’ve gone deeply personal. Which just adds to its charm.

52:45 MH: Let’s get real for a minute.

[chuckle]

53:00 MH: I’ve been hooked on cocaine since 1992.

53:00 MH: Moe how you do it in rehearsal is how you do it in performance.

53:00 TW: Hey Michael what are you wearing?

53:00 MH: Actually, you know what? I’ve got a new pair of jeans and they are really comfortable. I really like them.

53:09 MK: Maybe you should call him like the Tim Wilson of Australia or something.

53:13 MH: Who was that? And more insulting too…

53:18 MK: I don’t know, that’s why I like what it looks like. [laughter]

53:19 TW: Why don’t you call him the Tim Wilson of Australia and Moe.

[chuckle]

53:22 TW: Moe, this is where you just say, “I’m Moe Kiss.”

[chuckle]

53:27 MK: I didn’t know we were doing that today.

53:28 TW: Yeah that’s why I put it in there that way.

53:30 TW: Michael! , Michael can’t hear me.

[chuckle]

53:35 TW: Yeah! [laughter]

53:36 MH: Son of a bitch.

53:39 SR: Now Moe can’t hear you.

53:39 MK: Okay, now you’re back again. You went all gray.

53:41 MH: But now I’m back.

53:43 MK: No. Tim went gray.

53:46 TW: Yeah this is gonna suck.

53:47 SR: Moe, Tim can’t hear you.

53:48 MK: Oh man…

53:48 SR: Okay Michael, they can’t hear you.

53:51 MK: I can now, I’m back.

53:56 SR: Damn.

53:57 MK: But now Simon can’t. Simon’s refreshed.

54:00 TW: Simon’s still on. Okay. Sorry, yeah, I didn’t hear anything you said, Michael.

54:06 MH: No, that’s okay, just assume I gave an amazingly sensitive and gender balanced answer, and then go from there.

[chuckle]

54:16 TW: Okay, you know, this is going great, but we have gotta wrap-up.

54:20 MK: Hang on, Tim. Unit is dead.

[music]

54:24 MK: I can hear you two guys, I can’t hear [54:25] ____, so I have no idea what he just talked about.

54:28 TW: Oh, bollocks.

54:30 MH: Rag flag and tricky questions.

[music]

Podcast: Download | Embed

Subscribe: RSS