[music]

0:00:05.8 Announcer: Welcome to the Analytics Power Hour. Analytics topics covered conversationally and sometimes with explicit language.

0:00:13.0 Michael Helbling: Hi everybody, welcome. It’s the Analytics Power Hour, and this is episode 229. You know, the saying, nothing ventured, nothing gained. It goes all the way back to, I think the 1500s, but it actually would take quite a while for us all to really understand what that meant and start the first venture capital firms. That was back in the late 1940s. But now it seems like it’s a part of everyday technology life. You know, startup has an idea, they get venture backing, eventually they go get acquired or go public through an IPO, but who are these venture capitalists and how do they decide who to invest in? We thought it might include data, but we wanted to find out. Let me introduce my co-hosts. Hey, Val. Welcome.

0:00:56.8 Val Kroll: Hey, happy to be here.

0:00:57.0 MH: All right. Have you ever, uh, venture invested?

0:01:02.4 VK: Yeah, all the time left and right.

0:01:06.0 MH: It could be like a, there’s lots of people who do angel investing and things like that. And Tim, I didn’t expect to see you here today, but welcome.

0:01:14.5 Tim Wilson: Hello.

0:01:14.8 MH: Hello. It’s great to have you. And I’m Michael. Well, so far none of us are VCs, so we needed a guest, someone with some expertise. Samantha Wong is a general partner at Blackbird Ventures, a venture fund that backs the best Aussie and Kiwi startups, including Canva, Safety Culture, Culture Amp, Zoox and Halter. She’s also a former founder herself and product manager. And today she is our guest. Welcome to the show, Sam.

0:01:42.1 Samantha Wong: Thank you so much for having me.

0:01:43.3 MH: Yeah, well thank you for coming. We’re excited to chat with you about this. And maybe as a starting point, just for kind of level setting and things like that, sort of be great to hear sort of how you got into the world of venture capital initially. And then we can kind of dive into sort of more of the some of the data topics that we’ve got questions about. But maybe it’d be good cool to hear your story.

0:02:08.2 SW: I mean, you’ll hear sort of a lot of VCs say this, but for me it was like very accidental and non-intentional. So you did mention I was a founder, so I went through an accelerator program that one of the co-founders of Blackbird Ventures was then running in Sydney in 2014. So I got to know him really well through that process. I took a little bit of money from Blackbird and that accelerator start mate, but was between startup ideas working on something new, but it wasn’t quite all there yet. And so looking at getting a job and my main goal at that point was to get a job at a startup as a product manager at Series A stage so that when the startup was incredibly successful, I would be brushed with that glow a little bit and people would Angel invest just ’cause it was me, just ’cause I was the product manager of blah blah startup at series A. Because at at that point I really felt that that’s what I needed for my next startup.

0:03:05.7 SW: And I was chatting to Nikki about my grand plan. I had a couple of offers on the table and he was like, Hey look, if you’re not gonna work on your own startup, just come and work for us. Give us two years and we’ll back your next company. And I was like, oh, well that’s all I’m trying to really do here anyway. So, that’s what I did. That’s how I entered VC. It was a backdoor I thought to getting my seed round raised. And actually when I within about three or four months I just fell in love with Blackbird and being employee number one of a team of three and a half people, you kind of feel like a founder and I really haven’t looked back. So that was almost nine years ago now.

0:03:44.6 MH: Alright. So you mentioned a couple of words I just want to pull out really quick just for foundational purposes is Series A seed round? Could you talk through a little bit of like that structure? ’cause I think we hear that.

0:03:55.7 SW: Sure.

0:03:56.2 MH: Or we read news articles and we’re like, what is a series A? What’s a series B like? Yeah, that that’d be helpful, I think for the listeners.

0:04:03.1 SW: Yeah, and look, the, the vernacular changes every few years, which is kind of confusing, but essentially if you think about funding a business, some of these businesses might take 100s of millions to reach profitability, right? Rather than giving every person with an idea $200 million upfront or a billion dollars upfront, you slice it up into much smaller checks and feed it into the company every 18 to 24 months. And each time you feed it in, you call this a series of funding. And the very first round of funding is called the seed round, or it used to be called the seed round. Now there’s a pre-seed round but typically seed round used to sort of like, if you visualize it you put a seed in the ground and that gets it started. And then you enter the alphabet and Series A is kind of the round after that. But typically, I guess what we mean these days is seed round usually refers to like the first round of institutional capital. It’s the first bit of money that someone like a VC who manages money on behalf of other people will invest in in the company. And then Series A sort of usually signifies an inflection point. You’ve reached product market fit or high level of confidence that you’re on the road to product market fit and or you’re ready to scale your product market fit. Yeah, hopefully that helps.

0:05:29.4 MH: So just to close the loop on how that works, am I right in thinking that with each successive investment round, in theory the valuation of the company is going up and the risk of the investor is going down. So for every dollar you invest, you’re getting a smaller portion of the business. Is that… I mean that’s the balance you’re walking is invest early high risk, get more of the company invest later lower risk, but lower return?

0:06:02.4 SW: Yeah. In general terms, yes, but, I would say the thing you’re always trying to balance is what does long-term success mean? And at the earliest stages, one of the things you’re trying to think through is, like this particular team that I’m backing, how do we make sure that as investors we get compensated for the risk that we’re taking, which is very significant at day one without actually increasing the odds of failure because the founder is so diluted that five or six years into the journey they own so little of the company that even in the case of a good outcome, it’s not good enough for them. And so they give up or they sell too early for a smaller amount in year five or six versus going the whole way. So it is a little more nuanced than like, just try and get as much ownership as you can upfront, because what will inevitably happen is that that actually increases the odds of failure.

0:07:00.7 MH: Failure. That makes a ton of sense. I think where I was sort of heading towards, I mean being in a space, I mean this is an analytics podcast where we’re saying what historical data can we take where we build a model or crank out something like how is… As you’re investing in a company moving along, it seems like you have less, I would think you have kind of less data to work with. Like where, what does an analysis kind of look like? And you’ve already, I think, kind of alluded to some of it. There’s a lot that’s much more, I don’t wanna say qual… I’m doing a horrible job of articulating the question. If anybody wants to help me out or if that’s enough of a random ramble as I’m subbing in for Mo and ill-equipped. But how does… Like what do you analyze? How do you make the decision of what to invest, how much to invest, and how much you’re gonna offer?

0:08:03.6 SW: I mean it’s a great question and it really does depend at the stages you invest. So I did talk about how there’s the seed, the A, the B and so on, and typically the venture investing world is a different… It’s a different group and different fund structure and different portfolio construction. If you’re investing at the seed stage versus when you’re investing at C plus where both product, go to market motion, etcetera. Like all of that has largely been been de-risked. So what are we trying to look for? Well I spend most of my time in pre-seed to Series A stage. And one thing that we’re kind of, is like very known from the data, and actually there was a good thread on this kind of recently that basically looked at a portfolio of… A funder funds portfolio that basically looked through to 11,000 investments via funds and 53% I think it was, or roughly 50% of all of those investments don’t return capital.

0:09:08.7 SW: So that means if we invest a million dollars in the seed round, we get a million or less back from that. That is how we construct our kind of models at the beginning when we’re raising a fund, we assume half the portfolio will not return capital. That is the kind of failure rate that is expected for venture. But what that means is that the other 50% really have to perform in order to pay back the losses of the ones that don’t work out. Plus obviously more than that in order for us to do more than just return people’s money to them for us. And we do target a three times return. And so what that really means is even at the very beginning when it’s almost impossible to predict, does all of the ingredients of this startup team, product market, etcetera, have even the DNA to produce an outcome that could be 50 or a 100 times the capital that we invest in day one. And that’s kind of the starting point, I guess, of the analysis.

0:10:17.6 VK: And so it sounds like a lot of the analysis that you’re talking about has quantitative aspects to it, obviously, like looking at the total portfolio and understanding the 50%. Not expecting the return, but I’m also hearing some things that might sound like there’s some qualitative aspects to it and thinking about maybe the team obviously like there’s the confidence in that idea before you get to product market fit and looking at like, the addressable market and things like that. So how do those two sides come together, when you’re making some of those decisions?

0:10:52.2 SW: Well, you sort of… I don’t walk around every day going like, oh 50% of this portfolio will die. Like I just sort of have internalized that. And so for… So the other side of it, I guess we have a bunch of heuristics and every firm will probably, or every investor will basically start to establish a bunch of heuristics that correlate with success in their minds and that will be supported by data. But the problem is you don’t have a huge volume of data. And also there’s an issue I think with the availability of data. But anyway, one thing for us is particularly in the context of Australia and New Zealand, which is where I focus my time, it’s a total population of 30 million people between the two countries. So that’s like the state of Texas.

0:11:41.8 SW: So one of the heuristics we’re looking for is, is this addressing a global market? Because if you’re starting in Australia and New Zealand, or only servicing Australia and New Zealand, the chances of building a very big business that can 50 or a 100 X our money is relatively small or almost impossible. Like the probabilities are so small that it’s negligible. So that’s one of our heuristics is we’re looking for businesses that sell globally from the beginning or don’t have to reinvent the wheel on go to market or product for each new region that they go into. So Canva is kind of the iconic example. You throw up the the site and people around the world can become customers and very early in Canva’s life like Brazil emerged as one of like the top three markets for example. So that’s one of the heuristics we live by.

0:12:36.7 SW: The other one is kind of like, is it the founder’s life’s work? We know that these are very long journeys, very difficult journeys, and the really big 50 to a 100 X outcomes come from in the case of success, saying no to a 100 mill acquisitions or 300 mill acquisitions and really pushing for a very big outcome and, and not just pushing as in like having the grit and so on, but also not running out of ideas. Like having an idea of product roadmap that extends beyond three to five years. You’d be surprised how actually rare that is and how most founders are kind of incrementing from an initial solution to an initial problem, but haven’t really thought about what the 10 year state of that product could be. And what we tend to see is that when something is a product of all of your life experiences and passion and so on, like you kind of almost have to work backwards from the 10 year vision. Like you have this idea of what this big thing could be and you’re actually kind of incrementing back to what is achievable to build in the first 18 months of the company’s life. So these are hopefully a… Like they’re just some examples of some heuristics that we’ve kind of developed that we think correlate with long-term success.

[music]

0:13:57.1 MH: It’s time to step away from the show for a quick word about Piwik PRO. Tim, tell us about it.

0:14:02.9 TW: Well, Piwik PRO is easy to implement, easy to use and reminiscent of Google’s Universal Analytics in a lot of ways.

0:14:08.7 MH: I love that it’s got basic data views for less technical users, but it keeps advanced features like segmentation, custom reporting and calculated metrics for power users.

0:14:18.3 TW: We’re running Piwik PRO’s free plan on the podcast website, but they also have a paid plan that adds scale and some additional features.

0:14:25.1 MH: That’s right. So head over to piwik.pro and check them out for yourself. Get started with their free plan. That’s piwik.pro. Alright, let’s get back to the show.

0:14:32.7 MH: But with both of those examples, as you started talking, it had me thinking about if you think about it as like data quality, like the inputs to that. I mean, I’m sure every pitch you get it has a hockey stick, it shows the addressable market is the entire world and it shows that they’ve got 10 years of sort of growth. That is yours… Is that your starting point of data is what the founders are giving you, but you have to vet, like that’s a, like where does the… As you’re looking at like the addressable market, they’re gonna give you a number and it’s gonna be hundreds of millions of customers. Do you take that and say, how much do we believe it and need to scale it down or do you wind up going totally independently and saying, if given what I understand they’re pitching, this is what I see, like where does that… And the same thing I think for the how long of, how much is this, what kind of growth is this? Are they just making up stuff? Like they were smoking weed and they just came up with crazy stuff or is this a life’s work and they’ve got, this thing will last forever.

0:15:49.4 SW: Well, there’s probably two separate things like one on the market size, like again for the stage I invest at, which is very, very early, like you’re almost counting on a market emerging that doesn’t really exist yet, right? And again, for the kind of outcomes we’re looking for like 50 to a 100 x or beyond ideally multiple of capital invested. Like if I have to kind of get a calculator out to kind of calculate Tim, it’s not big enough, right? Like it should be very back of envelope. Hey, are there 10,000 potential customers for this kind of software in the world? And would they kind of pay $10,000 a year or $1000 a month-ish for this kind of software? Good yardstick is like, do they do that today or do they spend much more than that today because they have humans doing it and actually they would much prefer software does it.

0:16:41.3 SW: These are kind of like the very back of napkin kind of numbers. If I need to like drive to precision at pre-seed stage.

0:16:50.4 MH: It’s a problem.

0:16:50.5 SW: The tendency is basically to feel like it’s not big enough. It’s not big enough unless there’s a story around how the market is actually changing, behaviors are changing, something like that. Such that what is now a tiny market could actually eventually be a very big market. Like when Atlassian got started I guess bug tracking software was like incredibly niche, but bug tracking software became effectively software collaboration software and that turned out to be applicable to basically any team, any modern team in the world. So that’s a story of kind of a market emerging rather than kind of being obviously constructed up front.

0:17:38.5 SW: So that’s probably the first thing I’d say on kind of like market size. And I would say market structure is important too because venture is really built for businesses that grow very big but very fast. The sale cycle is an important factor. If you are trying to sell something for $10,000 a year, but there’s a lot of inertia around that sale or it’s very expensive to make that sale because someone needs to have three steak lunches and take it to the CIO or the CEO, blah blah blah, blah blah. Like all of that adds up to, wow, it could take us three or four years to get our first million dollars or something. And that for the most part doesn’t look like a venture style trajectory.

0:18:26.5 SW: So we’re often thinking not just about market size, but also how, readily can you grow traction in that market. And then the separate point I think is around founder’s life work or why is this the team to win? And for that, I personally believe you really can’t look at what the founder says sort of they’ll do in the future. It’s actually all evidenced in the past. So very often there will be a narrative in the founder’s life story that makes this all make sense.

0:19:02.0 SW: So Mel from Canva, for instance, she was teaching graphic design before she started Fusion Books. Fusion Books was the precursor to Canva, Fusion Books was a yearbook design company. Basically, the fundamental product philosophy or ideas were there, and that was started three or four years before Canva and so you kinda… And even like very little things like, Is there evidence of kind of like hustliness, and she was selling… I think her and Cliff were at festivals, putting tattoos on people to fund their travels and stuff like that.

0:19:58.5 SW: Just all of these sorts of… I know it feels like nebulous and not really great quality data, but it does kind of emerge over and over again, these patterns of why a bunch of different events have come together to result in these particular founders starting this particular business, and ultimately you’re placing a bet that these founders have a unique insight into the problem, which makes them better qualified than every other person who’s also had this idea to create the most elegant solution, and also they’ve internalised the buying behavior, that they understand the world of the customers so well that they can also best predict the best and most efficient channels to acquire customers, ’cause remember, you kind of gotta get distribution or go to market and product, like working in tandem, there are so many amazing products that just never… They just don’t get near an inflection point because they don’t find efficient pass to market, and so it’s just finding those two in one founding team, which is the magic.

0:20:56.6 VK: That’s super interesting, and I love the idea of that heuristic that you put together, and I’m curious, you’ve mentioned that you focus a lot in the seed in the Series A round, I’m assuming that the heuristic probably looks a lot different in future stages, like you even said, the calculations get different. Does that mean that you have a tendency to… Firms or portfolios would focus on certain phases just because you end up with so much evidence in these stories and this idea of the magic ingredients about what makes it successful, and that tends to be a focus for different VC areas or firms, is that a way to put it?

0:21:34.6 SW: Yeah, I mean, that’s definitely how the market has evolved, and particularly the more mature an ecosystem is the more specialisation kind of becomes a necessity, so in the US, you really can’t just be a seed firm, you have to almost specialise to I’m a seed firm, I only do vertical SaaS and I only do… I don’t know, the west coast, and then you’ll have the same… And even within that, that’s probably too broad, is probably like, Oh, I only do infrastructure and dev tools or what have you, and that’s because really, especially at the early stages, there is only actually quite a small amount of capital going, it’s usually two to three million is available really for the investing universe, and so there’s… For the very best companies, there’s a lot of competition to get into those rounds, and you really have to have an edge or an angle for why the founders should choose you, and usually that comes down to specialisation, like how can you be most helpful other than just the money to a really early stage company. Well, if you’re a dev tools company, you probably really want help with developer evangelism, introductions to early hires, be a very high quality signal to Series A and B investors who also specialised in investing in that kind of software universe, so that becomes a natural consequence of a developed mature ecosystem, Australia and New Zealand is not there yet.

0:23:07.8 SW: And again, we’re addressing a relatively small term, in terms of 30 million possible people, some tiny proportion of which become founders, and so we feel like we have to be a generalist firm, and by generalist, I mean invest in software, but also other areas like frontier technology, because we need to kind of cast the widest net, I suppose, for finding the best quality ideas and founders, and instead we sort of have specialisation emerge within our team.

0:23:37.5 MH: I wanna switch gears a little bit to talk about product-market fit, ’cause that’s something you hear people talk about all the time like, Oh, we wanted to try to get product-market fit, so as you start a company or as you come up with it, the idea that you’re creating a product and then you’re trying to find like, Who’s my right customer, but how does that intertwine with sort of like the work that you’re doing in the company, and then how do people know they’ve found it in terms of the data, I guess?

0:24:03.9 SW: It’s the million dollar question, right. And I think…

[overlapping conversation]
[laughter]

0:24:13.7 SW: And I’m gonna speak to what I know best, so I tend to do most of my investing around enterprise software. So for me, and I couldn’t tell you the precise point where something hits product-market fit, but what I know for sure is like if you have a rep, like an account executive who is consistently as in probably for more than three months in a row, exceeding their quota, and this is an account exec, not including the founder so like say your first sales hire, if they are more than three months in a row, hitting their quota, pour money on that thing, because… And again, my hypothesis here, and it’s kind of… I don’t have large in here, but probably a dozen companies. The reason is, as a very earliest straight shot at making your first sales hire, the chances that you have hired like the world’s best salesperson for your particular piece of software is very, very low.

0:25:21.6 SW: So you’ve probably hired an average to poor account rep in all honesty, and if they are able to exceed market average quota for that kind of software, you have produced something in the market that customers desperately want such that even if your rep is maybe not articulating it, the best they possibly could, and your marketing funnel hasn’t necessarily brought the absolutely ideal customer into the funnel for them to speak to, etcetera, etcetera, there’s such big margins of error all through that funnel, and they’re still kind of out-performing the average rep, you have got product-market fit. And what I see time and time again is like when you start to add more reps, your first account executive, which you previously thought was an absolute hero and legend of the game, and thought you were just so lucky to land, ends up becoming one of the poorest performers, once you’ve kind of brought in these other AEs who are actually better quality AEs, which kind of makes me feel really sorry for the first AE but…

0:26:27.4 MH: Well they had a good run.

0:26:27.6 SW: And sometimes the trick there is again, and this is probably peculiar to enterprise software, like probably the reason why rep A was so good in the beginning and less good later on when you add more AEs is typically the path in enterprise software is you start with a small feature set that a small development team can build, and then you slowly add on more and more features, and those features actually take you into a higher price point and a more enterprise-y looking customer and enterprise-y looking customers require to be sold to. And it’s a much more consultative sale or a problem solution-based sale, and you need a bit more of a skilled AE who can do that? And so usually the answer is to just make sure that rep A who’s so good at hitting quota with these small accounts, sticks to his knitting or her knitting and continues to really nail that.

0:27:20.0 SW: That’s one very kind of almost strict example, I would say in other areas, or maybe just like one yardstick if you just wanna look at product, ’cause a lot of businesses like defer revenue… What is the point of the product? What is the ultimate point of the product and what kind of product usage would indicate that users were really, really happy. So if I give a Canva example, in the early days that the hypothesis is that Canva makes it really easy for people to design fliers, invitations menus for their cafe, whatever, what at an initial kind of KPI, well before monetization that the team really looked at was the publish, publish or export equivalent button, because your engagement in the product alone wouldn’t tell you whether you were really solving the problem, like how many times have you spent a lot of time in PowerPoint or something like iterating on something and it just looks like crap, and so that wouldn’t have been a good measure, but if you’re publishing something, exporting it, sending off a print, PDF whatever you’re happy with it. It’s gonna get used. And so that was a really key indication of Customer Happiness and so… Yeah, I mean, hopefully those are two ways of thinking about product-market fit.

0:28:40.7 TW: No, that’s very helpful.

0:28:42.6 MH: On that example, to what extent was like… Was that Mel or Mel and Cliff saying, We think this is a good KPI. And then talking to their investors and saying, This is what we’re going to use, it may not be MAUs, it may not be revenue, it may not be new customers, and is there kind of a… Let’s pretend we’re not talking about that specific example, I’m not trying to uncover, but is there kind of a negotiation, an agreement where it’s like, these are gonna be the KPIs, we as the investors, are in alignment with this, and we agree that if we see this kind of growth and success, we think we’re on track and then do you stay on top of that? How does that dynamic typically work?

0:29:32.7 SW: Well, I mean, I would say in the ideal scenarios, it’s really more of a partnership, we’re getting together like every couple of weeks or every month, and as you are kind of getting ready to ship products like, Okay, what does success look like for this product, what do we think and we’re bashing around a bunch of ideas. And usually it takes a bunch of iterations to kind of really nail like, Oh, this is what success would look like for this particular product at this particular stage, or like our hypothesis is, these are our customers and this is why they love us, or why they should love us, and what would be the metric that kind of represents that, and rather than being kind of like a KPI where you kinda like, you’re bad, if you don’t hit this KPI, it’s really like, Hey, this is our hypothesis, this is what we think success would look like. Let’s just say that’s it, and we’ll report against that and in a month’s time, like we’ll have a discussion about whether you did and didn’t hit it, and more importantly why and develop new hypotheses and new experiments on new things that we can do to kind of incrementally get what we think customer happiness represents up and very often, maybe we got it completely wrong around why our customers buy us, and then you kind of have to iterate again on what customer happiness should look like.

0:30:49.4 MH: And so now let’s say we’re specifically not talking about Canva, so we’re talking about only generic other ones that have not been named, but if you’ve got the passionate founder who desperately, this is their life’s work and this is kind of their heart and soul, and you’re having the ability to re-negotiate this is from a broader analytics in general, you get the rationalisation, you have the wishful thinking, Oh, we completely agreed, we had this hypothesis, this was the KPI, we set, completely missed it. But we think… And they come up with like, do you have a harder time of figuring out when it’s the passion and the desire to have it work? Or maybe I’ll share my example. I was part of a startup that had a home pap smear test and we basically… It seemed really promising initially, and then it really wasn’t, but the OBGYN who invented it, the inventor and a couple of other people really wanted it to work, and so it made for that tension, like it seems like you have a harder… How much do you have to toe the line and say, I think you’re starting to, this wishful thinking on the explanation, that’s not what you get.

0:32:02.3 SW: Well, often it’s just, you know what I talked about before, product and distribution, so you really need to sort of have hypothesis all the way down the funnel, like What would customer love look like? And also, we have to acquire the right sort of customers for what we have our hypothesis that our customers can be really happy about, and I do think a very honest, empathetic but honest dynamic between your investors and the founders is really critical because you’re on the same team, you should all be wanting the same outcome, and you’re not really just like, not to pick on your OBGYN or Gyn but we’re not just investing you to kind of work your hobby, we are trying to build a really big business with a really big outcome, and find the truth of what the market wants and what we can build… I think about it as a truth-seeking exercise and try to have very, very honest conversations with founders and just make sure we’re aligned on that, and I think the best founders are not so in love with their idea that they can’t admit them hypothesis might be wrong, and really testing for that kind of growth mindset almost upfront and before investing is important.

0:33:26.3 VK: Probably one of the heuristics, one of the things that you test for right?

0:33:30.7 MH: Just to make sure for anyone who secured the HPV vaccine came out, as we were… Our market kind of like the future market shrunk drastically as our product, but that’s a whole other…

0:33:43.7 SW: That’s a great point because so much of why perfect simulation around venture is kind of impossible because there are so many things that can happen over a decade… Totally without, not within your control or even foresight, that can be fatal for the company’s success, and that is no doubt a huge contributor to the 50% that fail, but actually more often the 50% failure is actually just to find to a break up. That’s a huge proportion of it.

0:34:20.5 MH: Yeah, so one thing that’s sort of come out as we’ve talked, but we didn’t really address specifically, is like once you invest in a company, it sounds like your job is very much not done, you’re continuing on to work with that company, and it sounds like as you’re meeting, maybe it’s bi-weekly, monthly, whatever that cadence is. You’re having discussions about the metrics of the company, about the data behind it, to what extent are you as a venture capitalist involved in, it sounds like you’re sort of shepherding sort of a data culture in a way… I don’t know if you have thoughts about that, just because of you being involved and they’re… You hear stories from people like, Oh, I dread this, or the best founders are the ones that are able to give these regular updates and really be on top of their data and those kinds of things. And you mentioned sort of like, Let’s ideate and hypothesise around what these are, and then how do we experiment, and so that speaks to like, Well, we need to have a good data culture or an openness to discussing the metrics… I guess.

0:35:28.9 SW: I invest in technology, right? So people kind of self-select into that, I think a lot of the time, but it actually, rather than necessarily just being essential for an investor, it actually becomes essential for the team as you go from I don’t know three, four people to 20 to 50 people coordinating the activities of these people, it becomes really important to sort of have these abstractions, I suppose, of what success looks like without holding too tightly to a particular plan, ’cause you do want your people to be autonomous and experiment and find their way in a very abstract landscape to an outcome. And so having these kind of hypotheses associated with certain metrics, are very good ways of aligning teams on the journey, and often the kind of early hires would become very frustrated if this does not exist or they’ll take responsibility for inserting it into the business, if it doesn’t already exist. So I’d say it’s less of an issue than you might imagine.

0:36:40.0 MH: No that’s interesting, I feel like it’s a huge issue within companies, I was just fascinated by the dynamic that actually having a good culture between your venture capitalists and the founders is sort of an important thing.

0:36:53.9 SW: It’s definitely, when it’s not there, it’s very frustrating, right, because we obviously don’t work in the business day-to-day, we’re trying to be helpful, but it’s very hard to get visibility like…

0:37:03.2 MH: Well, I’m sure there’s things founders do that increase or decrease their confidence, your confidence in them as you progress. That seems like a normal, relationships progress or digress.

0:37:15.8 VK: I have a kind of a random question for you, and I might be a little bit of a loner or a weirdo on this one, but I often fantasise about taking the knowledge I have today and going back to my old jobs and being like, Oh my gosh, what I would do today if I had the knowledge I have now, and so especially curious, like any lesson learned stories, definitely interested in those, but especially in your past and former roles as a product manager, so what things have you learned in your recent years in this investment world that you would like to tell your product manager self… I’m very curious about that.

0:37:53.5 SW: Oh my goodness, I’m gonna immediately contradict myself, and I actually think we weren’t data-driven enough. It was also 100 years ago, I’m quite old. The data was, it was still emerging as all of this stuff we take for granted was like still very imagined like 13, 14 years ago. But yeah, I would say to be much more hypothesis and data-driven, I think sometimes I fell in love too much with my ideas, it’s really embarrassing to admit that now actually, and then I would say as well, like to be much less emotionally attached to the outcome of experiments and almost like try to make them fail.

0:38:41.2 SW: Like startups are so hard, you actually want to kill ideas faster. And I think as a founder, you get so excited and you want it so much to be real that you try to navigate the past rationally enough to like success when really you almost want to like smoke out all the reasons why it might fail like two, three, five years from now sooner. Does that make sense?

0:39:11.4 TW: Interesting. It makes a ton of sense.

0:39:12.4 MH: Does that, I mean the specifically putting a two, three or five year, that feels like there is a part of it where you want to find the, God, it’s gonna sound like I’m an MBA. Like it’s, you want the hypothesis with the most leverage. Like you want to find the ones that successful or not, it’s going to pay the greatest dividend over the long haul. Whether that means you stop and pivot sooner. So you’re of course correcting or whether you’re jumping out on an opportunity and you get a three month, extra few months on the competition because you found something faster.

0:39:58.5 SW: Yeah. It’s like you’re trying to bring forward like visibility of long-term success. Like sometimes, like your very early customers just don’t represent a larger market and you wait like three years, five years, whatever, to realize that actually it’s a niche, it’ll just never be bigger. And for quite structural reasons that you can’t change. And you could have probably gotten better visibility on that earlier if you’d spoken to more of these customers who were gonna say no and asked why. Or like spent more time like trying to understand whether that dynamic was going to change or was unchangeable for reasons that you can’t do anything about rather than like focusing on just selling to the kind of converter… Preaching to the converter… You know what I mean? Like, and you got, and look, this is super nuanced, right? Because like most of the time you do wanna try and get people to say yes and buy and so on. And so you just, you have to kind of hold these two states in your head at the same time. But in general, like I do think I wasted time and I do think a lot of founders waste a lot of time early on avoiding no or avoiding, yeah, when there should be curiosity about no as well as trying to get yeses.

0:41:23.5 MH: Well that seems like a, it’s kind of what’s endemic within a lot of enterprises. Putting aside the founder or the startup mentality. There’s a desire to do exactly that. And you’re bringing back some memories of organization, like large enterprises that have said, oh, we’re gonna have an innovation wing or a VC wing. And it’s the idea that they’re going to do that. And it just, it’s so hard to take that mindset of where are we heading? Find the nose, let’s have a hypothesis measure to.

0:42:01.4 SW: Endowment bias. Yeah. And people fall in love with their ideas, which is very human and, yep.

0:42:08.4 MH: And they can try to carve it off and say, we’re gonna have like this kind of in-house VC investment mentality, but then they draw from people who are sort of corporate. I mean, I think there are lots of reasons. I still think it’s a really, really helpful kind of mindset for anybody to be thinking that way. It’s just, it’s really hard to pull off. It’s hard.

0:42:33.4 TW: Speaking of hard to pull off. We have got to start to wrap up. This is so fascinating.

0:42:40.3 MH: I had a whole other avenue of wanting to figure out how you measure the portfolio’s performance, but we just can’t go there.

0:42:49.5 TW: We’re both gonna get to it. Sorry.

0:42:50.6 MH: And, but here’s my startup pitch. I’m just kidding.

[laughter]

0:42:53.8 MH: We do have to start to wrap up. Samantha, thank you so much for coming on the show. This has been a phenomenal conversation and I’ve learned so much.

0:43:03.5 SW: No, it’s been really fun. Great questions.

0:43:05.6 MH: I own or I’m the managing partner of a very small consulting firm, which is not something that VC’s invested, but I’m still getting so much value out of just listening to you talk about these things and it’s been awesome. So personally I’ve gained quite a bit from the conversation I think everyone has. But one thing we wanna do is go around the horn and maybe share a last call, something we found of interest that maybe we’d like to share with our audience. Sam, you’re our guest. Do you have a last call you’d like to share?

0:43:33.4 SW: Well, I recently read this like profile of Claire Hughes Johnson, who’s the COO of Stripe and a bit of a legend in startup circles. And one of the questions was like around what her most contrarian, high conviction opinion was. And she says that reading literature is critical to being a high functioning human. And I feel so validated by that because…

0:43:56.3 MH: I believe that too.

0:43:57.4 SW: Well, there you go. We’re in the Claire Hughes Johnson Club because I read a lot of fiction and like did a major in English Lit and I’m embarrassed about it ’cause I think I should have done like, I don’t know, data science or something like that.

[laughter]

0:44:10.3 SW: And, but I really do feel like fiction in enriches me and enables me to like run simulations on what different people would do in different circumstances and why people do the things that they do. And like all of that like adds up to better prediction around the future. Better understanding about people’s motivations and therefore better responses to all of that. And so anyway, I just like that has really stayed with me ’cause I’ve been wanting to feel better about the fact that I don’t read enough business books and an English Lit major.

0:44:41.4 MH: Well, someone who’s always looking for good fiction and I’ve gotten many of them through the Pikes. Can do a follow up last call and like a fiction book you’ve read in the last two years that you just can’t let go of that you would recommend?

0:44:57.3 SW: Oh, the one that’s like popping to mind is Three Women. I loved that. And oh, I forget the name of, it’s a Korean one that’s like a family saga. It was amazing. Can I follow up with like…

0:45:13.5 MH: Sure. [laughter]

0:45:14.5 TW: Sure, yeah.

0:45:14.6 MH: We’ll put them in the show notes.

0:45:16.5 TW: We’ll start a little email thread. No problem.

0:45:18.4 SW: Yeah, yeah, yeah. Let’s start the book club. I love it. [laughter]

0:45:21.3 TW: Michael are you still there?

0:45:23.5 MH: I’m writing things down. It’s like Three Women. Got it.

[laughter]

0:45:28.1 MH: No, there’s actually science behind this…

0:45:30.4 TW: Hey, Michael, guess what? We’re recording this, we have a record.

0:45:33.5 MH: It’s fiction. Yeah. But this is important. Fiction actually helps you grow empathy. That’s one of the things that fiction does. They’ve done studies about this.

0:45:42.5 TW: Not always. ’cause I read a lot of fiction and I’m still trying to bust through.

0:45:48.4 SW: It’s on a fiction too. Your sci-fi fiction doesn’t count.

0:45:50.4 MH: But we’ve got so much hope for you Tim. And by the way, Tim why don’t you go ahead and do your last call.

0:45:56.5 TW: I could recommend the, I’m terrible at remembering book titles ’cause I literally, the Anne Friedman weekly newsletter, she recommended a book that I’m now reading and it’s like weird and delightful. But that is not my last call. But I will put that book also in the show notes. I think I have done a last call before of the Choiceology podcast with Katy Milkman, but I want to call out a specific episode that was about a month or so ago that was called An Accidental Experiment and it had Steven Levitt and Solomon Ezra and Stephen Spector. So it was really kind of, I never heard natural experiments referred to as accidental experiments. And it does seem like there’s a little bit of a distinction, although Steven Levitt was kind of using them interchangeably. But it was a nice episode talking about some examples from history that I was not familiar with of natural experiments. And it did a good job of talking about how natural experiments can be handy and you kind of have to really look for them to see if they exist. So I think that might be kind of a double. I think I’ve talked about Choiceology before.

0:47:12.5 MH: Awesome. Alright. Thank you. Okay, Val, what about you? What’s your last call?

0:47:16.5 VK: Yeah, my last call it’s an article, a Medium article from the analytics team at Meta. It’s kind of a long title but it’s Notifications why Less is More, how Facebook is increasing both user satisfaction and app usage by sending only a few notifications. So yeah, quite a long title, but it opens by saying that this was written by the members of the Facebook notifications data science team, which just thinking about having an entire team, [laughter] focus on the notifications part. That was kind of interesting. But one of the reasons why I loved this read is because they talked a lot about how they used user research and they conducted multiple AB tests and that they were really thinking about the long-term effects of what they were putting out into the world. So not just looking at the short term AB test winner okay, roll with that conclusion, but understanding how the change in frequency of notifications was affecting user behavior. And again, like they said in that very long-winded title, the relationship with the product long term. And so it was a really robust way at conquering what seems like such a simple question but is so foundational to the product. And so I actually thought it was a really great read.

0:48:32.4 TW: And yet my Facebook app on my iPad always shows notifications and yet I open it and there’s nothing there. And I want to know names of the people on that team, but I don’t know.

0:48:42.5 VK: We’ll chase them down. We’ll chase them down to you.

0:48:44.4 TW: Michael, you have a last call. Sorry [laughter] It’s a little late here. I’m a little loopy.

0:48:49.3 MH: I’m confounding the data ’cause the first thing I do is turn off notifications for all apps. It’s just like, not gonna have that happen. Alright. Yeah, I do have a last call. So James Zhang at Instacart recently wrote an article about how they’ve been transforming their data at Instacart and leveraging DBT and Airflow to manage data transformation. And it’s a pretty fascinating article which I got from, well, I got some cool things out of reading it. So it’s a good read if you’re in the data engineering or analytics engineering space at all. There’s some good learnings they’ve, they got from their work and that was pretty, pretty informational. So yeah, I can recommend that.

0:49:32.6 MH: Okay. You’ve probably been listening and you thought, man, I had love to learn more about that. Or I wanna talk to these podcast people. Well we’d love to hear from you. There’s this great ways to do that. You can talk to us through the Measure Slack group, which is a place where we all are or on I guess it’s X now, not Twitter and Link or LinkedIn. We have a LinkedIn page where you can read us as well. Yeah, yeah. Tim, none of us are on any of your fancy also ran social media tools like Mastodon or Blue Sky or Threads or whatever.

[laughter]

0:50:07.3 MH: So dumb.

[laughter]

0:50:10.0 MH: Anyways, it’s what a world that’s a startup idea. Social media company. But a good one.

[laughter]

0:50:17.3 TW: Canva’s gonna become a social network ’cause everybody already has it. See there’s the… It’s on there.

0:50:22.1 MH: Yeah, I mean you’ve got all your art and everything in there. So basically you need to introduce a social graph and you’re good to go. Be like my space all over again. [laughter] You can decorate your page.

0:50:34.4 VK: I need Mower here to speak to that one.

[laughter]

0:50:38.4 MH: Yeah. Well hey…

[laughter]

0:50:39.0 MH: Good ideas can come from anywhere. That’s what I always say. Anyways, the point of all that was to say we would love to hear from you and please do reach out. Let us know. We’d like to hear from our listeners. All right, well, thank you once again Samantha, for coming on the show. Real pleasure. Felt like learned quite a bit and it’s a cool topic to learn more about. And thank you for spending that time with us.

0:51:04.3 SW: My pleasure. Thanks for talking with me.

0:51:05.5 MH: And no show would be complete without a huge shout out to our producer Josh Crowhurst. Thank you Josh for everything you do behind the scenes to make the show possible. We appreciate you and I know that I speak for both of my co-hosts, Tim and Val, when I say no matter what series or what your idea is, just remember, keep analyzing.

[music]

0:51:32.4 Announcer: Thanks for listening. Let’s keep the conversation going with your comments, suggestions, and questions on Twitter at @analyticshour, on the web at analyticshour.io, our LinkedIn group and the Measure Chat Slack group. Music for the podcast by Josh Crowhurst.

[music]

0:51:49.3 Charles Barkley: So smart guys want to fit in. So they made up a term called analytics. Analytics don’t work.

0:51:56.4 Kamala Harris: I love Venn diagrams. It’s just something about those three circles and the analysis about where there’s the intersection. Right?

0:52:05.5 TW: Just did, I hit record too late to get any good outtakes.

0:52:09.4 MH: Oh, well it’s…

0:52:10.4 TW: Sorry.

0:52:10.5 MH: That’s gonna be your problem too.

0:52:12.5 TW: Okay. Carry on.

0:52:14.3 MH: Who do you think you are, Mr. Dick stuff?

0:52:19.4 VK: There’s your outtake. Tim. [laughter]

0:52:21.3 TW: Yeah, there you go. [laughter]

0:52:28.4 MH: Rock flag and hypothesize.

Podcast: Download | Embed

Subscribe: RSS