00:00:05.75 [Announcer]: Welcome to the Analytics Power Hour. Analytics topics covered conversationally and sometimes with explicit language.

00:00:13.94 [Michael Helbling]: Hi everybody, welcome. It’s the Analytics Power Hour. This is episode 281. It’s finally done. The big report for the monthly business review. And you send it off. And over the next few days, you hope maybe you’ll hear something back. You might not hear anything. As analysts, sometimes we don’t get a ringside seat to how the business uses the data reporting and analysis that we provide. I mean, we certainly hear about it when something goes wrong, but how is the data being used? What strategy is it supporting? How does your work fit into the bigger picture? Hey, Moe, kiss. You’re an executive yourself now, but I think you remember a long time ago being an analyst with questions like these.

00:00:54.17 [Moe Kiss]: I still have these questions.

00:00:56.63 [Michael Helbling]: Let’s be real. Oh, okay. Well, yes. And my other co-host, Val Kroll. I know you also have a lot of executive experience, but I know you and I’ve shared many stories about learning the ropes as we climbed up. Stories or scars? No. Well, the best stories come with scars a lot of times. And I’m Michael Helbling. And I’m really excited for our guest because we’re finally getting some answers directly from the top. Anna Lee is the CEO of Flybys, Australia’s number one loyalty program. In her three decades of experience, she’s held CFO, COO, and other leadership positions at companies like the Iconic, Nora, and Groupon. And today she is our guest. Welcome to the show, Anna.

00:01:41.68 [Anna Lee]: Thank you so much. It’s such a pleasure to be here with you guys.

00:01:46.17 [Michael Helbling]: Yeah, I’m so excited to talk to you. I think this is a great opportunity for our audience to kind of see the other side of the business that we don’t get to interact with a ton. But as we roll into the conversation, I think it’s a good thing to set the playing field and for everybody to understand is, could you tell us a little bit about Flybys and your role there as CEO?

00:02:07.79 [Anna Lee]: Well, maybe the best way to do that is maybe provide a little bit of context of my career. Firstly, I’m a chartered accountant, so my background is actually finance, so very good with numbers. And so I spent many decades as finance director, CFO kind of roles, and that really helped me understand numbers and the depth of business behaviors and decisions and then I had an opportunity to move into the chief operating officer and that was when I was at the iconic which is just before flybys and that really allowed me to focus. and concentrate my efforts and my knowledge around operational breadth. So understanding how all the dots connected in the business, understanding how to really create productivity, efficiency, and drive performance. And then about three years ago, I had the opportunity to join Flybys as its CEO. And I think that really then became the culmination of those skills combined with leadership. Flybys is Australia’s largest retail coalition loyalty program. It is a joint venture standalone entity. It is owned 50% by Coles Group and 50% by West Farmers Group, two of some of the biggest ASX top 10 listed businesses in Australia. And my role is was to take the business into a more digitally and member focused direction. And of course, a huge part of that. is making it data-led.

00:03:43.93 [Moe Kiss]: How do you think your experience? I suppose one of the things that’s always top of mind for me is different execs seem to have different perspectives and views on data. Coming through the finance world, I feel like that would be a quite different perspective maybe than some other CEOs. When you look around your peers and how they got to their position, do you feel like finance gave you a special edge or Or is it like unique in any way or do you think that those skills just get learned regardless of how you come through the industry?

00:04:21.19 [Anna Lee]: I think CEOs come in different shapes and sizes. It really comes down to what is suitable for the business at its time. I think one of the things that many businesses that I’ve been part of have been quite digitally focused, so less physical footprint, more reliant on what the data is telling us. That was true by time at the Iconic, which was just prior to Flybuys. And of course, with Flybuys, even though you see it obviously at the supermarkets and bunnings at Kmart, itself is a digital business. And so the importance of understanding data and for things to be evidence and fact-based skews more towards that rather than, you know, perhaps some of the other pieces of data that are more physically demonstrated. So I think there’s no right or wrong on that, Moe. I think it really comes down to what is important. And I think What has been interesting in Flybys is it’s a brand that is very well known. It has almost 95% brand awareness and it has been around for 30 years. Almost every Australian knows Flybys, has seen it, knows about it and we are almost a 10 million active members. It is the scale of it in Australia is quite big. The good and the bad thing is that the good thing is that you have enormous amounts of information that talks to member behaviour and so on so that you can really create and unlock this value for them using personalisation and so on. On the flip side, Over 30 years, there’s a lot of myths and beliefs and stories that some are true, some are not. But they’ve been kind of the culmination of many different people and many different stakeholders. And so right now, what was important was to reset those stories and to ensure that they were properly fact-driven and data-driven. So right now, I think that is the right thing for the business now. I think the experience that I have around confidence in data and numbers and so on and the storytelling that it creates is what’s right for the business now.

00:06:46.76 [Val Kroll]: I love that. Resetting the stories. I’ve definitely worked at organizations and institutions and worked with companies that hang on to those stories and it’s to its detriment, right? And it’s so funny how quickly something becomes the truth if it’s set enough times in a meeting, right? I really like that that’s one of your priorities.

00:07:05.27 [Anna Lee]: Absolutely. And they also can go quite wide.

00:07:09.84 [Val Kroll]: Use as weapons.

00:07:13.10 [Anna Lee]: No, no, not at all. It’s probably more about I think people get quite. enthusiastic about the storytelling and perhaps sometimes, you know, further embellishments are made on stories that perhaps were a lot more simpler and potentially were quite fact-based and grounded, I’m certainly not suggesting any of it was, you know, not fact-based but over time some of those stories just become, you know, they become bits rather than facts.

00:07:43.08 [Val Kroll]: Sure. Well, so I’m going to ask one other foundational question, because I’m sure that this will be an important backdrop element as we get into some of the other questions and discussion. But 30-year-old company, you’re charged with bringing it more digital. I’m curious how you engage with data. So what is the current organizational structure if you want to talk a little bit about that? Or how do you interact with the data practitioners or analysts? or data scientists across your organization?

00:08:11.02 [Anna Lee]: Well, I think in the first instance, I have to acknowledge my role as the CEO. So it’s important that I seek answers that are evidence-based and data-led. So it very much comes from the top. So that creates a cultural conditioning. So that makes it very clear that We do not want to talk about things that are perhaps not so data-led. I mean, hunches are great, but let’s have a look at the data and understand it. What that drives is then the focus on ensuring that the data is easy to access. It’s reliable. It’s credible. And also, culturally, the business has a good level of understanding around how to use data and insights. So I think first and foremost, it’s multi-layered. So you set the tone, you have to acknowledge there’s a CEO, you have the… biggest way of influencing how it’s extraordinarily powerful. And I think being very clear that data plays such an important role in making decisions, that’s super important. I think the other piece that I like to, there’s a couple of other themes that I really like, which is productive curiosity. So I really want people to be curious about the why of things that they see. So if we’re seeing certain metrics behave or move differently, it’s really important to be curious, but then you don’t want to be curious to the point where it’s just interesting and it’s It’s just a wasted effort. And the last thing we need is people just running off and finding answers to things that don’t actually move the needle. So I kind of like it to be, there’s a real skill around productive curiosity. So embedding that type of mindset, that’s what I kind of demonstrate as lead by example. It’s like being very, very disciplined about the types of questions that I ask, not random things, but very focused on what the right thing. The other concept I really like is also what I call informed pragmatism. So again, data teams often love most just going to die when I tell them. But, you know, there is just this desire for a lot of data scientists and analysts to be really perfect and exacting and precision because that is a reflection on their own self-pride and there’s a lot of attached to things being right. And I totally understand that because I spent decades in finance and everything was about it had to be right. But, you know, at some point you’ve also got to decide What are the decisions that absolutely need precision? And what are the decisions actually just need enough to validate that it’s kind of legit and it smells right? What other information that you can add, which are non-data perhaps, more qualitative aspects that then combine together to give you a pretty solid position to be able to make a pretty good decision that feels informed enough, but is pragmatic so that we’re not sending people On three weeks of digging for probably not much more certainty than if you’d stopped now so you know that they’re kind of two concepts that I like to apply. I only just recently kind of put some words around that kind of concept, but that’s kind of the approach that I have. So in terms of back to the question around how the teams are structured, we have quite a mix. So we have a centralized team that ensures the platform and the data hygiene is there. We’ve just spent and invested significantly on our platform uplift, a piece of work that was very much needed over years, even though business has been around for a long time. It hasn’t actually had the platform to enable that. But as an enterprise capability, being data-led and understanding data is something that we expect of every single person. Now that has been a cultural transformation that we’ve been on. So we need to invest in the training for roles that previously didn’t need that. And then we also need to hero, I guess, the people that, and champion those people that do have those skills and how do we harness those skills such that it’s, you know, that skill is democratized.

00:12:55.62 [Michael Helbling]: You ever wish your data stack came with an easy button? Well, enter Fivetran, the data integration platform that lets you set it, forget it, and sync it. While you’re catching flights, catching up on sleep, or catching waves, Fivetran is catching your data from every source and pumping it straight into your data warehouse. No manual pipelines, no maintenance drama, no 3AM alerts because someone’s API changed again. just reliable, automated, always on syncing of your data. So your dashboards stay fresh, your analysts stay happy, and you, you get some well-deserved R&R. Take a break. Well, take two. 5Tran’s got you. Go learn more at 5Tran.com slash APH. You can see interactive demos. You can even get a 14-day free trial. That’s F-I-V-E-T-R-A-N dot com slash APH. Is your out-of-office hero that your Datastack deserves? Five Tran.

00:13:55.99 [Moe Kiss]: That sounds amazing. I honestly, I wish every… And I agree with you a lot more than… Yeah.

00:14:02.10 [Michael Helbling]: I’m stealing both of those phrases, by the way. Just FYI. Yeah.

00:14:07.00 [Moe Kiss]: Yeah, but the thing that’s actually that I love is like the self-awareness of, I think sometimes folks in leadership roles don’t realize like, they’ll be like, oh, this would be interesting to know or like, would it be nice if I could look at blah? And like, sometimes they don’t realize like teams hear that and they hear, I need you to look at X and I need an answer that says Y with like 100% confidence. And it’s like, I wish all, folks leadership roles had that self-awareness that sometimes they can create swirl and that’s not actually what we want. What we want is what is the right information that we need to make a business decision and that’s the point that we need to stop at.

00:14:45.03 [Anna Lee]: Exactly and I think there is also perhaps that’s where my experience around my operational experience as COO, I was in that role for four years and we were obsessed with like unit economics and productivity and so I now think about all of our team members’ time as value add or opportunity cost. So it’s got to always feel like this should only take you an hour or two. If it takes you longer than that, stop. Because it’s not like the value of what you’re going to come back with is not worth Moere than two hours so i think giving people a bit of guidance around is that and you know is that something really really super important that’s going to move a big needle or is that something that’s like okay that’s interesting. That will validate something else and therefore actually only need you to spend a very little amount of time on it i think it’s also. Certainly, in my CFO days, I used to have CEOs that would ask me all sorts of random questions, and they had reasons for them. They felt random at the time, but I legitimately know that they were. I think what I understood as the CFO was the complexity level of getting that information. So often what I would counter in my CFO days was, you know, before I run off and get my team to spend a lot of time trying to dig that information, what is the problem you’re trying to solve here? And it’s about the clarity around what is the problem. And if you’re clear on the problem statement, actually what you find is there are different pieces of information that you can provide that gives you the same answer. And often I would suggest something that was much more pragmatic that would still provide the answer rather than delivering the specific requests that the CEO provided. And I think having had that benefit of that experience now in the CEO role, I’m able to kind of counter not just request a specific, the danger is requesting specific metric, rather what I like to compliment that with is, hey, I’m wondering if you could tell me if there is a strong correlation or relationship between these two metrics, because what I like to know is does that have an impact on this strategic direction that we have? It allows people to understand why I’m asking the question rather than random curiosity. And also, I think it allows for, you know, whether it’s the, you know, our head of data or analysts or anyone to say, hey, I get what you’re trying to solve. But actually, that is not an easy metric to do that might take the team quite a lot. But actually, I can give you something else that probably give you the same answer. And I’m like,

00:17:47.30 [Val Kroll]: So I feel like Anna could have a golden dot off. So I’m thinking like, okay, a CEO is like, I like to articulate hypotheses for my data teams to help me validate. So I’m thinking, and I hope your last call is, here’s all the roles that I have open for my team, because everyone’s gonna come work for you after this.

00:18:06.99 [Anna Lee]: That’s awesome. No, I love it. And I think there’s also, I think the other piece that’s quite difficult as well, like, you know, different, again, different businesses, I’ve seen different setups, but, you know, having that capability across. the business, no matter what your role is, and it obviously will be different depending on your role. Some might just need foundational understanding, and of course, some roles will need to have a lot more sophisticated understanding. But being able to, I mean, my vision and ambition is for everyone at Flybuys to have the right level of competency around understanding data that it necessarily have to have founded. But they need to know what to do with it and whether that piece of information is important or not. That’s where we want to go. And so that’s been actually quite a cultural transformation in itself. And if you’re going to expect that, then as a business, you also need to provide the support for everyone to get there.

00:19:07.17 [Val Kroll]: I love that. One of the things I’ve been hearing you say since the top of this episode is evidence, and I love that you use that to frame out. Because you’re acknowledging there’s lots of different ways that we can get at this. There’s lots of different tools and methodologies. Lots of different levels of certainty, depending on the question that’s being asked or the decision we’re trying to make. But one of the things I’ve noticed hinder organizations from being so flexible about the evidence that they bring to bear is sometimes organizational structure. So if a senior leader asks a specific person or team that has you know, specific tools in their tool belt or like guild like groups, if you will. Hey, here’s a question that I have, but the most appropriate tool if you’re considering the economics, like you were saying, like the units is actually market research, but that team sits in the completely different part of the business. What are some of the ways, like if you were talking to another senior leader, because this is the type of change that as analysts, we always say, well, that’s something that has to come from the top. So here’s my chance to ask the top. What are some of the things that you would recommend or that you think we should keep in mind or think about to try to help change some of this culture to be a little bit more flexible with how we right size the different tools and methodologies with the questions of the business? And you might say that it does have to come from the top. Tell their CEO to call me. But I’m just curious. I want to explore this because I think this is really important and something that I think is a really important evolution for lots of data teams and practitioners.

00:20:35.97 [Anna Lee]: That’s a lot of unpacking.

00:20:38.39 [Val Kroll]: Just take that wherever you want to go.

00:20:42.28 [Anna Lee]: Well, maybe a couple of ways I might approach that. Not sure if I know the exact answer to that, but maybe a couple of ways that I think about things is I think the business needs to find the right balance between routine cadences and discipline and a culture of creating space for whether we call the practitioners or team members to um question and and find things I think like for example um like for me I think sometimes people ask me you know daily, weekly, fortnightly reporting what’s your what’s your view on that um and my view is is that um cadence is only important for consistency so I’m a real believer of discipline so I don’t want to be making decisions and the business being made decisions on ad hoc things that are fluid and so on. So I think some sort of regular reporting structure creates consistency and constant monitoring. So if you get invest right and you get the right metrics, then you should be looking at some of these things quite regularly. And that grounds people in the right evidence, but the forums which you talk about them create the space for people to ask the questions. I’m not sure if I’m clear there. But maybe an example would be like one of the things that we introduced in his business, which was probably some of the hardest things that we ever did was, as part of this data reset, was we actually hold fortnightly Valley Driver Tree meetings. And that is actually how we drive performance in the business. It’s 90 minutes every fortnight. We have allocated people who own each of these metrics. And as part of that, we needed to ensure that there’s a lot of time spent on what the right metrics are. It also drove a lot of time investment in getting those metrics reported easily, seamlessly. They didn’t exist the start and also required us to train and coach our people to talk to those metrics well and create the space where people felt they understood what that was. But the great thing with Valley Driver Trees is they really help you understand which of the branches of the tree are the most impactful. So it helps us also to prioritise the things that matter. So all of a sudden, Instead of talking about every single branch equally, that allowed us to say, actually, that branch contributes 85% of everything. Let’s make sure we talk about that. The other stuff is important, but actually, you could spend half an hour talking about one thing, but if it only is worth less than 1%, it’s probably not that useful to talk about. So that allowed us, so that’s been a real journey for the business and I think as leaders, as the CEO, I intentionally turned up at those meetings and try my best to kind of influence role model, how we talk about data in a way that is grounded in numbers, but actually the narrative and the discussion is about the business. So when we are talking about points issuance, we are just talking about how many billions of points we’ve issued and so on. What we connect it back to, you know, that’s been partner activity, how much value has that created for members? What’s the quantifiable amount? Is that up or down? Are we delivering, you know, more value to our members compared to last year? Those sorts of conversations. So I think it’s quite multi-layered, really. You’ve got to have the right systems and platforms. You’ve got to have the right metrics. You’ve got to have the right forums, and you’ve got to guide and share those meetings in a way that focuses people on the right conversations.

00:25:01.29 [Moe Kiss]: I love hearing how much personal responsibility you take about the data culture at your company. That is just a really incredible thing to hear. But I imagine in previous roles, There have been also, because it’s like it’s a two-way street, right? It has to be top-down and bottoms-up. And part of the data culture also comes from the data folks that you employ and their managers and all that sort of stuff. What are the things that you’ve, just in all of the roles that you’ve had, that you’ve seen really stand out as like what makes a really strong data team more looking at like the bottoms-up attributes versus kind of the top-down influence you can have? Like what are those things that we should really be striving for in our teams and as data scientists?

00:25:42.55 [Anna Lee]: I am a big fan of the so what like everyone in the business will know it’s like if you tell me something I’m going to go what’s the so what so often I’ll be you know listening to something or a reader a report or a document and you know we still find you know there are times where you know the author will at a table of information and very little narrative or commentary about what that information is. So I think one of the most important things, I don’t want information for the sake of information. Not only is that, I guess, not very evaluating, but actually it actually pushes a lot of cognitive load on the reader. So for me, I look at the information, I’m like, Oh, what am I meant to make of this? Like, is it, am I looking at this column or this number? Or what is it that I need to draw out? So I think there’s a real, real important skill for anyone, whether data is in your title or not, it doesn’t matter, like this applies to anyone, is what is the so what. And if there isn’t a clear so what, then actually just take the table out. I actually prefer because it’s just noise. So I think it’s It’s about actionable insights. I know that’s a cheesy word it’s often used, but that is actually how I interpret it. Is this information going to change or impact? Does it validate where we’re going? If that’s the case, then show me whereabouts in the data table that it shows that. something that we’re doing now. we should raise awareness that it’s like, oh, hang on a minute. This doesn’t look right. Should we be reviewing a decision that we’ve made? Or could also fit in the other category, which is like, oh, look, it’s quite interesting. Can’t tell right now, but we should watch that to see whether that does have an impact on something that a decision that we’ve made or we need to pivot or adjust or otherwise. So that’s my favorite question is, What’s the so what? And I think that is also, again, it pushes and conditions a particular type of thinking in the business, because if you can’t answer it, then you’ve probably got to work out a way to answer it.

00:28:25.43 [Val Kroll]: Obviously, there’s a lot of clarity, especially when you’re in these forums, you’re talking about asking those questions and keeping focus on what’s most important with the prioritization and the value framework that you talked about. But when data teams are partnering with some of the different functional groups, lower down on the tier, It’s a little bit easier to lose the thread sometimes, like connected back always to the strategic vision or the plan. Do you have any advice for data practitioners to make sure that they’re anchoring on something that is actionable that’s going to move the business forward? Because sometimes, again, it feels like you’re looking in a smaller piece of the puzzle. But do you have any advice for the analyst that wants to be kind of performing in the way that you’ve described? What are some things they could do?

00:29:14.55 [Anna Lee]: Well, I think there’s no right or wrong. I mean, there are times where it’s really great for data people to stay focused on one particular thing and not get distracted. So I don’t think there’s a black and white, but I think your question’s probably more on for those who, you know, want to kind of go deep and also and broad as well. I remember when I was the COO at the Iconic and actually armed all of my operations directors with an analyst because I wanted them to be able to use the information. I wanted them to have someone that they could go to that would help them connect the business decisions with the information that was coming out of our systems. So I think that worked really well because I was able to ensure that every operations director was very clear because we had OKRs, we had a whole framework that set up what we were trying to get at. And for us at the time, it was very much about cost per order. That was our obsessive thing. Everyone needs to be obsessed about that because we owned it, we were accountable for it. So that then meant that you know all the cost could be split up between the different areas and functions of operations and actually there was a beauty around the fact that you could have a group that focus on okay you guys are in charge of looking at the opportunities around delivery guys were all in charge of looking at the operations excellence opportunities around, you know, taking the photography images that appeared on our pages, you know, there’s a whole different set of functions that actually or ladder up to the collective costs and so. there’s a benefit of everyone being focused but then every month we would come back and actually review what everyone contributed to and and be very clear which components of the costs were driven by what you know whether that was up or down and we’d have real learnings about that so um and that was a collective conversation I would have you know, basically the whole operations leadership team on that core that would actually analyze all of that and everyone just knew it would just be one of those performance like what is that, what drove that and that really then honed I think all of the skills of not just the analysts but also our operations team members to learn how to talk about the results and the big picture. And also a lot of it was also like, you know, no particular cost is ever driven in isolation with one function. Often it’s upstream or downstream. So it was also this great camaraderie around this common goal. And we all wanted to collectively reduce costs by X and I think creating this culture that we’re all in together and collaboratively can solve it rather than create silos of like, oh, I overspent, that means I’m in trouble and another team underspent. And, you know, it was all about, okay, you know, being very open about, okay, we’re gonna have, we’re gonna have pressures on cost here this month. What can we do over here to compensate for that? And that was just a really wonderful team culture that we created around this common goal. So it’s a bit of clarity, but also collaboration as well.

00:32:42.68 [Moe Kiss]: It’s really interesting. What’s coming to my mind is the word trust. Because I think one of the things we often discuss as data folks is the fact that we need to give the business bad news. We need to give the teams that we’re working with bad news. We need to give leadership bad news, that sort of stuff. You’re actually talking at another dimension, so it’s not even the data folk doing that. It’s the business leaders and the different area leaders being willing to come to the table and have a discussion about, hey, this metric has gone down and here are the reasons why. But having the trust that everyone around the table is part of the same team, we’re all united on the goal. This is a really interesting situation to hear about because what I’m hearing from you is so much more of, I think sometimes in the data space, we think that data culture is our responsibility and it’s something that we need to drive across the business. And what I’m really getting such an appreciation for is how much the leadership team can play, but also how much team culture more generally plays a role of, you need to have those really high levels of trust if you’re going to have very robust conversations about what’s working well and what isn’t. And that’s what’s top of mind for me.

00:33:59.25 [Anna Lee]: Yeah, I mean, that’s really interesting. I feel it should never be the role of the Data Insights team to be driving the culture.

00:34:08.18 [Val Kroll]: Can you say that again for the back row?

00:34:10.07 [Anna Lee]: Because… That’s a very hard expectation. I think with everything, you know, I think as a CEO, you actually have to role model everything. So it’s not just about data and insights. Of course, I, you know, I have to role, I see myself role modeling everything. So that’s, that includes psychological safety. It includes what our member first and lead narrative within the business. You know, all of that comes from the leadership team. And I tell my exec team that we must be the highest performing team in the business. We need to operate at a certain level. We need to role model. We need to demonstrate so that people can see it and also you can play a role in coaching it. So I think with everything, I’ve not personally run a data and insights team specifically, but it’s very similar to a finance team. So I’ve run many, many finance teams and I was in finance leadership for over two decades. And it was the same. You don’t want the finance team to be the only people that care about money or can care about commercials that creates a lot of friction and in fact extensive. We also apply data on analyzing our engagement survey results as well. It’s very data-led there. But all of those insights will tell you that if you put the onus on specialist teams to drive deep cultural behaviors that you expect of everyone, that’s it’s going to end in tears because you’re just going to have a lot of frustration, a lot of resentment. So, Ashley, as a CEO, I actually have to spend a lot of time investing in what’s important for me. So, as a CEO, I have to role model clarity of strategy, you know, communication that is motivating and inspiring. Of course, the data approach and what my expectations are. So it has to start from the top. It absolutely has to. And that is a That’s an ongoing thing. You never get there perfectly because that’s what business is. Business is just solving different problems. Sometimes this is the most important problem right now. Sometimes it’s something else. Sometimes there’s a fire over there. It’s just a careful balance of all the levers that you have to pull. But I think for me, I think, again, fundamentally, we are a business that needs to be member first and it needs to be data led. They are the hallmarks of what Live Eyes needs to be. And so we need to demonstrate we can’t just be lip service to that. Like our vision is helping Australians get more of what they value beyond the checkout. And I think there are lots of businesses that will create these vision statements. And no one ever really looks at them and just goes, oh, yeah, that’s what it is. And it’s kind of philosophical and so on. But actually, one of the things that we’re changing this year, which we reset our vision a couple of years ago. And now we’re in a position where we’ve actually got three or four different measures around what member value is. So if we’re going to be real and say, we’re here to help Australians get more value, you know, more of what they value beyond the checkout, then shouldn’t we be demonstrating that we’re actually providing them more value? And I think there’s some basic things that we Often businesses just don’t, they don’t think about because it’s often it’s there and no one ever goes, oh, but do we? And I think for me, it’s like, no, no, no, I want to be able to speak to our people and say, this is our vision. This is what we’ve done in the last year, you know, and break down value in what the members perceive as value, whether that is value that they’ve redeemed or it’s value that they’ve earned through points. What are the movement of new members? What are they getting? What about existing members? Is that also showing the same vibe? It’s just about being tangible and being real about the things that you say. Otherwise, don’t say it. Otherwise, you’re just not true to what the purpose of the business is.

00:38:44.16 [Michael Helbling]: When you were talking about the iconic and how you were pairing your directors with an analyst, it got me thinking a lot about organizational structure. for data teams because we talk about it in the analytics space, analytics world constantly. Should we all be clustered in one group? Should we all be spread out into the org? From your seat and the things you’ve seen as you’ve led teams, led organizations, what preference model or what structure do you think serves the organization best from your perspective?

00:39:17.32 [Anna Lee]: The most important attitude is what’s right for the business at the time. There is absolutely no cookie cutter, what worked. at the time probably wouldn’t work at flybys now, for example. So at the time when I took on operations, one of the challenges was insufficient dashboards and metrics and understanding of that. But I needed to get that done very, like I need that skill capability to be done very, very quickly. So I needed to be able to talk the same language as my operations directors. And we were aligned in values, we were aligned in intent. Skill level was there because there were absolute legends and magnificent in running operations. but there was kind of like this language challenge and so the quickest way and it was a bit naughty because I wasn’t approved those those roles but I just went inside. Well, that was a huge advantage because I’d been the CFO and… Leaders leave, that’s… Well, I’d been the CFO and so I moved into the COO role and I kind of knew I was like, but you know, when I was CFO, people would kind of put on roles and they never like, you know, it was disgust, but it was like, okay, did you really need that role? But it wasn’t like, you know, it was just like a little slap. So I thought, okay, well, Now that I’m in the business, I wanna say bye. I’m like, I’m just gonna put these roles on and ask for permission, forgiveness later, right? Because if I didn’t and if I stuck to the hard rules, I just wouldn’t actually drive the results. I felt very, very confident that that was the right thing, and it really turned out to be. In fact, those positions paid themselves off in six months, like the visibility. And so you kind of need to know and back yourself. And of course, as a result, the savings that we generated, more than offset the investment in those people. So that was the right thing at the time because of the gap between, I guess, the competency of being able to talk to the data and what I needed to see. Of course, in this role now, what is right is what we’ve put in place, which is a bit of a hybrid. So we’ve got analysts that are better in the squads because they need to be able to provide very fast and relevant insights because they’re embedded, they know what the squads are achieving, they know what their priorities are and so on. And then we also have a centralized team that ensures all the infrastructure, all the systems and all the big insights that are strategic. are done and there’s still quite a lot of work around the platform that needs to still be focused on and so we have dedicated teams for that as well. You know, we had introduced the concept of chapter. We probably haven’t done as much as we’d like. That’s something that we just need to kind of resume. And we feel that’s right thing to do. I think conceptually, you can kind of go that that feels right. But in practice, sometimes you’ve also got to be adaptable and say, OK, maybe right now that that didn’t need the focus, but now kind of 18 months on from. the reset were probably in a right place to elevate that role and that type of framework could work. So I think the answer is that there is just no right or wrong answer. You’ve really got to understand where the business is at and meet it there to form the right structure. I also think that I really hate reporting lines. Like I just think Great businesses just it should just be spiderwebs like I the way I describe them is spiderwebs Because there should be relationships that transcend who you report to, who your boss is, like that stuff just exacerbates old school silo thinking. And of course, there is a role of your direct leader, your direct leader plays such an important role that your people leader, they’re there to ensure you’re constantly developed, your workload is balanced and so on. But the relationships that exist within the business that really succeed are a spider web. So people should be able to talk and connect with anyone in the business in order to get what they need and jam it out. So I know that you wanted a white answer. No, no. I don’t think there is a right answer.

00:44:05.15 [Michael Helbling]: I think an absolutist would be like, no, there’s got to be, but I actually completely agree. the structure and function of the company mandates or requires a response just like you’ve described. No, I actually totally love the answer. Thank you for confirming what I already think, but you say it in such a great way. Yeah, it’s awesome.

00:44:27.82 [Moe Kiss]: Thank you. I’m keen to leverage your amazing brain as much as possible for advice for those listening. You’ve talked a lot about what are the great qualities and the culture that we want to see in data. I think one of the realities is that data folks can lose trust and credibility very quickly with a couple of bad interactions. I’m curious to hear. and definitely not pointing fingers, but like, where have you seen things go wrong? Or like, I feel like it’s such a skill for data folks to learn that, how to do that so what, especially for an exec audience. And sometimes that takes time and sometimes there’s missteps along the way, but like, what are some kind of things that folks should avoid, particularly when they’re presenting or like communicating results to senior leadership to kind of keep that trust?

00:45:23.04 [Anna Lee]: Well, I might break that into two ways to answer that. So first of all, the first thing is what we discussed a couple of topics ago, which is around that. So what? So I think one of the hardest things is letting go of all of the wonderful data because sometimes to get those insights man you have like dug dug dug like there’s just mining information and so there is this desire to almost showcase like oh look at all the work that I’ve done but actually most senior stakeholders just like that’s great actually we trust that you know, because that’s why you’re in that job. So really being clear about the so what so you’re going to present something like what is the killer point? That’s why I say to people like what’s the killer message here? What’s the wow moment that you’re trying to produce or convey or influence. So it might look like. So we always thought that this was the case and that that data has previously shown that. But I’ve been monitoring this metric and I actually noticed this other thing that’s come up. And as a result, I’ve kind of you know, combine that with that particular insight. And actually what I’m saying is perhaps what we need to consider and have a conversation with, you know, say partner X is, would they consider doing this? Because if we did that, but yield a different, you know, activation rate or whatever and that would drive, you know, really connecting the work that you’ve done with the strategy and whether something actually could be changed in order to drive a different outcome. I think That’s kind of more the technical side. And I think on the other side, I feel like I never had any of my team members present. to senior stakeholders, you know, specifically like the CEO. So when I was in CFO role, CEO role without a lot of coaching. Because I think a lot of people also just kind of, it’s just, you’re literally just like throwing someone in the deep end. Like if you don’t, like if you’ve never engaged with the CEO before, you know, nothing about them. I think that’s just, I think that’s poor management. Like for me, if I had, whenever I had a team member that was going to present to our CEO because you wanted to come along to the meeting and give them an opportunity to shine, you never did that without prepping them, rehearsing, giving them coaching around this person. They really like this. They’re really particular about that. you know, this is how and also how we could play a role. So also, you know, it’s almost like there’s different roles, right? So you come in and let me open the conversation and talk to and reframe it such that there’s context that allows the team member to observe because so much of what we learn is on the job. So much what I learned was observing different behaviours and different things of different leaders that I really admired. So that allows our person to go, oh, I like how they’ve set up. I see how Anna set that up with and built that connection with the CEO. And then hand over and go, well, you know what, you know, Jane’s role is this, right? And then it kind of really sets the context and it allows that person to kind of focus on what they’re going to talk to without having the pressure of like, oh, God, I don’t know if you know this, but you know, and this is There’s quite a lot of energy wasted on setting up the context when actually it’s just quicker if I just set that up and just pass it on. So I also think if you are an emerging leader in data analytics, I think don’t be afraid to ask for coaching and asking for What should I do? And also, could you give me some feedback afterwards? So because those interactions that I did that were coached afterwards, there was just an open dialogue to go, oh, I think you did great here. This was a curly question. We probably didn’t expect that at the CEO. So that’s OK. And how would we manage that next time? Invite the person to say, what did you think of that? That creates just a really safe culture. There’s also, by the way, as the people lead it, it’s probably really smart to also say to the CEO, hey, I’m going to bring so-and-so in. It’s a development area for them. like ask your questions like you would because I don’t want you to be a different person, but also just manage your expectations on what this person is because everyone’s done something for the first time and the first time that they do anything, you know, it’s always going to be a bit challenging and there’s going to be lots of different nerves and emotions and lots of different expectations that will culminate. But they become big deals for those people because they’re like, wow, I got to present to senior stakeholders. So I think there’s also a job of the people leader to kind of go, hey, you know, and just set the space up properly as well.

00:50:57.28 [Val Kroll]: That’s a good one. I’m going to sneak in one more, Michael. You’ve got that look in your eye. I don’t think it’s going to be too bad. I want to ask you this question in particular because you’ve had the CFO, the COO experience before, being in the COO, and I heard you talk earlier about the investment in the operations analyst, like paid for itself in six months. You talked about the investment at Flyby’s in the platform and some of the data collection pieces. One of the things that some people in analytics, and I’ve spent some time more recently in experimentation struggle with is, how do I communicate the story of value to senior stakeholders? And one thing that we come back to a lot is, how do I quantify the value of a learning? So even if it wasn’t directly like, this is now the decision, there’s a fork in the road and we can specifically tied the dollars to those decisions. How would you suggest someone talk about or what are some of the ways that we could illustrate the value of helping kind of raise the institutional like learnings of the organization or for some things that are a little bit more hard, a little bit more intangible, but obviously are moving things forward. I’m curious how you would like to hear some of that or would be the best ways to illustrate it.

00:52:16.85 [Anna Lee]: Oh, that’s such a, that’s you’ve saved the last. Oh, did I? Okay. Sorry, go away. That’s like, no, no, it’s a good one to ask. I like a tough question. Look, value is, I mean, I don’t know if we’ve got it right ourselves. So I’m certainly not going to talk to this as an expert at all. I think we are. we’re productively curious about that question ourselves. But I think over my years as in finance, I think, you know, calculating something like an ROI or what, you know, the perceived benefits are on things. One thing I have learned is there’s no, again, there’s no, you know, one, one rule, size fits all for, for everything, because there are some things that are just going to be really easily tangible, like, you know, very classic, ROI measures, measure tangible savings from investments and decisions, very, very tangible. Then you will have ones that have much slower payback. I think an example would be educating this impact of the data platform and being able to lay the foundations and so on. I think that is a narrative around enablement. So, you know, in order for a lot of these other things that are in our strategy to actually be delivered, we will need to measure, you know, these metrics or otherwise such that, you know, and we need to be able to do those very, very quickly and very easily and therefore It increases the speed at which we can deliver or otherwise again. little bit tricky because, you know, sometimes someone would say that to me and I’m just like, that is all fluff, give me something more tangible. So obviously I’m not going to say that I would be. But then there are some times that I just go, yeah, yeah, no, that’s okay. That sounds plausible. And I appreciate that we can’t quantify that for now. But you just know, and I think that’s, I think coming back to what we talked about earlier around Not everything needs to be precise and completely evidence-based. We’re not here on a criminal kind of case where the burden of evidence is so, so high. Sometimes it’s just about being able to complement those hunches or otherwise with other information that could still you know, sound good and sometimes the trade off is, yeah, I’m going to take a risk there. And yeah, let’s let’s let’s put on another couple of people because you know, we think that that’s that’s probably going to yield some really great outcomes that we just limited in capacity now. So I think it just some of it is just about experience and expertise. And I’ve been in working for over 30 years. So sometimes there are things that you kind of just know intuitively and And you just have to kind of take a gamble sometimes. So it’s just moderated risk-taking in a way that kind of feels like, well, what’s the worst that can happen here? Not really a lot, frankly. Like we’re not talking about you know, a multi-million dollar investment here. We’re talking about, you know, some certain roles where we can be very clear and see. And I think that’s okay. There are some measured things that you can take risks on.

00:55:57.97 [Michael Helbling]: Yeah, that’s great. Okay, we do have to start to wrap up. This has been such an amazing conversation, Anna. Thank you so much. I literally have like a bunch of notes on my desk here, especially about productive curiosity and informed pragmatism. I think those are two amazing concepts. So thank you so much. But the whole conversation was amazing. And before we get into last calls, I’m going to jump over to take a quick break With our friend Michael Kaminski from ReCast, the media mix modeling and GLIF platform that helps teams forecast accurately and make better decisions. And Michael’s been sharing some bite-sized marketing science lessons over these next couple of months to help you measure smarter. So over to you.

00:56:39.55 [Michael Kaminsky (Recast)]: We’re all interested in doing good causal inference work, so let’s talk about its fundamental problem. Cause and effect are such a simple relationship, and yet formal scientific causal inferences almost impossibly difficult. At its core lies what philosophers call the fundamental problem of causal inference. we can never observe what would have happened to the same person at the same time under different conditions. So we always have the lingering doubt that the medicine make the patient better, or would you have gotten better on our own? In order to estimate the causal effect of some intervention statistically, we would have to control for every variable that could possibly confound the analysis. The problem is that in any individual there are unobserved or unmeasurable characteristics that make us unique and are impossible to statistically control for. This makes observational causal inference extremely difficult and is the main reason why you can read a new study every week about how coffee both increases and reduces life expectancy. Our inability to statistically control for the unobserved or unmeasurable differences in individuals experiencing some stimulus forces us to turn to randomized experimentation to differentiate between correlation and causation. Because controlling for everything that we can’t observe is quite literally impossible.

00:57:48.90 [Michael Helbling]: All right, if you enjoyed that many lesson, Michael and the team at ReCast put together a library of marketing science content specifically for analytics power hour listeners. From everything from media mix models in-house to communicating uncertainty to your board, head over to www.getrecast.com slash aph. That’s www.getrecast.com slash aph. Okay. Well, one thing we love to do is go around the horn, share something about your interest to our listeners that’s called our last call. Inna, you’re our guest. Do you have a last call you’d like to share?

00:58:23.48 [Anna Lee]: Well, I am a huge James Clear fan. I absolutely love his book Atomic Habits. And one of my favorite quotes for him, which I think is a really good counterbalance to all the evidence conversation that we had today, is many situations in life are similar to going on a hike. The view changes once you start walking. And I really love that because I think often we find ourselves under pressure to have all the answers and go, well, we don’t want to take action until we have everything that we need, perfectly validated and so on. And I’m of the view is just take action and it’ll just start to work itself out. And there are some things that actually you don’t know the answer until you actually start taking action because some of those things actually don’t happen until they depend on the things that you’ve taken action on. So I love the fact that the answers will just reveal themselves. That’s kind of a nice saying.

00:59:22.76 [Michael Helbling]: That is amazing.

00:59:24.14 [Anna Lee]: Michael’s got a big smile.

00:59:26.43 [Michael Helbling]: All right. I do. Well, because there’s a phrase that was really important in my life that’s very similar, which is you can’t steer a parked car, which is sort of in the same vein, but not exactly the same. I liked the idea of like taking the hike and watching the view change. So yeah, anyways, that’s really cool. Okay, Val, what about you? What’s your last call?

00:59:47.35 [Val Kroll]: OK, so mine is a medium article that was published on the UX collective. And never had I had a last call that was tied in more to an episode. I’m so proud of myself. But this one’s called, There’s Logic Behind Your Gut Feeling. And so this is all about abduction. It’s about Charles Perce. And so I’ll read you this one, a little two sentence thing. So he believed that truth wasn’t something that was handed down. It was something that you worked towards. And the way you got there wasn’t by doubling down. It was by willing to be wrong. For Perse, doubt wasn’t a weakness. It was the start of real thought. And so it breaks down like all these different types of thinking and reasoning. And he actually at the end ties it into why it’s really important for the way that we’re incorporating generative AI into our work. And so Michael, I know you’ll love that tie in too. But anyways, I found it a good read and it’s nice and concise. So that’s a good one.

01:00:46.85 [Michael Helbling]: All right, Moe, what about you? What’s your last call?

01:00:50.24 [Moe Kiss]: I have been thinking a lot about how AI summaries changes search behavior and folks coming to sites and how that’s evolving. The Pew Research Center just put out a really interesting article on that. And it was specifically about Google users. And it was showing that Google users are less likely to click on links when an AI summary appears in the results. So a lot of us would have noticed that happening now in our Google search results. and just has a lot of flow-on effects also, like I said, for how users come to our site, like the volume of traffic, the quality of traffic. So it’s definitely a topic that I’ve been getting around. It’s a really good kind of quick summary if you want to get some info on the topic. So I found that one really interesting.

01:01:36.81 [Michael Helbling]: That’s awesome.

01:01:37.72 [Moe Kiss]: What about you, Michael?

01:01:39.16 [Michael Helbling]: Oh, well, I’m glad you asked. So I read an article recently, and it’s also related to AI, but I love a good contrarian take, and this is one. Nancy Dooley, who I think is a listener, but also a good friend of the show, she sent me an article called The Hater’s Guide to the AI Bubble. by Ed Zittron. And what’s great about it is the author does an excellent job driving through sort of a financial analysis of the AI industry and sort of some lack of sustainability inherent in that he sees in it. And I thought it was actually quite well researched and done. I’m still pretty excited about AI and where AI is headed, but I really enjoyed reading and thinking, you know, it was provocative to think about that. So, And all these things that we just shared as last calls, we always put the links to those items and we’ll put a link to that James Clear quote in that book. All of those will be in our show notes on the website. And as you’re listening and you want to reach out, we’d love to hear from you. The best way to do that is uh through our website which i just mentioned analytics hour dot io you can reach out to us that way or through our linkedin group or um on the measure slack chat group so we’d love to hear from you feedback about the show and things like that or topics that you’d like to see discussed That’s a great way to do it. So as you’ve been listening, please reach out. Anna, once again, what a pleasure. Thank you so much for taking your time. It’s something we in the data space, we always want executive time and feel like we don’t get it. And so the chance to spend some time and talk about these issues and hear from you is something that’s not just valuable for us, but I think it’s really valuable for our listeners. So really appreciate it.

01:03:30.89 [Anna Lee]: Oh, thank you so much. It’s my pleasure. And you guys are such fun coyotes.

01:03:36.08 [Michael Helbling]: We try. We try. But thank you. We’ll take that. We’ll take that. Anyways, no matter where you are in your progression, whether you’re presenting daily to the CEO or you’re just starting out as an analyst and trying to get coaching to move up the ladder, I know I speak for both of my co-hosts, Moe and Val, when I say, keep analyzing.

01:03:58.52 [Announcer]: Thanks for listening. Let’s keep the conversation going with your comments, suggestions, and questions on Twitter at @analyticshour on the web at analyticshour.io, our LinkedIn group, and the Measure Chat Slack group. Music for the podcast by Josh Crowhurst. Those smart guys wanted to fit in, so they made up a term called analytics. Analytics don’t work.

01:04:23.09 [Michael Wilbon]: Do the analytics say go for it, no matter who’s going for it? So if you and I were on the field, the analytics say go for it. It’s the stupidest, laziest, lamest thing I’ve ever heard for reasoning in competition.

01:04:38.24 [Michael Helbling]: No, but it’s like the best. It’s sort of the best because you put it in your mouth. You can make it your own mouth. Like, don’t put it in your mouth. Don’t put it in your mouth. Don’t put it in your mouth.

01:04:46.77 [Moe Kiss]: Don’t put it in your mouth. Don’t put it in your mouth. Don’t put it in your mouth.

01:04:49.18 [Michael Helbling]: Don’t put it in your mouth. Don’t put it in your mouth. Don’t put it in your mouth. Don’t put it in your mouth.

01:04:51.70 [Michael Kaminsky (Recast)]: Don’t put it in your mouth. Don’t put it in your mouth. Don’t put it in your mouth. Don’t put it in your mouth.

01:04:55.10 [Michael Helbling]: Don’t put it in your mouth. Don’t put it in your mouth. Don’t put it in your mouth. Don’t put it in your mouth. Don’t put it in your mouth. Don’t put it in your mouth. Don’t put it in your mouth. Don’t put it in your mouth. Don’t put it in your mouth

01:05:01.68 [Moe Kiss]: I honestly wasn’t so and at the end of the show someone always has to do that embarrassing thing and it is almost never me and so I assumed it wasn’t me it’s like yeah it’s an inside joke but I was like it’s not going to be me and then in the last one someone really down to do it. So now I’ve got to do it.

01:05:22.38 [Michael Helbling]: So are we leaving? Yes, we are. I think now we’re committed because all of the outtakes are going to be this next year.

01:05:32.21 [Val Kroll]: And now I have to tell a quick story. So when I worked at UBS in the investment bank, I was in New York.

01:05:38.63 [Michael Helbling]: Do you want to stop the recording so we can just make sure people get uploaded?

01:05:43.16 [Val Kroll]: Yes, although I would love to hear Moe and Hannah’s reaction about yes, we can still.

01:05:59.26 [Moe Kiss]: Rock flag and pragmatic curiosity. Good one. Is that the actual saying?

01:06:06.31 [Michael Helbling]: No, it’s productive curiosity.

01:06:08.32 [Moe Kiss]: Fuck! All right, let me do it again. Fuck!

Podcast: Download | Embed

Subscribe: RSS