[music]

0:00:05.8 Announcer: Welcome to the Analytics Power Hour: Analytics topics covered conversationally and sometimes with explicit language. Here are your hosts, Moe, Michael, and Tim.

0:00:21.9 Michael Helbling: Hey, everybody. Welcome to the Analytics Power Hour. This is episode 215. You know, what is a podcast really, but a set of conversational elements constructed into an audio format. And we swap out the components from time to time. For instance, many guests who come on the show and sometimes this episode where we’re doing a little bit more and we run episode after episode and we observe the results. I mean, it’s not a simulation, but if we set it up in a systematic way to answer some sort of question, whether Tim should warn guests about the rock flag and eagle at the end of the show or not? Well, we could run some simulations. Let me introduce my co-hosts and we can talk more about it. I am really excited to introduce my guest co-host for this episode, Julie Hoyer. Welcome, Julie.

0:01:10.3 Julie Hoyer: Hi, Michael.

[laughter]

0:01:11.7 MH: You wanna try that again with a little more enthusiasm? Easy there?

0:01:17.7 JH: Yeah, sorry, I choked on my coffee right when you called my name.

0:01:21.7 MH: No, that’s totally okay. What I feel really awesome about Julie is that I technically am responsible for hiring you originally and I was like…

0:01:31.1 JH: Yes, you are…

0:01:31.2 MH: Oh, I feel really proud of that, which is like, that was a way smarter move at the time than I even realised. So welcome. It’s so awesome to have you. Julie is an analytics manager at Search Discovery and you’ve also heard her on some previous episodes and we’re so delighted for you to be here co-hosting with us as Moe is out. And it’s awesome that you’re here specifically for this episode I think, too. And Tim. Tim, I don’t know that I’m as excited to welcome you to the show, but I think it’s just because we’ve done it so many times before, so it’s okay. Welcome, Tim.

0:02:02.2 Tim Wilson: Hi.

0:02:03.0 MH: I know you’re excited about this topic.

0:02:05.5 TW: Hi. Hi. Hi, Michael.

0:02:07.1 MH: Yeah, exactly.

0:02:08.5 JH: Stop.

0:02:09.3 MH: The love is still there.

0:02:11.3 JH: I’m very excited to be here, okay?

0:02:12.7 MH: Yeah.

0:02:13.3 JH: I promise.

0:02:14.5 MH: The love is still there, Tim. Okay, so yeah, but Julie, I mean since you do have a master’s in Applied Mathematics, I think this is actually perfect for this episode. So I’m really glad you’re here. So anyways, we did also wanna bring in a guest to help talk about this topic of simulation. Somebody who could open up this world a bit more for us and for our audience. Frances Sneddon is the chief Technology Officer at Simulate, where for almost 25 years she has helped organisations leverage simulations to improve all kinds of processes and business functions. She’s the co-chair of the heads of analytics and operational research forum for the OR Society, Operational Research Society. And lucky for us today, she is our guest. Welcome to the show, Frances.

0:03:00.3 Frances Sneddon: Hi. Thank you for having me.

0:03:01.8 MH: It’s delightful. Also, I think to start, I think there’s a very important question that we will… Probably should start with what exactly is a simulation?

0:03:11.3 TW: And are we living in one?

0:03:12.6 MH: Yeah. [laughter] See, the whole podcast, I avoided that in the intro so much, Tim.

[laughter]

0:03:21.1 FS: Oh, I think that is a very, very good question ’cause simulation is one of these terms that can mean so many different things to different people. My kids play simulator games, they go down to the arcades and they play the flight simulators. You have pilots that are being trained and in airplanes you’ve got all these different types of kind of simulation. But the type of simulation that I focus on, and I’m admittedly biased, but I think provides the most benefit to businesses is what I’d call process simulation. And that’s the ability to look at any process within an organisation, within a business, and be able to create a simulation of it so that you can experiment with it and see what if? What if I double my staff? What if I double the amount of work coming in? Experiment with it and know what’s gonna happen in the future before you implement it in the real world. So a simulation, and at the heart of it, whether it’s a game or whether it’s for business, it’s about an environment that you can test in and experiment in safely.

0:04:23.4 TW: So you’re using Flight Simulator as an example, I mean, not… Of not necessarily the type of simulation, but I think of Flight Simulator as being, or even SimCity or the Sims or whatever it’s called, they’re at the core the process has to be fully modeled, all the rules of the interactions. Is that in a business or government or another… The context that you work more in, is that still the core of it, is you have to essentially have built out all the connection points and then when you’re experimenting, you’re saying, what if this was bigger or this was smaller or this was faster, and how would it ripple through the rest of the system? Is that fundamentally what’s happening?

0:05:12.8 FS: Yes, and no. So you can build simulations in that way, and some people do do them, and you maybe build them at an organisation level. But the key thing whenever you’re building a simulation, we always sort of advocate is, what’s the question you’re trying to answer? And then you build the simulation to the right level and to represent it to be able to answer that question. So an example might be, okay, I want to decrease the waiting times for patients in a hospital in my ER department. So then you need to model the ER department and you will have patients arriving and they’ll pass into other departments in the hospital. But actually the rates that they arrive in and how the other parts of the hospital perform doesn’t really help me answer the key question. So I can almost put them out of scope. So it’s a big part and it’s a big part of what makes the difference between a usable simulation and an unusable simulation is really understanding scope, what question am I answering, and only make the simulation as complicated as it needs to be to answer that question.

0:06:19.3 JH: Do you ever find it hard to get down to the question they’re trying to answer or do you have… I guess another way to ask this ’cause I feel we run into this a lot, oh, we want to work with your company and we want a model, or we wanna do machine learning, they choose the tool before they decide like, why? Do you ever run into that with simulation where they’re like, “Well, we want you to build us a simulation,” you’re stuck asking them, “Well, why? Well, why?” And it’s really hard to boil that down. Do you guys run into that much or how do you propose simulation to the different clients that come to you?

0:06:51.8 FS: Yeah, no, I think that’s a common problem. I think that’s a problem with probably any analytic type tool or anything that you kind of use. And so we’ll say a lot… What we typically see from people is that sort of idea like, okay, I wanna improve my waiting times in the hospital, and so therefore I wanna build a simulation for the entire hospital. And they wanna make it so complicated and they want it to answer every possible question they might have in five years or 10 years. So it’s almost like… The best way I could describe it, is they come with almost an IT project mindset of I’m gonna write this specification and if a simulation’s done well, that’s not typically how it should be done, and so it is that bit of that discovery where, okay, well, what is the question you’re trying to answer and let me see, kind of asking why, why, why, a lot and seeing this as more of a journey, a learning journey that you’re gonna go on rather than sort of building an IT tool. So, yes. And in my early days of doing consulting, that’s some of the hardest parts is just getting people down to that one question, and what they’re trying to answer.

0:07:54.7 FS: But, yeah. But one question then leads to another and another. So there’s also a preconception that you build one simulation and you often don’t. It’s maybe a series of simulations that you’re building because you’ve built one, you’ve learned something. Now there’s another question I wanna answer. Sometimes the same simulation will be able to answer it, sometimes it won’t. And then you’ll move on to the next one. So it’s a journey of discovery.

0:08:16.4 TW: So where, and the hospital wait times is a interesting one. I mean, I could see a Six Sigma Black Belt coming in and saying, oh, the way to solve that is we map out the process and then we… I’m not a Six Sigma deep, knowledgeable person, but they would come at it and say, it still comes to, we have to understand what the process and what the inputs and the outputs are and then we’d measure it. And maybe it’s, well yeah, there are different ways to answer, to solve the same problem. It feels like a simulation would say, “Yeah, we still have to figure out by mapping out the core part of the process and figure out the constraints.” But now you actually get to see, instead of somebody doing a math formula and saying, “Well, if we increase this by 20%, then in theory this other thing would go down,” and at the end of the day we’ve done the math and said that this will reduce wait times with a simulation. It feels like you give a little bit more of a tangible, “Hey, let’s build this virtual world and let’s try that universe and just see what happens.” Does that… How much of it is what’s easier for the stakeholders to actually process, internalise, buy into, to the point that they say, “Okay, now let’s go make that change.” Does that question make sense?

0:09:45.3 FS: Yeah, I think there’s a few questions and also… So bringing it back to kind of the core, you’re right. Somebody that was doing sort of a Six Sigma project, they’ll probably come in and they’ll sort of… They’ll map out the flow of work and then they’ve got some mathematical techniques about where’s the wastage, where’s the non-value add steps, here’s some of the tweaks that you make, and they can make some assumptions and some predictions about how if there’s some changes they want to improve, the impact that they’ll have. And people definitely use simulation in that way and it’s valuable to do that. I think sometimes what the simulation highlights is impartial because of the way things are modeled in it is… It can catch things that other techniques can’t. So when somebody maps out a process, they can map it out and you can see the rules for why somebody goes whichever way. We will push through that flow diagram though all the individual patients and when the patients are complaining, don’t get worried, it’s anonymised data, so it’s not about the actual patients but it’s representations of the patients.

0:10:53.3 FS: But we can then kinda say, right, okay, well this patient’s male and he’s 35, he’s got a broken leg and he’s previously had this type of condition, so he’s gonna go through this path, in any department or the ER, or we can say then the next patient, she’s female and she’s 6 and she’s got bad stomach pains, right? Okay, where she’s gonna go through. And so what the simulation can do, and then there’s… ‘Cause there’s all these complicated drills is that sometimes it can expose that, okay, in theory, the improvement that you thought was going to do is gonna work, but actually because of sort of the dynamics of the patients that are coming in, maybe at different times of day it’s gonna perform differently, as well, because, you get peak times of peak flows or because it causes a bottleneck in some other part of the process. So sometimes you’ll exhibit and catch things that other techniques won’t. But I think where simulation really then comes into its own, is that, you can really experiment with radical ideas. It doesn’t just have to be right, okay, well, what if we sort of decrease the time they’re spending in the cubicle by 10%, what’s then the output of that?

0:12:00.3 FS: So an example of that. So one of the great ideas I heard one time from a sort of healthcare provider was saying that, right, okay, what happens is, somebody comes into the emergency department and they are treated by a series of junior doctors who assess and sort of figure out what tests that they’re gonna do. They’re doing this barrage of tests to figure out what they think is the likely illness for the patient. And then eventually they get access to a senior doctor or a consultant. What if the consultant was the first person that saw them? And so a consultant with 25, 30 years of experience can sort of look and get guidance and go, “Actually, you know what, we’re not gonna do those five tests. I’m pretty sure it’s just these two tests that we need.” What would that do to the process time? Now, that’s the type of thing that you can’t just do by having mapped at a process and sort of saying that, “Okay, if there’s a 10% game here or 20% gain there.”

0:12:53.7 FS: So you can radically test that idea, you can stress test it out and then you’ve also got this communication tool to sort of take the stakeholders, take to management, take to the staff in the process as well and say, “Look, this might not seem the most sort of intuitive idea, but look what it delivers, what do you think?” And maybe that’s not the end result, but then we start to have a conversation and say, “Right, okay, well, I see where you going with that, that’s flawed because of X, Y, Z, but tweak it this way and it does start to work.” So suddenly we’re doing sort of step change process improvement and more than just sort of incremental sort of little gains.

0:13:31.7 TW: So let me ask on that, that one… So that example, it does think that if you shifted that part of the process, it’s an unknown as to whether if… Say they’ve looked at the data and said, on average we run five tests before it goes to the senior consultant, without knowing, there’s an idea or a hypothesis that if the senior consultant started that they would likely, and there may be some anecdotal evidence. So is that still become sort of the… Do you wind up with the core of saying, “We think that if we shift this order, it might be that… ” What would happen if that only reduced the number of tests by 10%? Then that may ’cause… So is that you bake in, and you’re saying if we made this change, we know there’s uncertainty as to what might actually happen at that change point. So we simulate a range of possible, what if we reduce the test by 80% versus 20% and run the simulation and say what’s the variability of the result? Is that how it works?

0:14:38.1 FS: Yeah, yeah, exactly that. So if you’ve not got the concrete evidence, and there. So then you’re effectively doing sensitivity analysis and saying, “Right, okay, what if it registered by 10%? What if it’s 20%, 30%,” and you can go all the way up. So then what you’ve got is you can actually say, “Right, okay, in best case, if it registered by 80%, here’s the gains in the process performance. Worst case, here’s the gains and process performance.” And that might mean you describe the idea, but you are sort of, you’re able to validate and give an ROI sort of prediction of the changes. And that might mean that that idea becomes the 10th one that you’re gonna try and you’re gonna try the other four that unless… So a lot of it’s about, especially when it’s improving processes, like this…

0:15:25.5 FS: So, there’s usually lots and lots of ideas from staff, from management at various different levels about what could we do. And simulation as much then can act as that facilitation tool and help you prioritise those ideas and sort of try and assess that impact and get a shared understanding. ‘Cause we can run it forward and we can tell you, “Okay, it’s gonna make a difference over a week of this, over a month, over a year, over 25 years,” you know what I mean? That if you have to wait and do that in the real world, you’re gonna be at minimum six months, maybe a year on to finish your prototype to then be going out. So are we gonna get it absolutely perfect with the simulation all time? No, but we can accelerate and we can make sure that what you implement has a far, far better chance of success. And knowing that really, really quickly, and particularly in today’s climate, where we’re all facing huge backlogs and whatever else. It’s just we need that speed and that agility to make an impact quickly.

0:16:19.4 JH: Oh, my gosh. Okay. So I have four different questions swirling in my head. So I’m gonna try to [laughter] prioritise which one I ask first.

0:16:26.4 MH: Hey, no pressure, Julie, because Tim will just string ’em all together. So really the bar is super low for you there.

[laughter]

0:16:33.7 JH: To try make really clear. Okay. Okay. My first one that came to mind was like, how… Okay, when you’re talking about even a hospital system, just that example, to keep building on that, I mean, there are just so many pieces of information and business rules. So the consultant, what do they do? How many years of experience, what is the experience of the people staffed in the ER? How many are there, how many people are in-fluxing? What is their… Like, what is the patient’s issue coming in? What is their process for someone who has a broken leg compared to 104 fever, who’s 3 compared to 60? How in the world do you get [chuckle] all of the rules to actually define the system? To me that’s infinite amount and then it blows my mind that you can build a simulation. So how do you talk to stakeholders, get all that information, hopefully they’re collecting enough of it, too. You get in situations where there’s these realities that how do you even represent them? So my brain was going crazy thinking about that, like how?

[laughter]

0:17:38.8 FS: Yeah. And it comes back to that understanding the question you’re trying to answer over simulation. And so therefore how much do I need to model this accurately? Because I don’t need to make this a 100% sort of accurate simulation. There are cases when we, and we have done that, we can do that, but you don’t always need to go that far. And the reality is, is that, simulation engines like ours are really, really sophisticated. So we actually can model all those types of roles that you’re talking about, so we can put in the different kinds of staff, we can put in their experience level, and we can see, hey, that impacts on the time that it’ll take them to do sort of steps in the process and things like that. So we can do all that kind of part. Some of it relies on yeah, sort of interviewing people and doing that kind of part, but that’s also where you get a lot of the insight. Back when I started consulting, sometimes you stood on the manufacturing line with a stopwatch to collect the data. We’re in a different world now. Often the problem is, whoa, that’s a lot of data. You’ve kinda thought… You’ve got all, kinda that. So, I think… So there’s lots of different things going on.

0:18:50.3 FS: So I think it’s understanding how much you need to model that. And I think the process is, it’s speaking to people, it’s understanding the data and then sort of understanding which rules are important ones and sort of then what you do. So you keep doing a process of validation and verification so that it… Until you’re getting it to the point where it’s representing enough of the behaviour in the real world, then I can start experimenting. But just to blow your mind a little bit, kind of more and add another bit of complication where simulation has got really much more sophisticated and I think it’s a real step change and what will make this a much more accessible technique to people and also a quicker technique to do is that. Now if you imagine our little steps, if you imagine our simulation looks a bit a flow chart and we’ve got this one step, somebody’s kind of coming out of triage, where do they go next?

0:19:35.4 FS: You’re right. In the old days, we would’ve written in all the different rules and the logic as to where they’re gonna go. But now we can call on machine learning algorithms and we can plug those in and we can say, right, okay, last time we saw a patient of type X exhibiting these conditions and these diseases, they went down path A, put them down path A. Next one comes down path B and stuff like that. So now you’ve got the flow of the simulation, you’ve got your data sets and all you need to do is tap into that data set so the time to build the simulation radically comes down. The accuracy probably will increase as well because there’s a lot of nuances where it becomes really complicated is things like, why does a consultant send somebody somewhere? Not ’cause there’s probably reason, not necessarily something you can articulate often as to why put that patient in this bed versus another with bed managers. So that power of that machine learning stuff to really step change and make this much more accessible to people is… Well, it’s not even the future, it’s the current, but that’s what’s happening now.

0:20:39.6 JH: Gotcha. That’s amazing ’cause then, okay, so then even thinking you’re running this into the future of 20 years, whatever, to see the change, could you then use machine learning to possibly also help represent the changing patient population you may have? Would you ever have to do that if you’re trying to run it so far into the future where you’re like, “I can’t assume my patient population or maybe even partly the staff is gonna look exactly the same.” So does that help there as well?

0:21:06.2 FS: Yeah, so what I would say is that there’s always a timeframe that you’re on simulations on. And so for example, in any department you would probably run a week, maybe a month, but run it repeatedly again and again and again because it’s about seeing the sort of availability and seeing how often does the work come in and where does it exit the system. And so most of the time nobody’s staying in any department for a long time. So you can rerun the same week of the month and sort of do it again and again and again and again. A factory, you would run whatever their scheduled orders are. But you’re right, there are other situations in other simulations that you then run over these 25 year periods. And so population health is a really good example of that.

0:21:47.7 FS: So I think a lot of the examples we’ve used just now are very operational process focused. But the same simulation techniques get used at a much higher level, almost a policy level. So another… Sticking with the healthcare example, another good example of that is managing your aging population or managing people with long-term conditions. So if you’ve got a long-term condition, you’re gonna be in the health service, not just for one iteration, it’s gonna be for a long time period. How effectively we treat you at the start of that long-term condition might impact your journey much, much further on in the system. You’re right, the different populations, the geography that somebody lives in can affect the makeup of the population and their health often. The services that are available in that geography will then also impact.

0:22:38.9 FS: So that’s an example of a type simulation where you would wanna run it over sort of 25 years because you’re looking at the lifetime of the patient and the lifetime of the population. And some of those simulations are… They’re much more abstract and you’re not dealing with so much of the individual rules and things that. But they’re fascinating to see how you make a change here and then it impacts 20 years down the line and then even sometimes creates problems. The better we treat long-term conditions, the more demand there is on the hospitals ’cause we have more people that are still… So there’s also some capable policy types and relationships you can do as well.

0:23:15.9 JH: Wow. Okay. I have one other one but Tim I will pause ’cause I feel like you have a question.

0:23:19.4 TW: No, no, go ahead.

0:23:20.2 JH: Okay.

0:23:20.6 TW: Continue.

0:23:22.1 JH: Last one that blows my mind is the assumptions going or the unknown relationships that just are… And I know you mentioned machine learning, so maybe this helps with some of it. But I keep thinking of this example. I was just talking to actually our other co-host, Val, the other day and we were having this whole conversation about levers people in an organisation can pull to move a KPI the business cares about.

0:23:43.3 FS: Yeah.

0:23:43.4 JH: And she was giving an example from one of her previous market research jobs I believe. And she had said how they had done this study and they found that if at a call center people had their call answered within 30 seconds of them calling, the end satisfaction and whatever else they were measuring was way, way better. [chuckle] So they report this piece of research back to the client that they were working with and so they implemented in their call centers. Well, ends up what they did in reality, I don’t know if we would’ve seen if we were simulating this, I don’t know. They were answering within 30 seconds ’cause their manager told them to and then they were putting them back on hold. So people actually were really pissed, [laughter] they were not happy and the results they thought they were gonna see, they weren’t gonna see. So that’s obviously a small example but how do you guys approach things like that? You have an assumption going into your simulation, what happens when unforeseen things come up, or have you run into an example like that where the simulation said one thing but then it hit the reality of the world and it was a little different or really different?

0:24:52.1 FS: Yeah, we call it moving the problem. And I think when people look at a lot of these process problems and they end up implementing in the real world when they haven’t done some of that simulation, you hear that story again and again and again. The classic example where you used to see it again and again is that on a manufacturing line, a sales guy would come in and convince you to buy the super duper new deluxe machine whose run through rate is 20 million parts per hour and you think, “Fantastic, this machine solves all that problems, got a big bottleneck there.” But the next machine can only process things at a hundred parts an hour. Well, okay, the process can only run as fast as the slowest machine in the process. And in your example, the way that would’ve got caught in a simulation is that you would’ve been looking at overall performance results. So you would’ve been looking at the time of the customer in the system as well.

0:25:50.7 JH: I see.

0:25:51.0 FS: So then you would’ve seen like, okay, “My fast response time, yay, fantastic.” But actually their timing system either hasn’t gone down or possibly worse, it’s increased because I’ve front loaded all my staff kind of now.

0:26:03.6 JH: Gotcha.

0:26:04.4 FS: And it’s that type of stuff. That’s what I mean about it being a journey of discovery a little bit, you know? So the simulation looks a flow chart but then we have if you imagine almost red sort of balls storing up in front of steps and that sort of visualises bottlenecks and things like that. So a big part you’re sort of running the simulation and you go, “Okay, look, great. All these customers are kind of coming. Hang on a minute. Why is there a big stack of red balls here? What’s going on here? Okay, let’s dig and investigate a little bit. Ah, I’ve moved the problem, I’ve moved the problem.” That’s what’s gonna happen. So now okay now we are back to the start again, I need to try another idea, what to experiment with and things like that as well. Yeah. Or to use that example then to use how they might have kind of solved it, what I’ve seen in some call centers unfortunately. I’ve modeled them is what they actually then do is say, “Okay, the last customer in is the first customer call that we will answer because then hopefully we’ll satisfy them a bit more. We will still lose a chunk of them but we satisfied this chunk. Is that okay?”

0:27:05.4 FS: And so that’s what I mean about you can catch a situation and then it makes you think about it in a different way. It’s just that ability to experiment with something and really see the impact. And it doesn’t need to be even 100% accurate to get that aha moment. It just needs to be accurate enough to catch the dynamics of what’s happening in the process.

0:27:26.8 JH: Man, I’d be so mad if I was that caller that got cut off the top, I’d been waiting two hours and they took the person that called last.

0:27:32.7 TW: But that’s fine.

[laughter]

0:27:33.8 FS: Yeah it is. It’s, yeah, in the KPIs. So, yeah.

0:27:39.2 JH: Yeah.

0:27:39.9 TW: It’s worth it. The danger of averages or whatever the median, that’s what you get into…

0:27:46.2 FS: Yeah.

0:27:46.5 TW: What’s your median wait time versus your average. That poor sap who was on the call for four hours.

0:27:52.4 FS: Yeah. And I’m the worst person, like when I’m in a processor, if I’m standing in a coffee shop, honestly I’m standing there just going, “I can fix this for you. There’s a reason you queuing.”

[laughter]

0:28:10.9 TW: Oh, no. This knowledge is dangerous.

0:28:14.9 FS: Yes.

[laughter]

0:28:15.1 FS: Disneyland, my husband hated going to Disneyland with me ’cause I’m standing and I’m gonna go, “Oh, that’s really clever. They’ve made the queue. Walking into two, sort of halves… He’s just like, he’s like, “Enjoy the ride. This is not what we came to Disneyland for.”

[laughter]

0:28:35.5 TW: I mean, it does seem like… Almost from a… You clearly and I feel the same way. It feels like you get a lot of joy from going in and just trying to break down the pre-work the. “Oh, ultimately we’re gonna do a simulation but I’m gonna get to actually understand your process and have a conversation with the domain experts,” which seems like that has very, very broad application. And you have in some of our pre-talk about this, you had had referenced that sometimes, not to put words in your mouth, you get the most value in just getting to the point where you can build the simulation before you actually build it, just doing the discovery with the client. Can you talk about that a little bit?

0:29:29.9 FS: Yeah, and I think… So I think there’s a misconception in simulation that goes back a long time that it’s about this sort of hard number crunching analytical tool. And it can be, and we build a lot of simulations like that when there’s definitely a place for them. But there’s what I would describe and I think a lot of data scientists just in general would identify with, this sort of the art of simulation is the way I would refer to it. It’s all the soft parts that go around it. It’s just the skills that you learn in terms of being a consultant and being a facilitator and everything else. But simulation is a very unique tool in that way because it has the power to just get people talking and get them thinking about things in a different way and and letting them see. I think a big part of it is the visualisation, it’s letting them see things that they maybe can’t. Because often in a modern day world, I mean, a call center process might span four or five different countries because calls are being farmed to different parts of the world at different times of day depending on what we can… So you can’t see your process anymore the way that you could in the olden days and sort of now really we’re just on the factory floor, whatever. So processes have got more complicated and businesses got more sophisticated.

0:30:31.6 FS: And yeah, so a big part of what and our users do and our team do when they go in with users is just talking it through with them and sort of… And you get a lot of the joy of that the problem formulation. And there’s a lot of value kind of in that and just kinda getting that to and putting to understanding and sometimes simulations even used. And the end goal is not to build the simulation, it is just to go in and get people around the table and just get them talking about ideas, see something that can kinda come behind or get behind and do some experiments with it. So there’s a few different examples where I’ve seen that done. I’ve seen that done when some sort of government kinda changes have been coming out and we’ve used it with a facilitation tool with the public to sort of make them understand why they’re making a decision that might be kind of a bit more controversial. Look, here’s what we think the impact will be, this is what we’re trying to do. And we’ve been sort of analytical about it. I’ve seen it where people, they just need to get a shared understanding across an organisation. They maybe have different departments whose processes kinda come together but they all work in isolation.

0:31:50.7 FS: So I’ve seen them come together and go, “All right. Okay, so when I make this decision here it knocks onto you two months down the line. Ah, right. Okay, yeah. Didn’t get that.” But why would they because they can never see that these departments are working in isolation or equally, we’ve seen it maybe sometimes in sort of healthcare services where they’ve just wanted to come together and just get everybody thinking, what can we do differently? How can we approach this candidate problem differently? So the simulation lets ’em see what they’re doing and then radically just sort of quickly check some different ideas and then go, “Okay, right, this one works, this one doesn’t.” And yeah, the people parts of processes and data analysis, they’re sometimes… They’re often the most frustrating but the most rewarding parts when you really help somebody get to that aha kind of moment, it’s just… Yeah, it’s what it’s all about.

0:32:47.2 TW: It seems like a really sneaky way to get people who maybe don’t inherently think process to actually talk process. ‘Cause it makes it feel they’re doing something.

0:33:00.0 FS: Yeah, and I think it’s… To use the coffee shop example, like to go back to it. So when I’m standing sort of in a coffee shop and I can see them starting to make my coffee first, so my orders come out of the coffee and I’m getting my sort of panini sandwich sort of thing that needs to go in the oven, and I can see them trying to make my coffee. But the slowest part is actually making my sandwich.

[laughter]

0:33:25.1 FS: So why are they making my sandwich first then come back and make me coffee? No, I’m making a joke and that’s a really small example, but I’ve been looking at processes for 25 plus years, that makes me feel so incredibly old. But the reality… So there’s loads of stuff that’s just intuitive to me and some people’s minds working that way. But we don’t need nurses to be process experts. So that’s where simulation can come in and it can show them things that are a little bit differently and just sort of make them realise, “All right, okay, if I change the order, I do the steps then, or if I bring all this equipment into the one room so that I’m not having to go to different places, that times the 50 times I have to do it today is gonna add up to breaking it apart.” So it’s almost like a learning tool. So it’s abstracted from the day-to-day. So you need that. We can help people improve processes when we don’t have the access to the experts of those processes. And those are people that are doing the job and they’re doing a hard job, but what we can do is help show them, “Right, look, but if you change what you’re doing, you can get more out of your system. You can get better outcomes.

0:34:36.5 MH: Yeah. And it seems like to start the process, you need some sort of a going in question or, we know we have this problem that we’re trying to solve, but also how often is it true that in the process of doing this simulation or whatever, people will actually find new things they never thought of and then how do you determine the validity or whatever of like, okay, yeah, that’s something we should address or whatever. So like, I don’t know the right way to ask that, but hopefully that makes sense.

0:35:03.7 FS: Yeah. I know it does. And it’s all the time, it’s all the time. And any piece, any data analysis you can do, I think that that’s true. You start off with one question, you learn something, you pick it a bit insight and that kinda takes you time. So you have to… I think that’s where it comes back to sort of the process that you’ve got around. So yeah, trying to think of that core question that we’re answering and it’s a bit the sort of the building as you learn look that we do, so start… It’s very kinda similar. Okay, what’s the question we’re trying to answer?

0:35:32.6 FS: Let’s build the simulation and do enough of the data analysis to answer that question. What have we learned? Okay, now next, that second iteration, is the question we’re asking still valid? So we’ve got further work to do and go around that. Or actually is there another better question that we should be answering? And so it’s continuously going through that kind of look. Now, I’m making that sound really easy to do. It’s not, we all get distracted and we go, “Oh, this is really good.” But yeah, but in principle, every time you have to just keep coming back. Okay, what was the question? Have I learned enough? Okay. Are we on our next question? And that’s what I meant earlier about, it’s not necessarily one big simulation because we might throw the simulation away in each of those cycles. So it’s just about doing enough to answer those questions and move on to the next part.

0:36:13.7 TW: And then do simulations provide the ability to give a scale of change or this impact level or is it… I don’t know the right., Tim, you’re better at this. Basically, if I run this simulation 50,000 times, it’ll tell me that if I change this part of the process, it’ll improve this by 15%. How much is that predictive or how accurate is that? Or do you have to sort of do… In other words, does it give out accurate results, I guess is the right way to say it?

0:36:50.0 FS: Yeah. It does. And so that’s part of the sort of the validation verification process. So you can build a simulation for a process that doesn’t exist and you’ll get a lot of value from that. But technically what people are doing is that, there’s a process that exists and so they’re building the simulation and what you do first is go, “Right, okay, these are the sort of inputs I’ve got. These are the sort of the key results I’m looking at. Does this match what I’m seeing in the real world?” And you’ll keep going back and sort of hone in the simulation until you get those output KPIs reflecting what you would see in the real world. Now that’s where the caveat comes in. ‘Cause it doesn’t always need to be exactly 100% accurate. There’s a scope and it comes back to exactly, am I all right with them being within 10% or is this number so critical that I need to get it down to the 1%? So there’s a bit of, yeah, figuring out what’s the right level.

0:37:46.2 FS: So then once I’ve got simulation and it matches what the world is and it’s sort of it’s the accuracy, then I can start making changes. And then where simulation really comes into its own is that we can do… We can rerun the same week one time, 10 times, 10,000 times, 20,000 times, 50,000 kinda times, however many times you need to do. And the more runs that you do, the more accurate it will be. And then what that ensures is that you’ve not run it for one good day, you’ve not run it for a bad day, you’ve not done it for an awesome day. You’ve taken into account the range of days that you’re gonna get, so that you’re able to then predict overall how the system will perform and how many runs to do is a question we always get asked. And that’s down to how much variability, then we can have the process. So there’s some processes, you might be running a really long process for years, therefore you don’t actually need to do that many runs. Or you might run one week, but there is an awful lot of variability in it. And so you wanna do lots of runs, but the good thing is it’s still really, really fast.

0:38:46.1 FS: So I’ve built simulations that run 25 years and in 25 seconds. So, it’s not like these runs are gonna take days and weeks and months to do because simulations can be valuable if they can be built and run as quickly as the decisions a business needs to make and at the pace that the business needs to make the decisions. ‘Cause otherwise you’re gonna go and go instant.

0:39:05.6 MH: The other question that’s sort of in the back of my mind is around uptake of decisions or information. So one thing that exists in the world of analytics for sure is people have preconceived notions of what they think the result is or should be. And certainly when we do other analyses or predictions or things like that, people will have a hard time believing their data scientist or analyst or whatever. I don’t know if you feel comfortable comparing and contrasting, but I wonder, to what extent do simulations help or hurt or do they feel the same? What has been your experience? I’ve literally watched executives watch a usability lab test and walk out thinking that user is so stupid.

[laughter]

0:39:56.8 MH: Instead of taking in the idea that, no, the user’s confused by your terrible system here that you need to then go address. But it’s like they just can’t put their mind around the idea that, “Oh yeah, maybe it’s my problem or my fault or my idea is not the right idea.” I don’t know. I’m just curious what your experience has been there.

0:40:17.1 FS: The phrase I would use to describe that is that you’re telling them their baby’s ugly.

0:40:20.9 MH: Yeah.

[laughter]

0:40:22.9 MH: Nobody likes to hear that. Yeah.

[laughter]

0:40:24.4 FS: Nobody likes to hear that. So there’s a lot. But I think it’s trust, isn’t it? It’s trust and it’s sort of building up that trust. And I think that is always a challenge and I think that’s a challenge in any project. You use an analyst, gets so deep into the numbers and you really understand all the parts and you know how you built the system, but you’re asking somebody to trust in that black box. I think that’s where simulation does have some advantages, because you’re not just presenting a chart and then I have to trust that you did the right analysis to produce the right chart. It’s the simulation’s visual, and so you’re seeing something that looks like your process and we’re gonna engage with you. And a big part of the simulation project would be working with your [0:41:13.6] ____, what do you wanna try? Okay, you wanna try this idea, you wanna try that idea? And so we’ll build little interfaces on top so that we can just radically try lots of different scenarios out. And that allows them to go on a journey of discovery themselves, but also get a bit of exposure to what’s happening behind the system and therefore build trust in it, so that outcomes are more likely to be implemented.

0:41:38.4 MH: And I think that might be it right there. A simulation visualises out this process where sometimes I think as analysts, we don’t get credit, and sometimes analysts don’t fully understand the process themselves. And so their analysis focuses on one aspect of it, but misses other pieces. And maybe that’s the… That’s where this can become a very powerful tool potentially then.

0:42:00.9 FS: Yeah, yeah. Definitely.

0:42:02.7 MH: Interesting.

0:42:02.8 FS: Yeah, it’s…

0:42:03.6 MH: All right. I’ll let Tim and Julie get back to the science questions.

[laughter]

0:42:06.5 JH: Perfect. No, just kidding. No, ’cause I actually do have one that’s maybe a little left field compared to some of the questions we were just talking about. You mentioned something. Oh, Michael, when you first asked your question about when you’re designing it and the accuracy and the ranges and you mentioned, when you’re building the simulation, you are looking for a tolerable range on those KPIs, those outputs you care about. And so it made me think, and I’m guessing Tim thought the same thing. Is there an idea of over-fitting in building a simulation?

0:42:40.8 FS: So by over-fitting, you mean having made it too perfect?

0:42:43.8 JH: Yeah. When you’re doing a model, if it perfectly matches the training, it’s too sensitive and then you put it with your test data and it’s gonna go crazy.

0:42:53.3 FS: No, that’s a good question. It’s not one I’ve been asked before. Do you get over-fitting?

0:42:58.0 TW: Well, I guess if somebody says, oh, you said we took our test data and we put it in and we’re within 10% and the client’s like, “No, keep iterating on the simulation,” and you keep tweaking things and say, “Well, now with our test data, the simulation is within 2% of reality.” But you were almost… It sounds you would have your spidey sense tingling saying, “But wait a minute, now when I make this change I might have over modeled the… ”

0:43:28.6 FS: Yeah, okay.

0:43:30.2 TW: The exact understanding of the history, which means when we try to change something, it may not be a good reflection.

0:43:38.3 FS: Yeah, I think there probably is an element of that. I think more where my clients come from is that it is the time that it’ll take to get the additional accuracy of 2%. Is that a worthwhile investment in terms of is it going to make the simulation significantly more accurate? The input, affects the inputs, but it means the delivery time and the simulation is much kinda slower. And it comes back to that, what I was saying about the simulations when we’re… A lot of our customers, some of them, that’s all we do. Some of our users, they’re just building simulations all the time and they’re supporting business with the decisions being made, all the time. So they need to do that really, really quickly. They can’t be going back weeks later with the sort of the decision ’cause the manager of the process had to make the decision already, you had to do it yesterday. We all have to do that. So, there’s an element of just, that’s what I mean about it and it’s sort of the art of it.

0:44:38.0 FS: It’s just making the simulation complicated enough. But there are times, and I think simulation is moving into different fields now where it’s not about building one-off simulations. What a lot of customers and users of it are now doing is sort of building operational tools. And so the simulation’s effectively making decisions in their process. And so, okay, what is the right bed to place a patient in and it’s helping with real time decision making. Those simulations need to be 100% accurate, and the time that it takes to get into that is worth it because that simulation is gonna be used 50 times a day plus or sort of, kind of, and there’s lots of different use cases. So it’s a data science project in that sense. Does the extra effort get me extra value?

0:45:29.7 TW: If you’re gonna spend so much time building the simulation that you could have just run an experiment, a well designed experiment and gotten the answer, then maybe you spent too much time trying to build the perfect simulation, I guess. Interesting.

0:45:44.9 FS: Yeah.

0:45:45.9 JH: Okay. The one big question I know we’ve talked a lot about simulation and what it is, how you do it, where you use it, but I would love to hear, and I know you talk about this a lot, I believe in some of your talks in other places, but where is the technology of simulation going next? What’s the future of simulation?

[laughter]

0:46:05.5 FS: Future of simulation. Yeah. That is the star gazing blue sky part. That’s one of my favorite parts of… [laughter]

0:46:13.2 TW: If you simulate… Have you built a simulation to simulate the future of [laughter] the different ways that simulations could go?

0:46:19.6 FS: We built a lot of simulations to table for different things, but yeah. But about simulating… Okay. We have a future simulation [0:46:27.2] ____… It’s that shift, we’re moving from using simulations to make one-off decisions, to make real time decisions. And that’s already happening. So people are already doing that. And it’s the ability to use process mining. So a technique that’s gonna allow you to analyse the data and find out the process flow. Then we’ve got machine learning, which can come in and mean that we can get the decision rules automatically. So now we can have self building, self adapting simulations, which means that a simulation can never go out of date. So we can now put it in the heart of tech status within organisations to make decisions. And that can be about understanding on a weekly basis, what’s the best sequence to put the cars through down the production line to make sure we hit our orders.

0:47:21.8 FS: It can be about a bed manager in a hospital understanding that, okay, the simulation’s running for them and saying, right, okay, based on the emergency arrivals that are forecasted, the planned surgeries over the next two weeks, the beds that are available, the rules that are there about gender or whatever else, put the patient in bed 341 because that’s gonna be the most efficient for today, but also for next week’s running of the hospital. So it’s that real-time decision making. So we can do that, but then where it’s going to go and where it’s leveraging up is then that we are running the simulations forward and we can tell you in advance, there’s a problem coming, a week, two weeks down the line, you’ve got a problem coming. We’ve done some scenarios for you. Here’s what the corrective and preventative action you can take so that you’re never gonna get into that crisis situation. And that’s where the future’s going. So cognitive simulations.

0:48:19.3 JH: I have so many more questions now.

0:48:21.4 TW: So that would be if your simulation has said, if you go back to the hospital example that when you say, well, let’s simulate a shock to it, a pandemic or even just a large high injury count and yeah, we’ve got the simulation, we’ve got this working well, but if this shock happens to the system, the wheels are gonna fall off. So now let’s do our disaster planning with that sort of thing and say, if this happens, then we need to do something else.

0:48:55.2 FS: Yeah. And you can already do that. So just have your disaster plan. That’s a big use case for what people do where… But it’s probably those things that sneak up on you. Actually, there’s just been a really high incident of emergency arrivals into the ER department over the last weekend. Turns out it’s because there’s been some mad music festival and something’s all gone haywire. We’ve got all the people kinda coming, but it’s gonna ripple on through the hospital for weeks and weeks to come because now we’ve got patients in the… So now we’ve got an elevated backlog, we’ve still got all our planned arrivals. It’s also Christmas time. So we know arrivals are always heavier at that…

0:49:36.5 FS: So it’s that type of stuff. It’s understanding that there’s only inputs that are changing and coming and shifting. And if they can come together and they can collectively… So sometimes it’s more the small things rather than the big things. It’s the small things that come in because people spot the big disaster. They spot the earthquake that’s happened. They don’t spot that there’s this myriads of little things that…

0:50:00.2 TW: The little thing that’s gonna…

0:50:01.9 JH: How do you sleep at night? My brain would just be going forever, [laughter] living in that world. I already feel that way. I’m like, good thing it’s 7:00 in the morning, [laughter] my shot of caffeine. Because now my brain’s just turning.

[laughter]

0:50:15.2 FS: There’s a lot of screaming at TV screens I have to say. You’re just like… [laughter]

0:50:21.5 MH: The curse of knowledge. All right, well we do have to wrap up. This has been an amazing conversation. Thank you so much Frances. This has been awesome to talk about this topic with you. It seems like a super powerful tool in the arsenal of a data professional and it seems well worth it to take somebody from the team and really get them focused on this and leverage this capability. All right. Well, we have to start to wrap up and one of the things we love to do is go around the horn and share a last call. Something might be of interest to our listeners. Frances, you’re our guest. Do you have a last call you’d to share?

0:50:58.4 FS: First of all, just to say thank you for having me and I’ve enjoyed it as well. But yeah, my last call would be that, simulation can do a lot of good in the world and we did a lot of good during COVID and we did a lot of pro bono work to help. And particularly as an organisation, we do want to help because we know that this isn’t always accessible to everybody. So, if people are working the third sector or whatever else, then we’ve got a Tech for Good program. Please do get in touch and let us help you if we can.

0:51:25.5 MH: Very cool.

0:51:27.2 TW: Yeah, that was almost a whole other potential topic was just the Tech for Good and simulator we’re like, “Oh, that could be… ”

0:51:34.2 MH: We’ll include a link to that in the show notes as well on the site so people can check that out. Thank you. All right, well Julie, what is your last call?

[chuckle]

0:51:46.7 JH: My last call is weirdly on brand for this and I really didn’t think about it until last night. I was typing up more notes about it and I realised it’s a simulation, so [chuckle] it worked out well.

0:51:57.7 MH: Nice.

0:52:00.2 JH: It is an article by Duolingo about a simulation they built to figure out how to move their daily active users, main KPI. They had seen a lot of growth in their initial years and then it was becoming stagnant and so they actually built what I think they call it their growth model. And I just… I love the article. It was really well-written, it was really simple and straightforward and I thought it was really, one, interesting. Two, it opened my mind up to, I think a lot of things that are applicable to what I’m trying to do at work, and then of course, realising it was a simulation. It was really cool to see how they were trying to figure out what other metrics can help us move our daily active users, and so by building out their whole process, mapping it all out, running a simulation on it, they were able to figure out which other metrics to go target and they’ve seen huge success from it, and then they talked at the end about how they’re trying to actually make it more in-depth and future proof it for the next time. Daily active users becomes stagnant ’cause they know they can’t grow to 100%, whatever that may be, so I thought it was a great article.

0:53:04.0 MH: Awesome. Nice.

0:53:05.1 JH: Yeah. What about you guys?

0:53:07.3 MH: Well, I’m getting to that. No, but I’m just… You’re setting the bar so high by tying it into the show topic. Everything is really…

0:53:13.9 JH: I really didn’t mean to. [chuckle] I was lucky.

0:53:15.6 MH: No, but own it. Just take it. Just take the victory lap. It’s awesome. You’re a natural. Okay, Tim, what about you? What’s your last call?

0:53:25.5 TW: Well, so I was gonna have just one, but now since I’ve got one I can tie into the show, I’ll do a quickie.

0:53:29.4 MH: Oh, okay.

[chuckle]

0:53:32.4 TW: This was… Again, I realised that, oh, this was sort of a… And this was a little bit of a slightly different simulation, but it goes back to Merritt Aho had started working on it. It has to do with sequential testing. If you’re doing classic A/B testing and, “Don’t peek, don’t peek,” and Merritt started asking like, “Well, what happens if you do peek?” He didn’t invent sequential testing. But recently, a month or so ago, Matt Policastro did a presentation to the Test & Learn Community where he talked about sequential testing, but part of that was he and Merritt had run sort of… Simulated like, “What would happen, what would your false positive… I think your… How would your false positive rate be affected if you peeked early without changing some constraints?” So I just have that picture in my mind of them showing all of the simulated tests and what would’ve happened and therefore what could they do to correct for that, and that introduced sequential testing. And I’m probably butchering it, but it’s on YouTube. There’s a video, so we will link to that.

0:54:37.5 TW: But my non-simulation last call was Nick Potter on the Measure Slack posted this, and it is 100.datavizproject.com, Viz with a Z. But it was… These guys took a six data point data set and they visualised it 100 different ways, which sounds crazy, but they were just like, “With just the simplest of data sets, how many different ways could we visualise it?” And they came up with 100 different ones. Some of ’em, I don’t think anyone would recommend, but they’re all pretty polished little vignettes. And it was an interesting exercise to think… I mean, I first read it and said, “You can’t possibly… These are gonna be simple little derivatives. They’re gonna have to be cheating.” And no, they came up with 100 distinct visualisations, so it was a thought experiment taken to an extreme that’s worth checking out. And you, Michael?

0:55:40.3 MH: Well, I’m so glad you asked.

0:55:41.2 TW: I’ll stop with two. Unless you wanna skip since I did two.

0:55:46.0 MH: No, it’s okay. No, I do have one briefly that I’ll share. So the Digital Analytics Association, of which I’ve been a part for many years, they’re having an executive leadership forum coming up in April, April 26th and 27th. So if you are an executive leader and you want to talk to other executives about simulations specifically, [laughter] I suggest you go and attend that. I don’t know if that’s one of the topics, but now that you’ve listened to this episode, I think you can go there and make it a topic [laughter] at the Executive Leadership Forum. But we’ll put a link to that in the show notes. It looks it’ll be a very good event to just think through some and talk to other leaders in different spaces about analytics challenges and solutions. All right, well, this has been awesome and we’ve… Really love to hear from our listeners. So if you’re out there listening and you have questions, you have thoughts, we’d love to hear from you. The best way to do that is to reach out to us on either LinkedIn or the Measure Slack group or Twitter. And we’d love to hear from you. I don’t know, Frances, are you active on social media at all? Is there someplace where people could follow you, or?

0:56:54.7 FS: Yeah, you can find me on LinkedIn or on Twitter, and yeah, Simulation. [chuckle]

0:57:01.7 MH: Awesome. All right. So I would say probably go follow Frances as well. It seems the wealth of knowledge lives there. Anyway, no show would be complete without a shoutout to our awesome producer, Josh Crowhurst, who works behind the scenes, and sometimes in front of the scenes, to make this show happen. So Josh, thank you very much for all you do. And once again, Frances, thank you so much for coming on the show. It’s been awesome to have you. Thank you for handling all of our questions and teaching us a little more about simulations today.

0:57:34.4 FS: Well, thank you for having me. It’s been great fun. And then you guys have asked really good questions, [laughter] really good questions. [laughter] Thank you for that.

0:57:44.3 MH: Awesome. All right. Well, I know I speak for my two co-hosts, Julie and Tim, when I say, no matter what the process looks like and the number of simulations going through it, just remember, keep analysing.

[music]

0:58:01.5 Announcer: Thanks for listening. Let’s keep the conversation going with your comments, suggestions, and questions on Twitter at @AnalyticsHour, on the web at analyticshour.io, our LinkedIn group, and the Measure Chat Slack group. Music for the podcast by Josh Crowhurst.

0:58:19.4 Charles Barkley: So smart guys wanted to fit in, so they made up a term called analytics. Analytics don’t work.

[music]

0:58:26.3 Kamala Harris: I love Venn diagrams. It’s just something about those three circles and the analysis about where there is the intersection, right?

0:58:36.0 MH: I remember when I was in college, I worked at a Starbucks, and I would work there and I was like…

0:58:43.7 TW: That explains so much. [chuckle]

0:58:46.3 MH: Yeah, I worked at… You know, we’d be at the cash register, getting people coffee, making coffee, and I was like, “Someone has had to have done some sort of modeling of who works where and what position in this store to make the flow work correctly.” And I was like… In my back of mind, I was like, “They probably did process simulations to basically say like, “All right, we need one employee to be here. We need to put this here, we need to put this here,” but yeah, when they added the sandwiches, the Starbucks, I think they threw it all outta whack, and then they make your sandwich at the wrong time. It’s a huge problem. I think you’re so right.

0:59:20.0 TW: And that’s when Starbucks became a horrible place.

0:59:23.0 MH: Oh well no, it’s when they took out the actual barista-making machines and made ’em all automated. That’s when it all went… So yeah, terrible. Just terrible. Used to be an art. Used to be an art to it. [laughter]

0:59:37.2 JH: Supposedly, I heard that Waffle House has some really impressive process they follow. They speak some shorthand to each other, like this crazy… If you listen. I mean, I don’t know if there’s any Waffle Houses around you to easily go to, but… [laughter]

0:59:53.8 TW: I mean, there are around me.

0:59:55.3 MH: My daughter loves Waffle Houses, so.

0:59:57.3 JH: I just know my mom and my stepdad were telling me, they’re like, “We sat at the bar seating and you could hear all of them talking and the waiters and they speak in numbers, letters and where you put the toast, tells you what order it is,” and they’re like, “It was amazing and everything came out so fast.” So they know something.

[laughter]

1:00:16.0 TW: It’s no botched cues at Disney. That’s for sure.

1:00:18.6 MH: Yeah, well, I think they’ve really made an art out of it. But it’s also when you’re sitting in a traffic light and it’s way too long and you’re like, “Who planned out this intersection?” You know, all that stuff, all synergy.

1:00:30.1 TW: You know, I worked in the traffic signal industry. When you were doing your barista-ing, I was in the traffic signaling. So we could talk… Rock flag and faster coffee through simulation.

Podcast: Download | Embed

Subscribe: RSS