#104 - Growing Through Experimentation - Lisi Hocke

[00:01:34] Episode Introduction

Henry Suryawirawan: Hello, my friends. Welcome to the Tech Lead Journal podcast, the show where you can learn about technical leadership and excellence from my conversations with great thought leaders in the tech industry. And I’m very happy to present you episode 104. If this is your first time listening to Tech Lead Journal, subscribe and follow the show on your podcast app and on LinkedIn, Twitter, and Instagram. And if you’d like to support my journey creating this podcast, subscribe as a patron at techleadjournal.dev/patron.

My guest for today’s episode is Lisi Hocke. Lisi is an active figure in the global testing community. In this episode, Lisi shared her lessons learned growing an experiment-driven quality culture in her recent years. Lisi shared why it is important to have an experimentation mindset before we adopt something new or any best practices, and to have a safe environment to execute those experiments. Lisi shared her advice on how to run an experiment from building transparency, creating hypothesis, getting buy-in, and understanding our biases, and in particular, the sunk cost fallacy. In the latter half, Lisi shared her personal transformation journey learning in public and shared her tips on growing technical confidence.

I hope you enjoy listening to my conversation with Lisi. If you also enjoy it, please help share it with your friends and colleagues who can also benefit from listening to this episode. It is my ultimate mission to spread this podcast to more people, and I really appreciate your support in any way towards fulfilling my mission. Before we continue to the conversation, let’s hear some words from our sponsor.

[00:05:06] Introduction

Henry Suryawirawan: Hi, everyone. Welcome to another new episode of the Tech Lead Journal podcast. Today, I have a guest with me named Lisi Hocke. Lisi is someone who is very active in the testing community. She actually has a lot of blog posts around testing. And in fact, the latest blog post when she attended the Agile Testing Day, that looks really beautiful. There are so many illustrations that she did herself. So make sure if you are interested in following agile testing presentations and materials, go check out Lisi Hocke’s blog and you can see so many beautiful illustrations on that website.

Lisi, thank you so much for attending this session. Basically, we’ll be talking a lot about testing culture. So you want to share a great sharing topic from your personal journey about experiment-driven quality culture. And later on, you also have some personal sharing, which I’m really looking forward to hearing from you as well. So, welcome to the show, Lisi. Thanks for spending your time.

Lisi Hocke: Thank you for having me. It’s great to be here.

[00:05:59] Career Journey

Henry Suryawirawan: So Lisi, I always like to start our conversation by asking your career journey. Maybe if you can share any highlights or turning points so far in your career.

Lisi Hocke: I’m happy to. So I think I wouldn’t have a career that’s quite straightforward, so maybe there’s quite a story to start with. I never thought, for instance, that I ended up in tech. I’m very happy I did in the end. But it’s not where I started off. I studied also a different subject, coming from a completely different background, basically. But when I reached the end of my studies, I really felt, okay, what do I do now? Because it was a quite generic study. There were many options, but what to pick? Where to start a career?

Based on the thesis subject that I had, which was computer games in China, I felt maybe, just maybe, because that happened, I could also try and see if I can get my foot into the computer games industry, because I have a passion there. And maybe I could combine that passion with my work, which would be amazing. If I don’t try, then I don’t know. Having that thought made me actually do it. I ended up in a little startup in the computer game industry. We were not developing games ourselves, but actually an AI middleware for computer game developers. So I was ending up in a software development company. Very small, complete startup. So that was my entry point into tech in itself, and also got me into contact with agile. What is Scrum? What are the different roles that we need? The different skills that we need. Everything. And I just loved it.

I actually started as a game designer because they honestly just gave me a chance. I’m not coming from that background either. But just a few months in, our tester quit the job. That was a turning point for me, definitely, or basically, my starting point. Because my manager back then called me in, into his room with a very serious face and he said like, “Hey, you know, what are you doing right now? You’re not doing so well.” And I was like, “Oh my gosh. Don’t tell me I’m fired. My whole world is scattering. I just love this place. Please, can I stay here?” And then he continued, “Testing could be something for you.” And in hindsight, I know he needed someone to take over that job. But for me, it was like, I have no idea, but let me try it because I definitely want to stay here. I love that place. Maybe it does fit or not. I don’t know because I have no idea what’s it even about. The thing is, he probably needed to sell the position, yes. But he actually was right that it was a better fit for me. Because suddenly, I could really use my strengths. I could contribute in many different ways that I couldn’t before. So it really played to my strengths. I realized this is something that I could imagine working in for a very, very, very long time.

So that’s sort of how I was gently pushed into testing and quality or fell into testing. But in the end, I really just found my place in tech, and realized this is something I never imagined I could work in. But it makes a lot of sense when looking back at my history. I also grew up with technology, grew up with computers. I just was socialized differently. So I never thought I could really work there. And that turning point, that really showed me how well I could work there.

Now, back then, there was no other person. It was a really small startup. So the other tester quit already. I was alone. We had some very old books in our very little library to learn about testing and quality. That’s where I also started to find the community outside. All the people that are sharing their knowledge and sharing the insights. I learned a lot by trying these things out. One of the big highlights back then was my product owner back then. We didn’t have a training budget in that very small startup, but we could at least buy books. And one of the books that I got was “Agile Testing” by Lisa Crispin and Janet Gregory, and that opened my eyes to a whole new world. And then I started following them on Twitter and seeing, oh, there’s so many other great people and so many resources and started to learn from there. So I received a lot from that community back then that I could actually try out because, again, I didn’t have anyone else to talk to or to learn from at work. So I got a lot from outside to try out those inspirations at work.

Well, that startup in the end, it didn’t make it. So it was sort of a roller coaster phase. But I wouldn’t miss it because I learned a lot back then about all things software development and teams and collaboration and a lot of stuff. Later on, I went to, let’s say, a consultancy. So working on projects, learned a lot about operations, third-level support, and so on. But it got me away from product development where I was before, where I was really happy. It also got me further away from testing, quality, and it felt like, no, I want to get back to that.

My last company where I stayed for six years, I was finally in product development again. I really realized then, okay, this is my happy place. This is where I’m thriving. This is where I can also contribute most. I had a great journey there. Also at that company, I had another turning point basically for me, because I became a speaker at conferences. So suddenly not only getting input from the community outside but also being able to give back, which was a journey in itself. But it was definitely a turning point in the sense of increasing my network, having a lot more access to knowledge this way, getting even more inspirations because speaking in conferences means you have like an entry ticket to a lot more conferences. Now I had a training budget but, obviously, no training budget is paying you like five, six conferences per year. Speaking really opened that door for me. I got to know a lot more people, a lot more inspiration.

I think the next turning point then was to level up in seniority and getting a fancy new title after senior. Back then in that company, it was principal. Titles are context-dependent. So it was just the next one after senior at that company. But with that, I was also suddenly part of leadership, of formal leadership. Not in a management role, but in that individual contributor role. So I started to make more intentional impact across the organization and seeing, “How could this look like?” I realized suddenly people also perceived me differently because I was still the same person, but suddenly I had that title. So that’s something that also opened my eyes a lot in the last years.

And I think the final one is now that I have just started at a new company to see how much value is actually my experience worth. Can you transfer it to another company? Because if you stayed for a long time with the last places, at least I start to wonder like, okay, is this actually applicable also in another context, or is it really in that context? Is it really constrained to that? And yeah, that brings me to today and our conversation.

Henry Suryawirawan: Thank you so much for sharing your journey, Lisi. I think some people might relate to that particular journey as well. So where you started in, probably not in tech, the background, but then you gave it a try. You didn’t find the first fit of the job, the game designer thing, right? Probably some people also experienced the same thing. But hey, sometimes opportunity happens. And if I remember correctly, Janet Gregory and Lisa Crispin themselves actually didn’t start to pick testing in the first place. So they somehow grew from opportunities to opportunities. Now they become thought leaders and I’m sure you are also quite active in the testing world and become inspirational for other people. So thank you so much for sharing this story. I’m also intrigued by your journey going through conferences over conferences. This is, I think, probably some people aspire to do as well. We’ll be discussing about that later.

[00:13:39] Sharing About Building Quality Culture

Henry Suryawirawan: But I just want to maybe go through our first topic where you probably are sharing some of the experiment things that you did in building some kind of quality culture. I think changing culture is hard, no matter which company you are in. Small or big, maybe big is even tougher. So tell us a little bit more about this journey that you have. Maybe a little bit of background about this story.

Lisi Hocke: So this is a story about the last three years at my last company. So from 2019 to 2021, basically. It’s covering that time span. Back in 2019, the company I worked at was in the phase where we had a lot of product teams, like I think 38 around that time. We had nine different domains. All those teams were really autonomous, or at least that was also our aim, always, to have them independent, to enable them to really make their own decisions in their own context. Because all those product teams really had a different context, and each of them owned like, either a full product or maybe a few services. But definitely, they were quite autonomous.

The result that we saw back in 2019 was: we actually had no real insight into how these teams were doing, especially regarding testing and quality, are they actually thriving and they could share more across teams like good practices and everything. Or are they having struggles? Maybe there is room for improvement, maybe we could help. But we just don’t know. It’s like a black box. Back then, our CIO then also sponsored an initiative to figure out, okay, how’s it going? Well, that job landed with me. So I was still embedded in one of the product teams, because that was the setup we had. But with my seniority level, I was meant to also drive these kinds of initiatives across teams, and well, in the testing quality space, that was dear to my heart. So I picked up that topic.

What I made really good experience with in my personal past basically, was to frame these kind of things as experiments with a real hypothesis, to know what I want to try and what I hope to have as an outcome from what I want to try before I do it. So I can really measure against, and I can also communicate clearly about all these things and also hold ourselves accountable as well. So that’s what we tried back then. The mission was really to level up the culture regarding testing and quality. We wanted to have those teams grow mature in the sense of: as soon as we’re there, we can really scale further because that was the ultimate goal of the company. The driver behind was we want to scale. We already had 38 teams, but we want to scale, like heavily. So we better know now where we are.

This led to an experiment and actually was like an experiment of a lot of small experiments. It was like an overarching goal to level up the knowledge, skills, and practices of teams, but also enabling them to experiment themselves. Because, again, their context was massively different. It’s not one size fits all that we could apply there. We went into the teams. We had some pilot teams to start with. Basically, to test our hypothesis if that could be proven or we need to find something else because that could also very well be, right, with experimentation. It could totally be that we’re on the wrong track. It could have also been that actually teams are thriving and they’re doing well. They just don’t share enough. Fine. No problem there. That would be cool.

What we found was, we went into four teams. Four different product teams, quite different aspects. So quite a nice subset of the teams that we had. We worked with each of them to, first of all, create transparency where they currently are. How do they build quality in? What are they currently do to test what they build? Are there any pain points? Because usually there are, right? And usually, teams know about that the best, because that’s their context. What could they try to improve? So transparency was one part. But they also needed to have the awareness of what kind of options they have to improve their situation, and then to experiment, to really try something new and see if that works or not. This is what we did with each of the four teams. Obviously there’s more details, how we created transparency, these kinds of things. But the end was that we helped them to design their own experiments, helped them support the run, following up and so on. And also design a second experiment for them.

This whole initiative span across like 10 months. We also had, like, a lot of reflection points. Because, again, in the end, we wanted to see is that hypothesis that we have, to help them create that transparency, help them spread their awareness and experiment, is that actually leading us further to that quality culture that we wanted to aim for? I would say, in hindsight, it was partly doing the job. Partly in the sense of it triggered a lot of great conversations about testing and quality and also the underlying problems in the team. Because usually when you start talking about testing and quality, a lot of communication issues are unveiled or collaboration issues, like deeper things, foundational things that need to be fixed first. And then we can talk about testing and quality itself. But it’s actually an interesting trigger about these conversations and really valuable. It also could raise a lot of awareness. We could break down some misconceptions. We could inspire teams, definitely, to improve their practices, which is great.

Well, unfortunately, that was the impact they had. We hoped for other impact as well, but we didn’t have it. For example, that not the whole team was on board. No matter how much we encouraged everyone to really join those sessions. No matter how much buy-in we got before from managers, from everyone. Not the whole team was on board in the end. Also, not all people opened up for new concepts. I mean, humans, we can’t force them, which is totally fine, right? But misconceptions prevailed in the end. But the saddest part was that the teams really struggled running their experiments, which was sort of the core of our hypothesis to enable them for continuous improvement, and to really look further. They really struggled. Not because they didn’t want to, but it was that fallback into everyday business, which so easily took over. Like that roadmap, that’s higher priority. So we need to work on that. Like, yes, it would be nice to work on this experiment, but the other one, we feel the pressure. Any other results or actions spoke a lot louder than words in that matter. And again, we have a lot of encouragement and a lot of support from management. But the decisions of the actual people doing the work painted a different picture. So there was something else going on. We felt, okay, we need to switch this around. It also was too heavyweight for scaling because we spent a lot of time with the four teams. So we had 38. How to scale that? It would’ve taken way too long.

So next experiments started and further experiments followed, especially from what we observed and spanned across three years, basically. I would not claim we have found the perfect way to build that quality culture. But it has triggered a lot of thoughts, a lot of conversations, and that itself was already valuable enough to always continue trying the next thing that could hopefully help cater to what we need and cater to what we need from a product team perspective, where we started, but also as we later found for our tech leadership, so my peers basically, because they had different needs. This underlying system that we all worked in where I realized there’s something systemic here. So let’s look at our reward system. What do we actually reward them for? Do we reward actions contributing to testing and quality? Or is it something different? Because usually people have good reasons why they show certain behaviors. So, yeah, that’s sort of the whole initiative in a nutshell, obviously, there’s a lot more detail, as a starting point.

[00:21:14] Experiment-Driven Culture

Henry Suryawirawan: Thanks for sharing this end-to-end. I’m sure that when you are telling the story, you look back and bring back those memories. So in the first place, maybe just to start from the beginning, many people want to drive culture change. Many people have great vision. I want to be a world class tech organization or tech teams. I want to follow Google or whatever big companies out there. So the vision is always great. Building the culture itself is tough. And you have this angle, which I find very interesting, not many people do so. You actually make it such that it’s an experiment. When you have an experiment, that means you have a hypothesis. You have things that you want to try to do, and you have some kind of measurement, right? The outcomes that you want to try to achieve and measure whether you meet that over a certain period of time. Tell us why experiment-driven is something that we should aim for when we want to build a culture?

Lisi Hocke: So, originally, I started out to use that because I just made personal good experience with it. But in the end, over the years, I learned that this is definitely something I want to build as a culture. That we do more experiments. The idea is that we focus on learning, on fast learning, that we really need to try things out in our context before we actually know if it works or not. So that’s also the case with what often are called best practices.

Yes, they might work in a majority of context. But does it work in my own context right now with these very specifics? I don’t know if I haven’t tried it. Also, when we think of, like, the Cynefin model of the different domains, in which, like, be aware, are we working on something that’s really simplistic? Is it something that’s complicated, but actually we have good structures and we know a lot of good practices that we can use? Or is it so complex that we really have to explore that space we don’t know yet? Mostly, what I face in software development, we usually always build something new. We work with new people. We work in a new domain. We try to discover new land. So it’s a lot of exploration we need to do. For that, we need to experiment. We need to try something new.

The other part of experiments that I really like besides that we obviously can really have them help our learning, is about how you frame failure. So, a lot of people are afraid to try something because we could get blamed. We could fail personally. Maybe I lose my face because of that, right? It’s like there’s a risk involved. So I think the most important part about building an experiment-driven culture is to make it safe to fail and to fail in good ways. Like that hypothesis that I came up with, I could not fully prove it. You could see it as a failure. You could also see as it’s not a failure because we actually learned. It’s a good thing that we learned that this part is not working. Other parts are working so we can find something better.

But there’s a whole range of how we can perceive failure. I really like, I think it’s from Amy Edmondson, like a Spectrum of Failure, where she ranges it from like, blameworthy failure, really stuff that could be avoided that maybe you should not go into again, to really praiseworthy failure, in the sense of experimentation, of exploration, to learn fast. And I feel building that safety in can help in so many ways and allows us to experiment and to allow people feel safe to try something, and not get the blame, but actually get the praise. So try something in a good way, in a safe way that also limits the fallout, basically, of what could go wrong. And I think we all need to learn how to do that better, and me included.

Henry Suryawirawan: I mean, the spirit of agile itself is basically a lot of experiments and trying out and you know, shorten the feedback loops and things like that. You brought up two good points, actually. When we hear about best practices, sometimes we think, okay, this is the way to go. But you brought up a good point that sometimes we have to think about the context, where you are at. Some people put it like you are not Google, so don’t try to be following the Google practices, for example. So some best practices, even though it’s best practices for many people, sometimes it doesn’t work for your context for certain situations. So thanks for highlighting that.

And I think I like the last part, when you frame it as an experiment. So experiments sometimes can fail. It’s not like a commitment that you will achieve what you intended to do in the beginning. So I think experiments allow you to frame it such that even if you don’t meet the outcomes, it’s kind of like a success in a way. Of course, you bring up the spectrum of failure. We all should aspire towards more praiseworthy kind of failure rather than some things that we underlook or something like that. So I think that’s definitely a different kind of failure.

[00:25:41] Building Transparency

Henry Suryawirawan: You brought up about transparency in the beginning. When we started about these experiments, you brought up the point of building transparency. Know where the team are at. What kind of pain points? And things like that. Why do you think this is important to build this transparency in the beginning of the experiment?

Lisi Hocke: So what helped me with that is to have a baseline to really have a look into where are we right now. Not only from individual point of views but also from a team point of view, because it might be a different perceived reality for everyone on the team what we’re doing right now. Honestly, I had teams where different roles were on, like for example, the UX person, a developer, someone from quality, someone from product. All of them share different things. What needs to happen, for example, to get an idea to production? Not any of them have the full picture. Only when they really put that together visually in front of them, they realized, oh, there are other things I never knew about or never thought about, and also pain points that come along with that.

So to have that baseline, for me, is really important to not only face the truth of our current and acknowledge where we are right now with the good and the bad. Because it’s just fact. And then we have a baseline to experiment from, because then you can see the delta, basically. You can really know how things improved or maybe not, or maybe they went in a different direction. So you have that information to evaluate your experiments. So transparency in itself, I find really valuable because you can really share also the pain and also hopefully inspire others to reduce the friction that you’re currently experiencing. But then when you start taking action on that, you see that delta. Are we going into a better direction than before? It doesn’t have to be perfect, just a bit better. Or are we maybe regressing? That could also be, right? But then, you know, otherwise how could you tell?

Henry Suryawirawan: So I think, yeah, this comes back to, if I’m not mistaken, it’s a scientific method in a way, where you need to know where you start, right? Because sometimes we all tend to want to build a culture. Yeah. We know there is a problem. It’s not written explicitly. We just kind of feel. It’s a gut feeling and we start our experiment and things like that. But at the end of the day, we don’t know whether we are progressing or regressing.

So building this transparency and baseline, I think, are actually very important in any kind of experiment. So that you need to know, at the end, whether you are actually progressing, making good progress and the changes is intended. Or is something that is completely out of the blue, where you didn’t intend it to happen, but it happened anyway? So I think, yeah, this kind of transparency I can see it’s important.

[00:28:08] Hypothesis

Henry Suryawirawan: I think after you build that baseline, you have hypothesis in experimental scientific method. You have this hypothesis. So tell us why we should build hypothesis when building culture as well? How should we come up with this hypothesis?

Lisi Hocke: The hypothesis part is actually one that many people struggle with. And for me, it’s also personally the hardest part. The reason I’m still doing that is it forces me to really write down what is it exactly I want to do? And what do I expect out of it? Just that thought exercise, for me, is actually the most valuable part. Even if maybe I haven’t framed it perfectly because wording is hard. Writing out a hypothesis is also something tricky. Again, they don’t have to be perfect, but it forces us to write things down, to have it not adapted afterwards as we go. But really see, okay, this is where we wanted to start out from. Maybe we discover better directions.

It’s totally fine to also cancel an experiment and try something else. But then it’s an intentional practice. So formulating those hypotheses in itself, I felt always extremely valuable to clear up my thinking. But also then, in hindsight, to hold myself accountable. Am I really working on that? Can I prove that? Can I disprove it? How is it going? This is what I wanted to set out. If I want to change directions, again, that’s fine. But I should be intentional about it and also be open about it.

Henry Suryawirawan: So, yeah, I think this kind of thought process, where you want to experiment something, but if you don’t write it down, sometimes it could happen also things change and you go into different direction unconsciously. I think this is what we don’t want to do, because that means you are running a different experiment altogether. I think having this kind of hypothesis stage where you write down, or maybe make it clear to people who are involved, okay, this is the thing that we want to experiment, and this is the outcome that we want to try. So let’s give it a go, everybody in the same picture and give a try.

[00:29:58] Building Better Hypothesis

Henry Suryawirawan: Another thing that you mentioned about this hypothesis is that we need to be conscious that it should not be too big in terms of hypothesis, or maybe too open-ended or generic. We should make it such that it’s small enough and we should aim for faster iteration or short feedback loop. So tell us more how we can build a better hypothesis, actually?

Lisi Hocke: Again, this is one of the key things I feel, if you heard my story right now, we started out, for example, with a huge experiment. It was very big, took a long time to get that feedback on and heavy weight. And I built this despite having practiced small feedback loops and everything before, and still, I caught myself coming up with that huge experiment. No matter how often I preach otherwise, I still caught myself doing that. In the end, it was too big, definitely. I think the most important part is to reflect on what we’re actually doing, also create transparency with ourselves, basically, to realize, okay, maybe I’m doing something differently than I speak. There’s a mismatch. What makes me do something else? Is there something that hinders me? Is there something that’s missing? Anything. So I can then, again, also change my own behavior and see what would be helpful to make it smaller.

The experimentation part for improvements, again, for this initiative, it was huge. It was cross-team. Many people actually have this kind of experimentation opportunity within the daily work in the teams. You also don’t have to be in a leadership position to experiment. It shouldn’t hold you back, that you’re not having the expected role or position or whatever. If you’re working, for example, in a team, you can use your cycle, whatever that is. Maybe you’re working in sprints. So maybe every two weeks you have a reflection cycle anyway, or you could use that timeframe to say, okay, we’re going to try this for two weeks. See what happens. It’s like this continuous improvement idea of agile.

You could also try it with yourself. So, one thing that I tried also, because of knowing that in the past, actually I came up with way too large experiments. What I tried now in my new job, especially my first month, tried something daily, just a very small thing, just a different thing every day, and see, does it help me? If yes, then I’ll just incorporate it in my daily habits. If not, try something else. So to make it really small and tangible, and also to rediscover actually better ways of working. Because yes, I’m working now for quite some years, I got used to certain working styles, certain things work for me. I know that. But maybe there are better things out there, so I should never stop learning and looking for better approaches. So these daily experiments really helped a lot, especially starting at a new company, which is like a natural point where I have a new setup. Maybe I work with new tools suddenly. So I have a change anyway in my life, and I could incorporate that.

GeePawHill has like a phrase that I really love. It’s “many more much smaller steps”. So ever since I read that, that’s sort of sticking in my head. I’m trying to use that as a guideline to come up with even smaller things. I’m not yet always succeeding. I think the only thing I’m aiming for, you might take that as advice or not, is to always think, can we make that faster? Can we break it down? Just like what we try to do usually with product development. Get faster feedback. Get learning quicker. But I haven’t found the golden rule yet.

Henry Suryawirawan: But I think that kind of golden rule that you mentioned from GeePawHill, I think many much more smaller step. I think that’s kind of like crucial, maybe in agile, maybe in experiments as well. I think it can be a golden rule, so we should not aim to start a big experiment where you have multiple teams and multiple activities. In the end, one big bang. So I think we should try to have more smaller steps, experiment and reflect. I think you mentioned about reflection.

[00:33:38] Getting Buy-In

Henry Suryawirawan: The other thing about doing all this culture change is about getting buy-in. In fact, in your story, you mentioned that some people understand about the value of doing the things that you are driving, but they fall back into BAU stuffs. I mean, life happens, right? You have so many other things. You may have incidents. You may have roadmap, features, and things like that. How can you drive people to be onboarded to your experiments, so to speak?

Lisi Hocke: So for this big initiative that really went across teams, we had that sponsorship from C-level from the start, which definitely helped because we could use it in our communication with everyone. But we also made intentional effort to get buy-in from VP level, for example. So, for example, from our VP for product who is basically also driving, for example, product roadmaps. To get that buy-in from the start so that there is not a complete clash of priorities that would instantly pull people apart, basically. Also, with the team’s managers, so to include that leadership level and the hierarchy. But in the end, we wanted to work with the teams themselves. So we made really intentional effort also here to not just select people or select teams top down, but to really investigate, which could be great teams? And then to get their explicit buy-in.

We had a short, very small interview round with a short list of teams that we had in mind. Like eight teams in that case, to interview them just for half an hour about a few aspects that interested us. Part of that was also: are they actually eager to join us in this initiative? Are they eager to try this out? Do they currently have the capacity? Because we also make clear you will need to invest time. Are you currently in a place where you could do that? And then, when we found, okay, we have now four teams, we got, again, the explicit commitment, and only then we started. I think even though in the end they still fell back into everyday business and still struggled, it was a better starting point to make them aware. Hey, you also want this, right? We’re here to help. Let’s do this together. We also had an explicit communication guideline. How to approach the teams because we didn’t want to impose ourselves. Like, hey, we’re like the experts. We tell you how things work. No. They know their context. They know the reasons why things are currently as they are. I bet there are a lot of things that are working, so let’s appreciate what’s working already for them in their context. That’s awesome, right?

So turning up the good, but also seeing if we can find custom tailored solutions. So something that’s really helping them. And I think that was big part of getting buy-in also from the single individuals on the team that were not coming from outside, basically. I mean, not outside from the company, but outside from their team, which is sometimes if teams are so independent, quite similar, but to not go there and, like, just overrule everything. But really make clear we are here to help. We’re here to help in your way, however you want to frame it. We have custom-tailored workshops or whatever you need. But in the end, it’s also up to them to take action and to really come up with those experiments. We’re there to help, but we’re not imposing. I feel that was a big part to first get management on board, but also to really how you approach the teams or how you approach people, I think, makes a whole world of difference if they would like to really join them and work together. Or if you feel, oh, there’s someone threatening us.

Henry Suryawirawan: You brought up this point as like an ownership thing, right? So the experiment should be owned, not by leadership management or the change agent group, but it’s actually from the team themselves. They really need to be on board first. You mentioned about they should get the buy-in, they should understand, and they should want this. When I read your blog, it’s about focus on inspiring them, getting them want to change, because they can see their pain points. Again, coming back to the transparency. Where they’re at? Their pain points, and they can see, really, this change affecting them. I also learned from my reading, actually, if you want to create long-lasting change, it should come from within. It should not come from external imposed. Maybe you have consultants. Maybe you have resolution from some other people. It should come from within and only then you can make a long-lasting change that can be impactful for your life. So thanks for sharing about this ownership thing.

[00:37:44] Sunk Cost Fallacy

Henry Suryawirawan: The other thing about experiment, right? You brought up about failure. Many people couldn’t forgo about this failure part, when you have spent your effort, spent your time and resources, and in the end it failed, we all have this tendency of sunk cost fallacy. There must be something that works. So we kind of like have our own bias. So tell us more about this sunk cost fallacy and what we should do instead in order to frame the experiment more objectively?

Lisi Hocke: Yes. That’s another part of the struggles when framing experiments. Also, when just coming up with hypotheses, what you want to do, to keep our biases out is really hard to do. I think that the first step is to be aware of our biases and reflect on them. For example, sunk cost fallacy. Well, we invested already so much, so we definitely want to bring it to the end. Although it might not be worth it, actually. It might not be leading us to better results or more learning.

So I think this one is a big one when it comes to having experiments framed too huge, too largely. So what I did basically in the beginning where it started out, where I had like, this first one, which is like across 10 months. There might have been a point where we would have needed to stop it. I did not make that call. Also biased in that very first one. Another bias also came in because I wanted it to succeed. So if you look up that hypothesis, the formulation is actually biased in itself because, I mean, it was my first initiative after being leveled up as a principal, and my standing, and so on. So I’m also part of this reward system. I really wanted to make this a win and you will see that in how I formulated that experiment. So that’s another bias that came in and that’s probably also why I didn’t then cut it at some point. Facing the truth that, okay, it’s too heavy weight. We should cut it here. Find something else.

Biases are really what drives us unconsciously and implicitly. So if we aren’t aware that these are in play, it’s really hard to catch ourselves. It’s still hard to catch ourselves, even though we know about them. So I think the key point here is to continue learning about biases. How our brains work, basically. I mean, they’re there for a good reason, but sometimes they get in our way. This is the part we need to learn about. I can only recommend, for example, Kahneman’s book “Thinking, Fast and Slow”. It’s one of the major resources. I gained a lot of insights about the cognitive biases. But, again, just knowing about them doesn’t help. We also need to continuously reflect on our own actions and really challenge ourselves. And that’s uncomfortable, right? To challenge ourselves.

The big thing that, like my work there, maybe, was just built on biases and maybe it’s not the best thing there could be. What made me really come up with that? Like for example, I wanted to earn my standing. I wanted to earn the appreciation of my tech lead peers. This definitely influenced my decision making. So yes, we need to become aware of the biases. We need to reflect continuously. I’m pretty sure I’m not catching myself always. I can just hope someone else might also give me feedback and call me out. Because, again, that’s part of learning to always improve ourselves as well, and our next experiments can only benefit from that as well.

Henry Suryawirawan: Thanks for reminding these biases. I’m sure that we all have biases due to our childhood, culture, school, and things like that. So I think the key thing is always be conscious about so many different biases. That’s the first thing. Of course, if you don’t have this awareness, it’s really hard to catch. You have always unknown unknowns in your life, right? In your knowledge. So, always continuously learn about those biases. And yeah, sunk cost fallacy is sometimes very dangerous. It could bring you to much more worse failure, so to speak, where you dig your rabbit hole and it goes nowhere. Thanks for being vulnerable and sharing about your personal agenda. I think sometimes we do have that kind of tendency, ego, and want to prove something and we just continue to march on. Knowing that one day we will achieve it somehow. But actually sometimes that’s not true, because that’s very dangerous. So thanks for sharing this beautiful thinking about biases.

[00:41:42] Learning in Public

Henry Suryawirawan: So Lisi, let’s move on to another topic which is more personal to you. So you mentioned in your career journey that you started from not having the tech background. You leveraged a lot from the community. You started buying that book, “Agile Testing” book. You followed community. You met Lisa, Janet, and all that. And you kind of like grew out of that experience. You made a pledge, so to speak, a few years back that you would embark on your journey of learning in public. I think many people would have heard about this term, learning in public. Tell us what made you make that decision? That’s the first thing, and why learning in public? For people to think about it.

Lisi Hocke: So once I learned about Lisa and Janet back then through their book and I realized, oh, book authors are on Twitter. They spread more knowledge and insights and what they currently learn. Back then, I also started to learn about, oh, there are conferences. Maybe I could join at some point a conference. That would be really cool. But it would also be scary a bit, because I don’t know anyone and I’m very introverted and also grew up very shy. I’m not that shy anymore. But it needed practice. But I’m still very introverted. So like big crowds, for example, like I love people, but it drains energy, so I always need to recharge. But still, I had so much inspiration from what I read and saw and I could instantly see like the results when trying things at work. I felt like, okay, I need to go to that conference where, for example, Lisa and Janet are.

So that’s when I started out to join my first conference. It was like in 2015, where I had, for the first time, a connection to the community. It really inspired me so much that I felt like, okay, let’s come back. Okay. Next year, same conference. Because I was really inspired. And in 2016, something happened. There was a really inspiring opening keynote about being brave and having courage and do something that scares you every day, like to really get out of your comfort zone. It was super inspiring and setting a tone and an atmosphere for the conference. And it happens that on that day I met another person in a workshop, Toyer Mamoojee and it sort of clicked. So next day, over lunch, we found ourselves in a nice conversation. We spoke about that opening keynote, like, okay, what is it that scares you? What is that?

We realized it’s very similar for both of us. We both thought about speaking. Speaking in front of people. Oh my gosh, this is so scary. No way. We also talked about writing a blog and these kinds of things, but they always came back to speaking at conferences. Suddenly, Toyer looked at me and said like, “You know what? Let’s make a deal. If you submit a talk for next year’s conference to speak at that conference that we’re currently on, I’ll do the same way too. But if I do it, you have to do it too.” Usually, I’m not the type of person who just agreed to such a challenge. It’s not me. But I was so inspired and like the whole conference and like having courage and taking a leap and everything, and I just decided, “Okay. Let’s be it.” I took his hand, we shook, made a pact. Toyer instantly announced it to everyone. I felt like, okay, I can’t back out now. It’s not possible. I don’t want to disappoint people. So we better do it.

This was the start of something beautiful happening for both of us. Because in the end, our learning partnership that started back then, over that handshake at the conference, grew into a lot of different personal challenges. We did actually manage to come back as conference speakers from then on. We both spoke now regularly at conferences. But we also took up other personal challenges to continue that learning in public. Because, well, actually that starting point was already learning in public in a sense of the whole conference crowd learned about our deal. So, again, I felt like I can’t back out. Everybody knows it already, so I can totally be open about it. So I tweeted about it and made it really public. Hey, we have this deal going on and Toyer did the same. So it sort of started that learning in public period. But, again, after we became conference speakers, we felt like this was so successful. It worked so well. We have that chemistry. We can really support each other. We hold ourselves accountable. What’s next? What else could we try? And then we started diverging challenges, but still supporting each other. And that made me then set out to having a lot of pair testing sessions, for example, with other people.

Other challenges followed, always continuing learning in public, publishing also more blog posts because that was a side effect, actually. At the same conference, I also started, okay. Now’s the time to start my blog. Something that I also thought about for years before, but I never dared doing it. But at that conference with the speaking pact, basically, I also had a great conversation with someone who said, who told me, “Why are you scared of starting a blog? Just do it for yourself. Just for your own learning. Not for anyone else. Just do it for yourself.” And that really freed me up to not think of an audience or who might that provide value for. All the thoughts that hindered me always from starting to put myself out there, basically, and just do it.

Ever since, only good came out of it. Then I can just recommend give it at least a try and see if it’s for you. Because, again, we’re all different persons. It might or might not work, but if you never tried, you don’t know. For me, it worked wonders.

Henry Suryawirawan: It looks like another experiment, but for your personal thing. First of all, thanks for sharing the story. It is very beautiful. So there are a few key things from me when I heard about your story. Accountability. So you have a partner whom you make a pledge to each other and you never back out. That’s another good thing, right? You never back out. Another thing is about getting inspired. So you were inspired by that first keynote, and you mentioned something about do things that always scare you. I think this is also mentioned by Tim Ferris, if I’m not wrong, that if you want to grow rapidly, we always have to try to do things that always scare us. I myself, personally admit I was also very introverted. I couldn’t bear standing in front of a lot of audience, even in meetups. I’d tend to sit in the corner or something like that. But then I think the challenge is do something that scares us. I think there are so many good things about learning in public. Many people preach about it.

[00:47:37] Growing Technical Confidence

Henry Suryawirawan: But many people get this first challenge. I want to learn in public, but the confidence is not there. Because when you start it, we can assume many people don’t have the skills. They don’t have the experience. They may not even know what to talk about, so they don’t have the confidence about doing this. So you have also a good advice here for people who do not have that kind of technical confidence. How should they actually start growing this confidence? Because also taking from your background, you didn’t start from tech. You may not have the background, but you did it anyway and look where you are at now. So maybe you can share a little bit about this growing technical confidence part.

Lisi Hocke: Absolutely. It sort of also accompanied me through my career. As you mentioned, I didn’t start out from that background. So I don’t have that argument basically to convince people, “Hey, you know, I can, actually. I have a place here and it’s justified.” So I had that conversation with myself for a long time. It was over years. I also presented myself as non-technical. Something I stopped doing for good reasons years ago. But for a long time, I was always like, “No. I’m not technical.” Having that sort of as a shield defense mechanism upfront. So to not let people attack me because I’m coming from a different background.

The more I said, “I’m not technical”, also the more people believed that, obviously. And they also saw me as non-technical because I’m stating it myself. I stated it out of that missing background, out of insecurities, uncertainties. I didn’t have any mentor. I was always a lone tester. I never knew actually, am I doing well in what I do or not? Maybe I’m doing something completely wrong, and I just didn’t know about it. That also played into the first conference speaking. Like, I mean, who am I to tell something about that? Until I realized I can just share my own experience because that’s valid. Anyway, nobody can challenge that because I lived through that. And maybe it helps people. I learned over the years, it helps people, actually. It can help a lot of people. I’m still just sharing my own experiences. Part of these experiences was my journey of growing technical in the sense of growing the confidence to know that I am actually technical and to be able to present myself as such.

The moment I started presenting myself as technical which, again, I had inspiration from Andrea Goulet, actually. I resonated with her journey. The moment I started presenting myself as technical, things changed. I grew my own confidence with that. Again, it was a scary thing to do because it didn’t yet fit my own internal beliefs. But I realized people started changing how they see me. I realized about myself. I suddenly dived into things that scared me first more and learned more about it and actually grew also my technical competencies. When my team changed or now starting at a new company, for quite some time, I practiced presenting myself also as technical, nobody questions me at all. It was all about perception and the confidence. I’m still the same me. The difference is how I talk about myself and how others talk about me.

I also realize I’m on my own journey there. I can only compare myself to myself. And as much as I would love to compare myself to so many others who are better there or who are like more advanced there, something that I grew up with, I always turn in circles because I can never be as good as someone else. What I learned is when I try to put that aside, sometimes these thoughts still come up, right? It’s still a learning journey. But when I try to put these kinds of thoughts of comparing oneself with others aside, and instead just focus on our own progression over the years, it’s a whole different picture. And then I can actually validly say, “Yes, I grew a lot in that area, also with my technical competencies.” That made me also start out to give a workshop about that to help others, because I see still a lot of people very unsure in whatever technical means, right? Even that it’s like, there’s so many things. Often, it’s just used as a gatekeeping term.

So, again, it’s about perceptions, about the confidence part. To grow that confidence that we can figure things out. I always get reminded about the quote I learned from Angie Jones, another person I completely admire and I learned a lot from. She said, “Actually, you’re never going to know everything there is to know in tech. Instead, aim for the confidence to know that you can figure it out.” Ever since she tweeted that, I’m always getting reminded of that. And I find so much wisdom in there to keep the confidence that we can still learn, and we can still figure things out. We can still grow. We’re not at the end. There will always be something we don’t know. Especially the more senior we grow, the more we know what else is there, and that’s okay.

Henry Suryawirawan: Thanks for sharing your perspective. I think this is very key for many people who aspire to start building content, learn in public, wants to be perceived as technical or expert in some fields. And yeah, I mean, being technical is just a matter of perception, right? So I think many people in tech have this imposter syndrome, where we feel somebody else is better than me. How could I be someone like that who presents in big conferences and things like that? I’m nobody. But I think you brought up a point where you always compare yourself to yourself, right? It’s not compared with other people. The key to that, you mentioned, is about changing your identity. I think this is also something I learned from “Atomic Habit”, James Clear book, for people who haven’t read that book. One of the suggestions that you have, if you want to change your habit to a better way, right? You don’t start with what activity or why you should do that, but you actually change your identity. So think of it like you want to be technical, right? So first of all, yeah, have that identity that you are technical and then you figure out how to be technical, and then you grow actually along with that kind of identity. Because you don’t want to confront your identity, so to speak. You want to follow based on the beliefs.

[00:53:12] 3 Tech Lead Wisdom

Henry Suryawirawan: So thanks for sharing that. Lisi, I think I learned a lot from your sharing, your journey. It’s been a pleasure. Unfortunately, we have to wrap up pretty soon. But I have one last question that I normally ask for all my guests which is about three technical leadership wisdom. Think of it, just like advice for you to impart to the listeners from maybe from your journey, your experience or lessons learned because you seem to have run so many experiments and you have met failure along the way and successes also at the same time. So what would be your three technical leadership wisdom, Lisi?

Lisi Hocke: So I’m going to draw them from another experiment that I actually did in 2021, where I could co-host and co-create a series of leadership workshops together with Shiva Krishnan. That’s also where I want to start, like where kudos belong credit, where credit is due. Shiva Krishnan is the one that I learned most about leadership from. So the three wisdom that I’m about to share is something that we also co-created and built on each other’s ideas. It’s still following me around also in everyday life. So that’s why, for me, they’re really valuable. I hope they’re also valuable for others.

The first one is actually starting with ourselves. Who you are is how you lead. That’s something to remind ourselves. Leadership starts definitely with ourselves. We need to reflect on our behaviors. We need to understand how we do things. What drives us? Which kind of values do we have? Knowing ourselves really enables us then to also increase the impact. But first, we need to be really aware of ourselves.

The second one is about the other people, the “us” aspect, basically. So leadership is about relationships. In most cases, we do lead other people. It’s really about building those effective relationships, to foster them, deepen them. That’s the key for a lot of things. If you want to initiate change, if you want to drive experiments, you first have to have those relationships with people, and it could be any initiative. But the same is true for a lot of technical topics as well. If you’re missing out on those relationships, good luck with fastening feedback loops, for example. We do have the power to influence there. No matter if we have a formal leadership position or not. So it’s about relationships.

The third one, the last one, is about our environment. Impact over intention. That is something that really drives me, especially in the last years where I had to learn a lot about systems, basically. So I hope people start out with good intentions. Most people do. But intentions alone, they’re not enough. Because even with the best intentions, you can still do harm. And especially when using your leadership skills, it’s really about not like reducing the harm, hopefully, increasing the positive impact. So let’s shape our systems and set the scene, basically grow the garden to nurture that culture where people can really thrive. To always look at the impact that our decisions and actions can have, at best, before we take them. But even if it happened, to acknowledge potentially harmful impact that we’ve done, acknowledge, apologize, do better next time. So that’s my third one. Impact over intention is sort of a guiding principle for me that I always try to live up to, and it’s not easy. But let’s do better than the last time.

Henry Suryawirawan: Wow. It’s pretty good summary. Very beautifully said. So thanks for sharing that. I like the last one, impact over intention. We all have good intent, like assume good intent, anyway. But sometimes we kind of like derail from the intention and maybe biases and things like that made us make the wrong decision. So I think, yeah, always try to focus on the positive impact that you want to create in maybe your life or other people’s life. Thanks for sharing that message.

So Lisi, for people who are inspired by you from all this sharing, if they want to talk to you or maybe continue the conversation, is there a place where they can reach out to you?

Lisi Hocke: Yes. The easiest way is via Twitter. In case you happen to have a Twitter account, my direct messages are open. So feel free to contact me through there. Another option is LinkedIn. However, for LinkedIn, I’m not as active. So, in case you are reaching out there, please leave a message as well. So I can really set that in context. Otherwise, I’m not really active as I shared. You can obviously also follow my content on my blog and website. Maybe we meet each other at one or the other conference or meetup. So that’s another chance to connect in person, or just join me on my pairing session, for example.

Henry Suryawirawan: So I’m sure if you still continue your pledge being in conferences, people would see you in any kind of testing or quality kind of conferences. And I do recommend Lisi’s blog as well. There are so many gist about testing per se. Like I said in the beginning, Lisi has summarized the latest Agile Testing Day in such a beautiful illustration written by her. Okay. So it’s created by her in real time. That is really an admirable skill to have. So thanks again, Lisi, for your time. I really enjoyed this conversation. Again, I hope people are inspired by your message today.

Lisi Hocke: Thank you so much for having me. It was a pleasure.

– End –