#268 - Don't Outsource Your Thinking: How to Lead in the AI-Native Era - Emilie Schario

[00:02:02] Introduction

Henry Suryawirawan: Hello everyone. Welcome back to another new episode of the Tech Lead Journal podcast. Today I have with me Emilie Schario. She’s one of the co-founders of Kilo Code. Maybe some of you have heard about Kilo Code. And recently, it was probably a few days ago, that Kilo Code was also just acquired by Anaconda. So congrats for that. And Emily, looking forward for this conversation.

Emilie Schario: Yes, thank you so much for having me. I’m really happy to be here.

[00:02:27] What Is Kilo Code All About?

Henry Suryawirawan: Right. Emily, in the very beginning, you know, like maybe introduce a little bit more about Kilo Code because maybe some people have heard it, but I’m sure many people have heard about Claude Code, you know, Codex and all that. But maybe some may have not tried Kilo Code. So tell us a little bit more about Kilo Code.

Emilie Schario: Yeah. Kilo is an agentic engineering platform where we focus on being everywhere developers are. So really leaning into the multi-platform, whether that’s your VS Code extension, JetBrains extension, a CLI/TUI, a cloud agent on the mobile app on your phone, wherever you want to be, Slack, Discord. But we also… what really differentiates us from those other tools that you mentioned is that we’re model agnostic, which means that we support over 500 plus models from so many different labs in the Kilo gateway. Or you can use Kilo with your own API key or with local models. So you’re not tying the software you’re using to the lab that’s creating the model.

[00:03:31] How Do You Avoid Getting Overwhelmed by 500 Plus AI Models?

Henry Suryawirawan: Right. Sounds very unique, right, in a sense that you support multi-model, that’s the first thing. Because some people, I assume if they wanna use multi-model, they have to, you know, use multiple tools. But this one is actually, you support 500 plus models. So how does it, you know, how can someone, you know, not being overwhelmed by the models that they are using?

Emilie Schario: Yeah. Most people don’t use 500 plus models, but I think what we really believe in is model freedom. And so we work really hard to bring those models to users, whether that’s in, via an AI gateway where we have a number of partners or building direct relationships with the model providers in the labs or inference partners. And so, we’re not only working to make sure we have all the models available on Kilo, but making sure that they work well on Kilo. One of the reasons those relationship with labs is so important is because that those relationships are what allows us to really optimize the model to work well in the Kilo framework.

[00:04:34] How Does a Multi-Platform Approach Benefit Developers?

Henry Suryawirawan: So one of the things, one of the things that you mentioned is also about multi-platform. I think this is quite unique, right? Because these days people either have to use an extension, something that is in their IDE or they have to use a CLI-based, terminal-based kind of like user interface. So how do you think this kind of a multi-platform approach actually help developers much better?

Emilie Schario: Yeah. You know, when we talk about developers and developer workflow, so often we’re talking about it like it’s one swath of samesies, right? And the reality is that developers, especially when it comes to AI engineering, especially when it comes to agentic coding, are all over the place. The stat from Stack Overflow is that like 50% of developers still don’t use AI at work every day. I met with a developer who’s a, he’s a platform architect at like a large investment group in the US, and he told me that their AI budget per engineer is $13 a month, which means that on the first day of the month, you run one prompt and your budget is gone, and they’re still writing 99.9% of their code by hand. Like this is 2023.

And so I say that because some developers are only ever going to want to use… The “AI” that they want is tab autocomplete in their editor of choice. For other developers, what they’re looking for is how do they run 10 agents across eight repos all at the same time, not on their machine, but in the cloud, but within a sandbox environment. Like there’s so many different, there’s a spectrum, and that’s really important.

So with Kilo, you know, we are in VS Code, we are in JetBrains with a native extension. We are in the CLI, if you prefer that interface. We’ve also got a cloud agent, which is when you’re running Kilo in a sandbox environment on the web. And then with our cloud agent, we’re able to meet users in a lot of different places. So through the Kilo mobile app, you can either communicate or kick off with a cloud agent, or you can connect to a remote agent, so an agent that’s running on your machine, but you’re talking to it from your phone. Similarly we’ve got our code reviewer and our security reviewer which are specified kinds of agents that run on the cloud infrastructure. And then finally, we’ve got our Kilo bot or Kilo Connect, which is in Discord, Slack, Teams, kind of all the places you are already working. Now you can bring Kilo along as well.

[00:07:19] How Does Kilo’s Agent Harness Differ From Other Tools?

Henry Suryawirawan: Sounds really comprehensive. I think it’s so unique, right, in the sense that you meet people where they are, and they can use variety of options to use it. So I think many people might have tried, you know, using all these CLI-based terminal. In fact, it gets– getting more popular. And I think one of the secret sauce of all these tools is actually they’re talking about, you know, the agent harness, right? You know, how the things actually work and do this looping, making sure that they actually meet what the developers want in terms of, you know, answering their prompts. So how does Kilo actually differ in terms of agent harness as well? Maybe if there’s any unique flavor that you guys provide compared to the others?

Emilie Schario: Yeah, so one of the things that I think makes Kilo really special for individuals is that all of our code is either open source or source available. So it… Our Kilo Code kind of surfaces, the CLI, the core, the VS Code and JetBrains extensions, are all open source. And then the proprietary code is still source available, which means anyone can see it, can read it, can contribute to it, but you just can’t use it. So I start there because this means that everything I’m gonna say next, if you don’t believe me, like go hook up an agent to the code base and double-check, which is always exciting.

You know, what we do, I think other labs, especially if they’re creating their own software, they’re spending a lot of time making their models really strong in that software or making that software really a strong platform for their models. With Kilo, we’re focused on making the best model agnostic piece, but that doesn’t mean we’re completely indifferent to the models’ performances themselves and the nuances related to that. So we work with many of the labs to adjust prompts, system prompts specific to their models. We’ve got a partnership where we can make things, we can… we’ve got a partnership with most of the labs where we get early access, so we can make sure that tool calling is working correctly. We can make any adjustments before models are available to users. So we’re really focused on providing not just a model agnostic experience, but a model agnostic experience that’s really strong in each of the platforms.

[00:09:51] Why Will Model Choice Matter Less Over Time?

Henry Suryawirawan: Nice. So one thing that I would like to ask you as well, right, because you mentioned multi-model and model agnostic, right? So probably some people are not used to this kind of, you know, doing switching of the models in practice day-to-day. How do you envision engineers actually use this multi-model approach in their day-to-day life here?

Emilie Schario: Yeah, great question. I think of it a little bit like using spoons to eat soup. Like most people do not care that much about the spoon that they’re using. Like, as long as it doesn’t have holes, right? They just wanna scoop and… And so any spoon, any size, shape, most people are like fine with it. And I think when it comes to models, we’re gonna see something similar over time. Today, people have opinions because they’re such different, drastically different quality in the different models. But in, I don’t know, the next couple of years, I think we’re gonna see models become a little bit more commodified, where you’re just gonna want a model that can get the task done, kind of like you want that spoon for soup. If it can pick it out of the bowl and put it in your mouth, that’s plenty sufficient. And the task that… or the prompt that you give, if the model can execute on that, you’re in great shape.

[00:11:20] What Tools Help You Pick the Right Model for Each Task?

Henry Suryawirawan: Right. Yeah, so I think currently, you know, model differs as well in terms of, you know, output, in terms of, you know, again the harness might be different, right? So definitely people might have preference. I personally also, I think I don’t like switching too much for doing one task, right? So it has like some cognitive load for me, like to know which kind of models. But I think when we speak about agentic, you know, workflow, for example, if you wanna automate, you know, code generation to the agent somehow and only review it when they get back. Any kind of difference here, because I think some people might say that, okay, you use one model for planning, you use one model for architecture, one model for writing the code, and so on and so forth. Any kind of things that you pick up from how people use Kilo?

Emilie Schario: Yeah, so we have, I think there’s two things I’ll highlight. The first is that we have a kilo.ai/leaderboard. That’s our leaderboard where you can see how people use different models in Kilo. Not just what models are popular and what are the most bang for your buck, which is true across all tasks, but also breaking it down by specific kinds of tasks, like planning versus coding versus debugging, where you’re gonna be in vi- very different workflows. And so, I could tell you, by the time this comes out, that data might totally be outdated because the models are changing so quickly. But I’ll just point you to the leaderboard where you’re gonna get live, up-to-date information on how people are using different models in Kilo.

The other is that we have a suite of what we call auto models in Kilo, where we can handle the routing for you based on your cost profile and how much you’re willing to spend. So we’ve got Auto Balance which is like focused on a best bang for your buck series of model. Auto Free, which is only using free inference, and then Auto Frontier, which is the state-of-the-art models in Kilo, where you can just select Auto Frontier, and we will route it to the right task for you. We also do something similar with a model called Auto Efficient. But it’s to the next level where a prompt comes in, what we do is we categorize that prompt based on a type and a subtype, using a small classifier model, and then we compare that type and subtype identification to our own internal data and benchmarking so that we can route it to the right model for the task. So that’s a little bit more advanced, and that’s truly focused on giving you the best bang for your buck. Focused on accuracy, but the most cost-efficient version of that accuracy.

Henry Suryawirawan: Wow, again, something very nice, so that you can actually meet people based on their use case, right? So, yeah.

Emilie Schario: They can go to the store and say, “Hey, I just want a spoon,” and we can help them get the right one.

[00:14:16] What Does the Anaconda Acquisition Mean for Kilo Code?

Henry Suryawirawan: Nice. So yeah, one thing that we know recently is that you guys got acquired by Anaconda. Congrats, first of all!

Emilie Schario: Yes, we just announced this last week.

Henry Suryawirawan: Yeah. So how does it change now that Kilo Code is part of Anaconda?

Emilie Schario: I think, the… This is one of those acquisitions where I’m so excited because one plus one is going to equal three. We get to bring the powerful coding agent that Kilo users know and love to the fast-growing, over 50 million Anaconda ecosystem, where I forget the exact number, but it’s, like, over the 90% of the Fortune 500 uses Anaconda. So being able to bring the magic of Kilo to them in a time where companies are already starting to think about token efficiency is really powerful for us. We also get the opportunity to see continued investment from Anaconda in Kilo and the coding agent. So I really am excited about how we’ll be able to bring all the power of Kilo’s coding agent with the secure managed package envi- packages and environments that Anaconda users know and love. And I think this is gonna be a really exciting opportunity for both teams to do more together.

[00:15:43] Why Should You Measure AI Spend Per Merged Pull Request?

Henry Suryawirawan: Yeah. You mentioned about, you know, token efficiency and people also talk about cost these days. I think the trend it seems like the cost keeps increasing and it might get increased even more. So what do you feel about this? Is this something that like developers should be concerned, starting to get concerned with?

Emilie Schario: Yeah, I think spend is a number that if you look at it in isolation is only gonna go in one direction. Kinda like compute, right? We never– I don’t think there’s any company that’s really seen their compute bills go down since, you know, the era of AWS. So I think looking at spend in isolation is a problem, because spend doesn’t mean anything. It’s not just about the spend that you care about, it’s the spend in the context of what it did for your org. If you could spend $100 to make $500, if you can get the ROI, great! No CFO’s gonna complain about it. But if you’re spending $100 to make 98, then we have a problem. So a metric that I don’t think we’ve circled around as an industry quite yet, but I do think we’re going to see people push more and more, is gonna be spend per merged pull request.

And let me tell you why. Broadly speaking, merged pull requests is the closest thing that engineering teams can come to, to measuring customer value in the short term, right? It’s– There’s a lot of better metrics, and really it’s like dollars and cents, but for companies that have long sales cycles, that’s a way off.

So merged pull request is probably the best metric for customer value shipped by an engineering or a product and engineering org. And so if you can take that AI spend from that org and divide it by the merged PR count, what you get is an average cost per merged PR. And I think this is the metric that matters. If it’s costing you $100 to create new customer value, that’s a really different conversation from it costing you $20 to create more customer value. On the other side of that, as we think about customer value, like that is also part of what we wanna see go up. So merged PRs should be increasing over time. And as AI spend continues to increase, if we’re also seeing the volume of customer value being created, I don’t think that’s a bad thing.

Henry Suryawirawan: Right. But one challenge definitely is about the size of the merge request or the pull request, right? Because some people can give you tons of, you know, code changes in one go. Some might do little code change. Yeah, how do you tackle all these such-

Emilie Schario: Yeah, it’s not, it’s certainly not a perfect metric. I think an organization needs to standardize for themselves on what your pull request size looks like. For companies that are already using DORA metrics though, this is probably the next iteration that helps capture customer value, tying it to AI spend.

[00:18:54] How Do You Balance Speed and Control in AI Development?

Henry Suryawirawan: Right. Speaking about delivering customer value, right? So I think people want to use AI to deliver a lot more customer value. But at the same time, I think people also think about consideration like security, you know, governance, making sure that the AI doesn’t come up with a bad feature. So how do you see this, you know, speed versus control kind of thing, right? Do you actually care about it when you build Kilo Code such that people don’t go off the rails, yeah?

Emilie Schario: Yeah, definitely. I I think the number one rule, and if people only take this from the conversation, let it be this, is don’t outsource your thinking. And so the human, the person is still the one prompting. They’re still driving the results. They’re still responsible for the architecture. They need to be thinking about the security implications, right? You should not… You cannot vibe code auth, for for example. You cannot vibe code billing. You cannot vibe code security. And so we need to be intentional. And I think having really strong engineers who understand where human judgment still needs to be crucial is key. A little bit of that is also a learning curve, and understanding that some mistakes might happen along the way, and giving people a little bit of room to experiment so that mistakes happen in safer areas than others.

Henry Suryawirawan: Right. Yeah, don’t outsource your thinking. Definitely it’s a very good reminder for everyone. Vibe coding maybe once in a while, yes, but not for everything, right?

Emilie Schario: Time and place.

[00:20:28] Why Should Companies Treat AI as Operational Infrastructure?

Henry Suryawirawan: Yeah. Yeah. Sounds good as well. So one other thing that you say you wanna talk about in this conversation is that you want to talk about how AI native organizations or, you know, organizations that have used AI in, you know, multi-facets of their organizations, how they run companies differently. And you even mentioned that we should not look at AI just for, I don’t know, like code, you know, software engineering, but also at– as the operation infrastructure. So tell us more about this aspect.

Emilie Schario: Yeah. I wanna tell you specifically about Arkadiy on our marketing team at Kilo. There are all these people posting on the internet about how they are using AI to source leads and things like that, and I don’t know how much of that is true. But what I know is true is that Arkadiy is using the Kilo coding agent to run our ads, like our ads that we’re running on Google and other platforms. And it’s because the work that he’s doing is very similar to the work that an engineer would do. You’re prompting an agent, you’re getting feedback, you’re giving it corrections. He’s still doing the thinking, but the agent is now a partner for him. And I think the future of work is one where every knowledge worker is working with a team of agents, not just engineers, not just marketers like Arkadi, but everyone will have three or four tasks that they’ve delegated. And over the course of the day, their interactions are going to be primarily with their agents, as they move into what are the tasks that only a human can do.

A couple of months ago, I wrote a blog post that the human role has changed from generation to reviewer. And we talk about this in engineering a lot, but I think this is true across the board. You see people draft, for example, competitive analysis, and then a human is doing the review of that analysis. You see a human or an AI might draft the initial version of a cold email sequence, and then the human is doing the review and the adjustments and the polishing and all the things that make it really unique and special. And I don’t think we’re in a place where we’re gonna outsource that. But when you’re working in an AI native company, you have these people who all have the same mindset of, how do I find more leverage in myself without outsourcing my thinking to an AI agent?

And I think that piece of it is also really important ‘cause in a lot of companies where people are just being pushed to adopt AI, you also are hearing these horror stories of, you’re hearing these horror stories around how people are getting AI slop generated back and forth, how docs are longer than ever, PRDs are longer than ever, Confluence or wikis are full of slop, and people don’t know what anyone’s written, let alone read. So I think we’re gonna find that sweet spot over the next couple years for sure.

[00:23:31] How Do AI-Native Organizations Run Their Day-to-Day Workflows?

Henry Suryawirawan: Right. And speaking about AI-native organizations, right, how much portion of things that are run by agents versus human, right? Because I would imagine many people talk about this as well, right? We have agents for everything. They run maybe in the background. You just submit tasks to them, and they’ll come back, you review it. And I think it seems like the speed might be, you know, exponentially different compared to traditional organizations. Like how do you actually see AI-native organization run, you know, day-to-day workflows and things like that, yeah?

Emilie Schario: Yeah, I’ll give you a really specific example that I hear across the board is like when people are going to log off for the day, whereas I think in 2022, people are saying, “Hey, I’m wrapping up for the day.” They’re closing out their work, they’re putting a bow on it. They’re, you know, making sure they’ve answered that last message so they can wrap up. Today what I hear from my team members is, “Actually, I’m using this last half hour to kick off three or four different agents that can run on long-running jobs overnight so that when I come back to my desk in my morning, I’m not kicking off new work. I am reviewing some work that’s already been started.” And oftentimes what that’ll look like is you’ve got the work you’ve kicked off from last night. You will start off a new batch of agents and then you’ll pick up that work as well, or you’ll pick up reviewing that work from last night.

Henry Suryawirawan: Sounds really like futuristic maybe for some people.

Emilie Schario: Very, very. And I, you know, that is something we can talk about just briefly. Something we talk about on the team all the time is that there is a whole spectrum of developers. I know we talked about this earlier, but, you know, even within the team where everyone is pushing the boundaries, we hear people talk about how this week they’re running five agents in parallel, and then next week they’re working on something that’s a different level of intensity, or they’ve come down with a cold and so their focus is off, and they’re running with, like, a, a different volume. And so I think not only are we figuring out what this future of work looks like, we need to recognize that it takes a different kind of energy from people, and it’s okay if we’re seeing some variation right now, which is why I focus on, when it comes to metrics at least, averages. Knowing that there’s gonna be some variation. Everyone has a good week, everyone has a bad week.

Henry Suryawirawan: Apart from running things, you know, with agents, any other, you know, AI features that is being used? Some that I may, might have heard is like, you know, like skills, you know, creating skills library within the organizations. Also how about cross-functional things that happen between other departments, right? So yeah.

Emilie Schario: I almost take that sort of stuff for granted ‘cause we’ve just operated that way at Kilo for so long. So we have Kilo, the product itself, has an MCP marketplace as well as a skills marketplace, and some additional tooling there that people are welcome to use, and that’s gonna make things available in Kilo, so to anyone in Kilo. Within the team, you know, we’ve got multiple skills files that people share. We’ve got a specific repository just for that kind of leverage. And not just on the engineering side, but throughout the org.

So I’ll give you a really specific example. I spend a lot of time posting on LinkedIn, but I was having a hard time taking the content, the written content I was writing, and turning it into PDF carousels. I wanted it to be better, more beautiful, but also wanted it to be easy. And there’s lots of tools out there, but I just didn’t have the time to sit there and design six graphics. So I built a skill that can, excuse me, that can take any blog post and turn it into a PDF carousel in the Kilo branding in the right font. It writes like the HTML in a viewer, and then it prints that to a PDF, and it works every time. And because of how I’ve set it up, I can even, like, tweak the copy, before I get those final PDFs. So it’s a really powerful tool. I’m not a marketer. Marketing is not my day job, but of course, that’s the kind of thing that I’m sharing, not just with the marketing team, but with anyone at Kilo who wants to leverage it.

[00:27:58] How Will High-Performing AI Organizations Differentiate Themselves in the Future?

Henry Suryawirawan: Yeah, so definitely I think it’s a great thing that you mentioned about the example you publishing your post, right? I mean, it will get exciting if you work in these AI-native organizations. Let’s say if you have plenty of skills published by, you know, other people who have the experience and the expertise, and you can leverage on that, right? So that everyone can use the same kind of like standard, the same kind of like expertise to generate the same kind of output as well. So I think this definitely change the profile of high-performing organizations. Like one thing for sure, like if organizations can already leverage this, it’s like more high-performing. But how do you see high-performing organizations later on will differ versus each other? Because I’m sure everyone will start to leverage AI a lot more. And like what’s the secret sauce here now?

Emilie Schario: I think a failure mode that I’m definitely seeing is companies taking their existing process and saying, “How do we tack AI onto this?” And that’s not gonna be really effective for the org, for the people involved. It’s, I mean, you see this in lots of SaaS apps where they’re like, “So let’s add a chat interface.” Like that’s the way everyone’s just tacking AI on. I don’t think that’s the right model. I don’t know what it means to have an AI-native org if it’s like a 20-year-old company though, and I don’t think we’re seeing that yet.

What… It’s definitely a mindset. It’s definitely up here more than it’s in the specifics of the tooling. And so it’s not about a subscription to Kilo or some other coding harness. But it’s about looking for ways to find leverage in your work. I think the people we’re seeing who are these early adopters of AI tools, who are making the biggest differences at their organizations, they are also the people who were pushing the limits of finding efficiencies before AI. And so the mindset was always there. Now they’ve got stronger, better tools that can make them even more powerful.

[00:30:11] What Is the “Killing Problem” Now That Shipping Software Is Free?

Henry Suryawirawan: Yeah. I mean, speaking about efficiency, one thing that also you wanna discuss is about, you know, because these days everyone can churn out, you know, code pretty fast, which code here assumes like more features to be built, right? So you are saying that people have now the killing problem, right? So tell us a little bit more about this problem. What do you mean by killing problem?

Emilie Schario: Shipping software is free now. it used to be that software development was a funnel where you had product managers who were filtering out ideas and then there were specs writ- there were specs written, and then software that was actually getting shipped was the very bottom of the funnel. So you never had people work on things that you didn’t think were going to drive notable impact. And now that funnel looks a little bit more like this, which means it is easier than ever to just ship at the speed of idea. It’s a little bit of a hyperbole, but not really. You can ship more than ever before, and with a strong engineer and a really great prompt, it’s absolutely mind-blowing what can be done.

So the question becomes, if you’re shipping all this stuff, do you want all of this stuff in your product? And I think what we need to internalize is that now we are prototyping and building and pushing things out. And then we’re testing it in production. Does it get users? Does it see adoption? Can people leverage it? Are they interested in using it? Rather than us having to do all this expensive discovery beforehand, I think discovery has moved to after the feature has gotten shipped. What that means is that we have to be willing to say, “Hey, this did not stick. Let’s pull it out.” Because you don’t wanna end up with a Frankenstein product that has no coherency to it. So it’s gonna take a lot of intentionality to work, to identify exactly what it is that still belongs in the product. It’s just where that filtering mechanism is after the feature has landed, which means we’re gonna push things out, we’re gonna let users use it, we’re gonna collect feedback, and we’re gonna use that feedback as a dis- as a guide to decide whether we invest in it more or we revert the change in the first place.

[00:32:40] How Do You Shift to a Metrics-First Mindset?

Henry Suryawirawan: Wow, interesting change of, kind of like way of delivering software, right, where you probably don’t do discovery first because the everything now kind of like cheap, right? Or the way you want the features want to be built, you can just trigger an AI agent and they will come up with something. And you’re saying like testing in production. But I think also one important thing in all these lifecycle is that you need to gather the metrics, right? For some probably is not in their DNA yet, right? When you deliver features, you want to measure how it is being used, how many people actually love it, and things like that. So how do you start thinking about changing this kind of mindset? Because I think for some this might be new.

Emilie Schario: Yeah. It is a key part of it, right? When I think about how we’re gonna capture that feedback. It is qualitative feedback. It’s users saying, “Hey, I do or I don’t like this thing.” And for us, that’s coming mostly through Discord where our community is, but not exclusively. But it also means that we’re getting that quantitative feedback. So, you know, a feature’s not shipped unless it has metrics. How many people are using it? How many people are interacting with it? What does adoption look like? Not just like week one, but also are people gonna use it day one and day two and day three and day seven and day nine, or are they gonna give it a spin once and then never again? Because the way you build, has to… you have to understand how your users are using product in order to decide what’s gonna stay and what’s gonna go. And it used to be that you did that through these, this user research and these big conversations and lots of discovery, and I just, I think we’re still doing that. We’re just doing it at a different place in the cycle where we have different kinds of input.

[00:34:24] How Is the Product Manager’s Role Changing in AI-Native Teams?

Henry Suryawirawan: Right. So I like that you mentioned, right, the metrics has to be there before the features get, you know, maybe shipped, right? So how does the role of the product manager also change because I think they play probably a much bigger role in, you know, deciding what features to be built. I mean, kind of like traditionally. Like in the future, do you see product manager also still kind of like hold this role or, you know, or engineers actually also has to evolve and taking some of this ownership as well?

Emilie Schario: Yeah, all of the above. Product managers, which is I think a good place to start, are gonna zoom out a level. And the thing that product managers are going to be really key for, and this is true especially in developer tools when you’re building for a technical audience, is that developers, engineers have really good judgment at building for themselves in a world where you’re building in developer tools. You are your own customer first and foremost, and so that can… A lot of the product management that has always been done by traditional PMs can now move on to developers. What has changed is that product managers now need to help connect what is useful to what generates revenue. And that judgment still belongs with product managers. What pricing looks like, what gets shipped where, what features you prioritize because they are going to unlock revenue, not just because they’re focused on developers. And developer, especially in developer tooling, I think different pricing models generate different approaches.

What we’ve done at Kilo is follow a buyer-based open core model, which means that our free users, individual developers don’t pay a subscription fee to Kilo. Using the software is free. You just pay for the consumption that you drive. So we charge for cloud infrastructure and inference cost. So what you use is what you pay for, not for software. The team- people that are paying for software are teams and enterprises who are getting SaaS features that they need, like management, authentication, billing, invoicing, audit logging, all of those features that a business cares about. And I think in that framework, it’s even more important that the product manager is there to serve that like revenue use case, because we’re not selling to developers, and so developers aren’t the great guide on who, what they would be willing to pay for.

[00:37:06] How Do You Balance Rapid Feature Delivery With Product Quality?

Henry Suryawirawan: So yeah. Speaking about, you know, this feat- number of features shipped and all that, right? So I think one common example that people also posted over the internet is about, you know, the speed how Anthropic actually delivers features, right? So I think even some people post like a graphic showing a timeline, you know, how many features they ship within, I don’t know, X months, right? So definitely shipping the features seems really, really fast. But also if you look qualitatively in some other people’s feedback, right, some– the quality might not be there for some, right? Or there’s long requests that people have, you know, asked for, but they also don’t actually build that. How do you see this balance, right? Because we can certainly easily go into this trap, like delivering more features because, you know, it seems useful, it seems cool. But at the same time also some of the quality not so great. You know, some people also want something different. And how do you actually, you know, decide to actually kill some of the features that actually don’t get used, right? Because I think this life cycle gets very shortened really, really fast, and I think some people might get confused how to actually tackle this speed.

Emilie Schario: I can’t speak for other organizations. I know that at Kilo we’re really focused on how users can have speed and quality. You know, in the past, when you think about the quality shortcut that came with speed, it’s not necessarily a quality shortcut, but it’s a we’re not gonna handle those scenarios, right? When you are shipping a feature and you’re thinking about all of the edge cases one through ten, you’re like, “We’re gonna handle scenarios one through eight, but we’re not gonna handle nine and ten.” And I think that is a super reasonable approach if you’re focused on serving just users one through eight. You have to be willing to say that nine and ten scenario is one that we’re not willing to serve. That’s gotta be the thing that we’re honest about. I think where people get frustrated is that they communicate or they hear from companies or products, “Hey, we’re serving one through ten.” And then user nine comes in and they try to do the thing, and it doesn’t work for them. So it’s important to kind of be really clear on what portions are and aren’t about serving your user base. And I think it’s actually more honest to focus on serving a few things really, really well than doing too many things, not that great.

Henry Suryawirawan: Yeah. And I think also from my user experience as well, right, sometimes too many changes in such a short time actually confuse me as a user, right? Because, what do you mean? Like a few days ago, maybe this works this way, but, you know, now it seems to be different. How do you actually now educate users that this is probably something that is more expected?

Emilie Schario: Yeah. Yeah, I mean, we have this problem a lot with our docs too, where we ship so fast it’s hard to keep our docs up to date. Even if you have an AI agent working on this, it’s just like a chronic problem. And so I don’t proclaim to have all the answers. I think what we are trying to do at Kilo is get really crisp on who we do and don’t serve, and this is something we’ve been doing with the Anaconda team as well. It’s like we serve the builder. This is someone who is building things. They’re making things. They’re– whether it’s a computationally intensive program like a data scientist or a developer who’s working in VS Code to ship a SaaS app, like being really clear and crisp on who that user is and what their use cases are can help you make sure that when you’re cutting scope, you’re cutting that extra scope. And I think this is where speed really matters. The alternative is something that isn’t shipped at all. If the alternative is that a ship– a feature isn’t shipped at all, that’s worse for all 10 of those personas.

Henry Suryawirawan: Yeah, so definitely we all want to deliver more things faster, right? But sometimes there needs to be a balance, right? Maybe in terms from your team as a, also from the organization’s perspective. And also don’t forget about the users, right? So I think what you mentioned about be really clear on, you know, the users who you focus on and not to focus on, I think that’s, I think super important in this era as well.

[00:41:48] How Is the Software Engineer’s Role Evolving in the Age of AI?

Henry Suryawirawan: So speaking about, you know, building things, right, writing code and things like that, I think one of the realization that many people have is that engineer’s role is now kind of like evolve, right? So no longer that you see them as, you know, the code producer, they are moving into a higher level, maybe like a reviewer or some call it system architect and all that. Like how do you see this engineer’s role now change because of the AI and agentic development?

Emilie Schario: When I think about what engineers have most adapted to AI well, it’s these really senior engineers who, what role were they playing pre-AI? They were reviewing everyone’s code. They were meeting with team members, architecting big features, and then handing off the work. They were, people were coming to them with questions. They were pointing or kind of guiding the answer, but they weren’t necessarily writing all of the code across the org, but they still kind of had their own projects. Who are– Why are they– Why is that feature set so valuable? Because that’s the same things they’re doing with AI agents. They have internalized how to work with AI agents because they have always been working with team members that way, with more junior team members.

And so I think the opportunity comes from thinking about an engineer not as an IC anymore, but as an IC who’s manning– managing their own team of agents. And when you come to it from that mindset, you can help coach them into that tech lead opportunity, that tech lead role that we’ve seen engineers move into, you know, five, 10 years in their career. Now they have to jump into it faster, sooner, more efficiently. But they are definitely, to the point we were talking about earlier, we’ve gone from producers to reviewers.

[00:43:57] What Is the Best Advice for Junior Engineers Entering an AI-Native World?

Henry Suryawirawan: Right. So yeah, I think this is probably something that, you know, especially for those engineers who just came into the industry, right? Probably it’s a bit of a challenge. What do you, what message do you wanna give to the, these, you know, new people, juniors who, you know, not necessarily having some ideas how software development was done before and now they have to code with, you know, AI speed, AI-native kind of way of building software.

Emilie Schario: That is a great question. I don’t know is the honest answer. I think hold your head up high, keep experimenting, and keep shooting your shot. The average engineer at Kilo has 10 years of experience, so… Or the, the average engineer at Kilo has fifteen years of professional experience, with the least experienced engineer having ten years. So we’re a very senior team, and I think that’s part of the reason this has worked really well. I’d, I– we didn’t have to teach anyone how to work. But when I think about moving into a world where, we’re hiring, say, college grads or people earlier in their career, I know that we’d have to be more intentional about bringing them along and coaching them how to use these tools. But I don’t think it’s that different from bringing an IC engineer who is AI-resistant or doesn’t understand the value and bringing them along as well.

Henry Suryawirawan: Right. Yeah, so I also don’t know how to actually give a good advice for these juniors, right?

Emilie Schario: Yeah, it’s so hard.

Henry Suryawirawan: Yeah. And especially I think some organizations also kinda like, you know, I don’t know, like focus on hiring seniors or they stop hiring more juniors simply because they think, you know, seniors can work with AI much better. So I don’t know about all this, but I guess I think the key message is like, you know, adapt and try to also adopt this AI, you know, engineering practices, right?

[00:45:51] How Can Developers Handle the Cognitive Load of Code Reviews?

Henry Suryawirawan: So speaking about code review, I think one of the challenge is when AI produce a lot of things, especially if you work with multiple agents, you know, spawning multiple agents, and the agents come back to you, it’s like there’s a lot of cognitive load that, you know, engineers now have to bear in terms of responsibility to review the code. Some may actually give up and just accept whatever AI agents suggest to them. So what problems do you see with this kind of, you know, behavior? Like is code review something that we all still need to do for all changes or is there any better way of doing it?

Emilie Schario: Yeah. I was on a panel last week with the Head of Product at Replit, and he shared that they have a category of pull requests that if they don’t touch things like auth or billing or any of these sensitive areas, an agent reviews it, and if the agent recommends it gets merged, it gets auto-merged. There isn’t even like a second person who reviews it, which is wild to me. This is not advice, and I’d encourage teams to understand their compliance obligations before making any changes to their review process.

But broadly speaking, I think, the important… we need to recognize that the volume of code that’s being shipped is more than ever before, and so we cannot change how code is generated without also changing how code is reviewed. So at Kilo, we use Kilo Code Reviewer as the first-pass reviewer. We also have a local code review agent that team members will often leverage. We recommend you use a different model for code generation than you do code review so that you’re getting multiple perspectives.

Our code reviewer has different focus areas you can give it so that it can focus on security or SQL injection risk or whatever you guide it to. And then for really high-risk problems, you can also roll out a council of reviewers. So making, having many different agents review it, they all come together and kind of make recommendations on what areas need to be changed. So this is like having a code review, but with many agents at the same time, and it’s really a powerful option.

Your organization’s risk tolerance, appetite for new tooling needs to drive this decision. What are your customer obligations? What are your compliance obligations? What is your organization’s tolerance? How– what does your history of shipping look like? What does your uptime look like? These are all, I think, important considerations as a company lays out their policy. But I think the most important thing is the recognition and alignment that we cannot change code generation without also changing code review.

Henry Suryawirawan: Yeah, so I think that’s a very good tips, right? especially, you know, when you have so many lines of code being produced, I think definitely it’s worth to think about how do you want to review all this such that you, it doesn’t become a liability when, you know, things go bad, you know. Like especially when it affects users, you know, compliance and all that.

[00:49:11] How Should Docs and Code Attribution Evolve With AI Agents?

Henry Suryawirawan: I think one other challenge that, you know, we, I also discussed this with another guest in the past is that this thing called cognitive debt, right? So with so many things get done in such a short time, probably our attention span is also kind of like short. And we tend to also forget, you know, what has been done. Maybe even like if AI agents sneak in one line of code, we don’t know actually how it comes there in the first place. So how do you tackle this within your organization? Like do you also think about improving documentations or, you know, capturing important stuff as part of the PRDs? So how do you actually tackle this problem?

Emilie Schario: There’s a lot of things that we’re doing, and I think the most important one is experimenting. So for example, we were for a while there, we were including all of the specs for a project in the code as well. And then that worked for a while, and then the models started to pull in the files in weird ways, and so we stopped doing that. We have certain workflows where skills are included and certain models get prompted to pull the skill in. And then that might work for a little bit, and then it doesn’t. And so the really important thing is like this willingness to experiment and try and see what’s working and recognize that just ‘cause it worked, you know, six months ago doesn’t mean it’s gonna work today. That MCP server that was awesome is now getting really expensive ‘cause it’s eating all the context. Whatever it might be, I think it’s really just coming down to experimenting, seeing what’s making workflows better, and then continuing to push on that.

That being said, the other thing that we are trying to work better at is making it really clear when lines of code are changed by a human versus by an agent. And for a long time, if you had an agent running on your machine, that meant that code was committed by you. And if that’s the case, that’s great, but that means you’re responsible for it. And if you’re putting up the PR, you’re the author on it, you’re responsible for it anyway. But it can also be helpful to know, yeah, lines one through four were me, but lines five through 876, that was actually the agent that wrote that.

Henry Suryawirawan: Yeah, I think it will be exciting, seeing things like this, right? Because I think, yeah, not knowing where the code comes from is still a big challenge, right? Especially if you’re new to all this new workflow. So yeah, very,…

Emilie Schario: Yeah, I mean, building on top of that, one thing that I find is that even a year ago, users would say, “I want the– I don’t want people to know how much of a coding agent I’m using, so I don’t want the commits to come from the bot. I want them to come from me.” And it is crazy how much that has changed in the last year, that people are totally comfortable with the bot doing everything and no– ev- it being very clear that their job is just to prompt the Kilo agent. And I think that’s just an incredible shift where people are really buying into the future of what the next stage of work’s gonna look like.

Henry Suryawirawan: I think we don’t hide anymore. Everyone uses AI these days.

Emilie Schario: Nope.

Henry Suryawirawan: Yeah. So I think that’s a key thing. And I like what you mentioned, like experimenting, right? Because with so fast pace of changes that is happening and also in terms of AI model also change fast, it’s just given that we have to experiment and change a lot if things change, right? So I think this is probably a new way of working as well, that people have to really think about how do you actually adopt things fast, change it when things work not the same way as you expect it to be. So definitely very true that we have to experiment a lot.

[00:53:06] How Can AI Improve Your Life Outside of Work?

Henry Suryawirawan: So I think we have discussed a lot. Is there anything else that maybe you wanna, you know, touch on about Kilo, maybe or about new way of AI-native organizations work, before we wrap up the last question?

Emilie Schario: I think the last thing that I’ll share is that I don’t believe that AI is just for work. You know, we’ve talked about it a lot in the context of coding agents and Kilo and customers and work, but I also think one of the areas that I have found the most impact in AI is at home. So being able to leverage AI to make running my household smoother, managing information around my kids, coordination with my husband, being able to delegate to an AI agent at home the same way I have at work has really unlocked a new level of productivity. But it’s like a freedom. It’s less time that I have to spend on tedious house-household stuff and more time that I get to spend actually enjoying my family and time off. I think that’s not an understated accomplishment. The other thing that’s really nice about using AI in this context is that there’s no security guardrails, right? You don’t have to go through the IT team to get access to some tools. And so if you want to experiment, I think being at home is a great place or applying these AI tools to some workflow in your household is a great place. It’s a great way to experiment with cutting-edge tooling.

Henry Suryawirawan: Yeah, I think, that’s a very interesting thing, right? I personally have been using AI a lot more for, you know, health, you know, stuff, right? There’s this new thing about AI coach released by Google, right? So been playing around with it. I think it’s kind of like useful as well, and especially if you have all the metrics, all the data available that they can leverage on. Yeah. So I think these kind of things also is worth to experiment, you know, outside of work, to improve your personal life. So thanks for sharing that.

[00:55:04] 3 Tech Lead Wisdom

Henry Suryawirawan: So Emily, I think it’s been a great conversation. We learned a lot from you. And also for people who haven’t really checked out Kilo Code, please do give it a try, right? So before we wrap up, actually, I have one last question. I call this the three technical leadership wisdom, I ask to all my guests, which is kind of like an advice you want to give to the listeners. Maybe if you can share your version today, that would be great.

Emilie Schario: Yeah. The first, and this is not mine but I can’t tell you where I heard it, was, “If you’re gonna be a bear, be a grizzly.” So if you’re gonna do something, do it all the way. Go aggressive, be ambitious, don’t settle for anything other than the biggest, scariest, biggest bear there is. So if you’re gonna be a bear, be a grizzly.

The second is don’t hesitate to ask for help. I think all people struggle, whether that’s professionally, personally, you’ve got an issue at work, find your community, find the people you feel comfortable going to, build your network, and the group of people where you can, where you can turn to for questions, community, camaraderie, both professionally and personally.

And then finally, number three, last but certainly not least, is that it is really something to applaud when someone changes their mind. Especially in startups, it’s important that we optimize for speed. So we’re gonna make decisions off imperfect information. But as time goes on, more information is going to be made available, and we should make– we should be willing to reevaluate past decisions when there is new information. And it’s actually quite admirable when a leader is willing to say, “Hey, I was wrong. Let’s– We’ve got new information. Here’s what changed. Let’s change course.” And people are afraid to do that ‘cause their ego, and they think that people are gonna look poorly upon them. And I actually think it’s quite admirable and a show of strength.

Henry Suryawirawan: Right. Really love that, right? Applaud when, you know, someone changes their mind. Of course, if it’s for good, right? Not for the bad reason.

Emilie Schario: For sure. For sure.

Henry Suryawirawan: So Emily, if people would like to, you know, reach out to you, ask you more questions, is there a place where they can find you online?

Emilie Schario: Yes, LinkedIn is the best place. My name is Emilie, E-M-I-L-I-E, Schario, S-C-H-A-R-I-O. Outside of LinkedIn, I have a newsletter, Emilie, E-M-I-L-I-E.substack.com, where I write Playbooks and Priority, a newsletter on parenting, working, and working parenthood. And users can– people can feel free to reach out to me. My email is emilie@kilocode.ai

Henry Suryawirawan: Nice. I’ll put that in the show notes later on. So thank you again so much for this conversation. So very excited to see where Kilo Code goes after the acquisition. Hopefully, you know, you guys, can achieve bigger things, you know, talking about grizzly just now.

Emilie Schario: Thank you so much. Thanks for having me. And I really enjoyed this.

– End –