#13 - Startup Growth Strategy & Building Gojek Data Team - Crystal Widjaja

 

“Goal on the behaviors that matter. Don’t goal on your vanity metrics. Figure out what it is that, not just works for your product, but also for you as an individual."

Crystal Widjaja is a startup growth advisor and a Forbes 30 Under 30. She was most recently the SVP of Business Intelligence and Growth for Gojek. She is also the co-founder of Generation Girl, a non-profit organization that introduces young girls to STEM.

In this episode, I had a fascinating chat with Crystal on many things about startup and her exhilarating journey with Gojek as the first data hire. We started with the recent trends of the startup funding in US and Southeast Asia, and the impact that COVID has brought to the startup scenes. Crystal then shared her insightful tips on startup growth strategy, including the common pitfall startups need to avoid in their strategy. She also gave practical tips on how a startup can start its data analytics journey. We then talked about her recent Gojek career when she outlined her amazing journey building Gojek data team from one person (herself) to 200+ people, the challenges she had to go through and how she overcame them. Last, Crystal shared about Generation Girl and why it is an important cause to help Indonesian young girls to succeed in STEM.  

Listen out for:

  • Crystal’s career journey - [00:06:04]
  • Why Crystal quit Gojek and moved to VC - [00:09:26]
  • Startup trends in US and SEA - [00:13:16]
  • Impact of COVID to startup growth and funding - [00:20:10]
  • Reforge - [00:25:56]
  • Tips on startup growth strategy - [00:28:32]
  • Strategy for building data analytics strategy - [00:36:55]
  • Building Gojek data team as the first data hire - [00:45:54]
  • Generation Girl - [00:59:30]
  • Crystal’s 3 Tech Lead Wisdom - [01:08:22]

_____

Crystal Widjaja’s Bio
Crystal Widjaja is a startup growth advisor, Reforge advisor partner, and Sequoia Scout in the SF Bay Area. Crystal has been recognized as a Forbes 30 Under 30 and was most recently the Senior VP of Business Intelligence and Growth for Gojek, the leading on-demand multi-service platform in Southeast Asia committed to empowering informal sectors and MSMEs through technology. She is also a co-founder of Generation Girl, a non-profit organization that aims to introduce young girls to Science, Technology, Engineering, and Math (STEM).

Follow Crystal:

Mentions & Links:

 

Our Sponsors
This episode is brought to you by JetBrains.
Do you want to learn to code? Do you have friends who are looking to learn how to code?
Our sponsors at JetBrains recently launched JetBrains Academy, an education platform that offers interactive, project-based learning combined with powerful, professional development tools. Advance your Java and Python skills, with more programming languages to come.
To get an extended 3-month free trial on JetBrains Academy, go to https://techleadjournal.dev/jetbrains-academy.

 

Like this episode?
Subscribe and leave us a rating & review on your favorite podcast app or feedback page.
Follow @techleadjournal on LinkedIn, Twitter, and Instagram.
Pledge your support by becoming a patron.

 

Quotes

Difference Between US and SEA Startups

  • Almost everyone in the US speaks English and has access to online learning platforms, has access to English documentation, and ability to go through different tutorials in their native language.

  • In Southeast Asia, you don’t have the same resources or access or opportunities to the same support systems that you would in the US.

  • In Southeast Asia, the people have genuine unbridled creativity. I think people are inherently creative. They’re able to solve things, not just with their communities, but independently as well. The community aspect is one of the most interesting parts of Southeast Asia, that people are actually willing to come together, and go through a solution regardless of the backgrounds of whether or not someone is technical. People in Southeast Asia seem to be able to get things done with far less resources.

On Collaboration Tools

  • The mindset of collaboration tooling has generally been around project management, or around “how do I get alignment and share the discussions that we’re having internally with the rest of the organization”. So collaboration management and tooling has been very traditionally thought in these ways. It’s about getting alignment and reaching consensus.

  • The new wave of collaboration tooling is very specific to a specific team. It’s something that a team can put together. 5 to 10 people can look at a screen, start whiteboarding together, people can provide comments on design, for example. It’s become much more for a unit of work, rather than an entire team’s collaboration management.

Impact of COVID to Startups

  • If there was any time better for Southeast Asia to get funding, it should actually be now. Because startups in the US, unless they are completely remote friendly and primarily for a remote workplace, they’re not able to get to scale. They’re not able to launch and do their offline campaigns. Where you are able to do this, is in places where the virus has been relatively contained, where people can go about their regular lives, F&B is open again and thriving, SMEs are starting to go back to work. This is actually the right time for Southeast Asia to actually dominate more of the investment pipeline, because they’re able to go back to business, and show results and growth.

  • You’d have to be a very tenacious founder to want to start at this time. That helps with some of the self-selection, that only the best will be ready, and I think determined enough to want to work and try to build a company in this type of environment. So you have the best types of founders starting in this environment. The people who are starting a company right now are likely to do pretty well, because they are so determined. And when there isn’t a pandemic, they’ll only do even better.

On Reforge

  • I believe that what Southeast Asia often needs is just opportunity. They need resources. They need help with whiteboarding ideas, bouncing ideas off of advisors, working with people who have seen it before.

On Startup Growth Strategy

  • My biggest tip that I see most startups get wrong is: do not goal on retention itself. A lot of companies will set their OKRs around 3-month retention. It’s a metric that you should absolutely be monitoring, but it’s not something you direct a team towards.

  • We should look at the behaviors that correlate with retention, and we use those as the metrics to goal on instead.

  • A setup moment helps the user and the product get to and reach the aha moment.

  • What I tend to look at is identify those habits and those moments and behaviors that correlate with retention.

  • Most companies will look at 3-month retention and say, “Great. We have either done it or we haven’t.” What happens though is that when teams are given this pressure to get to 3-month retention, they will start trying anything and everything to get there. They will spam promotions. They will send out tons of push notifications. They will create irrelevant content, and send users on this wild goose chase that actually confuses them about what the product is about.

  • What’s important is to be able to identify the behaviors that correlate with retention positively, that have strong correlation.

  • It’s important that the team knows the specific behaviors to goal on, rather than retention itself. Because when they goal on retention, they end up going in a million different directions, rather than focusing their resources on trying to identify a specific behavior.

  • When you have this very concrete and admittedly smaller space to ideate on, you actually come up with more ideas. You come up with more concrete ways to achieve the goal.

  • I would encourage companies to figure out what are their use cases, what are the frequencies for my product, and what can I do to keep my user engaged in a way that doesn’t cause them to churn, that would cause me to have to reacquire them later.

Building Data Analytics Strategy

  • For companies where the use case is extremely infrequent, find the use cases that are adjacent to the product that can be more frequent, that we can use to collect data on a more regular cadence.

  • My starting point is always for new users that have experienced our core product value prop, how frequently are they coming back to our app?

  • There’s going to be an inherent use case frequency that we need to align ourselves to first, so that we know the cadence of communication. We know what retention looks like, so that we can start goaling on the longer term retention rates.

  • Start with the founder’s proprietary hypothesis. That’s actually where I always start. I always start by asking teams, what do you think drives growth? What is the aha moment? When does it feel like a user is gaining a super power by using your product?

    • What I would encourage companies to do is first figure out what those hypotheses are.
    • Then I would encourage the teams to figure out and track on a per user basis, when are they using the product in that way?
  • There is a universe of behaviors that we could look at. But what’s best is usually we start out with our own personal experience.

  • You don’t need thousands of people to respond to a survey to get a good enough hypothesis. The more people you ask, the more specific your answers become, the more specificity you gain out of this data. But the difference between 30 people and 1,000 people is just that, specificity. You still get the overall same theme, the same insights, the same ideas behind the hypothesis.

Building Gojek Data Team

  • The first part was trying to make it clear why the data was so important. Creating a shared sense of urgency.

  • By providing them those insights time after time, my goal was to make it clear what their impact was, based on the engineering work that they were doing, based on what they were delivering.

  • The sign of a good data team is that people care. Because if your data is a mess, if no one respects the databases, if no one’s willing to fix the problems, then the real issue is that people haven’t been able to realize their impact through data.

  • People want to hear the impact they make to the company. And they want other people to know that they are impactful.

  • The best way to tell the impact is through the data. I always started out with making sure that the data was used in a way that incentivized people to keep the data clean and organized. And then secondly, that it helped people make better decisions.

  • The primary initial part of the journey: hiring out someone who understood the business process to help with automation. Next step was creating a team for the data lake to move the data to one place. And then after that comes the tactical hires. It was bringing on people with statistics background. Someone who understood how to make a meaningful dataset into an actionable, statistically significant and rigorous analysis that would direct us towards optimal gains in the business.

  • “Everything that’s happening this quarter was already set in stone last year.“ ~ Jeff Bezos

  • Lessons learned from “Crucial Conversation“

    • It’s about how to get to the right objective. You have a goal, however, opinions differ and emotions are very charged. Everyone thinks they have a right way to do it, and their reputation is at stake.
  • I always have to be cognizant of the fact that in startup world, you do have a million things happening, and not everyone is going to get the perfection that they want. The product team itself is probably not releasing the best version of the products that they actually want because of limited time and constraints. So I think the first point is to just be a realistic.

    • Create the standard, make it super easy and available and accessible to the teams.
    • The next thing is to create plans. The data is not always going to be tracked correctly. Give people an easy way back to the golden era of data.
    • We try to make it super easy for teams to integrate their data. If something happens, we give them the benefit of the doubt. We make sure that we’ve done what we can to bring them back to the data cleanliness side.

On Generation Girl

  • If you are a woman, you don’t have quite as many role models as a male would.

  • It’s just a reality that in our culture, we don’t have the same examples for girls to go into STEM fields.

  • Generation Girl tries to normalize the idea of women and young girls being part of STEM fields (Science, Technology, Engineering, and Math). Our goal is to give them a safe, inclusive, and a confident and inspiring space for them to fail, and to try, and to be successful in STEM.

  • What we want to do is to give these women the opportunity to try STEM, to see whether or not they like it. And then to give them opportunities to continue learning if they choose to do so.

  • Where I think we can do better is in investing in creating opportunities for women to be part of the workplace.

  • More diverse teams create better products. A more diverse team is only going to make the product better. If teams are committed to creating the best possible engineering team, that gets the best possible product teams, then they should be investing in opportunities for women to be part of that equation.

  • It’s a cultural endemic issue that there aren’t enough women in STEM, and therefore we don’t hire them, and therefore women never join STEM. It’s a self-perpetuating problem that we don’t invest in them. We don’t make it easy for them to be part of those communities. That’s why we lack the talent pool for them.

Crystal’s 3 Tech Lead Wisdom

  1. Goal on the behaviors that matter. Don’t goal on your vanity metrics. Figure out what it is that, not just works for your product, but also for you as a person, like what works for you as an individual, what makes you the most successful. Figure out what those behaviors are that make you the best to your abilities. Figure out what those are, and operate, and optimize for that.

  2. Figure out what you’re not great at, and find people who can help you fill those gaps.

  3. Listen and be a rational person.

Transcript

Episode Introduction [00:00:56]

Henry Suryawirawan: Hello, my friend. Welcome to another episode of the Tech Lead Journal with me your host Henry Suryawirawan. Thank you for tuning in and spending your time with me today listening to this episode. If you haven’t joined any of the Tech Lead Journal social media channels, I would encourage you to take a second right now to click on the links in the show notes, where you can find the podcast, either on LinkedIn, Twitter, or Instagram. I’d really love to see and interact with all of you there. Every time you post and share something about this podcast on your social media, it makes me extremely happy. And I do hope that more people will be able to find this podcast and learn from the amazing guests that I have on the show. If you have other creative ideas on how to grow this community bigger, please feel free to reach out to me and share your ideas. It would really mean a lot to me. And if you would like to pledge your support and make contribution to the show, you can do so through our patron page at techleadjournal.dev/patron. It would help me tremendously towards achieving a goal that I’m currently running on the patron page.

Our guest for today’s episode is Crystal Widjaja. Crystal is a startup growth advisor, Reforge advisor partner, and Sequoia Scout in the San Francisco Bay Area. She has been recognized as a Forbes Indonesia 30 under 30 in 2019, and was most recently the Senior VP of Business Intelligence and Growth for Gojek, the leading on demand multi-service platform in Southeast Asia. She is also a co-founder of Generation Girl, a non-profit organization that aims to introduce young girls to Science, Technology, Engineering, and Mathematics.

In this episode, I had a fascinating chat with Crystal on many things about startups, and her exhilarating journey with Gojek as the first data hire. We started our conversation with the recent trends of the startup funding in the US and Southeast Asia, including the impact that COVID has brought to the startup scenes. Crystal then shared her in-depth and insightful tips on startup growth strategy, including the common pitfall that startups need to avoid in their growth strategy. She also gave practical tips on how a startup can start its data analytics journey. We then talked about her recent Gojek career when she outlined her amazing journey, building Gojek data team from just one person, which is herself, to about 200+ people, the challenges that she had to go through and how she overcame them. Last, Crystal shared about her non-profit organization, Generation Girl, and why it is an important cause to help Indonesian young girls to succeed in STEM.

I hope that you will enjoy this great episode. Please consider helping the show in the smallest possible way by leaving me a rating and review on Apple Podcasts or the other podcast apps that allow you to do so. Those ratings and reviews are one of the best ways to get this podcast to reach more listeners, and hopefully the show gets featured on the podcast platform. I’m also looking forward to hearing your comments and feedback on the social media, or directly to me at techleadjournal.dev/feedback. You can also post your learnings and even creative pictures of you listening to the podcast, following the trend that Jerome has been doing since the last few episodes on LinkedIn. So let’s get the episode started right after our sponsor message.

Introduction [00:05:00]

Henry Suryawirawan: Hello, Crystal. Welcome to the Tech Lead Journal, very excited to have you here.

Crystal Widjaja: [00:05:06] Thanks for having me, Henry. It’s super exciting to be here with you too.

Henry Suryawirawan: [00:05:10] So where are you in the world now?

Crystal Widjaja: [00:05:12] Right now I’m in Boston. I am on the East Coast, but I actually just moved back in January. The point when everything in the world was starting to go a little bit crazy, I moved back from Singapore. And it’s super funny because I remember thinking, “Oh wow, there’s this virus thing going around and it doesn’t seem like people in the US are really aware of it yet.”

Henry Suryawirawan: [00:05:33] Is it something that you plan to stay there for long or is it just for temporary?

Crystal Widjaja: [00:05:37] That’s a good question. I actually moved back to be closer with my family and to travel a little bit in the US, and then, of course, that plan was no longer possible. But I’ve actually started working with a couple of different startups in the US and in LATAM. So right now my plan to do is decompressing and understanding and surveilling the US startup ecosystem where I came from, and it’s where I first started, and then move back to Southeast Asia in a year or two.

Career Journey [00:06:04]

Henry Suryawirawan: [00:06:04] Alright. Cool. So let’s start with your career journey. So Criss, you have a very wonderful illustrious career so far. Would you be able to share your career journey, maybe highlighting some of your major turning points so far that are interesting for the listeners to know about?

Crystal Widjaja: [00:06:19] I’d love to talk about that. I guess it would all first start when I was actually attending community college. A lot of people don’t know that I actually transferred to UC Berkeley. I had initially wanted to do something in rhetoric or logic or even philosophy. And I ended up majoring, or deciding my major in political science, because that was going to be the fastest way to graduate. So based on my test scores and my AP tests, I was able to skip a lot of the prerequisites, and decided to go to UC Berkeley for political science. Really the goal was to be like Nate Silver. And we all know Nate Silver’s skill set in being able to determine who is going to win the next election. How are precincts going to be voting? And why that is statistically significant. And so for me, that was exciting. I loved being able to figure out and predict what would happen based on heuristics and quantitative datasets. The initial thinking was, I’ll do some education research, I’ll learn about how data can be used to derive a conclusion, and I actually ended up in investment banking instead. Not too different maybe, when you’re using numbers to figure out what the best next bet is. So we did VC Financing and M&A Advisory, and the investment bank that I started working at in San Francisco. But what’s interesting is that I didn’t realize that was directly my first foray into startups. I ended up learning a lot about what metrics we should be asking different companies. We interviewed people like Summon, which was the Uber before Uber existed. And it just gave me a really good view of what the right company should be, if I were to try to work at a startup.

So I ended up going to Indonesia for the first time in 2015 with my parents. And I was exceedingly excited. I think there’s just so much opportunity in Indonesia and there’s so much growth. At that point in time, I decided that I did want to join a startup. I wanted to do something within data. I tried to create a SQL database for the bank that I worked at. I actually Googled “top startups in Southeast Asia”, and I did my investment banking thesis. I figured out “what were the metrics for each of these startups?” I try to figure out their funding. What their backgrounds were. I ultimately ended up picking a couple that I emailed. And Gojek happened to reply.

When I went there, I was the first data person on the team. There was a whole slew of data. We were growing super fast already, but there was no organization or strategy around how to use this data to grow more quickly, more effectively. So when I first joined, I hired a bunch of different people to help build a data warehouse. We built out a data infrastructure team. And when you have access to all of this data, you quickly realize how much fraud is in the system. And so I ended up also taking up the fraud and risk team, building that team out. Moved on to portals, performance marketing, and then of course growth, which I’m primarily focused on today. So it was an incredible journey. Over five years, I essentially grew the teams from 0 to about 200+ across BI and growth, helped hire our Chief Data Officer. And now I’m working with the Reforge team on growth. And I’m also a Sequoia Scout for the US, where I help identify pre-seed and series A startups for investments. And I’m also advising a handful of companies.

Why Crystal Quit Gojek And Moved to VC [00:09:26]

Henry Suryawirawan: [00:09:26] It’s a very wonderful journey. I didn’t know that some of the things, for example, you actually come up with a research before you joined Gojek, utilizing some of your knowledge before. I think it’s very interesting way to find a job definitely. And then, what made you decide to quit Gojek and move out to the VC side?

Crystal Widjaja: [00:09:43] Yeah, that’s a good question. So I’d spoken with the COO of Stripe, Claire. She’s amazing. She spent nearly a decade at Google. She’s built Stripe to a global company. And one of the things that she said to me when she was in Singapore was, “what’s really concerning is whether or not I was just really good at working at Google”. That was something she didn’t want to feel, given all of her time there. That reminded me a lot of my experience at Gojek. I’d spent a really long time there. Definitely not a decade, but certainly a long enough time to have felt like my influence and impact was more of a factor of my legacy rather than my actual abilities. I didn’t want to feel like I was primarily working around in the politics, or that I was navigating seniority in the right ways. But I wanted to feel like I was actually being impactful and building my skills. Not to say that I wasn’t building any skills at Gojek, because certainly I did, but I wanted to make sure that it was transferable enough to other companies. I felt like I had to leave because I felt very comfortable, to be honest with Gojek. It’s such a great company and honestly, anyone listening who is from Gojek, it’s an amazing place! I think it’s pretty magical. But I also knew that it was time for a change. I was starting to feel restless. I wasn’t sure exactly what I wanted to work on next.

So I actually left without really a clear plan of what to do next. I just moved back to the US. Thankfully, I have the privilege and the luxury of my parents being able to let me come back home and spend time with them. I worked with a SaaS startup for a couple of months. Decided that I didn’t want to do an operations role, or a Head of Operations role. And then ended up deciding to just help or continue helping the companies that were already reaching out to me. By the time I left Gojek, people would be asking things like, “Hey, you left Gojek, but I’d still like to learn what you did there, and how you did growth. My team is growing, we don’t really know how to look at our numbers. And so after enough time, I ended up just helping these companies full time, and going through their data, advising them on where their users were getting stuck, and where users were retaining, what were the behaviors that correlated with retention, and helping advise them there. The Sequoia Scout program is super interesting. I think it’s a lot of founders of Sequoia companies. They also have a pre-founder program that’s coming out very soon. But it’s an amazing experience where given your network and the people within the startup ecosystem, they know that Sequoia is too large of a firm to really get access to some of these friends and family rounds. And so that’s where the Sequoia Scout program comes in. They’re looking for someone who has access to these founders, to people who are just starting a new company and are just thinking of building something and incorporating it in the US. My work there is really keeping tabs on amazing people that I know were starting companies, making sure that I can be helpful to them, whether that’s in product or just trying out their MVP. That’s been super exciting, and so I’ve made a couple of investments there so far.

Henry Suryawirawan: [00:12:35] I think it sounds very interesting, like accessing the friends and family stage startup. So are you predominantly doing it for the US? Or is it also global or Southeast Asia?

Crystal Widjaja: [00:12:45] That’s a good question. So the Sequoia Scout program, I believe was started in the US. And so I’m part of the US cohort. So that does mean I’m only allowed to invest in companies that are incorporated in the US, which is unfair because I know so many amazing founders who are starting something in Southeast Asia. But I’ve actually been able to help share the Southeast Asia scout team, which I think just started at Sequoia Scout program in Singapore and Indonesia. So I’ve sent them a handful of startups as well, hoping that they’ll also continue to provide support in Southeast Asia for new startups.

Henry Suryawirawan: [00:13:16] So, you have been in Southeast Asia with Gojek for quite some time in the startup landscape there. And now you are in Sequoia Scout in US. What are some of the things that you can see, like maybe, trends that are different between in the two regions? Or are there any similarities? Or maybe just different things. So maybe you can share your insights here.

Crystal Widjaja: [00:13:34] Definitely. There are a couple of things that have been immediately different. I think first, the fact that everyone in the US, almost everyone in the US, basically speaks English and has access to online learning platforms, has access to English documentation, and ability to go through different tutorials in their native language. That is such a strong leverage point that I think many people in the US do not realize they have this privilege. Because the amount of people who are in non-technical roles, but have technical skills, like they can write in JavaScript, or they have SQL background, or they have skills in being able to write quick scripts in Python. That I think is far more common in the US than it was in Southeast Asia. In Southeast Asia, you don’t have the same resources or access or opportunities to the same support systems that you would in the US where everyone, at least in the Silicon Valley, is very… coding first. There’s resources and opportunities and anyone that you can talk to is just an arm’s length away, at least back when we were in the office. But you don’t really have that same opportunity in Southeast Asia. So I think they don’t appreciate that quite as much in the US, the fact that everyone is so technical. And I think that makes the development work a little bit easier in the US, because everyone kind of understands how tech works. Everyone has an idea about what can be automated, and what effort it would take to automate something. Whereas in Southeast Asia, what you do have though, I think is genuine unbridled creativity. I think people are inherently very creative. They’re able to solve things, not just with their communities, but independently as well. But I think the community aspect is one of the most interesting parts of Southeast Asia, that people are actually willing to come together, and go through a solution regardless of the backgrounds of whether or not someone is technical. People in Southeast Asia seem to be able to get things done with far less resources. I think that’s pretty remarkable as well.

Henry Suryawirawan: [00:15:30] So what are some of the trends in your work now? Like you see that the VC is investing more in. Is there any industry? Is there any technology that are outstanding?

Crystal Widjaja: [00:15:40] I have a lot of opinions on this. So I have definitely noticed is collaborative tooling. I don’t know if I see this quite as much in Southeast Asia. I’m not sure why, but I see a lot more collaborative tooling being picked up in the US. Things like Figma for designers. One of my new personal favorites is actually Excalidraw, which is a online white boarding tool. I see a lot of these new startups coming out that provide remote based collaboration tooling. And it’s a good framework. I think I see more investment here. I see more people also entering with the consumer first mindset. Things that were usually traditionally just a B2B sales model, is now being driven by a Slack like growth model, where you have a certain department or team that adopts the tool. The tool usage ends up becoming widespread in the company. And then the company is all but forced to pay for the enterprise plan, because there are so many people at the company already using it. And I think this is one trend that I see happening much more quickly in the US than in Southeast Asia. But I hope we’ll see that trend, and just the better tooling come to Southeast Asia soon.

Henry Suryawirawan: [00:16:50] This is interesting, because we also see a lot of big enterprises. Microsoft obviously has been around industry for long. And Google has its own G Suite, or what is called Google Workspace now. Why do you think there are some rooms for these new startups to play in this collaboration tools?

Crystal Widjaja: [00:17:06] That’s very true. The mindset of collaboration tooling has generally been around project management. Or it’s been around “how do I get alignment and share the discussions that we’re having internally with the rest of the organization”. So collaboration management and tooling has been very traditionally thought in these ways. It’s about getting alignment. It’s about reaching consensus. And that is the type of collaboration tooling that many companies in Southeast Asia are drawn to, and are using today already. But this new wave of collaboration tooling, what I’ve noticed is, it’s very specific to a specific team. It’s something that a team can put together. 5 to 10 people can look at a screen, start whiteboarding together, people can provide comments on design, for example. But it’s become much more for a unit of work, rather than an entire team’s collaboration management. And so I see less management tooling, which I think is a lot of what Microsoft and Google build, and instead a lot more of these real time synchronous tooling systems that let people have discussions in a better way, live together. So I do think this is where I’d love to see more people come together and adopt these types of tooling, because I think it cuts through a lot of the communication that happens on video, where people are trying to explain things. But honestly, some people have very different learning styles. When I started using things like Excalidraw, I ended up realizing that some of the people in the teams that I work with are very visual learners. They don’t really see the connection of the arguments that I’m making, unless there’s a diagram, or like literally a drawing connecting these two points of a funnel together, and illustrating why the drop-off rate is so important here. So these are the ways that I’ve seen collaboration evolving from the overall company perspective, down to how can we make individual teams more effective.

Henry Suryawirawan: [00:18:58] So you mentioned it is growing there in the US. How about in Southeast Asia? What is the most recent trends?

Crystal Widjaja: [00:19:05] I think I have actually noticed wellness and mental health becoming more widely accepted and widely discussed. I think this is super important. It’s something that we actually tried to implement at Gojek early on right before I left. It’s actually a fairly new phenomenon. The startup scene has just started. I know some people will have been there for a decade, and say, “No, it’s been here always.” But really at that point, in which there was a unicorn in every country in Southeast Asia, that’s I think when the pressure really becomes overwhelming for a lot of people. Because there is that pressure to compete. There is that pressure to be compared with your competitor, not just in your own country, but then in a neighboring region as well. And so there is a lot more pressure these days, I think for startups. And I think that has helped introduce or create an entry point for these wellness startups that are beginning. I think Siu Rui (the CEO) at Carousell recently shared one with me, that he’d recently invested in, and I think they’re super powerful. I’d love to see more wellness and mindfulness startups take place and grow in Southeast Asia. I definitely see that trend happening now.

Impact of COVID to Startup and VC [00:20:10]

Henry Suryawirawan: [00:20:10] So I wasn’t aware about that. I think it’s very interesting for me to check out after this. The recent COVID pandemic, do you think is there any impact so far with the investing world, VC world and the startup growth in general?

Crystal Widjaja: [00:20:22] This is such a good question. I actually was talking with someone recently about this in the Philippines, that if there were any time better for Southeast Asia to get funding, it should actually be now. Because startups in the US, unless they are completely remote friendly and primarily for a remote workplace, they’re not able to get to scale. They’re not able to launch and do their offline campaigns. It’s become very difficult to reach mass market, and have sales meetings, and things like that. And so where you are however able to do this, is in places where the virus has been relatively contained, where people can go about their regular lives, can go to restaurants, F&B is open again and thriving, SMEs are starting to go back to work. This is actually the right time for Southeast Asia to actually dominate more of the investment pipeline, because they’re able to go back to business , and show results, and show growth. I’ve been a part of the 500 Startups investor discussion groups on Slack. We have a pretty huge directory of, I think maybe close to 500 different VCs. And it’s been an incredible place where I’m noticing more people provide those opportunities and reach out to more Southeast Asia startups, because they know that’s where the growth is right now.

Henry Suryawirawan: [00:21:32] I’ve read somewhere before that after these kind of a crisis, for example, like financial crisis last time, there will be lots of new startups, especially growing to become unicorns. So after this pandemic, do you foresee some new startups popping up and become unicorns?

Crystal Widjaja: [00:21:47] I think that you’d have to be a very tenacious founder to want to start at this time. And I think that helps with some of that self-selection, that only the best will be ready, and I think determined enough to want to work and try to build a company in this type of environment. And so you already have just the best types of founders starting in this environment. The people who are starting a company right now are likely to do pretty well, because they are so determined. And when there isn’t a pandemic, they’ll only do even better. So I am excited. I think we’ll see people who have had the time to really refine their product during the quarantine. To do very close on user experience surveys, product reviews, and just have time to develop heads down. And then when the pandemic is lifted, and hopefully we’re all back to normal soon, then they’ll be able to go full force, and the next founder could be creating the world’s next unicorn for sure.

Henry Suryawirawan: [00:22:43] I’m looking forward to see that coming. So obviously, hopefully this pandemic is ending, maybe next year, early next year. So we all hope so. And we will start seeing new startups, new kinds of startups, where probably they will dominate some areas like remote, obviously it’s one thing, remote working. And then maybe digital kind of payment, e-payment, wallet. I know some countries have just started, especially in Southeast Asia, around digital banking.

Crystal Widjaja: [00:23:08] One interesting place where there will probably be new startups needed, is in the entrepreneurship segment. There are a lot more people who are now out of work, who are maybe like me, want to go and see if they can make something of their own. Maybe they’re not necessarily building a huge company, but they want to provide their own services. They want to be an independent contractor. They want to try to provide and build something independently. Not unlike the open source community, where they see problems, they want to create something to fix it. And of course they will want to be paid for it, in many of these cases. And so I think platforms that help people be self-sustaining, that help people become maybe more independent, better business people, collect invoices on time, etc. I think these platforms will also see a new rise as well.

Henry Suryawirawan: [00:23:57] So, this is an interesting point you made, because these type of, I call them Indiehackers, or independent, maybe small businesses, doesn’t necessarily aspire to grow as big as Gojek or Uber, but they want to be successful as well, maybe a million dollars type of valuation. What are some of the things that can be done to support these types of entrepreneurs? Or is there any some kind of funding available for them, because I’m not sure whether the VCs are interested as well for these type of businesses?

Crystal Widjaja: [00:24:24] Exactly. I think that VC financing is not the right place for this. I think that there are plenty of good ways for governments to support these industries. By providing work loans, by providing business loans that carry low interest rates, that help people ideate and figure out whether or not their idea works, and to provide them with that initial investment. That will actually pay off, because they’ll be building an independent employment sector that could eventually be not employing maybe thousands of people, but like you said, employing maybe 5-10 different freelancers, 5-10 different engineers or designers. I think that this is definitely one place I see growing. There was a very good book written just about Korean fried chicken restaurants that were surging in ‘97, because of the financial crisis where a lot of these Korean families no longer had an opportunity to go to work, and instead decided to open a restaurant. Now, the problem with that use case is that it was completely saturated. You had a mom and pop shop on every block selling the exact same type of food. But I think in Southeast Asia in general, especially in the digital services economy, you have people who can provide a wide range of services. There’s a ton of work in data consulting, and designing, and branding, and PR, and agencies. I would hope that we have a lot more of these solo entrepreneurs or small group, I guess not really co-ops, but kind of a co-op, where it’s a couple of people who want to work together as a cooperative and build a product together. I definitely encourage this kind of collaboration.

Reforge [00:25:56]

Henry Suryawirawan: [00:25:56] Cool. So Crissy, like you just mentioned you work with Reforge. So maybe, can you tell me about Reforge more, and how do you actually interested to join that? And what kind of, some of the interesting activities that you have done with Reforge?

Crystal Widjaja: [00:26:10] Absolutely. I’m super glad you asked. I, for one really believe that what Southeast Asia often needs is just opportunity. They need resources. They need help with whiteboarding ideas, bouncing ideas off of advisors, working with people who have seen it before. In Southeast Asia, because everything is so new, because Gojek was the first unicorn in Indonesia, there was no one we could ask, " Hey, how have you solved this problem before?” Because we were the first people to see this in Indonesia. Just that lack of perspective, I think was a real challenge for us. And I always reached out. Back when I would travel back home to the US, I would reach out to anyone who would answer my WhatsApp message, my LinkedIn message, my cold call email. I would ask, “Hey, I have a list of these questions, can you help me? What do you think?” And so that actually is why I’m really interested in the Reforge team, because they are composed of some of the world’s greatest growth companies. We have the CPO of Eventbrite. There’s the Head of Growth from Slack. And there’s just a ton of resources available for people who are within this community. We recently launched a Reforge beta advisor Partners Program, where we were accepting applications, and we got hundreds of applications from around the world, of startups that are looking for growth advisory help. And we’re not taking or charging salary. We are taking a very small percentage of equity based on the very commonly accepted FAST template agreement. But what this allows for, is that startups who are growing, who just need an hour or two of time to whiteboard, to figure out growth loops, to figure out growth models, are able to get access to this amazing resource, this team of experts who have seen it before. That’s not to say that we’ll have all the answers or that we’ve seen every problem that is admittedly going to be very different in every country, but at least they have that opportunity. With the Reforge team, we’ve really been helping a handful of startups that are in the growth stage of their life cycle, are trying to meet and close that gap between product and marketing, and I’ve been super thrilled to work with them. I actually recently published a blog post on why most analytics efforts fail. A lot of it is based on the learnings that I’ve had at Gojek about how to track in-app events. How do we create a data platform that works not just for the engineering team, but for the business teams so that they can also make decisions on the data. So this has been super exciting work the past year.

Tips on Startup Growth Strategy [00:28:32]

Henry Suryawirawan: [00:28:32] So one of your expertise obviously is around startup growth strategy. And I’m sure you play a big role as well in Reforge to advise startups on this area. Maybe can you share some of your biggest tips for startups that are looking for growth?

Crystal Widjaja: [00:28:46] My biggest tip that I see most startups get wrong is: do not goal on retention itself. A lot of companies will set their OKRs around 3-month retention, 2-month retention. And to me, it’s a metric. It’s a metric that you should be monitoring, absolutely. But it’s not something you direct a team towards. Every quarter, I do this exercise with the companies that I advise. We look at the behaviors that correlate with retention, and we use those as the metrics to goal on instead. You could look at a company, let’s say Facebook, they had something like 10 friends in 7 days. That is not an aha moment, contrary to popular opinion. That’s actually what I would call a setup moment. A setup moment helps the user and the product get to and reach the aha moment. And so for me, to create a good user experience in a platform like Facebook, the setup moment would be “how do I get the user to get 10 friends in 7 days”. Once they’ve gotten 10 friends in 7 days, now it’s time to reach the aha moment. The aha moment is correlated with, let’s say, seeing 3 newsfeed posts in the first one day, and they have to be relevant posts. And that’s not going to happen, unless you get those 10 friends for that user. And so that’s why these behaviors are actually correlated. The first behavior is getting 10 friends in 7 days. The next behavior for the aha moment is probably something like getting 3 posts in the first day. And then the long-term retention metric is something closer to 7 active days in the first 10 days. What I tend to look at, and what I tend to do with my companies is identify those habits and those moments and behaviors that correlate with retention, rather than saying, how do we get to 3 months of retention itself?

Henry Suryawirawan: [00:30:32] So why do you think 3 months retention is not enough? Because based on my limited knowledge, obviously looking at the 3 months retention is probably quite close in terms of seeing the growth of your products, or the growth of your users. So why do you think it’s very important to see the behaviors that probably correlate with the long-term retention of the users?

Crystal Widjaja: [00:30:51] You are asking such good questions, Henry. Most companies will look at 3-month retention and say, “Great. We have either done it or we haven’t.” What happens though is that when teams are given this pressure to get to 3-month retention, they will start trying anything and everything to get there. They will spam promotions. They will send out tons of push notifications. They will create irrelevant content, and send users on this wild goose chase that actually confuses them about what the product is about. And so when I do this correlation analysis, what’s important is to be able to identify the behaviors that correlate with retention positively, that have strong correlation. And most importantly, that we realize at what point does it not really matter if a user gets to let’s say 8 app opens in the first three days. Does it really matter if we get them from 7 to 8? Probably not. What really matters is that we get them to the aha moment at least 3 times. And at that point, you then realize what the content strategy should be, that you don’t need to spam your users, that the natural frequency of usage of this product is no more than 3 out of 7 days. And for a company or team to ignore that, and create a completely different experience for the users, that’s disruptive. It creates churn. People uninstall the app. You don’t want that. And so for me, it’s really important that the team knows the specific behaviors to goal on, rather than retention itself. Because when they goal on retention, they end up going in a million different directions, rather than focusing their resources on trying to identify a specific behavior.

So this is the exercise that I actually do with these companies. I asked them, and we can try this too, Henry. Close your eyes and imagine everything in the world that is orange. So you’re thinking, maybe a basketball, okay. What else? An orange itself. So there’s maybe two things that you think of in these 30 seconds. But then, let’s try this again. Let’s think of everything in the world that’s orange, but within a construction site. And so now when you have this very concrete and admittedly smaller space to ideate on, you actually come up with more ideas. You come up with more concrete ways to achieve the goal. The difference between saying “let’s think of everything in the world that would get us to three months of retention” versus “what are the things that we can do to get a user to experience our newsfeed 3 times within the first 7 days”. It’s a very different experience that the team comes up with.

Henry Suryawirawan: [00:33:25] This is very interesting exercise, definitely. I understand how constraint can actually make you creative, not just in this type of exercise, but basically almost everything, including like technology, teamwork, or even like the company policies. But the tendency of these startups, for example, if you take E-commerce or maybe ride sharing or whatever that is, they just spam promotions or maybe emails. Everyday I receive tens or maybe twenty emails every day, just offering promotion discounts or whatever that is. So what are the things that some of these companies can do actually to use your behavior correlation analysis?

Crystal Widjaja: [00:33:59] Yeah, that’s a really good question. And yes, you’ll get emails from companies like Zara sending here’s a coupon for a new jacket, a new shirt, every day. And then you have to ask yourself, what does the natural frequency for me to be buying things? Is it really supposed to be every single day? I don’t think so. And so you get to this level of just spam. And I think what these companies haven’t done, is likely they haven’t done the analysis of what is the short-term gains versus long-term gains I am getting with this strategy. I’m actually hoping to write a blog post about this very soon, so stay tuned. But if a company were to take the time to analyze for every message I sent, how many uninstalls am I getting? How many people are unsubscribing? How many people are turning off notifications? What does that add up to in the long-term? If they weren’t going to purchase today, they’re probably not going to purchase tomorrow. If I don’t even know the right frequency of purchases, I’m going to lose these customers. The goal is actually to identify how do we balance this mindset of I need to send a notification because if I don’t, no one’s going to transact in the short term, versus if I nurture my users, give them the right frequency of engagement, I acknowledge that the use case is defined by the users’ need, and I try to match that need and slowly nurture it upwards. What is then my long-term output? Because if I have to re-acquire these users time and time again, because they are churning, they are leaving, and I’m annoying them. The long-term impact of that means that I’m literally pushing people away to my competitors. They don’t want to be part of my product. So I want teams to actually conduct this use case frequency analysis, thinking through what are the use cases of someone wanting to buy a shirt? What are the use cases of someone wanting to buy ski clothing? Companies like Patagonia actually do this really well. They know that you’re not going to buy mountaineering clothing everyday. Instead what they do is this very wise use case transition into their social media. They give you something that you can consume every day. You can always consume super inspiring landscapes through Instagram. There is no one who can’t do that every day. And so they’ll send you there. They’ll keep you top of mind and they’re providing you with actually valuable content. And what they do there is that they prevent needing to reacquire you every season that their use case is available. And so I would encourage companies to figure out what are my use cases, what are the frequencies for my product, and what can I do to keep my user engaged in a way that doesn’t cause them to churn, that would cause me to have to reacquire them later.

Henry Suryawirawan: [00:36:32] It’s very insightful. So the way I think of why these startups doing it, or maybe even like big companies doing it, it’s because they come back to your 3-month retention growth strategy. You just want to see the numbers keep growing. If you spam them enough, probably people will just either unsubscribe, obviously that’s one thing, or just based on impulsive behavior, or based on some discount or promotions that they give, hopefully the numbers will just increase and increase.

Building Data Analytics Strategy [00:36:55]

Henry Suryawirawan: And the other thing that probably I can guess as well, because you’re saying it’s a long-term strategy, and probably you need to collect a lot of data and events of how users are using your products or your systems. So this is also coming back to the strategy, right? How do you actually build your data analytics?

Crystal Widjaja: [00:37:11] That’s such a good question. And yes, like you said, it’s super hard to do this and be patient for use cases that are very limited. Things like buying a new car. Going to the doctor. These use cases are certainly much more difficult to do this analysis for. But I think what’s important is that they start building the frameworks to understand how often users perform these behaviors. I think I did a Twitter poll, maybe last month asking people “how frequently do you go to a doctor?” I asked, “Do you go every five years? Every one year? Every week? Every time you’re sick?” And the amount of people who said five years was huge. And so when I was talking to the company that I’m advising around healthcare, I was saying, “Look, people are saying they only really naturally use you every five years. What does this mean for your business? It means you’re reacquiring people for five years, or you’re trying to figure out how to keep them engaged for five years.” And so the question is then, do you actually need to reframe the use case? Is the use case of going to the doctor the primary use case, or is there actually a different use case that has a higher frequency? Something like every time I’m sick, where do I go to check my symptoms to make sure it’s not serious? Now that use cases actually far more frequent. What we do is it depends on the product. For companies where the use case is extremely infrequent, then what are the use cases that are adjacent to our product that can be more frequent that we can use to collect data on a more regular cadence.

Now let’s say you have a regular cadence of data. What do you then do from here? So my starting point is always for new users, that have experienced our core product value prop, how frequently are they coming back to our app? Within the first 7 days within the first 28 days. And within the first 34 days, I think . So I’m always looking at, are users coming back to my app? Let’s say in a seven day window period. Are they coming back three to four days out of the seven? That tells me that the use case is every other day. Are they coming back only one or two days out of the seven? Then this told me that the use case is actually closer to once a week. Then we opened the window. In the first 28 days of a user experiencing the core value proposition, are they only coming back three to four times in those 28 days? Then yes, it’s confirmed that we have a weekly use case, which means we should design our communications around that there is a schedule by which the users need us, whether this is a product for coffee, or for creating Jira tickets. There’s going to be an inherent use case frequency that we need to align ourselves to first, so that we know what cadence of communication. We know what retention looks like, so that we can start goaling on the longer term retention rates. So if we know what the short-term retention is, let’s say it’s once a week, every 28 days they need to be seen on our app or our website at least three to four times. Then let’s look at what long-term retention looks like. So this is another cut of data analysis that they have to do. They’ll have to then expand the cohort retention lines, and we’ve all seen these charts where it’s six different lines, all starting on the left hand side of the axis and it looks like a ski slope going down. And the best companies that looks like a smile, where you have retention going up over time. But that’s very rare, to be honest. But let’s say we’re looking at that retention slope, at some point, either it’s weekly or monthly, you’ll see that it starts to flat line. It’s stable. It holds. Let’s say it’s 30%, that most of your users by week four or by week six, end up having a 30% retention rate cohort after cohort. And so now that we know that inflection point at which attention is stabilized, that’s our long-term goal. That’s the point at which we know, users are most likely to retain. And again, instead of goaling on that six week retention rate, what we do is we do that analysis again. And so we’re literally working ourselves to the top of the retention funnel by doing the same behavioral correlation analysis. We then look at for all of the users who have retained in week six, what are the behaviors that they did in the first 28 days, and the first four weeks, and the first five weeks maybe. We’ll look at things like they need to have had two unique days of activity every week for the first five weeks, then correlation with the six week retention rate is at least 70%. We’re always looking for these different metrics. Again, nothing is ever static forever. We have to do this exercise every quarter because our users change every quarter. But this is the analytics journey that we go through.

Henry Suryawirawan: [00:41:35] This sounds like a pretty hardcore data analytics, or maybe even science thing. So for those new startups out there, how can they execute this kind of analysis? Where do they start? Or maybe there are some tips from you, like how can they begin this kind of journey? Is it like collecting data, building some kind of data warehouse capability? Because in the beginning, you probably don’t even have resources to just sit down and analyze all these complex behaviors.

Crystal Widjaja: [00:41:58] That’s true. I think it always, like you said, they don’t have the time. To be honest, this isn’t actually a very challenging data science problem. I think it sounds like a very challenging data science problem, especially for a company that has only been goaling on retention itself. Because it’s such a different way to look at the problem. Instead of saying like the standard metric is 3-month retention rate, let’s just look at that. That’s super easy. Every data platform has that built-in. What they don’t have built-in though is the founder’s proprietary hypothesis. That’s actually where I always start. I always start by asking teams, what do you think drives growth? What is the aha moment? When does it feel like a user is gaining a super power by using your product? Is it when on Facebook, for example, you find out what a friend is doing after 10 years of not talking to them. That’s an aha moment for you. And so how do we then translate that into metrics that we collect? There are problems, absolutely, where a company that I’ve been working with doesn’t collect any data. They don’t have any behavioral data. Then that’s a problem. But that’s why we always start with these hypotheses around. What do you think are the magical moments in your product? What are the behaviors that you would say, okay, I know someone is experiencing the aha moment on our product, because let’s say for Pinterest, for example, while they always come here to create and plan a new design project. That’s how I know someone has felt the aha moment of Pinterest.

With Gojek, it was actually a user comes here when they know they need to get something done that involves someone bringing something over a distance, cause we are a logistics platform. For us, a user has felt core value when they use us for transportation in the morning. The morning is when you are in the most urgent rush. You need reliability. You’re not going out to meet your friends. You’re going to work and you need to be on time. You need everything to be reliable. That’s how we knew that the aha moment existed for these users. And that actually did correlate with some of our long-term retention rates. It just wasn’t necessarily causation. And so we would know if someone reached that retention rate level, if they were able to have that experience on our platform. So what I would encourage companies to do is first, figure out what those hypotheses are. Some people have a very power user mentality of their own products. And they’re like, “Oh, they’re only experiencing core value when they’re using 90% of the features.” That’s not really true. There’s a shallow use case that exists where someone’s using the product and they’re like, “Oh yeah, I like this. This works for me.” What are those moments? And then I would encourage the teams to then figure out and track on a per user basis, when are they using the product in that way? What day is it? What time of day is it? How frequently have they done it in the past seven days? And I have done this analysis for teams in literal Google Sheets. This isn’t anything super fancy. If you know how to use Google Sheets, if you know how to put all your users in one column, if you know how to figure out the goal. You put a zero if they didn’t do it, and a one if they did. That’s all you need to get started.

Henry Suryawirawan: [00:44:49] I really like this framework. So come up first with your hypothesis. So the founder’s hypothesis of what those aha moments could be for the typical users. Either then you build some kind of data analytics collections in your applications or your systems or whatever that is, or even you could just start by interviewing qualitatively with some users, understand their behaviors when they use your product. But I really like it that you actually start with hypothesis rather than spam and see how it goes.

Crystal Widjaja: [00:45:15] Exactly. I mean, there is a universe of behaviors that we could look at. But what’s best is usually we start out with our own personal experience. Honestly, I think Evan (Williams) told me this, our old head of research at Gojek. He was saying that you don’t need thousands of people to respond to a survey to get a good enough hypothesis. The more people you ask, the more specific your answers become, the more specificity you gain out of this data. But the difference between 30 people and a thousand people is really just that, specificity. You still get the overall same theme, the same insights, the same ideas behind the hypothesis. But again, just because you have 30, doesn’t mean that it’s invaluable data.

Gojek Story [00:45:54]

Henry Suryawirawan: [00:45:54] So let’s go to your, you know, go back a little bit in time to your Gojek story. Interestingly, you were the first data hire in Gojek right? Can you share with us, what was that challenge, in the beginning where you were the only person?

Crystal Widjaja: [00:46:08] Oh gosh! The first part was actually trying to make it clear why the data was so important. And so I would look into our databases. I think at some point in time we had archived our customer table, and suddenly everyone was a new customer. Looked at their data, I was like, “Oh, that’s weird, why do we have thousands of new customers today out of nowhere? There’s no marketing campaigns happening. What’s going on here?” And so I reached out to the engineers. I said, “Hey, looks like there’s a new customer table. You did know that I was counting new customers based on that table. And now that table is no longer there.” And they’d look at it and they’d be like, “Huh, you’re right. Oh, well. " And they would just continue on their day as if that was not a problem. I think it was really the first part was creating a shared sense of urgency. And so I would tell them, “Look, I know you worked really hard to release this new redesign, but unfortunately I cannot tell you how many new people are using the platform as a result of it, because of the way that you have structured the data now.” I think by providing them those insights time after time, my goal was to make it clear what their impact was, based on the engineering work that they were doing, based on what they were delivering. And I could only really do that if we had a logical database of data. And so I think they quickly realized when I told them then, now that we’ve made this error, I can’t actually help you do your job. I can’t actually help you see the impact that you’re having. And that actually hit them much harder, then they were like, “Oh wait, I’m sorry. What do I need to do to fix this?” So I think the sign of a good data team is that people care. Because I think if your data is a mess, if no one respects the databases, if no one’s willing to fix the problems, then the real issue is that people haven’t been able to realize their impact through data. We always were very careful to make sure that when we use data, when we ask someone for a good database, when we ask someone to give us access to data that they’re collecting in a new feature, we make sure that one of the first things we do is we say, “Hey, thank you for giving us this data. Now what we want to show you is the impact you’ve had on the company. Your feature has been used by 20% of users. The users who use your feature are more likely to actually be stronger power users on the platform.” People want to hear these things. And they want other people to know that they are impactful as well.

The best way to do that is through the data, through data telling you itself that you are an impactful part of this company. And so I always started out with making sure that the data was used in a way that incentivized people to keep the data clean and organized. And then secondly, that it helped people make better decisions. So when people would create new experiment groups, or try to do a redesign, then I’d help them out and say, “Hey, if you’re launching this new feature, if you really want to know how impactful this feature is and whether or not you spent the right resources on it, and whether or not you should spend even more resources on it moving forward, why don’t we create a controlled experiment?” So helping people to set up their randomized experiment groups. Making sure that they felt confident about what their investment and of resources was really paying off in terms of return on those investments. That I think helped us become a very data driven organization, especially at Gojek. We definitely didn’t start out being data driven. We started out with a Post-It note with a symbol for what was a completed booking and what was not a completed booking. But we got there. And I think that it’s really to the effort of the team knowing that we had a lot of opportunities, and there were a lot of places where we could make the wrong calculated risk. And so we wanted to be absolutely confident. We could only really do that through the data.

Henry Suryawirawan: [00:49:35] Super interesting story definitely. You also grew the team from zero, or yourself only, to about 200, just now you mentioned before you left. So what are some of your critical hires or what are the things that you hired in the beginning in order to be able to scale that big?

Crystal Widjaja: [00:49:50] That’s a really good question. So when I first joined, the first person I realized we’d need was someone to help with the automation. When I first joined, I think someone was executing a SQL query every day. Around 5:00 PM, they would export a CSV. They would email the CSV manually to someone else who would then copy paste it into a bigger worksheet on Excel. And then that person would put it in the graphs, would update the charts, and then send that out again manually. So this whole process that probably took an hour and a half of everyone’s time, and it was happening every single day. So the first thing I did was hire someone named Yoim, who’s still at Gojek, and he’s fantastic. He was able to say, “Okay, these are pretty basic reporting. We can just put this into Tableau. We can create a Cron job report.” And that way, we can actually cut out that first person who was manually writing the SQL queries. Now it’ll just go directly to the person who’s creating the visualizations.

So it was just piece by piece. We didn’t come in and we didn’t say everyone stop what you’re doing. We’re going to completely revamped this process. No. We went step by step. I think that’s important. Making sure that we actually do the process ourselves to see what is meaningful, what’s not meaningful, what can be accelerated, and what are the edge cases in this behavior. Because there were cases when… What happens on the weekend? Someone’s still doing this on the weekends. What happens on Fridays? What happens on Mondays? Are you doing this new chart for all the weekend days, or is someone still doing this every single day, even on the weekends? So understanding that process and coming in and taking it down step by step, I think it was the first strategy.

The next part was then, okay, if we’re going to be scaling to new databases, we don’t actually have access to all of the data in one place. We need to create a data lake. And so I hired, admittedly, a bunch of people from XL Axiata. And they were great people. I always credit Johanes, Fadli and Dimas as three people who were key hires, who we could not have done this journey without. They helped me transform the data lake from our Postgres database to a BigQuery database in record time. We were scaling so fast that when we decided to move all of our MySQL and MongoDB databases to Postgres, it actually only lasted us around eight to nine months, because we were growing so fast that the queries were no longer running on time. We had to create some more sophisticated architecture and orchestration, and we had to very quickly move to the petabyte scale database and Google.

That was I think the primary initial part of the journey. It was hiring out someone who understood the business process to help with automation. Next step was creating a team for the data lake to move all of the data to one place. And then after that, I think then comes the tactical hires. Then it was bringing on people with statistics background. Someone who really understood how to make a meaningful dataset into an actionable, statistically significant and rigorous analysis that would direct us towards optimal gains in the business. Someone to look at, “Hey, when users do this versus that, this is what the outcome will be. We can project and predict how many users are going to retain in six months.” That’s the beauty of it. And I think Jeff Bezos for Amazon has said the same thing. He says that everything that’s happening this quarter was already set in stone last year. And that’s actually what we tried to do at Gojek as well. Everything we were doing, we were planning for a year in advance. We were building things that were about “how do we retain a user one year from now”. We know what the adjacent user is doing today. The adjacent user being the users who are just starting to understand Gojek’s core value prop. But how do we keep them engaged one year from now? Are we building products and features that will allow us to gain access to the next market? As we expand our services, as we expand to different types of consumers. In the beginning, it was primarily tech savvy, young entrepreneurs who were using Gojek. But now we were, helping the “emak-emak” who are going to the Farmer’s Market or going to the wet market, buying and selling groceries there, and just wanted a way to bring all of their groceries home. And so for us, it was really, how do we then start bring in more of the qualitative data? Helping build the research and analytics team there with Ramda, who is amazing at leading our research efforts. And then marrying that with the statistics. Making sure that, okay, we may have these qualitative insights, but they are statistically rigorous in the way that we are defining them and projecting our growth moving forward.

Henry Suryawirawan: [00:54:09] Really some great tips there. I mean, super interesting journey from nothing out of, even just one person, and now it can scale for million of transactions per day. I have one question though from your story, for example in the beginning, you have to get the buy in from the other teams, either engineering or maybe even the product team to actually know that data is actually important. And then now, the data has become the blood of the company. But I’m sure in the process, throughout the journey, there are some challenges. And I just want to understand. There are many companies out there that I see is that their data strategy is actually first, it builds as a silo. Or secondly is that they just don’t interact enough with the product team or the engineering team to say that, okay, we are rolling out this new features. I probably will need these kind of data in order to come up with this kind of statistical or analytics kind of a model. So how do you actually do that in Gojek? Do you always have presence whenever new features or new products are being designed, and instrument that in the engineering side?

Crystal Widjaja: [00:55:05] I personally live and breathe the philosophy in this book called “Crucial Conversations”. It’s about how to get to the right objective. You have a goal, however, opinions differ and emotions are very charged. Everyone thinks they have a right way to do it, and their reputation is at stake. And so for me, I always thought about it from vantage point of I have a goal. My goal is for the company to be as effective and data-driven as possible. I want long-term for data to be the founding factor by which we make decisions, and by which we determine who is successful and who is not. If that is my goal, what needs to happen? I need to get that, like you’ve said, the buy in of not just the engineering team, but also the product and business teams. To get that buy-in, I have to recognize that I don’t actually have all the answers. There is that ego part of me where… do I have the best practice and perfect standard of data instrumentation that I would like to see? Absolutely. Is it realistic for the teams to implement it at this time, given everything that’s happening? Maybe not. And I think I always have to be cognizant of the fact that in startup world, you do have a million things happening, and not everyone is going to get the perfection that they want. The product team itself is probably not releasing the best version of the products that they actually want because of limited time and constraints. So I think the first point is to just be a realistic. I don’t know if that’s like an underrated advice that I can give. But I actually think I don’t see a lot of data teams sometimes being super realistic about the capacity of the team or the expectations that they have. But hey, if you have a standard, that’s great. Get as close to the standard as possible. But also keep in mind that pushing against some of these people who want to be able to meet the demands is not possible. And in fact, it’s detrimental to their ability to build the product itself. And you want the product to be excellent. So instead, what I do is, create the standard, make it super easy and available and accessible to the teams. We created this in Confluence, where we had essentially a sample data model. We had an event tracking template, which I’ve actually published a blog post on, and provided to the rest of the world. And at least made it clear, made it super easy. They didn’t have to have a meeting with us in order to figure out what does the data team want. The standards were all already there and the documentation.

The next thing is then to just create plans. The data is not always going to be tracked correctly. Sometimes there’s a typo, or someone used created_date instead of date_created. And their ETL is no longer automated in our systems. Those cases happen and I think what’s important is to make sure that there are disincentives for that behavior. For example, they wouldn’t get their pipelines automated in the first day of their feature release, but instead we had, not necessarily a retribution policy, but we made sure that there was that disincentive of I’m no longer going to get access to my data as quickly as it could have, if I follow the protocol. But instead I’m going to have to work with the engineering and BI team to make sure that pipeline is created in a custom way for my feature, and I’ll get that data in maybe 5 to 10 days instead. So I think making sure that we still service the team. We don’t completely blacklist them from the data access itself. That would actually be detrimental to the overall strategy of the team. If we immediately said, “Oh, if you’re not going to follow our processes, we’re never going to give you data.” Then everyone would just be like,” I don’t need it then.” So I think it’s about just giving people an easy way back to the golden era of data. How do you give them that lifeline or that path to goodness again? We try to make it super easy for teams to integrate their data. And if something happens, we give them the benefit of the doubt. We make sure that we’ve done what we can to bring them back to the data cleanliness side. But beyond that, I think it’s really just being a practical person, being reasonable with those expectations, and making it as easy as possible for them to follow.

Henry Suryawirawan: [00:58:49] The way I see is, it’s like you implementing the tips that you mentioned in the beginning. You are doing this behavioral analysis. How do you get the other teams to have the aha moments to see, “Hey, actually, this data is important and let’s work together in order to achieve some kind of insights and aha moments for our users.” So I think that’s one of interesting that I listen from your conversation.

Crystal Widjaja: [00:59:08] You are absolutely right. To be honest, I had not actually thought of it in that way. There like you said, there was a startup moment for the data to be correctly architected, for the teams to put the right instrumentation, the right definitions and naming conventions. But the real aha moment is, “Oh my God, I launched my feature and first day my dashboard is automatically created! That’s amazing!”

Generation Girl [00:59:30]

Henry Suryawirawan: [00:59:30] So let’s move on to your other activities, which is about Generation Girl. So you co-founded this nonprofit organization, right? So can you tell me, or maybe the audience here, what is Generation Girl?

Crystal Widjaja: [00:59:41] Yeah, great question. Generation Girl is my selfish plug to our goal. I think actually many of your listeners probably know this as well. If you are a woman, if you have a little sister, if you have a niece, you know that she doesn’t have quite as many role models as a male would. There aren’t as many people in society that say, “Yes, this girl’s first hobby should be with computers. Let’s give her a Linux based laptop and see what she does with it.” That’s not a common vantage point for a parent or for our culture. And I think that, what’s missing is, I don’t see it as negative intentions. It’s not a problem with people wanting to be discriminatory. It’s just a reality that in culture, we don’t have the same examples for girls to go into STEM fields. So with Generation Girl, what we’re doing is we’re trying to normalize the idea of women and young girls being part of STEM fields <Science, Technology, Engineering, and Math>. And we first started out with focusing on the 12 to 16 age group. Our goal is to give them a safe, inclusive, and very confident, inspiring space for them to fail, and to try, and to be successful in STEM. It’s success for us if these girls come into our programs, which are free, they’re usually one week long for every subject. So they can be up to, I think, five different weeks where they’re learning how to code and program a moving robot. They’re learning how to build a website. They’re learning how to create a UI/UX prototype and wireframe. And so for us to give them the opportunity to try these things. What’s important for us is that these girls feel the agency, and feel like they have the opportunity to say, “I like this field, or I don’t like this field.” If they come out of these programs saying “I don’t actually like STEM, I’d actually rather do something in literature.” That’s great. But what’s important is that we have given them the chance to try this themselves, to feel like they’re not being intimidated by a bunch of people who are over their skillset.

This is actually stemming from, no pun intended, stemming from a study within Carnegie that showed that women who were given the opportunity to do a pre-class for the CS106 seminar, they were actually graduating at the same rate as men. Most women were actually dropping out without this pre-class, because they felt like they didn’t have the same skillsets as the men in their class. They were falling behind. They weren’t feeling prepared. And just culturally, a lot of women are told at a young age that “You need to be perfect. You need to be prepared. You can’t fail.” Whereas young boys are often taught, “Go ahead, fall down, pick yourself back up, do it again.” And so we’re trying to create an environment that gives them the best opportunity to see good examples that they can look up to, who are young women themselves. We partner them with mentors, who are typically in university who understand them. Because to be honest, if I tried to mentor these girls, one-on-one, I don’t know the latest TikTok stuff, and I don’t get it. I’ve tried. So we thought it’s better to just pair them with someone a little closer to their age. The goal of it is to give them literal generations of role models, not just university students, but we have women, who are fantastic leaders in STEM that helped talk to them about their role. What is it like to lead a team of 20 engineers as an engineering manager at Quora for example. What we really want to do is give these women the opportunity to try STEM, to see whether or not they like it. And then to give them opportunities to continue learning if they choose to do so. We have programs with Tokopedia, with Traveloka that let you partner with someone in their engineering team and build a project over the summer. So that was last year. We were actually looking for mentors right now. So if you go to generationgirl.org, and you’re a university student, or someone who is able to provide mentorship to the girls for five days a week, couple hours, no big deal, but you get to help hundreds of kids learn how to use and be proficient in STEM. It’s a good deal.

Henry Suryawirawan: [01:03:41] Wow! This is such an inspiring activity definitely. Is it just for Indonesia, by the way?

Crystal Widjaja: [01:03:46] It’s primarily focused on Indonesia. We did have a couple of students though, come in from Singapore or maybe even Malaysia, I think. So if you’re interested, come join. We do have half English classes, and half of our other classes are in Bahasa. And if you are looking to be a mentor, male or female, you realize your mission is to help give these girls confidence. You should apply to be a mentor as well.

Henry Suryawirawan: [01:04:08] What are some of the inspiring stories that have come out of Generation Girl?

Crystal Widjaja: [01:04:12] Oh, it’s just been really crazy to see the level of engagement. There are just really are not that many opportunities, specifically for women who speak Indonesian to come in, and take these classes for free. And so we had, I think in our first year, a student who flew in, from I think Medan, or somewhere else very far, flew in to Jakarta, stayed with her aunt and uncle for a week, just to join our program. We had another student who initially wanted to do a journalism major, and after a program, she was like, “I am going to be an engineer. I’m gonna learn to code fully. Build my own product.” And so she completely changed her idea on what her major should be. We had students from Indonesia who are now going to university, join one of my last data science classes. And I had two students, one that woke up at 4:00 AM, and one that woke up at 5:00 AM her time, because they were like, “I really want to join.” So I’ve just been really blessed to see that people want these programs that they want to attend. And so we’re very careful about the mentors that we select. We really want people who are going to be excited about giving these opportunities and sharing their knowledge with them.

Henry Suryawirawan: [01:05:19] So how do you see these women actually in the industry, and maybe specifically in Indonesia. Are they getting the same opportunities? Or are there enough supply of women in technology or STEM in general? Are they brave enough to give it a try now? Seeing all these startups starting to bloom, and Generation Girl, probably there are other organizations that help them. How do you see the trends of these women specifically in Indonesia?

Crystal Widjaja: [01:05:41] I think in Indonesia in general, it’s definitely gotten better. I think Indonesia in particular is a place that is, I think, pretty progressive and forward when it comes to women having children, needing to take time off, creating good integration programs for when they come back. I think Indonesia has always been actually much better than the US at this. Where I think we can do better is in investing in creating opportunities for women to be part of the workplace. We all know that more diverse teams create better products. Imagine we’re creating a healthcare app and all of the engineers are men. You’re going to miss some pretty vital parts of a woman’s healthcare needs in a team that’s just filled with men. And in any case, every product is going to be hopefully 50-50. Men are going to be using it. Women are going to be using it. A more diverse team is only going to make the product better. If teams are really committed to creating the best possible engineering team, that gets the best possible product teams, then they should be investing in opportunities for women to be part of that equation. If that means, like you said, having to create bootcamps, or create programs that make it easier for women to be facilitated into the program, then they should be doing that. If the talent pool is a problem, then what is the company doing to help solve that problem? Because it’s really a cultural endemic issue that there aren’t enough women in STEM, and therefore, we don’t hire them, and therefore women never join STEM. It’s actually a self-perpetuating problem that we don’t invest in them. We don’t make it easy for them to be part of those communities. And that’s why we lack the talent pool for them. And so things like supporting programs like Generation Girl, we have amazing sponsors, from Sequoia in Singapore, Tokopedia, even XYZ. We have great sponsors who realize that this is a problem, and commit time with us to make sure that we can afford to bring on hundreds of these girls into our programs, so that talent pool pipeline can be expanded. And it’s not going to happen until they decide to invest in it. Gojek for example has their own Ruby bootcamp that we partnered with them for our Electives (program) this season. And they made sure to put in the resources to make sure that women can join this program, that they will learn Ruby as a programming language, and they’ll be able to be proficient and hopefully part of that talent pool from next year.

Henry Suryawirawan: [01:07:58] So I hope to see this kind of organization, nonprofit organization, starting to pop up in multiple countries within Southeast Asia at least, because the opportunities are still very big, and some of these women are actually not yet immersed into the mindset of, engineering is a good industry to be in for them. They can perform as equally good as men. And yeah, hopefully we can see or hear some of these inspiring stories coming out of this region definitely.

3 Tech Lead Wisdom [01:08:22]

Henry Suryawirawan: So Crystal, I’ve really enjoyed our conversations, going from VC stories, startups, product growth strategy, Gojek story, and now to Generation Girl. But due to time, I think we have to wrap up the conversation. But before you leave, normally I would ask my guests this one question, which is, what are your three tech leadership wisdom that you want to share with the audience, for them to learn from.

Crystal Widjaja: [01:08:43] I think the first one, actually based on our discussion, is really to goal on the behaviors that matter. Don’t goal on your vanity metrics. Figure out what it is that, not just works for your product, but honestly, for you as a person, like what works for you as an individual, what makes you the most successful. It’s probably not doing your work a hundred percent of the time, or working 24/7. It’s actually probably things closer to getting 7 hours of sleep. Having a maintained diet. Working out. Feeling healthy. Figuring out what those behaviors are that make you the best to your abilities. Figure out what those are, and operate, and optimize for that. That’s probably the first thing.

The second one is probably figure out what you’re not great at. And find people who can actually help you fill those gaps. Like I said, I couldn’t have done all of the data modeling and strategy at Gojek, and gotten it to the scale that it is today, without people like Johanes and Dimas and Fadli. They were people who could help fill the gaps for me. We had a vision together that we were able to execute as a team, because we had the skillsets that complimented one another.

And then, I think the last one, to be honest, is to listen. I think the first mistake I made, as an early manager, was probably not to listen enough, was to think that I knew what the right data model should look like. I knew what the best practices were, and that team should just follow this. I shouldn’t listen to their excuses. But in the long run, it’s actually, that’s not the case. And so being mindful of that, listening and being a rational person, I think is something that I would equate with my success as well.

Henry Suryawirawan: [01:10:19] Thanks for sharing that. So Crissy, where can people find you online if they want to interact with you more?

Crystal Widjaja: [01:10:25] Yes, please. You can find me at crissyw.com, or follow me on Twitter @crystalwidjaja. I’ve been tweeting more. I don’t think I’m going to get the hang of TikTok and Instagram to be honest, because I’ve given up. But I’m still using some of our more traditional mediums, so building my own website.

Henry Suryawirawan: [01:10:40] Okay. So thanks so much Crystal for your participation today. I hope to see you in future episodes, and good luck with whatever things that you’re doing now.

Crystal Widjaja: [01:10:48] Thank you so much Henry. Time flew by, and it was such a joy. Stay safe out there.

– End –