#12 - Singapore's Open Government Products - Li Hongyi

Episode Introduction [00:00:43]

Henry Suryawirawan: Hello, my friend. Welcome to another episode of the Tech Lead Journal with me your host Henry Suryawirawan. Thank you for tuning in and spending your time with me today listening to this episode. If you haven’t joined any of the Tech Lead Journal social media channels, I would encourage you to take a second right now to click on the links in the show notes, where you can find the podcast, either on LinkedIn, Twitter, or Instagram. I’d really love to see and interact with all of you there. Every time you post and share something about this podcast on your social media, it makes me extremely happy. And I do hope that more people will be able to find this podcast and learn from the amazing guests that I have on the show. If you have other creative ideas on how to grow this community bigger, please feel free to reach out to me and share your ideas. It would really mean a lot to me. And if you would like to pledge your support and make contribution to the show, you can do so through our patron page at techleadjournal.dev/patron. It would help me tremendously towards achieving a goal that I’m currently running on the patron page.

For today’s episode, I had an amazing and inspiring chat with Li Hongyi. Hongyi is the Director of Open Government Products, a division of GovTech Singapore, where he leads an experimental team of engineers, designers, and product managers to build technology for the public good. The team has produced a number of cool products that have brought tremendous impact to Singapore citizen, such as Data.gov.sg, Parking.sg, FormSG, Go.gov.sg and Isomer.

In this episode, Hongyi shared with me his journey on how he started the Open Government Products initiative, the kind of challenges that he needed to overcome at the beginning, facing all the challenges within the Singapore government to adopt new technologies. Since he started, Hongyi has successfully scaled up his team, and he also outlined with me his long-term vision for Open Government Products, which includes open sourcing its products for other governments around the world to adopt and implement. He also touched on Singapore government’s challenges in terms of cloud adoption and also hiring engineering talent. Do not miss Hongyi’s explanation on the term “bureaucratic deadlock”, that he beautifully explained as one of the major challenges that he has to overcome when starting Open Government Products. I hope that you will enjoy this episode, and I’m looking forward to hearing any of your comments and feedback on the social media, or you can go directly also at techleadjournal.dev/feedback. So let’s get started right after our sponsor message.

Introduction [00:04:02]

Henry Suryawirawan: Hi Hong, it’s very good to see you. Welcome to the Tech Lead Journal.

Li Hongyi: [00:04:08] Thanks for having me, Henry.

Henry Suryawirawan: [00:04:09] So Hongyi, I know you work quite recently in the Open Government Products space, doing a lot of innovations for Singapore, especially during this tough times in COVID-19. There are so many innovations coming out from your group and also a lot of products, apps being churned out to help people and citizens to cope with this kind of situation. And I really appreciate all your effort and your team’s effort in producing all this to make us feel slightly more bearable in terms of these tough times.

Career Journey [00:04:36]

Henry Suryawirawan: Before we start with all that innovative journey, if you can probably share a few minutes about your career journey, maybe sharing some of your highlights and your major turning points in life as well, that people will be interested to hear about.

Li Hongyi: [00:04:48] Yeah, of course. So let’s see, where should I start? When I was in college, I started out actually as a Economics major rather than a Computer Science major. I was doing some Computer Science in some various other classes. But I guess my interest in tech really kicked off when I got an internship at Google one summer. I was a product manager intern on the Android team. I was working on what’s now like Google Keep and like the Google Play Store. I think that was a really big part for me because the main thing about that internship that really changed my mindset was that, I was really surprised because even though I was just this intern working alongside all these other professional engineers and PMs, no one treated me like an intern. They treated you like a full time employee. They gave you serious work to do. And if you did your research and you put forth proposals, they took them seriously. It’s just an internship, so you don’t have that much time to work. But that really changed my mindset, because I was really amazed that even as an intern, there was so much real work that there was to do. So I started thinking that, “Yeah, I think maybe I should start getting into tech.” Because my previous option was, I was thinking of just going to become an academic and going to do a PhD and stuff". And you spend months writing a paper only for just your professor and maybe your parents read it, like that’s about it. And then on the other hand, if you were working in tech, you would propose some ideas to an app or some changes, and within a couple of weeks, they will be live to like millions of people around the world.

That was a really big turning point for me, sort of recognizing how much there was to do and how much influence you could have if you build the right things. Soon after that I switched doing Computer Science when I got back to school. I went back to Google for a couple of years after graduating. This was a gap year before I came back to serve my government bond. I worked on the infrastructure and image search teams. I guess even though it was quite a few years ago now, I think it still has a very big impact on me today.

First part, I think just recognizing that I had no idea what I was doing when I started out. Like some of the people I was working with, wrote papers that I was just studying the year before in school, so like it was quite intimidating. But the reason why this is really important to me, because Google is sort of very famous for being this full of very smart people and very high functioning company, all of these organizational things figured out. And the thing that was most surprising to me was that even at Google, people didn’t have everything figured out. Like they had a lot of stuff. They ran the company pretty well and they did quite lot of good things and people are generally happy, but there was a lot of stuff which, yeah, if you did some research and made a proposal and looked at it, they’re like, “Yep. That could work. Why don’t we give it a try?” And I think that helped me on a few fronts. The first one was that I realized that if even at Google, people hadn’t got it all figured out, and these people are really smart. At a lot of other places, no one else has got it perfect. And I think that’s one of the biggest things that I realized that you can always do better. I think at a lot of organizations, there’s this sort of myth that the way the organization runs is this very finely tuned machine that has been honed over many years of experience, so don’t mess with it. But if even the geniuses at Google haven’t got it figured out, the likelihood that any organization is immune to criticism, I think, yeah, that falls on its face. And importantly, I think the really big part of it is that I could see what a really well run organization looks like. Because in bad organizations, I think one of the common things that people say is that, “Oh, this is the way it has to be. That’s the way things have to run. There’s no other way of doing it.” And I can see how if you’ve never seen any way else of running, it’s hard to criticize, right? Because bad organizations do run for long periods of time and they do stay relatively successful. Even if people are very unhappy. But because I’ve seen it, because even though it was just a couple of years, even though it was just two years, almost a decade ago now, I still remember that it is possible, that you can run an organization where you communicate clearly, and you treat people fairly, and you try to set people up for success. It’s one of these things where as much as people tell you otherwise it was not possible, it’s not possible. It’s like I have seen it. I have seen it work and I can make that happen.

And to this day, a lot of that philosophy of trying to run a good organization and trying to treat people well and taking people seriously, that is because I remember that it is possible. Maybe I was very fortunate. My manager, he was very supportive and his attitude was very much, “Yeah, let’s see how we can make things happen.” You know if you had an idea, his first approach would be like, “Alright, let’s see how we can make this happen.” I think that was very, very valuable.

After a couple of years at Google, I came back to serve my bond in the government. For the first two years in the government, I really had no idea what I was doing. Like not at all. I had this sort of existential crisis, you know, you go to work, you have meetings, you make slides, but you don’t really know if you’re making any progress. You don’t really know if anything’s being done. And so this is like a weird existential crisis. But I think there was one point which made a big difference, which was we asked to take a look at Data.gov.sg. It’s the government’s data sharing portal. I was supposed to just write a proposal of how we’re supposed to manage like, which vendor to sort of award the contract to. But I looked at it and I realized the proposal, they were just looking at using an open source library, and because I had engineering experience, I just followed the instructions and I put up a simple instance of data.gov. It’s just a standalone. It’s done. You don’t need to pay someone a few million dollars to do this. It’s actually quite simple. You can just follow the instructions. It was a bit of a struggle even then, because the government wasn’t really set up to do things internally at that point. A lot of it, they saw that as like, “Oh, that’s cool. I want you to go tell the vendor to do this.” I’m like, “No, you don’t have to tell the vendor. I can just do this for you. It’s quite straightforward.” So we got it off the ground, I think that was a big thing.

Going from there, and for a while it was just me. I was just one guy in the government maintaining one website, and trying to keep it running as fast as possible. But I remember there was one meeting we were at, which was a big deal where we were in a meeting with one of the ministers, and he was talking about how open data and data sharing platforms are going to be a cornerstone of our Smart Nation plan, and all that sort of thing. And then I raised my hand and I was like, “Okay, that’s all good. But if this is really so important, I’m like literally the one guy right now working on Data.gov. If this is really a cornerstone of your plan, I’m going to need more than just one person. That was where I first got a team. Not a big one. I think we hired two or three people just to start working on Data.gov. But that was the start of the project. And then we maintained, and we built that, and we developed it up. It’s grown pretty big now. It’s a fairly popular site now, so quite proud of it still.

From there to today, that’s five years ago. Just fast forwarding a bit, basically what we did was we officially never had mandate to do anything beyond Data.gov. But the team, we had a lot of things that we wanted to do, a lot of ideas we wanted to push. So we kept building prototypes and proposals. We will make a lot of these little simple prototypes of ideas of things people would try out, demos of things that we could do. And we had a lot of them, like dozens of them. And most of them didn’t see the light of day. But I remember there was one time where the minister from MND was visiting, and we showed him one of our demos, which was for a parking app. Instead of using coupons, you could just have an app on your phone, and it will automatically pay for your parking coupon, pay for your street parking. And it’s like, “Oh yeah, that’s pretty interesting. Why don’t you go take a look at that?” And it wasn’t smooth sailing from there. There was a lot more convincing and persuasion to do, and a lot of things to work through. But eventually after a year or so, we managed to get something launched. It was pretty basic. And people liked it. That’s probably what we’re most famous for today still, like Parking.sg, but it’s actually one of our smaller apps that we do today.

Since then, after we got parking running, that gave us a bit more momentum, and we managed to get a few other projects off the ground. And so we work on Parking.sg, but we also worked on FormSG. So if you fill in a digital form in the government, like at one of those web forms nowadays, they are mostly FormSG forms. It is the digitized government paper forms, as well as some government websites and things like that. We got them running.

And I guess the last big turning point was about two years ago. I was feeling stuck in the government. We were sort of making apps here and there, but it always felt like there was this whatever progress you made, it was against this tide of bureaucracy and like sort of incumbency that just didn’t want to change. And I was thinking about leaving the government, because alright, I’ve done what I can. Maybe I should go back to the private sector. But I decided since I’m planning on leaving anyway, I might as well just write up what I want to do. And so I decided to just put together a big proposal, which I documented from my perspective, all the problems I saw about how government did technology, like all the ways it was hindering us from moving forward, and what I actually wanted, and what I thought we actually needed to do in order to make progress, and it was very specific. We talked about how expensive government infrastructure was. We talked about like our regulations being out of date. We talked about a lack of technical influence in sort of senior leadership, and things like that. And basically the proposal was, “Look, you don’t have to change the whole government, but just give me like a small team, and give me a little bit of flexibility to show you that this can work, and we’ll make something happen. If it works well, then you can change the regulations, and everyone can benefit. If it doesn’t work, you can shut us down. It costs you almost nothing.” It wasn’t approved straight away. It took quite a long time to push, and even to this date, it’s nice to believe that there’s a single incident where people say, “yes, approved”, then you move on. But no, like every week, every day is a fight for another small bit of space on something that you thought you had previously agreed on you have to do. But we got approved. Then we officially launched on July 2019. July 2019 was when we officially existed. That was the last big thing that we’ve done.

We’ve grown a lot since we’ve split out as sort of this independent unit. We now officially exist. So previously all our projects we were running, they were never part of our portfolio. They were just these like side projects that had grown big enough that people had to take them seriously to some degree. But now that we officially exist, we have an official mandate to build these sort of projects. It’s been a big help. The team’s grown a lot since then. We’re about 42 people now. We do pretty good work. We’ve digitized, I think, about 50,000 government forms. We’ve got about 70 something government websites that we’re helping to operate. Maybe like Go.gov. So if you see those QR codes Go.gov.sg that you’ve seen, that’s something that our team built. Yeah, so that’s where we are.

Henry Suryawirawan: [00:12:45] Thank you for sharing that. I think there are a lot of interesting stuff, which I personally didn’t know about as well, and I’m sure there are many people who didn’t know how tough was it to actually start this all innovation within the government. I also like the points where you make, like your experience in Google actually opens up your perspectives, that there are things that are not perfect. Like people are still need to figuring out, like they are smart people, but still, there are so many things that you can innovate, and you can propose, then you can try out and prototype, and things might work, and that’s how I think innovation happens. All these stories that you mentioned and you share, I think it inspired me as well. Like how you managed to tackle the bureaucracy, the challenges, the inertia in the beginning from the traditional way of running things. Up to now which I think, due to this COVID, we can all see this benefits of how your team has started and how your team has produced a lot of things.

Government Challenges Adopting New Tech [00:13:33]

Henry Suryawirawan: But initially when you set up this proposal before you obviously try to leave your position, before you start, what problems were you proposing to solve if you can share?

Li Hongyi: [00:13:43] Yeah, of course. It’s pretty straight forward. So the first one was that, the most obvious was that our infrastructure was just too expensive. Basically government infrastructure costs between, I would say ten to a hundred times more expensive than private sector infrastructure easily. So you’re talking about a few thousand dollars a month for it. If you get a CPU on AWS, a single core, it’s maybe 50 bucks a month for a decent sized computer. In the government, that same computer can cost you between $500 to $5,000 a month quite easily. There’s a lot of reasons for this. There’s like sort of security, and like scale, and different assurances, and all these other things. But basically, it’s very expensive. And the reason why we have so much digitization, and the reason why tech can take off, is because computers are cheap. If computers are expensive, if a computer costs more per month than a person, there’s no reason to digitize. And so a big part of that is showing that you can use these cloud services, and it will function just as well, and like them being cheaper, it doesn’t mean they’re worse, you can still secure them. Because you really need to get the cost of infrastructure down. If computers cost a hundred times more than it did today, then you’re basically doing computing like it was in 1990s, 1980s. And you can’t do big data. You can’t do AI, if CPUs cost $2,000 a month, $5,000 a month. We just can’t.

Another thing was the regulations. A lot of the regulations were out of date. It’s not that they were bad. One of the good things about the government is that, it takes like best practice and codifies it, so that it gets done very consistently. But the problem was that the best practice was best practice in computing like 15 years ago. It was still talking about setting up firewall rules, having a web server, having an application logic server, having a database, all by firewalls between them, and very strict architecture. As anyone who develops code nowadays, like that’s not how you build the web service. You have Elastic Beanstalk, and you will have a CDN, and you maybe even have a single-page application where you have most of your logic client side as opposed to server side. But because tech moves so quickly, the regulations just hadn’t caught up. And so the regulations essentially not only were not supportive of new architectures, they actually enforced old architectures, which prevented progress. It’s not that we’re moving too slowly, but that by design, we were enforcing what was best practice at the time, but it’s now obsolete.

I think finally, the most important one was that we weren’t focusing on recruiting. I think that was an issue. As everyone in tech knows, the big battle in tech is not for markets, but for talent. For all the talk about, you subsidize, and you try to go into a market or develop product, all that is just a manifestation of who has better teams, because the people with the best teams make the decisions and know how to win. That’s why tech companies fight quite a lot for the best people. And as a result, they get very good people coming in. The government just wasn’t interested in recruiting. It was not that we had bad people. Government officers they do good work, and they handle it. But if you had the best cybersecurity people, they were no longer coming to the government. They were going to Google and Amazon and in fact, Microsoft. Again, if you’re the best cybersecurity person in the world, why would you work for the government? Similarly, if you’re the best infrastructure person in the world, why would you join the government? You could join Google, Amazon, and get paid like literally ten to a hundred times as much, and have a great work environment. Even at the low level, like just basic things, like bringing in really good mobile developers, really good front end engineers. There’s a lot of really good young talent, and you need to focus on renewing. Again, it’s not that the people in the government were bad. It’s just that because everyone else is stepping up their recruitment efforts, if you don’t step up accordingly, you’re just going to get left behind. Maybe 10 years ago, our recruitment strategy was okay because the competition wasn’t so fierce. But now, anyone who’s decent is going to go through the people, fighting the hardest for them, and you have to keep up. You can’t just hope that your same strategy works out, because these are not small employers.

So yeah, those were the main things we wanted to fix, like infrastructure costs, regulations, as we mentioned before, lack of technical influence in decision making, and then finally like we needed to improve our talent recruitment. Those were the main things we wanted to do.

Henry Suryawirawan: [00:17:04] Yeah, I can see the talent fight actually is pretty big, especially as more and more investment coming to Singapore. All these big startups. All these giant MNCs coming to Singapore, set up Asia Pacific Center of Excellence. All these obviously is a fight for talent, and I’m sure government will find it some kind of a challenge as well to attract, first of all. And like you said, the best cybersecurity, infrastructure people, developer out there, they want to work with more progressive and more advanced technologies.

Li Hongyi: [00:17:29] Yeah. Even the good people that you do have, they’ll start leaving. Because it’s a fight. You don’t own your employees. If other people start fighting harder for them, you’re going to start losing them, and you need to fight harder to keep them.

Biggest Hurdles Hongyi Overcame [00:17:38]

Henry Suryawirawan: [00:17:38] So when you proposed this, you mentioned about a lot of bureaucracy, and maybe traditional mindset. Maybe what are the biggest hurdles that you need to overcome during that time? Like when you propose it and get it approved, sort of like, “Yeah, you can do this now.”

Li Hongyi: [00:17:51] The biggest hurdle I would say is… There is this very interesting phenomena, which I call “bureaucratic deadlock”. It’s hard to describe, but let me see, let me do my best. Basically there’s an idea that you want to try. Because you think it will create some benefit. And they’ll say, “No, we can’t do that for everyone. That’s too risky. You can’t change everything all at once. Everything’s working fine. Why do you want to do this thing?” And you’re like, “Okay, so why don’t we do an experiment? We’ll do a small experiment. We’ll test to see if this works.” And they’ll say, “No, no, no, we can’t do the experiment. The policy stops you from doing the experiment.” And then you say, “Okay, let’s change the policy, so that we can experiment.” And they’re like, “No, no. We can’t change the policy. We don’t know what effect it will have.” And you can see the deadlock forming very tightly, right? " Like every step of it is very logical, but it forms a deadlock. And you get this bureaucratic deadlock, and this is the biggest challenge. Because you’ll propose that, “Hey, why don’t we form an experimental unit?” And they’re like, “No, we can’t do that. We don’t know what effect it will have.” We said, “Alright, why don’t we just test it even smaller? We’ll just do a small test, and they will give a small amount of reasons. “Oh, we can’t do that. The policy doesn’t allow for this experimental unit to exist.” “Well, okay, let’s change the policy to allow the experimental unit to exist.” “We can’t do that. We don’t know what effect your unit will have.” And even for an experimental unit, almost in a very ironic fashion, this was the biggest concern. It wasn’t that we would cost too much. It wasn’t that we weren’t good at our jobs. It wasn’t that. Because we had shown success in the apps we had. It was the fact that just it was different.

My dad told me a story once about when he was in the army. There was a shortage of boots for his company or something like that. So he went to the quartermaster, and he had to go up to the quartermaster to try and get some more boots. And he was told by the quartermaster not to bother putting in a request because they were out of stock. There was just no stock overall, so don’t bother putting in a request for boots. But when time came to order new boots, for the quartermaster to order new boots for the army as a whole, they’ll look at the outstanding requests, and because there were no outstanding requests, there was no demand for boots, and so they didn’t order any new boots. And so you got stuck in this deadlock. It’s a very simple illustration of how you can get stuck. Because something is not immediately obviously beneficial, you then just sit there, and do nothing, and that’s where we got stuck quite a lot.

Henry Suryawirawan: [00:19:41] I can see why it is becoming a deadlock. Maybe if you can share as well, what made the deadlock got resolved. How do you break it?

Li Hongyi: [00:19:47] Okay. So let’s see, how do we break the deadlock? Basically we have to break the deadlock by like just freeing up some extra resources here and there, and doing things out of band, if that makes sense. So, what we did was, all the projects that we did to sort of prove that this works. They were side projects. They were not endorsed. They were not blessed. We had to prove that we could get things done, and we would work on projects, and show that they were successful even without having them officially funded or resourced. We figured out where there were pockets of funding for like small bits of innovation. For example, most of the Parking.sg, the prototypes were built on less than a thousand dollars. There was no big funding or anything. It was really just finding bits and pieces. Similarly for FormSG, we just used an open source library, put it together and did something from there. And that helped loosen the knot a little bit. Like it helps show, okay, maybe there’s some value here. We went to find other examples of similar experimental teams in other countries, as well as in Singapore. So experimental teams have always existed. They’re usually small, so it’s not like they’re very well known. But we went to find examples of them to show that maybe this isn’t as novel as you thought. This isn’t as scary. I think a lot of it was just trying to explain to people the deadlock. And I explained the deadlock, and being like, “What do you want me to do? You want me to just go in circles?” It’s just persistence to a very large degree. Because it’s a pain to do this, most people don’t bother. You just need to be okay being annoying. You basically just need to keep meeting with people, and pinging them, and going around, and having people realize that “Okay, this is not going to go away. He’s going to keep trying this until we do something.” And just having some degree of persistence.

Henry Suryawirawan: [00:21:00] Yeah. I agree with that. Like pestering people, trying to follow up every now and then.

Open Government Products [00:21:04]

Henry Suryawirawan: So maybe this is also a good time for you to share about your team, which is Open Government Products. How does it differ with other government technology initiatives? Like we have GovTech in Singapore as well. And within GovTech, we have a lot of subdivisions. I may not be familiar with all of them. How does it differ? Your team, Open Government Products, with the other?

Li Hongyi: [00:21:23] So firstly, OGP, Open Government Products, it’s a division within GovTech. So we’re not entirely separate from GovTech. We are a division within GovTech, but we are an experimental unit. So we still belong to GovTech, but we have some flexibility on certain regulations, and we run our own comms system and email system, and things like that. But we’re still part of GovTech just to be clear. But the key differences are, the way we think about it is that, GovTech is meant to take the government and digitize it. So you take the whole government and digitize it. OGP is meant to be an experimental unit, which means we take the other direction, which means that we start from a modern tech company position, and then try to figure how to backwards engineer that to the government. So you are going two directions. You’re going from taking government and try to digitize it. And then taking a tech company and try to “Governmentise” it, if that makes sense. And this is very useful, because if the government wants to transform, it’s very hard for the whole government to shift to something, like we said before, if they don’t have somewhere to test the new ideas out. Because otherwise you get stuck in a deadlock, where we can’t move because we can’t, it’s too risky to move everything all at once. But by having a small space where things can be tried out, and new ideas can be tested, it reduces the risk for the broader government to adopt them. We are very intentionally small in size, so that the risks are very small, and then we try things out. And as we figure out what sort of practices work, how we do cloud security, how we do product testing, and things like that. That can be more easily mainstreamed with less bureaucratic fear, if that makes sense. So that’s the role of us vis-a-vis the rest of government. It’s not that we are competing. It’s that we are symbiotic, where you need someone to do the massive thing. You need someone to do the trying out ideas, so that it de-risks the mass adoption.

One of the things that we do, as a result of this, is that compared to other government agencies, we are far more proactive in figuring out projects. So most government agencies, the projects are big established projects. An agency needs a sort of management system for this or that, and then they’ll come to GovTech, and GovTech will be, “Okay, this is what we can do, blah, blah, blah.” OGP on the other hand, we don’t wait for people to come to us for projects. We go. We look around at the government and see what people are doing. Build prototypes and then go to them with those prototypes and say, “Hey, why don’t you try doing this?” Almost every one of our apps has worked like this. We run a hackathon every January, where we go visit all the different government agencies, then we sit down, and for one month, we built different apps and prototypes. And most of them don’t see the light of day, but a lot of them actually do turn into real products. So Go.gov.sg, for example, our link shortener. The QR code generator that you see all over the place, that was from a hackathon. It’s not something that someone came to us and told us to build. It’s something that we went around and as experimental team, try to identify what the opportunities are, and try to build those out. And I think similarly along those lines, our focus is very much more user driven rather than top down. So for the other government agencies, there are sort of leadership discussions and meetings, and figuring out what the strategies would be, and then they direct that to the engineering teams, and engineering teams build that. OGP on the other hand, because of our proactive, experimental nature, what we do is we go around, we talk to users and see what users want and need. Try to solve problems from a citizen centric perspective, and I take those, and then bring those up to the government leaders, and be like, “Hey, we found this opportunity that we think will really help users. What do you think of it?” So you’re going the other way. You’re going into the ground, figuring out small opportunities proactively identifying them, and experimenting, and seeing what can work, and then going from there. Those are the key differences.

How Products Get Approved [00:24:08]

Henry Suryawirawan: [00:24:08] Thanks for sharing the process. So I have one question mark. When you have all these ideas, prototypes, you mentioned a lot of them through hackathons, or talking to the users, and all that. What made the government decide to actually, “Okay, let’s maybe formalize this. Publish that to the citizen. Do you need to do some kind of tracking with the users? Like early users, any good stats? How did you do that? How did you get the product approved?

Li Hongyi: [00:24:30] Yeah. So this is actually a big part of how the team operates. The team doesn’t operate on a project basis. It’s not like we go do an individual project, get the project funded, and go for it. The team is funded as a whole. So, we’re just block funded. And then our job is, given that we’re funded as a whole, to maximize our impact given this funding, which means that it is our responsibility to make sure that the things you’re working on have maximum impact. But it has the benefit of maybe we don’t have to convince people for every single project idea, as long as we can make it happen. A lot of this is just going piece by piece. It’s not like we will start out, identify the opportunity, and say, “This is going to be the next big thing.” We don’t know what the next big thing will be. To give you an idea, I think from a hackathon at the start of this year, we had about 25 different projects come out of it. Of the 25 different projects, I think maybe 3 of them are now live as real projects, which is pretty good. And it’s not that we chose the three very deliberately. It’s that we did 25 projects. We tried a whole bunch of different things. And the ones which seemed to have more promise, we kept doing the ones, which we tried for a bit and didn’t really get very far, we put on a shelf, and maybe we’ll look at them next time or later. You sort of have this backlog of new things that you’re trying. And as things become successful, you naturally grow them. And if they stop growing it’s okay, you just put them on the shelf and you shut them down. It’s a very organic process. It’s not so much about someone decides that this will be a big idea. It’s like projects, which naturally grow, naturally get more resources.

Examples of Open Government Products and The Impact [00:25:38]

Henry Suryawirawan: [00:25:38] I see. So maybe for those of us, the listeners, who are not aware of some of the innovations coming from OGP, I know you mentioned some of them already throughout this conversation. But maybe can you share with us here? What are some of the flagship products that you have published? And maybe give some stats how impactful those have been to the citizens, the public and the users.

Li Hongyi: [00:25:57] So let’s see, where should I start? The product we’re most well known for is Parking.sg. We launched that a few years ago. And it’s still got a lot of users. We have about 36 million parking sessions to date, since our launch about three years ago. So about 12 million parking sessions a year thereabouts. We have about 1 million parking sessions a month. I would say more than 80% of the cars in Singapore use it. At least, we’ve seen them on the app. And it’s pretty good. But Parking.sg now is actually one of our smaller apps. It’s got maybe I think one person working on it at the moment, unfortunately. We would put more, but we have too many projects to do, so we have to reduce the head count we had on it. And we’ve given $9 million back to drivers. So that’s one of our proudest things where we were like, the whole point of doing it was to make sure that people didn’t have to overpay for it. And we would give people refunds, and we refunded about $9 million so far. Not a lot in government terms, but still, it’s a good win for a couple people’s worth of work.

The biggest app we have right now is FormSG. So FormSG is Google Forms for the government. There’s Google Forms. There’s Typeform. There’s Paperless Post. There’s a whole bunch of these web forms that people do nowadays, that people use in their daily lives. But the government, for a long time, still had a lot of paper. And because they couldn’t use a lot of these cloud services. And so we basically built a Google Forms-like solution, but tailored towards government agencies. The plan for it is to digitize every single paper form in the government. The idea is that there should be no undigitized forms in the government in a few years time. And we’re pretty close. So we’re close to 50,000 forms digitized right now. So there are about 50 million submissions through our platform. Which is 50,000 forms, 50 million submissions is pretty good. And nowadays, whenever any government agency, if they need to collect data, it needs to have people register for something, or sign up for something, or organize something, you can just create a FormSG form. In fact, one of my favorite things that we do is when we’re demoing the product to new agencies, we create a feedback form for the presentation live on FormSG, while we are giving the presentation. And so you can show, within 20 minutes, you can have a digital form up and running. It used to take several months, right? You have to gather requirements, go hire a vendor, manage the vendor. Minimum three to six months to get a digital form running. Now you can do it within 20 minutes. And that’s a really big deal.

Go.gov is the other more well-known products. It’s actually not that big a product in terms of head count, only like three people. It’s a link shortener basically. We launched this as a hackathon, last year’s hackathon, and we’re now at about 74,000 different Go.gov links. So they are link shorteners. And there’re about 76 million clicks on links so far. It’s pretty good. The big reason why Go.gov exists is because QR codes are very useful, but there are a lot of QR code generators, but we needed an official government link shortener. Because the problem with private sector, using private link shorteners is that anyone can create a private link shortener, and it’s very subject to phishing attacks. Anyone can go, and create a Bitly or a TinyURL or whatever link, and say it’s a government link, and people get phished and tricked into going to these scam sites. Whereas Go.gov.sg, you can verify that, “Yup, it is a .gov.sg”, and only government officers can create Go.gov links. And so you have a link shortener and QR codes all built in, and all your link tracking, and everything obviously is built-in, so you know how many people have visited your website, things like that, without having that sort of officiality of things, without needing to do too much. And that’s been pretty big.

We have a lot of products, we’ve got about maybe 15 different products, so I just talk about some of the more well known ones. But hopefully that gives you a sense of the kind of things that we do.

The last one I guess I’ll mention is Isomer. So Isomer is a pretty simple idea. Basically government websites are not famous for being very usable. And the reason for that is because, like we said before, a lot of the regulations require old web standards and stuff, which were best at the time, but not so good anymore. And so the idea was rather than building the same website over and over again in different ways, we will just create one template. One template which was a decent website. Now it’s not perfect. It’s not going to be ideal for every single website’s different designs and needs. But it is a decent website that loads quickly. It’s easy to navigate. It has accessibility features, and things like that. It’s just a simple website template. And so if you, as a government agency want to launch a website, you just use our template, and you launch the website like that. All you need to do is change the text. You change the wording. You change the content. But the deployment, the management, the monitoring, the loading times, the image compression, all that is handled for you. We have 76 websites today, so it’s pretty good. They’re nowhere close to all of the government websites yet. But Isomer websites load very quickly and cost very little, because they are basically just the same template over and over again. The bet I made was that, most people don’t care how good looking a government website is. They just want it to load quickly, so they can get the information. It’s usable and loads quickly, move on. It’s okay if it’s not quite on brand.

Henry Suryawirawan: [00:29:49] Yeah. I don’t think that’s a wrong hypothesis.

Li Hongyi: [00:29:52] Yeah. So basically launching a website now on Isomer, the shortest one we did was like 1.5 days. Within 1.5 days, a new website was out. It’s pretty fast. It’s pretty good.

Henry Suryawirawan: [00:30:00] Some pretty cool stats and projects that you have there. I’m sure there are many other innovations. I encourage the listeners to actually go check it out, open.gov.sg, if you’re interested to see more and more products coming out of this team.

Scaling Up OGP [00:30:11]

Henry Suryawirawan: So Hong, maybe you can also share, in the beginning you mentioned you were the only person. And now I don’t know how many people you have. Maybe you can share the growth in terms of teams, in terms of scaling up. Maybe technology wise, what did you adopt if you can share? Some of these probably interesting for us to know.

Li Hongyi: [00:30:27] So let’s see, we’re about 40 people now. When I started 7 years ago, it was just me. When I got the Data.gov team, that was about four of us. I think that was about 2015, 2016. From there til about the end of 2018, we slowly added more and more products, and built like FormSG, and things like that. And each new product, we had another team of three or four people. About two years ago, we were about 26 people. And then now we’re about 40 something. 40 people is quite a lot, but each project, because we have so many projects, that each project team is actually pretty small. To give you sort of an idea of the team sizes we have in this. Parking.sg is about two people on it at any one time right now. FormSG is about five people. And again, we don’t hire external vendors or anything. These are engineers and one PM usually. And we have some designers shared between teams. Isomer is three people. Go.gov is two people. So just to give you an order of magnitude. The largest team we have is six, which is an internal data sharing platform. But most of our teams are five or less. And that’s by design. If you have very tight teams with a very clear ownership of product, actually adding more engineers doesn’t make the project go faster. It just doesn’t. We have this philosophy of very like tight, small teams.

OGP Vision [00:31:30]

Henry Suryawirawan: [00:31:30] So what are some of your future plans and directions for this team? Is it like more churning out products? Making more mature products? Or is there any like big vision you have?

Li Hongyi: [00:31:39] The big thing that we need, that we’re trying to do right now is just sort of build familiarity. So we managed to build quite a few good products. And we’ve done, I think fairly useful what we did, but they’re fairly well defined. But if our goal is to have long term sort of digitization of the government, and improvement of how a government process operate, it can’t just be like one team doing it. It has to be the whole government being, understanding that this is how modern organizations run with technology. It used to be that, “Oh, that’s just Google. Only Google did this crazy mode of operating.” But now they’re looking more like, that’s how almost all organizations run nowadays with technology. Grab is a tech company, but they’re also a logistics company and a food delivery company. There’s no such thing as a tech company anymore. All companies are tech companies. It just depends what industry you’re in. And I think similarly for the government. There’s no such thing as, “Oh, we don’t do tech. We’re doing healthcare.” No, tech is a big part of healthcare. There’s no such thing as, “Oh, we’re an education startup, not a tech startup.” If you’re doing education, you need to know how to use technology. I think that’s a big part of what we’re trying to do right now. Which is not just about building products, but like sharing how we operate, talking about, that we do hackathons, about how we run project meetings, about how we do user studies. So that people across the government, even if you’re not traditionally a tech person, you sort of understand how to use technology effectively in your domain, and how to run a team so that you get good technological projects out of it. This is one of the things that I think people don’t realize is that because software is very different from traditional industries, you need very different management techniques and very different organizational designs in order to be successful. If you look at traditional organizations which tried to go into software space, but even very big and rich ones, you look at the airlines, look at the banks, you look at big established companies. They can spend a lot of money, but their website or their app will be terrible. And it’s not because they don’t spend that money. It’s because you need a different managerial approach to getting good software. So that’s the first big thing, evangelization.

I think beyond that, what we’ve done so far is on the product side of things. We’ve worked on very horizontals, like these very common components that agencies can use. Where we want to go next is to help deep dive in some of the more operational systems, like core data systems, your business processes, helping fix those. The idea here is that the way we work is that, when we build a product, we try to generalize it so that more people can benefit from that product. For example, we work with MCI to build a mass WhatsApp messaging system to let people know about the COVID updates in Singapore. And that people know what was going on. But rather than building that just for MCI, we realized that mass messaging was a problem, not just for MCI. Let’s say MOH, they needed to send out a swab test result. Or MOE who needs to send out exam results. Or MOM who needs to communicate with workers. Mass messaging is a general problem that people have. And so we took what we built for them, and then we generalized it into a product that we call Postman. And this is one of our new products, we just launched it a few months ago. That is a very simple mass messaging tool for government agencies to either email, SMS, Telegram to people. That’s how we are planning to go more in depth. We’re like, you find the specific use case. You solve the problem. And then you try to generalise it, and productize it so that more people can benefit. Yes, every agency at some point will have some niche use cases. But overall, you realize that actually a lot of agencies have a lot of common problems, that if we solve, we can solve for everyone.

I think the end goal from here, and maybe a few years from now is that once we get really well established, we want to open source a lot of the things that we build. Cause I think just as different government agencies have similar problems, I think different governments across the world have similar problems as well. And if we solve these problems in Singapore, and we do a good job here, then actually we should share what we build with other governments. Because you can imagine that if you’re a government in, let’s say a country which doesn’t have as big a tech team or strong a tech emphasis, let’s say you’re an engineer somewhere in South America or in Europe or in Southeast Asia, and your particular government doesn’t have the resources to give you a big engineering team. You can take some of our open source projects. And just like, when I was just getting started, followed open source instructions, and got some first approximation stuff set up. These people too, could help the governments run better with a good parking system, or a good digital form system. Cause there’s no reason why governments nowadays shouldn’t have decent websites and good digital forms. You just need to follow open source instructions.

Yeah, so that’s the long term goal. That you build this baseline of tools where governments, not just our government, but governments all around the world can use each other’s tools to run better societies. It’s one of these things where I think now we’re realizing, especially during this COVID period, how important having a well functioning government is. There’s a lot of things, when there’s no crisis and stuff, government can be a bit inefficient, can be a bit slow, and it’s not too bad, the country still goes along, the private sector carries a lot of it. But when there is a crisis, you need your government to run. You need to be able to communicate information. You need to be able to get information from people. You need to be able to share data. You need to be able to coordinate. A lot of governments just don’t have that, and we need to get that ready.

Henry Suryawirawan: [00:35:44] That is truly inspirational! The way you mentioned about open-sourcing your products for other governments, for other citizens to use. So I’m sure if this is done right, more and more people get the benefit out of it. And also those governments can also transform to become more digitized, and they don’t have this hurdle of dealing with government bureaucracies, and old legacy systems that are not necessarily friendly to use.

Cloud Adoption in Government [00:36:04]

Henry Suryawirawan: Singapore I think is one of the most forefront countries in terms of digital adoption. I think the story is pretty remarkable in the last few years. And I think it is even more amplified during this COVID times, where technology has become more prevalent, like you use this TraceTogether, you use this QR code to check in, and now e-payment seems to be like quite a norm. And I think also government has this open stance now about using cloud within the government. If you can share a little bit, how much of the cloud actually transforms how government works? You mentioned earlier about infrastructure cost was too expensive, maybe ten or a hundred times more expensive. Is it now becoming a norm for government agencies to use the cloud. And if it is, what are some of the biggest challenge for people to adopt within the government?

Li Hongyi: [00:36:46] So I would say using cloud services as a whole is the biggest potential for how the government can improve. If you just go about your own personal life, there’s no such thing as software that you install anymore. If someone tells you to go download this package, and run the exe on your desktop, that’s a bit archaic. Almost everything is a website. And even apps that you install rely on internet connection. If you don’t have internet, most of the apps in your phone just don’t do very much anymore. So this is the biggest, most important thing that I think where most of the value will come from. Like on-prem solutions, there’s just no future in them, unfortunately.

Now in terms of how successful we’ve been, I think we’ve made some progress, but I don’t think we’re anywhere close to done. Basically, the government has started moving some services towards sort of hosted solution. You’re hosting on AWS, and Google, or Azure, or one of these other cloud service providers. But we’ve made very little progress in terms of adopting SaaS solutions. And by SaaS, I mean things that you just go to a website and use. The reason for this, in my opinion, is because the government’s mental model for IT is still very much from the early 2000s, late 90s mode, where you had to build solutions and host them. And moving to hosting on a different platform, like hosting on AWS, hosting on Azure, hosting on Google Cloud Platform, that makes sense. They can think of that, and they can do that shift over, and you get some gains. But this concept of just going to a website, subscribing to a service, and using that web interface to run task lists for your team, or to manage your HR, that just quite far outside of our current mode of operation.

I think the biggest hurdle by far is our security policy. The main thing that we’re worried about, basically, is that if you are a private entity, you go on the cloud. AWS has the commercial interest of protecting your data, and that’s fine. But if you are a government entity, you have, what do you call, persistent threats that are trying to compromise your data, and that’s why they are really scared of it. There are some things which are a very big deal. If you’re in the foreign affairs, or defense, or all these things, then yes. You need to be very careful. You might want to take sort of extra precautionary measures. Not that the cloud is more vulnerable. It’s just that the cloud is particularly vulnerable to particular nation state level threats. But if you are, for example, doing education, or healthcare, or these things which are not national security issues, actually moving to the cloud is far more efficient. We just need to be able to delineate between those two cases, between the national security kind of cases, where you do need to be almost paranoid, and then the important and privacy aware, but otherwise day to day business, where actually you can use cloud services fairly aggressively. So that’s one, which is like our security policy where because a lot of government agencies have not had to think about this very much, we basically just put everything in one big bucket and be like, everything is super important. But I think if we want to capitalize on the progress that the internet provides, we do need to be clearer in separating out and classifying our things, so that we keep our most important stuff protected, but we make progress on a lot of the things that actually aren’t that critical security wise. Maybe critical privacy wise, but not national security wise.

Another big hurdle, I would say, is finding the kinks in our own regulations. So security policy aside, I think the broad direction has been given that we should move on to the cloud. But you find that when you’re still moving onto the cloud, and you’re trying to migrate, and try to move your agency services onto the cloud. You’ve been given the direction to do so, but you’ll find a regulation which actually says, “Oh, you can’t put this there. Or this needs to be linked to this. And you need to reconcile these.” There’s still quite a bit of friction in terms of working through all these problems that we need to resolve. So I think those are the main hurdles.

Talent Challenges [00:39:42]

Henry Suryawirawan: [00:39:42] Thanks for sharing that. So you mentioned, especially for government to thrive digitally, you need all these talents, maybe locally, maybe from other parts of the world to come and join for this government team. What are some of the challenges for you to find the talents these days? Do you still experience that? How do you attract those talents to actually come and join government, and build these products and team and make impact to the citizens?

Li Hongyi: [00:40:04] So there are a few challenges in doing recruiting. The first one is just having people know that you exist. If you’re Google, everyone knows you exist. You don’t need to do anything there. But if you’re a tiny experimental team within the government, people don’t even know you exist. Let alone want to work for you. So you need to take some effort to be part of the community, share the work that you do, and just let people know that you exist in some form. Once you’ve established existence, then there are three things. There is how nice you are, how nice a place you are to work for. So this is basic stuff like you have a good work culture, people are happy, you treat people well, that sort of thing. Second is your professional excellence, that you know that you do good work. And third is your sense of mission. And this I think is our biggest selling point. Because for a normal tech company, let’s say competing with Google. They obviously have a very famously good work environment, and they are obviously professionally very excellent. They have a lot of smart people doing a lot of cutting edge work. But I think one of the reasons why people might join us, and this is what we’re trying to sell them on, is that there is a lot of good work that needs to be done in the government. I think people are seeing that now. Yeah, you go and work on Google, you can work on making more money from ads, or having people access Gmail a bit faster, which is good and it’s important to do. But at the same time, running our COVID Management Systems. We need to get our public transport efficient and up and running. We need to figure out how we coordinate across different ministries, and like, let’s say social workers, make sure they share information with each other correctly, so that the family who needs support across, let’s say Ministry of Health, Ministry of Education, and Ministry of Community, you have a holistic picture of a person, and so that no one slips through the cracks. That’s one of the things we do.

In terms of like specific things that we do to do this. The first one is that we try to give talks a lot. I encourage everyone on my team to go and share what we do. I think it’s very beneficial professionally to be comfortable talking about your work and sharing it. I personally think we do some pretty cool things on the team, and some pretty technically impressive stuff. When we go out and we share, when we talk about it, firstly, people recognize us and build familiarity, but they also see that, “Oh cool, this team and the government is doing some pretty neat work. Maybe I might want to work for them.”

In terms of the mission, the mission is the big important part, because that’s the biggest advantage we have of why people might want to join us. And you need to make that very clear. So if you go to a website, internal documents and everything, we write it out. This is who we are. This is why we do what we do. This is what we’re trying to get towards. We emphasize it very strongly. Because if you don’t have mission, then what are you doing here? And that’s our push for hiring talent.

Henry Suryawirawan: [00:42:03] I fully agree with that. So Hong, thanks so much for all your sharing.

3 Tech Lead Wisdom [00:42:06]

Henry Suryawirawan: My last question, as usual for all my guests that I have in the show, would you be able to share with us here, what are three technical leadership wisdom that you have, that you want to share with other people?

Li Hongyi: [00:42:16] Of course. Okay. So the first one I would say is, you need to start small as far as possible. I think one of the biggest reasons why I see tech projects fail, is because they start too big. I have never seen a project fail, because it starts too small. Because if a project is small and it does well, you can always grow it. There’s no problem. If you’re a successful small startup, and you’re doing well, you can always get funding. It’s quite easy to grow a project. On the other hand, if anyone’s worked in a big tech company before, worked in a corporate organization, you sort of see how projects can very quickly snowball to very large sizes. And then they get so big. And if all these different requirements, it is not focused anymore, just topples over of its own weight, and eventually launches, but it’s just a mess. That’s the first thing I would recommend. Start as small as possible, and do that small thing, and once you’ve done the small thing, then you can look into doing bigger things.

Next is seek out problems proactively. Don’t wait for someone to sort of hand something to you. Actively try and figure out what’s wrong with your product. Actively test, because the thing about software development is that it’s by definition building something no one else has built before, almost certainly. Because if someone else has done it before, then you should just copy paste, that you shouldn’t be building a new thing, generally speaking. It’s not about avoiding failure. You will fail, and the idea is to fail efficiently. You want to fail fast, and you want to fail small, so that you can fail as many times as possible, as quickly as possible until you figure out what the right answer is, and then move on. That’s software development. Even the smartest people in the world don’t come up with the correct architecture or the correct design or the correct thing on the first try. No one writes bug free code. You are going to have bugs. Accept it, both at the technical level, at the design level, at the product level. In order to succeed, it’s not about avoiding bugs, it’s about finding bugs as quickly as possible.

And the last thing, I would say the most important thing is, you need to really focus on hiring people and treating them seriously. The software is honestly the least valuable part of a tech organization. The people are the most important part. That if you took someone’s code, and you got rid of the team, within one or two years, the code will stop working, because OS updates, security flaws. But if I got rid of all the code and I kept the team, you could reimplement the code base. It would be a pain, but then you can reimplement the code base fairly quickly. And in fact, you might reimplement it better, because the team now knows what the problems are in the first code base. The people are super important. And you need to structure the organization around this. A tech organization is a knowledge organization, which means that it’s not about taking people and having them follow instructions. It’s about having people think for you. It’s about maximizing the amount of effective brain power working on a problem. And so you need to follow a managerial approach that gives people autonomy, that gives people creativity. You want people to be smarter together as a group, and optimize designs for that, as opposed to a military design, which is more about command and control. Tech organizations are less about command and control, and more about relinquishing control so that you have more brains thinking in more directions all at once. All the coaching, and flexibility, and good environment, and all is around achieving that outcome. And that’s it, those are my three things.

Henry Suryawirawan: [00:44:50] Yeah, I fully agree with the last one. People are the most important aspects in any software development team or tech startup company. I think it’s pretty cool that you shared this insightful thought. I really enjoyed this conversation. I learned a lot about your initiatives, your challenges especially in the beginning, how you overcome that and also make these innovations coming up out of a Singapore government. I think it’s truly inspirational. I wish you good luck for your future projects. And I hope to see more and more innovations coming out from OpenGov or even GovTech and Singapore government in general.

Li Hongyi: [00:45:18] Thanks so much. Thanks Henry, it was really good chatting with you. And thanks for having me on today.

– End –