Rust efficiencies at AWS scale

Tim. Fourth time lucky.

I’m so sorry, Gerhard. Honestly, I wanted to be there the first time, I wanted to be there the second time, I got injured, I couldn’t come on the third time… But now I’m here, and I want to know that – like, my time is yours. Yeah, feel free to use it however you think would be most suitable.

Well, I have to say thank you very much, Tim, to our listeners. That’s how badly I wanted to have this conversation. I tried until it happened. Okay? And we have been delaying it, I think by a few weeks, and we spoke about it briefly in September, and then holidays happened, and then all sorts of other things happened… And I said, “Okay, this has to happen.” And again, there’s like another big event coming up for me personally; we will talk about it a bit later, I think in episode 90. But for now, what’s important is that I really wanted us to have this conversation with Tim. So thank you for joining us. Welcome on Ship It.

No, my pleasure. It’s my genuine pleasure. I really love the way that the show has progressed, and it’s quite a privilege to be speaking to you here.

Thank you, Tim. I appreciate it. Now, how are your ribs, your physical ribs? [laughter] That was attempt number three, right?

Right, right. So what happened… As context, I was riding a mountain bike along the river, came across a ditch, and I thought, “Oh, look, that’s looking a little bit deep. I should slow down.” I didn’t quite slow down. I thought, it’d be fun to try and get through the ditch. My front wheel went down and kind of got stuck on the other side, but my body kept going. So I spent about 10 hours in A&E the night before one of our interview slots. And I think I texted you at about 1:30 in the morning, saying “Look, I’m–”

You did, actually… So I knew that you really cared about it, right? I mean, if you’re like in a hospital, and you remember to text me “Hey, Gerhard, sorry, I will not be able to make our slot.” I mean, you have no idea how much I appreciate that, Tim. Thank you.

Yeah. So ribs are okay, actually… Bruising is fine, that will go away. I’ve injured some ligaments that connect my ribs and my spine, which makes it quite hard to breathe in, and also to do things like lie down or put on my shoes… But I’ve been told that I will heal; bodies are crazy, they’re good things. And yeah, this is all part of me understanding my own limits. I’m no longer sort of 19, and –

And Superman, so we clear that, right? You’re not Superman… [laughs]

Yeah…

You can’t fly. Well, you can, but…

Multiple years since I’d been on a mountain bike, and one needs to appreciate one’s own limits sometimes.

Right, right. So how are you with laughter?

Laughter is okay, actually. I’ve been surprised…

Dang it!

Yeah, you really – but sneezing… Like, let’s say there’s some dust in the room - that would basically send a dagger into my ribs, which…

I’m not sure what I can do about sneezing. I could have helped you with laughter… But let’s see. I can only try. So you are a public figure. I enjoy reading your tweets very much. I’ll have a question related to your Twitter handle, which I think is great. However, you’re tweeting when – I think it was like the first time that we were supposed to record, that you have a very early morning meeting. So I think you set your alarm clock for 5:15 AM. Now, you’re ahead of everyone, because you’re in New Zealand. So no one can catch up to you. You’re always ahead of the whole world, okay? We try, but you’re many hours ahead of everyone. Was that early morning meeting worth it?

Yes. So, I am in a very privileged position where most of my team – so I can work remotely from New Zealand. This is very rare within AWS. Most of my team are based in the West Coast of the United States, and the remainder are in Europe. And so my day, or my week is typically staggered so that I will have one or two early morning meetings. 6 AM Here is 9 AM in the West Coast of the United States. And over where you are, Gerhard, depending on daylight savings, it’s either 10 or 12 hours away. And so someone has to be up late, or awake very early.

[05:56] Typically, people are very understanding of the fact that I’m in a very strange timezone… The only trouble is if I wake up early, and then also don’t get to sleep properly, because I’m sort of mentally preparing for the upcoming discussion, my brain doesn’t really have a very good off switch. That’s the thing that bugs me.

I know what you mean. Yeah, it just wakes up; the brain wakes up separately from the body, it starts churning, and then it wakes up everything else. And usually, it’s like work-related thoughts, experiments, things… “Oh, have I said that? Have I done that?” I know exactly what you mean. I get asked quite a lot, too. Okay, it’s not just me; that’s great to know. And I’m sure others listening to this will be able to relate. So you mentioned that you work at AWS… What do you do?

So my job title is senior software development engineer. My job role though is a little bit broader than a typical software person. So I work within a central team, supporting the development of the Rust programming language within Amazon. That includes AWS, as well as retail, and into like Amazon Go stores; if you’re in the States, you might have gone into it, in a physical store… And there are bits and pieces of Rust working in embedded – in the shops themselves… All the way through to some of our kind of flagship services, including Amazon S3, have components written in Rust. And this is now a technology that is strategically important enough for the company to have two teams actually working on the language.

So the team that I work on is mostly centered on internal support. So we do some technical things around like supporting the internal build system, we have a mirror of the open source sort of package ecosystem… Crates.io is the Rust equivalent of let’s say npm, for sort of the Node ecosystem, or PyPy in Python… And the other team is primarily focused on, let’s say, the compiler, or some other core components of the Rust ecosystem, like the Tokyo runtime. The division isn’t clean, but that’s the broad division of work.

The thing that is slightly unique to my job is I’ve increasingly become seen as the person who is driving education, or adoption of Rust at Amazon. So this year, I’m actually going to be leading an education project for like “How do we teach tens of thousands of programmers Rust?” I mean, it won’t actually be that big, but this is kind of the overall goal…

The scale, yeah…

…that we want to create a pathway for teams that are looking to adopt Rust, to be able to give them a path forward… Because most services at Amazon are written in Java, and teams choose their own tools. So there’s no way to sort of centrally dictate that such and such a team or such and such service must be implemented in such and such language. Instead, we kind of need to work kind of slowly, organically, with teams that are interested, but trying to work through their pain points with the language and with the ecosystem.

And one of the things that we’ve found is that teams that are looking to adopt Rust will typically have one or two people who have been tinkering with the language, and they’ve found that things run really quickly, or that they save a lot of memory, and that they reduce the operational burden… So their systems are more stable, even in a prototype stage. But then there’ll be a point at which they hit this kind of organizational inertia, where at like the engineering manager level, or let’s call it the middle management tier, will suddenly – the immune system of the organization will respond. And it will say, “Whoa, whoa, whoa, whoa, whoa… This is making me really nervous. You’re implying change, and change is very dangerous.” But at a more senior management level, some of the – Rust is being talked about so pervasively now that it’s impossible for senior leaders to ignore it, and they’re really sold by performance benefits, by safety benefits, by security.

[10:17] And so these are kind of the internal dynamics, that we have teams that are very optimistic about their futures, and they’re keen to experiment; let’s say one step above or beyond them might be a little bit of resistance, or a skepticism of a lot of the claims. The Rust community seems to make these ridiculous, audacious claims, like it will save you, let’s say – actually, I’ll talk about my own ridiculous, audacious claim, which is that “Why don’t we as a software industry reduce our energy use by 50%?”

Now that’s something worth doing. Right? Forget about your build times, right? Forget about latency; how about we save the planet first?

We can. And actually, my personal view is that software engineering is now in a significantly important part of social change, because all new products, I’d say the vast majority of them are software-first, or at least software-enabled. This actually means that the people who implement software are an integral part of social change. And I am one of these crazy people who, for example – like, I have the electric car, I cycle to work whenever I can…

You work from home, what are you on about? [laughs]

I work from home, but there’s an office though, like further down [unintelligible 00:11:44.26] I do primarily work from home, so this is a bit of a cheat. Dang it…!

Quick, Tim; think of something else.

I was on a roll. I was gonna say, I’ve been a vegetarian for like two decades now, and it was primarily for ecological reasons… And so I am not normal from that regard, but I still think that there was an argument that I feel quite persuasive; that we as a software industry can do better. We can respect our users by giving them secure software. We can respect our businesses by reducing waste, and also – and primarily from the Rust perspective, this comes in memory usage.

So a lot of these cloud platforms or cloud services are very – if you notice the way that their pricing structure works, typically compute scales quite well, so it’s quite cheap to oversubscribe CPU. You know this if you’ve ever managed your own Kubernetes clusters; you can kind of oversubscribe CPU quite well. But one thing that is much harder to oversubscribe is memory. Because if suddenly you have two applications that want to use all the memory, neither does very well. Whereas if you have two applications that suddenly require 100% CPU, things sort of slow down, but they don’t halt.

Crash.

They don’t crash.

OOM. There is no OOM for CPU.