How Mat writes HTTP services in Go

Hello, listener. Welcome to another episode of Go Time. [laughter] Do you guys like the dramatic pauses in between?

Yeah. This is now a True Crime podcast.

A True Crime podcasts, yes. And joining me as suspects - or co-hosts; I mean, depending on how you look at it - is Ian Lopshire. Ian, how are you?

I’m doing great.

Good. Good to hear it. And also, returning after a hiatus - well, I guess not so much of a hiatus now, because you did do the live episode from GopherCon EU, right, Mat? Mat Ryer, welcome.

True. Hello. Yeah, I did do that. That was fun.

That was fun.

You were there, Johnny?

Yeah, yeah, I was there. I was there…

You were on it.

…having my delicious alternative to brushing my teeth. Good ol’ coffee.

Delicious.

Deliciousness. Deliciousness.

No, but a lot of people – I said that joke, I said “Coffee is a delicious alternative to brushing your teeth.” I’ve had messages, people telling me “No, you do still need to brush your teeth.” So please… Yeah, just pass that on. If anyone asks, you do still need to brush your teeth.

[laughs] Those things are not mutually exclusive.

Yeah. And also, everyone, you can stop messaging me about that now, if you like. That’s fine. Also fine.

Oh, man… That’s funny. That’s funny. Yeah, you shouldn’t have made that joke. Now people are gonna chase you down. So what is not a joke is this article you keep writing and rewriting and updating and refreshing every couple of years about writing HTTP services. Now in its 13th year, in - what was it, second or third refresh? Is that it?

Yeah, I think it’s the second refresh. There was an early version, and then I did a refresh of that. And it just basically – it’s the same thing, I just updated the title with the different number of years that have passed… [laughter] It’s the gift that keeps on giving.

When was the first one? The first one I encountered was eight years. It was the one before that?

I think that was the first. There may have been the seven year’s, and then eight year’s was the one that was there for a long time… Because it didn’t really change much. And then I felt like enough it evolved, that it was time to do a refresh, and… Yeah, give it a new life, I suppose, a bit.

So the obviously the link to the new one, which actually contains the links to the old one, will be in the show notes. But yeah, Ian and I were thinking we’d kind of go through and call out the interesting bits. So what were the sort of the major changes with this refresh? What was so different? Why did you change the approach?

One thing that changed was preferring HTTP – I used to prefer http.HandlerFunc. So I would use that type everywhere. And then at the points at which you needed to interface with HTTP handler, you just, you can, because it does implement that. But then if you’re taking – so sometimes it made bits easier to write, but it actually made it… Whenever you have to interact with third party languages or third party libraries, you’d then have to do extra gubbins in places, and it kind of looked quite ugly. So I think I just decided I’ll just go to HTTP handler, because that’s a simple type.

Tom Wilkie, who’s the CTO at Grafana, also said that same thing to me a long time ago. Actually, I met him in an airport. We were both on our way to a conference in Lviv, actually, in Ukraine… And we’d missed a connection, and then there was just like a person talking behind me in the queue about Go. And I can’t resist; I can’t just let that happen, if they say something I don’t agree with… So I spun around, and then that’s where I met him. That’s actually where I met him, and that led to – now I work at Grafana Labs. So I don’t know how many people y’all meeting in airports, but… Get out – get to your local airport, I say. You never know what’s gonna happen.

Find yourself a job. [laughter] Go to your local airport to find a job, you know?

Exactly.

That’s how I’m connecting those dots.

You do have to miss a connection, I think.

Right, right. And do you think you have to be headed to Ukraine for that to also align things for you?

Good question. I’ve only done it that way. So that works for me. I suggest you do exactly the same. Don’t change anything.

Might not be worth it right now.

Yeah. True.

Right. Indeed. So away from HandlerFunc, and embracing the good ol’ HTTP handler interface. And obviously, you added some additional opinions to the testing stuff… And we’ll get to that. But yeah, I was looking through the examples that you had on the blog post… This new server constructor and all these arguments… So if we deviate slightly from the HTTP services sort of conversation and talk about sort of how many arguments should function signatures have and support… The reason I’m calling this out is because I’ve been a part of many language communities where there’s an optimal number that you should have, and if you go beyond, then you get a side eye from your fellow developer, kind of thing.

[00:06:25.29] Right, right. So yeah, this comes from – I used to have a server struct that would have all the dependencies in there, and then the methods would hang off that server type. And that meant you could just put dependencies like the logger, or a database connection thing - you could just put those into the struct, and then they’re available to all of your handlers. So that was kind of a nice convenience. But actually, what I’ve found over time was when you come to actually write a test for it, one of the nice things about a list of arguments is it’s very clear what this thing needs. And even if in your test you know it’s not going to be used, you can explicitly mark that with a nil; you pass a nil in, usually, or something.

So when I would do that - and sometimes it would still fail, and it would fail because it was using a dependency that I didn’t remember, because it was in the server struct, and there was nothing to prompt me to do that… So I had to live through the panic. And that was all, really. It’s not the end of the world. A lot of this whole blog post is kind of personal tastes and things that work for me. But this led to then you have a server constructor that takes in loads of arguments sometimes, because all your dependencies are going in there. Your logger, some config, you might have some database stores, you might have loads of various services… If you’re sending email, you might have an email sender service kind of thing, or a comment service. And depending on how you structure it, you can end up with these long argument lists.

So the question is, do you like long argument lists? Is there a limit where you think “This is silly. I’m gonna put it in a struct”? Do you just always go for a struct, wherever you can, immediately, and just be done with it?

I’ve done it a few different ways. And I think I will sort of tilt one way or the other depending on sort of how explicit I want to be or not be with things that can be nil, or the things that have defaults sort of within a struct. So my personal preference in this particular case would not be to have a new server with all these things I can pass in, and sort of passing in nil when I don’t need something… In this particular case I might perhaps choose the sort of functional options pattern, basically… And for those who are not familiar, typically you’ll invoke your pseudo-constructor, if you will, and then you can have some with options. With a router, with a muxer, with a store, with a whatever it is. And then pretty much internally whatever the defaults are for the things you don’t specify, they can kick in sort of automatically, and you’re just specifying a width sort of option for the things that you do want to override. So I tend to prefer that approach, as opposed to having long argument lists where I’m sending nil for the things I don’t need.

Yeah. So that is nice, because in that world you don’t mention the things that you’re not concerned with. So you have good sensible defaults, and then use those options to set the specifics that you care about. Yeah, I don’t mind that at all either.

I’ve heard a lot of people push back against these sorts of – you know, there’s a bit of a discovery issue with it. You can’t easily just see a list of all the possible options that you can set or pass in. A struct at least gives you that. But yeah, it’s kind of trade-offs and taste, I think; I feel like. But I think a lot of this blog post might come down to that, and I might be able to convince you with some good arguments. But if you’ve got a good reason to not do this – I think I do say this in the blog. If you’ve got a good reason not to do it, then you’ve got a good reason. That’s it; that’s what you should do.

[00:10:13.14] Because people sometimes will stick very rigidly to what I’ve written, and when they’re then battling something and it doesn’t quite fit, they’ll sometimes reach out, and there’s just a much easier – usually there’s just a much simpler way they could have done this, and they were kind of bending over backwards to follow the advice. But that’s just all part of, I think, learning and using these sorts of blog posts properly, which is not really to take them literally, but to get ideas from them and apply them in your case, where it makes sense.

With this long list, have you – what’s like the max you’ve encountered, and were you okay with it? If we ran this way – you know, we run like a pretty big monolith, and there would be 50 things in this… That’s just not doable.

50 I’ve not seen. I think, honestly, 15 to 20, and I was okay with it. But I should say, I tend to format it differently as well. So it’s not like continue – gofmt also doesn’t mind this, but you can basically put each argument on a new line. So it’s just like a list. A bit like when you do a variable list, or even a struct, when you’re declaring lots of things. You kind of have an indented list of items. So in that way, it’s not that different from using a struct. It’s just that you are forced to provide – you’re forced at that point to provide an answer.

And you know when you come to changing things, and you’re getting the compiler giving you warnings, errors - it won’t build because this signature is wrong - obviously that’s very brutal, but that is really how you find all the places. And as soon as you’ve fixed all that, everything kind of works… Assuming you’ve got all your dependencies of different types, so you can’t end up with mixing the order up, or something, which would be an absolute nightmare. Assuming not. Assuming that [unintelligible