Rails as a day job, Diesel on the side

Alright, I’ll jump right into it. [clears throat] Added helper types of inner join and left outer join. Diesel debug query has been added as replacement for debug SQL. Added support.

Is this your changelog?

Yeah. Is this not where I come on and dramatically read Diesel’s changelog? [laughter]

That was pretty good.

That was good, yeah. Keep going!

Well, I’m way unprepared if that’s not what this is about. [laughter]

I was entertained for a split second.

Literally, the changelog. Love it.

Sorry, I had to get one dad joke in early on. I came up with that one two days ago, I’ve been waiting for it. [laughter]

I like it. We should leave that in.

I don’t know why – this is taped, man. Let’s go!

Alright, let’s go. Well, now that we know what kind of guy you are, Sean, you’ll fit right in here on the show… But we have two subject lines; I believe they’re intertwined (as most things are), and that’s the subject of you full-time on Rails, and your work there on ActiveRecord, and then secondly, what you’re up to with Diesel, which is an ORM and query builder for Rust, and the interrelation between those two things.

But let’s start off learning a little bit more about you and your path to what we often call living the dream, which is working on open source full-time, and in a way, that seems like a pretty awesome way. So tell us about your work with Rails, how you got involved and how you ended up full-time at Shopify.

Sure. Let’s see - so I got started with Rails four years ago now, I think; I’d done a commit here or there, but nothing particularly major… But I’d been on some projects for a while that all sort of had these weird, similar needs in terms of modifying attribute accessories on ActiveRecord objects, and I think it was Ruby Conf like 4-5 years ago - Ernie Miller gave a talk where he was talking about how Rails is missing a lot of API’s that are kind of one level down in abstraction… All this culminated in me wanting to build out what is now today known as the Attributes API.

It just so happened that Aaron Patterson was – I lived in Denver at the time, and he was in town for a local meetup to give a talk. I went out for drinks with him afterwards, and was like “Hey, I have this idea for this API.” It sounds polite, but I was actually really drunk and kind of shouting at him about this API…

[03:58] Whatever works.

But anyway, so he was like “Yeah, sure. Whatever. Do it.” I think he was partially just trying to get me to stop asking him about this idea that I had, since the majority of people who do that are never gonna follow through…

Anyway, that resulted in a pull request that was way too large to be reviewed, and then a series of smaller pull requests over time, and ultimately I vastly underestimated the amount of work that implementing that was gonna require, and when it was all said and done, I had rewritten a significant chunk of ActiveRecord for non-association and non-Arel-related things… At which point I was like, “Well, crap. I guess I’m maintaining this code for the rest of my life.”

So I assume it got merged, then.

Oh yeah, definitely.

What was that like, to feel like you’re maintaining it for the rest of your life? Are you being facetious or are you being serious?

I mean, both… I didn’t really go “Well, I’ll maintain this for the rest of my life”, but yeah, it ultimately was part of why I ended up sticking around, it was just because I felt a responsibility to be around to fix the bugs in this code that I had submitted, and make sure that it will continue to evolve in the way I envisioned.

So just real quick for clarity, for those who are not in the Ruby on Rails scene - ActiveRecord is the ORM or the database library that ships with Ruby on Rails and is a significant portion of the Rails codebase, and probably a significant portion of the complexity in a lot of the bugs, so rewriting any small subset of ActiveRecord itself is not a small undertaking.

And probably, Sean, you know better than anybody at this point as a maintainer of ActiveRecord, while thousands, if not tens of thousands or hundreds of thousands of people who use ActiveRecord, there’s probably a group of people that you could count on one or two hands that are intimately familiar with the way that it works. Is that fair?

Yeah, I think so. And to give some actual numbers, I don’t know what it is in the codebase today, but around Rails 4.2, so not including action cable and not include active storage, ActiveRecord accounted for about 80% of our codebase, and back then I would say it was about 80% of our issues, as well.

Certainly, I would guess the percentage of the codebase has gone down, just because we’ve added two additional libraries, but probably still similar… Probably like 70%, I would guess. Although interestingly, I’ve been noticing that it is starting to become a smaller percentage of our opened issues, which is good.

Yeah, you can probably see that as a win, since you’ve been dedicating a lot of your time there…

Yeah.

So you opened up this massive PR, this idea that you were challenging there, and Patterson, who was then very involved with Rails - I’m not sure of his current status; in fact, he was full-time on Rails I believe working for AT&T Interactive, or somebody was paying him full-time to work on Rails back then. He now works at GitHub, and I’m assuming he’s still very much involved. So he said “Go ahead and do it”, you went out and accomplished this goal, and now a lot of the ActiveRecord codebase is your own, so you begin maintaining it. That doesn’t get us to full-time at Shopify, so tell us the personal/business end of that same timeline, how you ended up being able to do this as your job.

[07:37] Yeah, so I guess the first thing is when I decided that this is what I wanted to do… Because it’s funny, you call it “living the dream”, and to a certain extent I kind of call it “living the nightmare”.

That’s the great irony, isn’t it?

Yeah… But basically, I’m bad at work/life balance, and I started to realize that if I didn’t do this as my full-time job, I had two full-time jobs. So I set out to switch to doing it full-time, simply because I’m bad at convincing myself to not work, and if I didn’t do it during work hours, I was just gonna be doing the same amount of work on my own time. So it was purely a move for my own sanity.

I’d sort of really started to get a taste for like actually spending a significant amount of my time doing it. I was working at ThoughtBot as a consultant at the time, and I had some significant downtime between projects. That was when I started to try and spend more of my work hours doing it. I’ve been looking for ways to convince management that it was a worthwhile for me to be able to continue doing. Ultimately, the interest didn’t align there, so I ended up leaving.

I ended up at Shopify because basically I went to a conference and begged a lot of people to give me money, and Shopify ultimately was the company that saw the value proposition. Because when you say full-time open source, it’s a convenient way of kind of summing up a job with the emphasis on the focus of your job as working on open source, and it does not mean that I spend 100% of my time working on open source.

Open source is more than just code, it’s a lot more than that - it’s documentation, community, and even research and development, I’m sure. It’s not just the things you do.

Yeah.

How long was this timeframe of like you petitioning the community, so to speak, with what you wanted to do and how you wanted to do full-time? How long was that span of time?

About six months.

Not too bad.

And you were full-time at ThoughtBot during that time?

Yeah.

Let me just do something unprecedented here at the Changelog. With your permission, Sean, I’m gonna read Sean’s LinkedIn bio. I mean, he was reading the changelog to us –

Oh man, I haven’t touched this in a long time. Yeah, go for it.

I’m assuming you haven’t changed it for a while, because it sounds like it’s still in your petition phase.

Oh, I had that there just to get recruiters to stop spamming me.

Oh, boy…

I enjoyed this. I knew that Sean was at Shopify, but I wanted to see where he was previously. ThoughtBot makes a lot of sense, because that’s probably where you started doing The Bike Shed… So I saw this on LinkedIn and I thought it was funny. He said “I’m currently only interested in opportunities that will allow me to focus primarily on open source contributions. The best way I can contribute to the community is with my work on ActiveRecord, if that includes your company. If you agree, drop me a line.”

This is where it gets good:

“My dream stack is the one where recruiters leave me the hell alone. Please read the previous paragraph. If you think that ‘Focusing on open source contributions’ working at a company that uses open source are the same, then go away.”

It’s interesting there, because you were very clearly stating at that time - and probably to this day - that you were interested and you were set on opportunities that allow you to do open source contributions full-time; or ‘primarily’ is what you said, which is probably more fair than full-time.

Yeah.

And then of course, the call to get recruiters to leave you alone is hilarious, because I’ve been on LinkedIn for many years, and the only people who’ve ever contacted me on LinkedIn is the recruiters. I haven’t had a legitimate request, ever.

So I thought that was kind of funny, and perhaps written back when you were ready to go from consultant at ThoughtBot to maybe on the path that you’re down today. So tell us, at Shopify - you got the job… Can you tell us what your official position is? You said it’s more than just doing open source stuff. What all is involved in your role at Shopify?

[12:02] Well, my official position is 10x hacker ninja guru.

Oh, good.

If you’re wondering what that means, it means that I once thought it’d be funny to see what happened if I put that on a government form, into my work permit, and it turns out what happens is that I cannot legally work in Canada with any job title other than 10x hacker ninja guru.

Oh, my gosh… [laughter] That is so true!

Right now I’m the only person who focuses on rails as their full-time job, and one of the biggest challenges that we face as a project is just issues and pull requests come in at a greater volume than we can generally process them. So I try to keep it at the very least one of the baseline things I do as part of my job is day to day - or, I guess, week to week - open issue count doesn’t go up, and open pull request count does not go up.

Probably the majority of my day is just doing issue triage more than anything else, and reviewing pull requests and merging or closing them.

Does it feel like you’re making significant progress? I mean, when you say that the majority of your time is issue triage, and when I think about different ways that people work on open source, and the ways that I think companies should invest in open source, I think the way that Shopify is going about it is necessary in terms of pushing this thing forward, because certain projects, certain tasks and needs are so intellectually – not stimulating, but stressful; they require so much thought that you couldn’t possibly one day a week, or one week a month, or two hours at the end of your day make significant process, like for instance on a rewrite of ActiveRecord, or a portion of it.

I think we need people who are thinking about these things all the time, because that’s how you move software forward - you have to have the entire system in your head, and I like to think “Hey, at least Sean Griffin has ActiveRecord in his head and he’s moving it forward.” And then when I hear you say “Most of my time I’m triaging issues. It’s not that I’m working them down, it’s that I just don’t want them to get any bigger”, and I wonder “Wow, are we moving forward or are we just maintaining the status quo?”

I mean, a little bit of both. I think that moving forward also has a different meaning for ActiveRecord today than it used to. If you guys go to a Ruby Conf or a Rails Conf, you’ll notice the tone is very different than it was a few years ago. There’ll be a lot of people like “Oh my gosh, is Rails dying? Is Ruby dying?” I think the notion of that in general is sort of silly, but I think what a lot of that comes from is a misinterpretation of the signals that come from just “We, as a community, are no longer at the point where we’re going to be hip and breaking new ground.” Ultimately, Rails has shifted to a mature, stable framework.

It’s interesting, because I’ve been finding it harder and harder as time goes on to convince people why they should upgrade to new Rails versions, because we just don’t have as many killer features as we used to. But there’s a lot of work that goes into each release, and I think a lot of the things that people are missing – at Shopify I end up pairing with some people quite a bit on trickier test failures that they run into as part of upgrading Rails versions, and one of the ones that I’ve seen a lot recently was I changed how Dirty behaves in after_save callbacks in ActiveRecord. Basically, if you ask if an attribute has changed inside an after_save callback in 5.2, that’s going to behave the same as if you had that code, rather than in a callback, if you actually just put that code directly after the call to save, so it behaves as if it was after save.

[16:01] In 5.1 earlier, the way it works is we did a persistence, we would run the after_save callbacks, and then we clear the dirty flags. In 5.2, the way that’s gonna work is we do the persistence, then we clear the dirty flags, and then we run the callbacks. So it’s a bunch of new methods that are also more clear on what question you’re trying to ask.

The new methods are all named “Do you wanna know if there is a change that has yet to be saved, or do you wanna know if there is a change that was just persisted?”

That’s nice.

And all of the places that I’ve run into on Shopify that I’ve inadvertently broken things are places where there’s a bunch of code that is like in an after_save callback, mutating something and then expecting changed attributes to include both the things that were just saved, AND the thing that has yet to be saved. That actually just doesn’t work in 5.1. That change will only be reflected in the “Do we have an unsaved change method?”

Anyway, so every time that I’ve gone in with somebody to look at how to fix this test failure, ultimately we’ve realized that the code was really funky to work around this quirkiness, and once we fixed the test failure, we actually were able to just delete a bunch of this hacky code that was working around a quirk in ActiveRecord.

I think that’s a thing that a lot of people don’t realize going from version to version. For me, a successful Rails version is when you can upgrade and delete a bunch of your hacks.

So that’s definitely progress, especially when you talk about moving from franticness or from a lack of clarity or conciseness to a state of like a good, thought-through API. And we have that many people using different versions… Just moving a thing from step 1 to step 1.5 perhaps requires a lot of work, such as clarifying those specific – even just the method names in 5.1 versus previously.

So a lot of work goes into it. On the outside, when you’re looking at the new feature, it has less of the wow factor, less of the big wins that we were getting perhaps in the first ten years when there were obviously things missing and huge gains to be made.

Yeah. Certainly I think probably active storage is the thing that was most obviously – that we’ve added since Rails 5, it’s the first time we’ve had something I think everybody would agree was just a thing that almost every application needs and was obviously missing from Rails.

Yeah, tell us about active storage. This is the cloud storage, the after type stuff?

Yeah, it serves a similar purpose to Refile, or… Why am I spacing on the name of the other one that’s really popular…?

I’m still using Paperclip. Does that make me old-school?

It does, yeah. [laughter] CarrierWave is the other one I was thinking of. Paperclip is a thing that – it used to be CarrierWave and Paperclip, but I don’t think anybody is really using Paperclip anymore.

Well, you found him.

On that note, though - maybe this is a wrench to some degree in the conversation, but you said (and maybe it’s even something you can’t speak about) how you left ThoughtBot because management didn’t align, which I’m not gonna assume exactly what you said, but it was something about your desire to do this full-time… But then, Jerod, you use Paperclip - that’s a ThoughtBot project, right?

Right.

And there’s so much open source out there around Rails that’s almost default for many Rails developers that’s from ThoughtBot. So you would assume almost that ThoughtBot and management of ThoughtBot would see eye to eye with your future, Sean, and what you were trying to do; in hindsight, seeing this and this conversation has reminded me how prolific ThoughtBot is to Rails and open source, and… I don’t know, I kind of have a question mark on that.

[20:12] I don’t wanna undersell their dedication to open source…

Right, of course.

…but there’s a big difference between a developer who is billing and works on open source one day a week - because ThoughtBot just does not bill on Fridays… So there’s a big difference between that and somebody who is not billing at all.

You needed to be somebody that was doing this consistently, day-to-day, full-time, and that just wasn’t possible.

Yeah, exactly. So it’s a big question of just – I don’t wanna be working somewhere because it’s charity. I think that for the right company I very much do earn my paycheck and bring sufficient value to the company to justify my job’s existence. So when I say “Our goals didn’t align”, I mean that it just was not a situation there where I was gonna be able to justify my paycheck there.

Sorry, I actually asked that because I was like – you know, hearing Paperclip here in this conversation reminded me that they do such great work, and that’s why I just had that question mark.