37signals Podcast transcript

Subscribe to the podcast in iTunes or your favorite feed reader.

Episode #20: Programming roundtable (Part 1 of 3)

Matt Linderman: Welcome to the 37signals podcast. We're doing a round table with three of our programmers here at 37signals. Who are you guys?
Jeffrey Hardy: I'm Jeff Hardy.
Jamis Buck: I'm Jamis Buck.
Jeremy Kemper: And I'm Jeremy Kemper.
Matt: We're sitting at our new offices in Chicago. What do you guys think so far?
Jamis: Loving it.
Jeremy: It's pretty awesome.
Matt: We've got a specific room for recording audio and podcasts, and we're using that. We're going to read out questions that you guys posted at Signal vs. Noise, so why don't we get started? A question from Maksymilian Sleziak: "Tell us more about you. Who are you? Where did you work before 37signals, and how did you get involved into 37signals?"
Jamis: I'll start. This is Jamis. I was originally working at Brigham Young University, and I met David at the 2004 RubyConf when he was just introducing Rails. I'd been working on the SQLite3-Ruby gem. He was looking at the time to try and bundle a database with Rails, and so we talked. I wrote the first adapter for the SQLite stuff for Rails. He liked what he saw, and he and Jason flew me out to the building of Basecamp in February of the next year and made me an offer. I've been working here ever since.
Jeremy: Similar path of meeting David through Rails. Got involved with a couple of Rails ventures and ended up here at 37signals. I did web stuff before then, so it was a pleasure to discover Rails and to discover a company like 37signals where we can work this way.
Jeff: Yeah, my story's pretty similar, too. I started a company a long time ago just doing web design. All of our customers needed web apps, web programming done, and I didn't know how to do it, so I went to the bookstore and bought some books and taught myself Perl. That's how I got into programming. Then when Rails came out, I did Rails consulting for a while and eventually came to the mothership of Rails companies.
Matt: Alright, Jacob Bandes-Storch asks, "What do you use for version control? How do you have testing deployment set up?"
Jeremy: We use Git. We used Subversion for a long, long time, and we were pretty happy with it. Then we took the plunge at 37signals to get I think it was...
Jamis: It might have been through me.
Jeremy: But it's been a great change, going from something that seemed like it was plenty. Subversion just works. But something like Git, you don't really realize what the features will do for you until you're using it. The flexibility that gives you, it's hard to quantify when you don't know that certain things are possible.
Jamis: The first thing we really noticed moving from Subversion to Git was the ease of branching. That was the thing that really sold us initially. But like Jeremy said, there's a lot that you notice as you start using it that you just never noticed before.
Matt: Dan wants to know, "What does your deployment process look like?"
Jeff: [Joking] Cap production deploy. No, it's Capistrano, which Jamis wrote. That's probably pretty much like what every other Rails developer is doing.
Jamis: We do have a staging environment that we deploy for larger changes. For small, easy bug fixes, we'll often deploy straight to production, but for the larger things we'll deploy to staging and test and then move that into production.
Jeremy: We don't have a fixed process, so we have a checklist for ensuring that a change doesn't break something inadvertently. So rather than going through a pre-deploy process, we'll deploy and then watch and make sure that nothing bad happened after the fact. It keeps the deploy process pretty lightweight.
Jeff: It's not uncommon to deploy many times a day. A normal day can see upwards of 10 deploys across all apps.
Jeremy: That's one really nice thing about having everything super-automated. You can just deploy, and it's trouble-free. We can deploy in the middle of the day, and it's not a big deal.
Matt: Rupert wants to know, "How do you decide which new technology you use and what not to use?"
Jeremy: That's a tough one. It's a value judgment. If you go to the supermarket and see a fruit that's unfamiliar to you, what are you going to do? If you're feeling interested and you're up for something, you give it a shot and just see how it feels. [overlapping talks]
Jamis: If you hear enough people talking about it, then you're more likely to want to try it out.
Jeff: Git's an example of that. We weren't using it. You start using it for your own stuff, and eventually we were like, this looks like it would be a good fit, a good thing to replace Subversion. Maybe it's that it has to prove itself, and once it does, it's a pretty easy decision to make.
Jamis: We don't usually jump on the brand-new thing the first time we hear about it. Usually, one of us will play with it on the side a little bit and either come back with a recommendation for or against.
Jeremy: I think Redis is a good example of that. Redis is a persistent key-value store that got a lot of talk. There's a lot of buzz about it, and teasing out the reality from the buzz, it turns out all the buzz was really true. It's a really sweet tool. But it took us experimenting with it and really getting a sense for what is this, really, beyond a feature list so we could see how it would work in our products.
Matt: Julian wants to know, "What testing frameworks and processes do you use?"
Jamis: I love this question, because people ask it fairly often. We use Test::Unit right out of the Ruby standard library. We'll decorate it a bit with mocking frameworks like Mocha, and we use the testing extensions that Rails provides. But we don't really go into Cucumber.
Jeremy: We're pretty old school in that respect.
Jamis: We would like to experiment some with frameworks like Selenium and some of that, but we haven't gotten there yet.
Jeremy: I think what Jamis started with, it's really interesting that it's such a common question, testing frameworks as opposed to many other tool choices you could make. I'm not sure why that is. It might be because it's such an aesthetic choice and also such an unimportant choice. Ultimately, it's that you're testing at all, that you have an idea of what your software is supposed to do and that you're writing it out so you can programmatically test it. So it becomes a place that's just primed for bikeshedding.

There are so many different possibilities, and it's such a programmer-friendly kind of thing to work on, that there are tons of different libraries scratching everybody's individual aesthetic itch. It becomes a question of whether you like periwinkle blue versus cornflower blue. In the end, you're testing.

Matt: Hmart asks, "What apps have you migrated to Rails 3 already? Can you share tech details/lessons about it?"
Jeff: We haven't migrated any apps, but we've built new apps on Rails 3. So all of our big-paying apps are still Rails 2/3.x. Our Answers board is Rails 3. Our...
Jamis: Launchpad is Rails 3, isn't it?
Jeff: No.
Jeremy: It is converted, though. Actually, we've migrated some things, but we haven't launched them. So Launchpad, which is the main screen you go to for your 37signals ID, is converted to Rails 3, but we haven't deployed it, for one reason or another. We did a major effort converting Basecamp. We started early before the Rails three beta to prove out where we'd arrived with Rails three to see whether Rails was still something that felt right.

Basecamp is the ultimate test for us. It's been around since before Rails. So if Basecamp feels good on that version of Rails and you can deploy it, the whole thing works, then we know we're on the right track. That exposed a lot of issues with Rails going into the beta and thankfully helped polish it up.

But Basecamp's pretty big, and there's a lot of old stuff in it, so it's not deployable yet. It's going to take a bit of work to fix up old things that are tied to specific internal APIs and all the things you're not supposed to do but everybody does a little bit.

Matt: Jake asks, "How do you balance maintenance and utilities versus working on new strategic things?"
Jamis: A lot of that is taken care of by the system we implemented this year or late last year with the teams. Every two months, the programmers and the designers get shuffled like cards and dealt into new teams. The last month and a half I've been on support, so I've been dealing a lot with the maintenance side of it. But before that, I was working on new features and so forth. So we just all take turns on about a two-month cycle.
Jeremy: Part of the reason for doing the cycles was some frustration that grew, especially last year, where quite a few of us, actually, all three of us here, were working on some big internal infrastructure-type work. It's difficult when we're doing infrastructure work and we're very much a product development company. So feeling that push and pull of where our attention lies, especially when we're working for months on internal stuff and feeling pressure to, one, be doing product development - that's like the whole point of 37signals. I feel like teams are a way of addressing that, kind of time-slicing.
Matt: Tommy asks, "What type of development process do you use? Is it something very formal that's heavily documented or is it more informal?"
Jeff: Very informal. No formality.
Jamis: Calling it a process is generous. [laughter]
Jamis: But it's driven by the culture. We're all sort of the same way, so this is an unwritten process. If any of us were to use a formal process, that would be weird.
Jeremy: It would stand out, yeah.
Jamis: Yeah.
Jeremy: There are certain patterns to the way you do things, and if you describe a pattern and it becomes...or if you try ascribe that pattern to how other people should be working, it seems weird and it seems formalized because it's not natural. But if somebody were to come in as a software anthropologist and look at how we were working, I'm sure that they could derive some patterns.
Matt: Is there such a thing as a software anthropologist? [laughter]
Jeff: Job there for me.
Matt: Dave Tolsma asks, "What tools do you use?"
Jeff: Computers. [laughter]
Jamis: Our tool set is almost...I mean, by tools, we all use Macs. Our environments are probably almost identical, except I use TextMate as a text editor, you guys use Vim. It's not that exciting.
Jeff: We use Git. I've been using EngineX lately for local development, for running the stuff, and Passenger. A lot of you guys run the apps locally.
Jamis: I use Passenger.
Jeremy: I use Passenger. Otherwise, my favorite tool and the one that's most important to me is Vim. I'm living in a text editor, like, getting things done. It's just a way of navigating and moving around code you're working on. The other tools are just kind of out of necessity. You have to be able to do other stuff.
Matt: Kang Chen asks, "Do you guys do any A/B testing? If so, what do you use to help you decide whether something is valuable to the company?"
Jamis: We've done some A/B testing on the marketing side, which doesn't impact the programmers directly. The closest I've come to actually doing some A/B testing is on the signup page, which actually did require some changes to our signup code underneath. A/B testing had shown us one was more effective than the other, and so we went with it. The challenge from a programming point of view is trying to figure out how to make Rails work nicely with, you know, what percentage of the time do you show one view versus the other?
Jeremy: Generally, we don't A/B test big decisions. We'll just pull the trigger and we'll just go with it. We've considered it several times for certain features where we were internally kind of wondering whether it was the best idea. That's a way of pushing off that decision-making, pushing it off to - let's A/B test it, or let's do a support bill with code paths, let's grandfather in certain people doing things one way and new people doing things another way. It's really avoiding that decision.
Jamis: It's expensive. Like you said, you have to maintain multiple code paths. So unless there's a lot to be gained from it, there's really not a lot of point in doing it.
Matt: Ryan asks, "In a Rails app, do you put modularized code in lib/, or do you generally keep those abstractions/modules in vendor/plugins? What indicators do you use to determine if it's plugin worthy, so to speak? Also, what plugins do you use that 37signals was not responsible for writing, if any?"
Jamis: First things first, I guess. We don't use lib/ very much anymore. We used to use it more. It seems like more and more, we've started using app/concerns for putting modules and so forth in there, and then plugins for the code that doesn't really fit there.
Jeremy: There is kind of a path that we'll see a bit of code follow as it becomes extracted from an app. If we abstract it a little bit, it would be something that would usually be a concern, that would live in that app. If we find we need to use it in other apps, we'll pull them to a plugin. Otherwise, it's a fairly natural progression. Like, if you need to use it in just the app in multiple places, you'd abstract it and put it in a concern. Sometimes you'd use Lib if it were a standalone kind of object rather than decorating some existing object. There's no hard and fast rules. Things could just kind of graze up and it ends up where it ought to belong.
Matt: OK. You got more?
Jeff: I was going to say, as far as plugins being in-house...
Jamis: Oh, yeah. Most of our plugins are things that we've extracted out of the apps, so when all the apps have something in common, that makes its way into a plugin there. There aren't a lot of other plugins that we use that we didn't write, and it's not really intentional, like we're going to have a way to not use other people's stuff. It's just usually we already have some abstraction.
Jeremy: Our apps are pretty old now too, so a lot of the stuff we did predates what...I mean, there was no other choice at the time. So something I noticed - Pratik implemented most of Sortfolio, and that was built from scratch, and we have the full realm of plugins and existing tools to choose from and we reused a lot of them. So, starting from scratch, it was a lot easier to make those choices: let's just use this, use this, use this. Just take them off the shelf. Seeing that kind of fresh look at what we could use and couldn't use, I think most of us, having worked on the older apps, we more naturally reach for our own internal tools rather than even looking at what's in those other toolboxes.
Matt: Patric Schmid asks, "How do you decide what feature for which product to improve/implement next?"
Jeff: We've got a list. [laughing]
Jeff: Whenever someone is dissatisfied... like, one of us is dissatisfied with a product, we kind of throw it on a list somewhere and then when the teams get shaken up, the first thing we do as a new team is sit down and look at the lists and say, "OK, what do we want to work on?" It's guided some by, like, David and Jason and Ryan. They'll prioritize things a little bit.
Jeremy: So they'll do some curating. Generally things will be bubbling up, especially new product development. New features need a lot of work on. Somebody will champion them. I think for things that are annoying people, it's a little bit more difficult to prioritize, because they're smaller chunks of work. You have to kind of pack multiple things into a two-week iteration and it becomes more difficult. Then you just make the choice: this iteration's going to be about cleaning things up in this application, different pain points we've accumulated over the past month or so.
Jeff: And time boxing is a whole other topic. It's kind of hard to figure out sometimes what the scope of some of this is, but we're getting better at it.
Jeremy: But otherwise, that decision-making is pretty much decentralized. Things just kind of bubble up and we don't have an internal process or ticket system for making those decisions.
Matt: Is it easy to get consensus on that, or is there ever push-back in between the members of the group?
Jeff: Never has been, in my experience.
Jamis: I mean, usually we're all on the same page. If we have a general sense that something needs doing, and like Jeremy said, it just bubbles up to the top and it gets grabbed.
Jeff: And if someone is really passionate about one feature that they want to work on, the rest of us are usually fine with it. It's not like you have two big ideas that are conflicting.
Jeremy: Yeah, it's actually kind of a blessing, and having some passionate idea floating out there, because it's much easier to run with that. If you have just a bunch of ideas, they're just floating out there and it's hard to decide what really matters. But that passion attached to it makes the decision a lot easier.
Matt: Alright, great. We've got more questions but we want to continue that on the next episode of the podcast. If you want, you can go to We'll post related links to this episode. We also post transcripts and you can find all the previous episodes there. We'll be back next time with more questions for the programming team.

More episodes of the 37signals Podcast.