Developer.
126 stories
·
55 followers

#define CTO

4 Shares

I joined Stripe as an engineer in 2010. I began by working on the backend infrastructure: designing the server architecture, creating our credit card vault, and producing internal abstractions to make people’s jobs easier. I loved writing code, but I also spent a bunch of time on other things: figuring out our recruiting program, shaping the culture, or making our first T-shirts (which have been banned since we hired our first designer). I wasn’t doing these things particularly because I preferred them to coding: instead, I had a very strong vision of the environment I wanted to be a part of, and I was willing to go out of my way to make it exist.

As time went on, I accumulated more and more responsibilities which were not strictly writing code. As Nelson Elhage liked to put it, my job became full-time “early employee”. My days were filled with writing cultural guides, acclimatizing new people, running our recruiting program, and the like. I’d often think it was time to give up on coding altogether, but I somehow always found a way back to it.

About a year and half ago, we officially declared me CTO. It was really just putting a word to what I was already doing — the most common reaction was “wait, I assumed Greg was CTO already”. This post is the story of what happened next: finding a partner to build our engineering team with, and figuring out my role as the organization changed.

Around last summer, to keep up with the needs of our engineering team, I started doing 1:1s with everyone. I would stack them all on a Tuesday, and be completely burned out by the end of the day. By the time I was recharged and productive again, it’d be next Tuesday, and it’d be time to do it all over again.

Through this time, I knew I was faced with a choice: the technical route or the people route. I’ve never found anything I loved more than writing code, but at the same time I knew we had a responsibility as an organization to support the amazing people we’d hired. I was talking with one of my friends, the CTO at another company, who told me about how transformative hiring a VP Engineering had been for him. I’d heard of VPs Engineering before, but I’d never really considered that you could actively go and get one. So I resolved to find a VP Engineering, and convinced our CEO and others internally that this was a good idea.

Internally, no one wanted the role. We’d hired primarily great individual contributors, people who like me wanted to be building, and correspondingly hadn’t hired anyone who really wanted to be managing the entire group. So it was clear we were going to have to hire from the outside.

Over the prior year and half, I’d been meeting with a bunch of professional managers to ask them for advice. A very small number had really stuck out to me as the kind of people who, if they were ever available, I’d work with in a heartbeat. But timing never works out that way, so I prepared to spin up a recruiting firm.

And not to be too anticlimactic, but during that time one of those managers, Marc Hedlund, coincidentally became available. I talked to everyone on the team to talk about the idea of hiring a VP Engineering generally, and this candidate particularly, and then all 25 engineers sat down to talk about it as a group. We didn’t know exactly what the role would be, though we knew one thing it would entail: lots of 1:1s (primarily with engineers today, and primarily with managers as we scaled). So we figured that the best strategy was to focus on whether the candidate would be great at 1:1s, and then once he was here figure out the rest of the role together.

We then set up interviews with Marc: four days of back-to-back 1:1 or 1:2 meetings with everyone on the team from 10a-6p, as well as a talk to the whole company. I asked him several times whether this grueling back-to-back interviewing was actually ok: it seemed like the kind of thing that would require superhuman stamina. But he reassured me that yes, this was totally fine. And sure enough, he was great.

I also talked with a bunch of people who had worked with him previously, some of whom were references given to me by Marc and some of whom were backchannel. There were very clear themes in the feedback: “he’s been an amazing influence and mentor for me”, “I still keep in touch”, “I’d love to work with him again”. I very much wanted to be working with someone who others say that about.

Finally, just as important was figuring out how well Marc and I would work together personally. Between the two of us, we’d have responsibility for growing and shaping the engineering team, and we’d be in trouble if we weren’t acting in concert. We spent a bunch of time together, talking about some of the problems Stripe was facing, Marc’s view on management and leadership, and the like.

I asked him, “How would we resolve disagreements?” His answer: “Well, we’d talk about it, like we’re doing now. The ideal setup is one in which we trust each other to work to make engineering better, and if either one of us has significant concerns about an approach, we talk about it, and if those concerns remain then hold off on that solution.”

Later, when I asked him what he thought about the process, his answer was “you folks have clearly never hired a manager before”. He took on the task of designing our management interviews. Ironically, I’d had much the same reaction to my own interview process back in the day, which had motivated me to work on our engineering interview design.

So we brought Marc in. In advance of his start, I added him to the email lists and added him on IM. We spent many hours talking over what was going on at the company, how to approach particular issues, reviewing email drafts for each other, and the like.

Once he started, I immediately transitioned all the 1:1s over to him. He also told me that I should feel free to send him any other stuff that I wanted off my plate (music to my ears given how overwhelmed I was at the time). Recruiting in particular was one that he suggested could make sense. I was actually kind of surprised: I’d never really known where recruiting fit, and had never really expected to be able to stop running our program. But it turns out that recruiting is typically part of the VP Engineering’s role. In fact, I slowly realized that the role I’d developed organically actually was almost entirely the usual VP Engineering role.

As time went on, one important shift was making sure that when people would take problems or issues to Marc rather me. The best thing I did on this front was take a vacation (my first ever) to Hawaii, and then holed myself up with a few engineers building CTF3. I also transitioned more and more responsibilities: because Marc was now talking to so many people, he was much better situated to know the problems any given group was facing, or how an organizational change was playing out.

The integration for any new executive is never smooth. There are false starts, and the rockiness of any change — in some places the culture needs to change to match the executive, and in others the executive needs to change to match the culture. I suffered from something I think a lot of engineers do, which is having trouble distinguishing “not the way I’d do it” from “bad”. Marc would run recruiting, or hold the team lead meetings, or communicate, in a very different way from me, and it took a while for me to really grok that everything was still ok.

The best advice I’ve gotten here is you have two traditional choices for how to delegate: you either delegate completely (and maybe you define some principles, but otherwise you have to leave the execution alone), or you stay involved in all the details. The latter model can work — think Mark Zuckerberg’s involvement in product — but you really only get to do it for one area. (There’s also a non-traditional approach, which is to train some people to be really good at simulating you, and have them run the delegated function. This may work but is not necessarily healthy.) But universally, “sparse micromanagement” (the best term I’ve heard for jumping in to some random issue, overturning all the decisions, and then disappearing) is the worst.

The way Marc and I worked through this was actually pretty simple: we spent way more time together. We started doing two hour-long 1:1s a week, one on Monday and one on Friday. There would be times when we wouldn’t have a ton to say, but it was incredibly important that we’d spend a bunch of time saying what we were worried about, what we were working on, where things are going. As a result, I went from having a bunch of issues where I didn’t know how Marc would react, to never being more than a few days from finding out, to eventually having a good enough mental simulator of him that I’d basically know how he’d respond. We employed a few other communication tricks, such as both getting IM on our phones or using the pattern of “I plan to do X thing; will time out and go for it in 24 hours barring objections”.

When you have a rapidly-growing company, you’ll find the organization itself constantly changing out from under you. As Marc learned to make changes in the organization, I found myself having to re-learn the same. We went from being an organization of close friends to being a post-Dunbar’s number company, where we had to figure out how to operate well even though some people have never even met. Having a partner in this (especially one who had run larger organizations) was incredibly valuable, and I don’t know how I would have made it through otherwise.

I’d expected that I would find myself suddenly with a role vacuum, and I’d get to fill it however I wanted. This was hugely appealing: since about a year in, my role had been almost entirely reacting to the problems of the day. This would be the first time I’d have a chance to define what I wanted to focus on.

I went on a bit of a CTO vision quest, asking about 20 different CTOs and VP Engineerings to describe their roles. I went in expecting every CTO to be chief architect. But I wanted to know how they pulled it off: I was trying to be chief architect in addition to “early employee”, and I had no idea how to do that job well part-time. (And most scarily, I felt our systems were being held back because everyone was assuming that I would be on top of the problems.)

But to my surprise, I found only one CTO for whom that was true. Everyone else viewed themselves as the facilitators of the technology organization. Sometimes this was about connecting senior engineers. Sometimes it was mentoring. One thought-provoking case for me was a CTO who was effectively head of product. I realized that there’s a huge difference between head of product and head of infrastructure: a good product is simple and digestible, so one tends to quickly distinguish bad from good. In contrast, you can tell good infrastructure only by building on it extensively. So it’s much more tractable to be part-time head of product than to be part-time chief architect.

I realized the most important thing to do was to empower our engineers to make big changes and improvements (plus, we’d been hiring some especially amazing engineers recently, and anything else would be kind of nonsensical). Marc and I set to work on enabling and encouraging major projects to improve the core of Stripe. I also started an “architecture working group”, a group of engineers available to help others build better systems, and a forum for figuring out where we’re taking our infrastructure.

However, one thing I hadn’t anticipated was suddenly losing my feedback loop. I had a snapshot of the organization from back before Marc joined, but I wasn’t getting any updated information. If I tried convincing someone that X was a problem, or Y was the right approach, all my anecdotes and evidence were from the before-time. It was like being a ship formerly tied to shore, but now adrift after someone cut the lines. And most worryingly, the problem was getting strictly worse with each passing day.

I spent a bunch of time thinking about how anyone can even know what’s going on in an organization, or what the problems of the day are. I came up with at most four possible mechanisms:

  1. Doing the work: if you’re, say, writing code, you have direct experience with what’s good and what’s bad.
  2. Talking to lots of people: you get an aggregate view and can tell what the sentiment is.
  3. Observing the work: maybe you could try to be architect-y and read all the diffs. (I actually once spent a month doing this, but it felt like a waste of time — most of the work was reconstructing the author’s state of mind.)
  4. Plan the work: you could try to plan everything out, though I’m actually not sure that this is sustainable without some other feedback loop.

I realized I was doing none of those. And looking at that list, I knew exactly which one would make me happiest. The question was how to get there.

I played with a variety of patterns for writing code. Many of them didn’t work: for example, “go off in a corner, come back with a prototype, and dump it onto an engineer to maintain (or just leave it unmaintained)”. I know there are CTOs who do this, but I’ve never found a variation that feels great (if you’ve seen this working, please let me know). An incrementally better version was “work with an engineer from a team, and build a new prototype together”. But this was also a bit of an uphill battle, since it required constantly fighting for people’s time.

I’d met another CTO at a dinner who told me he’d recently gotten back into coding. I met up with him asked him to reveal his secrets. How had he done it? He looked at me and said, “Coding’s the thing that I love. I knew I’d do a far better job at writing code than at anything else. I just needed to figure out how to get leverage out of it.” For him, that was improving both his company and the industry better by building an open-source platform for their infrastructure.

This was last month. I’ve started working on a team and writing code again. It’s not been easy: there are a million other things that come up. But the more time I’ve spent immersed in producing code, the better perspective I’ve had on the organization, and the better I’ve been able to support people, get them excited about what we’re building, and help them improve their work.


Sometimes I wonder whether I’m justified in making this choice — am I missing higher-leverage activities by focusing on code? But one of the best pearls of advice I’ve heard (from yet another CTO) is that it’s not about time management, it’s about energy management. It’s important to find activities that recharge you (independent of leverage) so that you have the energy to deal with the high-leverage draining stuff.

What my role will look like from here, I have yet to define fully. I’m too busy coding.

1,523

Kudos

1,523

Kudos

Read the whole story
Share this story
Delete

OpenStreetMap can’t grow

1 Share

I’m a huge fan of OpenStreetMap but the organization is a mess. Last year I fished around thinking I should get deeply involved with OSM, it’d be a good use of my time. But I gave up on the idea because I didn’t like what I learned about the culture. I think OSM could grow to be as important and influential as Wikipedia. But not with the current trajectory.

The problem boils down to a question of scale and influence. OSM has accomplished a huge amount with very little. No full time staff, lots of of borrowed server resources, annual budget of less than $200,000. Think what it could do with more! The impression I’ve got talking to the folks who make OSM work day to day is they’re perfectly happy with the current scale. The de facto leadership, the most active mappers, sysadmins, developers, don’t want a change. And there’s no single visionary leader to bring things forward.

There are related problems with OSM. There’s a strong anti-commercial bent which not only results in an awkward license but also an inability to engage with potential partners like Apple or MapBox. The community itself has some toxic elements; I gave up asking questions on the IRC channel after the seventh time someone implied my questions were dumb. And right now there’s a bunch of drama around elections for new leadership that indicates structural problems, years-old grievances getting aired ineffectively on mailing lists.

I don’t have a solution to get OSM to grow into the massive influence it could have. I worry there can’t be one, that culturally the active OSM members want to remain small and unsullied by commercial interests. I could say and do a lot more to try to help, but I don’t think it would get me anywhere.

Read the whole story
Share this story
Delete

Twitpic given eleventh-hour reprieve as Twitter saves all the pictures

1 Comment and 2 Shares

Photo sharing site Twitpic will not be deleting its substantial archive of tweeted pictures after all, it announced today, after coming to an agreement with Twitter.

Twitpic announced in September that it would be closing down and deleting all the pictures it had hosted—rendering millions of historic tweets meaningless—after a trademark dispute with Twitter. Twitter issued Twitpic an ultimatum: drop its trademark claim to the word "Twitpic" or lose access to Twitter's API. Twitpic opted for the latter, promising to close down on September 25.

This crisis appeared to be averted on September 18 when Twitpic founder Noah Everett announced that the site had been acquired and would live on. However, the details that he promised would follow never materialized, and on October 16th Everett said that, once again, Twitpic was to close down, this time on October 25.

Today, however, the destruction of Twitpic's photo database seems to have been averted. The Twitpic domain and photo archive is being transferred to Twitter. The site will remain active indefinitely, albeit without allowing any new pictures to be posted, thus preserving the pictures that many Twitter users have tweeted over the years.

Read Comments

Read the whole story
rafeco
4 days ago
reply
Sounds like Everett was basically extorting money from Twitter, especially given the earlier blocking of archive efforts.
acdha
3 days ago
Yeah, I don't know what he intended but it's definitely raised a huge red flag for any future businesses which he's involved in. Hopefully it has encouraged people to think about their exposure to a single service shutting down – I believe one of Dropbox's biggest selling points is that a full copy of everything is on every synced computer, making an outage or bankruptcy very easy to handle. I hope more companies appreciate that draw.
Share this story
Delete

E-Mail Analytics at Kickstarter

1 Comment

I spent the summer working as an intern at Kickstarter through the HackNY program in New York City. When I started, Kickstarter was gathering minimal data about e-mail activity on its platform. E-mail is a key part of the lifecycle of a Kickstarter project — it's a crucial part of project updates and backer surveys, for example. 

My project for the summer was to create a system that Kickstarter engineers could use to gather more granular e-mail data. The goal was to better understand how users are interacting with the e-mails Kickstarter sends.

Kickstarter uses a cloud-based SMTP provider called SendGrid to manage the delivery of e-mails to users. I built an e-mail analytics system for Kickstarter that integrates with SendGrid's webhooks. Webhooks enable SendGrid to send us data about e-mail events from their service, such as click, open or delivery events. That data is received by Kickstarter, where it is parsed and sent to our data stores. 

For example, if the recipient of a backer reward survey opens that e-mail, SendGrid is notified of that event. SendGrid then posts data about that event to our callback URL. When Kickstarter receives this data, our system adds additional properties to the data and then sends it along to the analytics service Mixpanel and our data store at Redshift for our further analysis.

This is one of the first graphs I made using our new e-mail data, which shows the volume of Kickstarter e-mail events by type.
This is one of the first graphs I made using our new e-mail data, which shows the volume of Kickstarter e-mail events by type.

We wanted more information about the processed e-mail messages than was offered by SendGrid's event notifications alone. So, we decided to add an X-SMTPAPI header to our e-mails. This header contains data about the associated e-mail, such as a project's id number. SendGrid saves any data encoded into this header, for the purpose of sending it to us as a part of the regular event notification data.

The data that is sent in the X-SMTPAPI header consists of what SendGrid calls Unique Arguments and Categories. The Unique Arguments can be any data that consists of key-value pairs. The category is a string we send containing the name of the type of email being sent.

So, we can send Unique Arguments associated with a given email that look like this:


And we can send the Category for the email like this:

You might be wondering when we're actually building the X-SMTPAPI header that gives SendGrid all of this useful data. To do that, I built an interceptor class. An interceptor is a feature of Rails ActionMailer, which gives you models and views to use for sending e-mail. An interceptor gives you a way to perform actions on e-mail messages before they are handed off to delivery agents. The interceptor class that I built creates and populates an X-SMTPAPI header for any e-mail marked as needing it. The interface for that class looks like this:


Once this header is built, the interceptor sends the modified mail message on to the delivery agent as normal. 

Adding an e-mail to this analytics process amounts to one method call in its mailer method. The method tags an e-mail as needing analytics. The interceptor then adds the X-SMTPAPI header containing our unique arguments and category data to any e-mail with the analytics tag, before finally sending it to SendGrid for delivery to the recipient. 

So, to sum everything up, this is how the e-mail analytics system I started building this summer works:

  • An e-mail is marked as requiring analytics in its mailer method. 
  • The e-mail is intercepted before it is handed to delivery agents, using a special interceptor class that builds the X-SMTPAPI header. 
  • SendGrid sends the e-mail.
  • SendGrid receives event notifications about the e-mail (delivery, click and open), which are then posted to our callback URL by their webhooks. 
  • Our callback URL parses the event notification data and hydrates it with new, richer data. 
  • The hydrated e-mail event data is sent to Mixpanel and Redshift for further analysis.

I came into this project with very little knowledge of the complexities of e-mail delivery and tracking. This new feature touches many parts of Kickstarter's larger e-mail system, so having the chance to build something that could interface with that in the simplest way possible was a fun challenge. It's also exciting to have built a resource that enables Kickstarter to understand what kinds of e-mails are most meaningful to users — by analyzing open rates, for example. I'm happy that I succeeded in seeing this feature through to completion!

Read the whole story
rafeco
4 days ago
reply
What I like about Kickstarter is that they do a great job of avoiding building things themselves from scratch
Share this story
Delete

Putting armor to the test

3 Comments and 4 Shares

European suits of armor always look so impractical when you see them in museums, but how did they perform in combat? Well enough for the wearer to do jumping jacks and move quickly on the battlefield.

(via the kid should see this)

Tags: video
Read the whole story
rafeco
4 days ago
reply
So cool
Share this story
Delete
2 public comments
diannemharris
22 days ago
reply
cool.
glenn
22 days ago
reply
Apparently you can also summersault in full armour. Also, your ass is quite exposed to swords
Waterloo, Canada

The iPad Zombie

3 Comments and 4 Shares

Allen Pike:

The only thing we can do as developers to disavow support for these devices is require a version of iOS that won’t run on them. Unfortunately, Apple will surely continue support for the A5 in iOS 9. If they do so, we won’t have a mechanism to cut off support for these old iPads mini and iPods touch until iOS 10 has reached wide adoption, likely in early 2017.

2017.

Read the whole story
rafeco
12 days ago
reply
My iPad Mini doesn't feel obsolete to me at all.
wreichard
12 days ago
I tend to agree, though I'm starting to notice the low RAM and how every time I switch apps, it has to reload whatever I was doing. But that's worse on 8 than before. In any given app, it's rarely an issue.
Share this story
Delete
2 public comments
kiyote23
13 days ago
reply
As an iPad 1 owner, I hear you, but I think the problem the devs are facing is that while the device might run the OS, it might not support the features they need for their app. Then you're adding additional overhead to your code to check what device is being used, and what features are available.

But yeah, it strikes me as whiny, too.
Iowa City, IA
jepler
13 days ago
reply
f---- all companies with aggressive discontinuation policies. wikipedia's iPad mini article (updated with ipad mini 3 information) says the A5-based ipad mini is still in production. Google's cached version of the apple store page from earlier today shows it for sale (the apple store itself is down for me right now)

And developers are already bitching that they can't discontinue app support of them, like, now? For a device that Apple was 6 hours ago? "Powerful processing and fast graphics make everything quick, from launching apps to browsing the web. And its power-efficient design delivers up to 10 hours of battery life.2"

(sarcastic remarks about android 2.3 phones to appear below vvvvv)
Earth, Sol system, Western spiral arm
acdha
13 days ago
I can understand *game* developers needing a better way to present minimum hardware requirements for high-end games. As far as I can tell, this guy makes two apps which give you different ways to queue and play music from your iTunes collection.
trekkie
12 days ago
My son uses an iPad 2, my wife has an iPad mini. They have no problems with the apps they use. I'm surprised at all the bitching, you'd think Apple would have the same problem but amazingly all their apps work just fine. not even 'super lag but fine' but they just work too. Odd.
Next Page of Stories