154 stories

How do we build encryption backdoors?

1 Comment and 2 Shares
(photo source/cc)
They say that history repeats itself, first as tragedy, then as farce. Never has this principle been more apparent than in this new piece by Washington Post reporters Ellen Nakashima and Barton Gellman: 'As encryption spreads, U.S. grapples with clash between privacy, security'.

The subject of the piece is a renewed effort by U.S. intelligence and law enforcement agencies to mandate 'backdoors' in modern encryption systems. This is ostensibly a reaction to the mass adoption of strong encryption in smartphones, and a general fear that police are about to lose wiretapping capability they've come to depend on.

This is not the first time we've been here. Back in the 1990s '90s, the Federal government went as far as to propose a national standard for 'escrowed' telephone encryption called the 'Clipper' chip. That effort failed in large part because the technology  was terrible sucked , but also because -- at least at the time -- the idea of ordinary citizens adopting end-to-end encryption was basically science fiction. seemed like a problem for the future.

Thanks to the advent of smartphones and 'on-by-default' encryption in popular systems like Apple's iMessage, and WhatsApp, the future is here. Americans are finally using end-to-end encryption at large scale and scale, and they seem to like it. This is And this is all scaring the bejesus out of the powers that be.

Hence crypto backdoors.

As you might guess, I have serious philosophical objections to the idea of adding backdoors to any encryption system -- for excellent reasons I could spend thousands of words on. But I'm not going to do that. What In fact, what I'd like to do in this post  here is put aside my own value judgements and try to take these government proposals at face value.

What I'd like to do here is tackle the purely technical side of the question, since nobody in government seems to be doing this.

Thus the question I'm going to consider in this post:
Let's pretend that encryption backdoors are a great idea. From a purely technical point of view, what do we need to do to implement them, and how achievable is it?
First some background.

End-to-end encryption 101

Modern encryption schemes break down into several categories. For the purposes of this discussion we'll consider two: those systems for which the provider holds the key, and the set of systems where the provider doesn't.

We're not terribly interested in the first type of encryption, which includes protocols like SSL/TLS and Google Hangouts, since those only protect data at the the link layer,i.e., until it reaches your provider's servers. I think it's fairly well established that if Facebook, Apple, Google or Yahoo can access your data, then the government can access it as well -- simply by subpoenaing or compelling those companies. We've even seen how this can work.

The encryption systems we're interested all belong to the second class -- protocols where even the provider can't decrypt your information. This includes:
This seems like a relatively short list, but in practice w're talking about an awful lot of data. The iMessage and WhatsApp systems alone process billions of instant messages every day, and Apple's device encryption is on by defaultfor everyone with a recent(ly updated) iPhone.

How to defeat end-to-end encryption

If you've decided to go after end-to-end encryption through legal means, there are a relatively small number of ways to proceed. proceed

By far the simplest is to simply of these is just to ban end-to-end crypto altogether, or to mandate weakencryption. There's some precedent for this: throughout the 1990s, the NSA forced U.S. companies to ship 'export' grade encryption that was billed as being good enough 'good enough' for commercial use, but weak enough for governments to attack. The problem with this strategy problem, of course, is that attacks only get better -- but legacy crypto never dies.

Fortunately for this discussion, we have some parameters to work with. within. One of these is that Washington seems to genuinely want to avoid dictating technological designs to Silicon Valley. More importantly, President Obama himself has stated that "there’s no scenario in which we don’t want really strong encryption". Taking these statements at face value should mean that we can exclude outright crypto bans, mandated designs, and any modification has the effectof fundamentally weakening encryption against outside attackers.

If we mix this all together, we're left with only two real options:
  1. Attacks on key distribution. In systems that depend on centralized, provider-operated key servers, such as WhatsApp, Facetime, Signal and iMessage,** governments can force providers to distribute illegitimate public keys, or register shadow devicesto a user's account. This is essentially a man-in-the-middle attack on encrypted communication systems.
  2. Key escrow. Just about any encryption scheme can be modified to encrypt a copy of a decryption (or session) key such that a 'master keyholder' (e.g., Apple, or the U.S. government) can still decrypt. A major advantage is that this works even for device encryption systems, which have no key servers to suborn.
Each approach requires some modifications to clients, servers or other components of the system.

Attacking key distribution

Key lookup request for Apple iMessage. The phone
number is shown at top right, and the response at bottom left.
Many end-to-end encrypted messaging systems, including WhatsApp and iMessage, generate a long-term public and secret keypair for every device you own. The public portion of this keypair is distributed to anyone who might want to send you messages. The secret key never leaves the device.

Before you can initiate a connection with your intended recipient, you first have to obtain a copy of the recipient's public key.This is commonly handled using a key server that's operated by the provider. The key server may hand back one, or multiple public keys (depending on how many devices you've registered). As long as those keys all legitimately belong to your intended recipient, everything works fine.

Intercepting messages ispossible, however, if the provider is willing to substitute its ownpublic keys -- keys for which it (or the government) actually knows the secret half. In theory this is relatively simple -- in practice it can be something of a bear, due to the high complexity of protocols such as iMessage.

Key fingerprints.
The main problem with key distribution attacks is -- unlike a traditional wiretap -- substitute keys are at least in theorydetectable by the target. Some communication systems, like Signal, allow users to compare key fingerprintsin order to verify that each received the right public key. Others, like iMessage and WhatsApp, don't offer this technology -- but could easily be modified to do so (even using third party clients). Systems like CONIKSmay even automate this process in the future -- allowing applications to monitor changes to their own keys in real time as they're distributed by a server.

A final, and salient feature on the key distribution approach is that it allows only prospectiveeavesdropping -- that is, law enforcement must first target a particular user, and only then can they eavesdrop on her connections. There's no way to look backwards in time. I see this is a generally good thing. Others may disagree.

Key Escrow 

Structure of the Clipper 'LEAF'.
The techniques above don't help much for systems without public key servers, Moreover, they do nothing for systems that don't use public keys at all, the prime example being device encryptionIn this case, the only real alternative is to mandate some sort of key escrow.

Abstractly, the purpose of an escrow system is to place decryption keys on file ('escrow' them) with some trusted authority, who can break them out when the need arises. In practice it's usually a bit more complex.

The first wrinkle is that modern encryption systems often feature manydecryption keys, some of which can be derived on-the-fly while the system operates. (Systems such as TextSecure/WhatsApp actually derive new encryption encrypton keys for virtually every message you send.) Users with encrypted devices may change their password from time to time.

To deal with this issue, a preferred approach is to wrap these session keys up (encrypt them) under some master public key generated by the escrow authority -- and to store/send the resulting ciphertexts along with the rest of the encrypted data. In the 1990s Clipper specification these ciphertexts were referred to as Law Enforcement Access Fields, or LEAFs.***

With added LEAFs in your protocol, wiretapping becomes relatively straightforward. Law enforcement simply intercepts the encrypted data -- or obtains it from your confiscated device -- extract the LEAFs, and request that the escrow authority decrypt them. You can find variants of this design dating back to the PGP era. In fact, the whole concept is deceptively simple -- providedyou don't go farther than the whiteboard. 

Conceptual view of some encrypted data (left) and a LEAF (right).
It's only when you get into the details of actually implementing key escrow that things get hairy. These schemes require you to alter every protocol in your encryption system, at a pretty fundamental level -- in the process creating the mother of all security vulnerabilities -- but, more significantly, they force you to think very seriously about who you trust to hold those escrow keys.

Who does hold the keys?

This is the million dollar question for any escrow platform. The Post storydevotes much energy to exploring various proposals for doing this.

Escrow key management is make-or-break, since the key server represents a universal vulnerability in any escrowed communication system. In the present debate there appear to be two solutions on the table. The first is to simply dump the problem onto individual providers, who will be responsible for managing their escrow keys -- using whatever technological means they deem appropriate. A few companies may get this right. Unfortunately, most companies suck at cryptography, so it seems reasonable to believe that the resulting systems will be quite fragile.

The second approach is for the government to hold the keys themselves. Since the escrow key is too valuable to entrust to one organization, one or more trustworthy U.S. departments would hold 'shares' of the master key, and would cooperate to provide decryption on a case-by-case basis. This was, in fact, the approach proposed for the Clipper chip.

The main problem with this proposal is that it's non-trivial to implement. If you're going to split keys across multiple agencies, you have to consider how you're going to store those keys, and how you're going to recover them when you need to access someone's data. The obvious approach -- bring the key shares back together at some centralized location -- seems quite risky, since the combined master key would be vulnerable in that moment.

A second approach is to use a threshold cryptosystem. Threshold crypto refers to a set of techniques for storing secret keys across multiple locations so that decryption can be done in placewithout recombining the key shares. This seems like an ideal solution, with only one problem: nobody has deployed threshold cryptosystems at this kind of scale before. In fact, many of the protocols we know of in this area have never even been implemented outside of the research literature. Moreover, it will require governments to precisely specify a set of protocols for tech companies to implement -- this seems incompatible with the original goal of letting technologists design their own systems.

Software implementations

A final issue to keep in mind is the complexity of the software we'll need to make all of this happen. Our encryption software is already so complex that it's literally at the breaking point. (If you don't believe me, take a look at OpenSSL's security advisoriesfor the last year) While adding escrow mechanisms seems relatively straightforward, it will actually require quite a bit of careful coding, something we're just not good at.

Even if we do go forward with this plan, there are many unanswered questions. How widely can these software implementations be deployed? Will every application maker be forced to use escrow? Will we be required to offer a new set of system APIs in iOS, Windows and Android that we can use to get this right? Answering each of these questions will result in dramatic changes throughout the OS software stack. I don't envy the poor developers who will have to answer them.

How do we force people to use key escrow?

Leaving aside the technical questions, the real question is: how do you force anyone to dothis stuff? Escrow requires breaking changes to most encryption protocols; it's costly as hell; and it introduces many new security concerns. Moreover, laws outlawing end-to-end encryption software seem destined to run afoul of the First Amendment.

I'm not a lawyer, so don't take my speculation too seriously -- but it seems intuitive to me that any potential legislation will be targeted at service providers, not software vendors or OSS developers. Thus the real leverage for mandating key escrow will apply to the Facebooks and Apples of the world. Your third-party PGP and OTR clients would will be left alone, for the tiny percentage of the population who uses these tools.

Unfortunately, even small app developers are increasingly running their own back-end servers these days (e.g., Whisper Systems and Silent Circle) so this is less reassuring than it sounds. Probably the big takeaway for encryption app developers is that it might be good to think about how you'll function in a world where it's no longer possible to run your own back-end data transport service -- and where other commercial services may not be too friendly to moving your data for you.

In conclusion

If this post has been more questions than answers, that's because there really are no answers right now. A serious debate is happening in an environment that's almost devoid of technical input, at least from technical people who aren't part of the intelligence establishment.

And maybe that by itself is reason enough to be skeptical.


* Not an endorsement. I have many thoughts on Telegram's encryption protocols, but they're beyond the scope of this post.

** Telegram is missing from this list because their protocol doesn't handle long term keys at all. Every single connection must be validated in person using a graphical key fingerprint, which is, quite frankly, terrible.

*** The Clipper chip used a symmetric encryption algorithm to encrypt the LEAF, which meant that the LEAF decryption key had to be present inside of every consumer device. This was completely nuts, and definitely a bullet dodged. It also meant that every single Clipper had to be implemented in hardware using tamper resistant chip manufacturing technology. It was a truly awful design.
Read the whole story
Share this story
1 public comment
3 days ago
Everything old is new again, even the clipper chip...

Siracusa Hangs It Up

1 Comment

John Siracusa:

Those who listen to the ATP, the weekly podcast I host with Marco Arment and Casey Liss, know that I’ve been contemplating hanging up my OS X reviewer’s hat for some time now. Producing thousands of words (and hundreds of screenshots) about each major release of OS X was my first real claim to fame on the Internet. The prospect of stopping has made me reconsider my public identity and sense of self. Who am I if I’m not “that guy who writes those OS X reviews”? But when I finally decided, the relief I felt let me know I’d made the right choice.

His collected reviews to date constitute a remarkable body of work. They’re not articles — they’re effectively books, each one written to John’s own impeccably high standards for thoroughness, accuracy, and writing quality. Writing a good book every year, about a moving target, on a tight deadline — that is tough.

But I’m going to miss it. A new release of OS X isn’t going to feel the same without a Siracusa review to go with it.

Read the whole story
3 days ago
I'm pretty surprised Apple has never hired Siracusa.
3 days ago
why buy the cow when you get the milk for free? Plus they'd never get him to leave the east coast.
Share this story

New York Times Story Asks: ‘Should Grown Men Use Emoji?’

6 Comments and 8 Shares

Matt Haber, writing for the NYT:

Given their resemblance to the stickers that adorn the notebooks of schoolgirls, not to mention their widespread adoption as the lingua franca of tweens and teens everywhere, some people wonder whether grown men should be using them at all.

Where is that “Reversed Hand With Middle Finger Extended” emoji when we need it?

Read the whole story
16 days ago
Grown human beings can make up their own mind without the NY Times' input.
Arlington, VA
16 days ago
Chico, CA
17 days ago
I love emoji.
Share this story
3 public comments
13 days ago
An evening with Matt Haber, 💤
Mother Hydra
14 days ago
Completely crazy notion right there, NYT.
Syrup City
16 days ago
I'm anything but a prolific user of emoji, but the idea that its use is tied to gender is beyond absurd.
Earth, Sol system, Western spiral arm

Positing That Tesla Is a Battery Company

1 Comment

Jeremy Welch:

Tesla Motors started as a Car company, but they should now be considered to be a Battery company for three key reasons:

  1. Tesla leadership has expertise in batteries and energy systems.
  2. Batteries are the most important component of an electric vehicle (EV).
  3. Tesla can enter other markets with the battery tech they developed while building EVs.

I’m always a little wary of any such “__ isn’t what you think it is” arguments, but, it’s interesting to note that the company is named after an energy pioneer, not a car pioneer.

Read the whole story
17 days ago
Does Tesla have any non-automotive products yet?
16 days ago
They have a house-based battery system either in development or nearing release. They've discussed it. I disagree, however, that they are a "battery company". Apple also has expertise in battery and energy systems, and likely as much as Tesla, and they are certainly critical to almost all of their devices, yet that doesn't make them a "battery company".
15 days ago
@petrilli I am legitimately curious how Apple designs their batteries or if they contract with a distributor for a specific size battery. Does an Apple engineering team design and create the specifications of their batteries and then Apple hands those to a manufacturer for fabrication or does Apple have a distributor supply them with X number of Y batteries. Given the large range of battery issues that I have heard of (I can't say specifically because I don't have any battery issues with my iPad) with iPhones shutting down with 4-8% battery and sometimes as much as 20% battery showing. That is a large variance in an Apple designed battery for Apple designed hardware running Apple designed software.
Share this story

Every Exit is an Entry Somewhere

1 Share

A little over nine years ago, RedMonk made its first analyst hire. As an aside, if that number makes you feel old, well, you’re not alone. Anyway, our choice for the first non-founder analyst was a then little known BMC software developer based out of Austin, who was perhaps best recognized for his rather irreverent technology blog, Drunk and Retired. At the time, there was some consternation in the industry about the idea of hiring a developer to be an analyst. We fielded a lot of questions about the selection, but the quality of Cote’s work pretty quickly put those to rest.

It is probably in part due to Cote’s success – both with RedMonk and in his subsequent career at Dell, The 451 Group and now Pivotal – that we didn’t get nearly as many questions when we hired his replacement out of the Mayo Clinic. Superficially it might sound odd to hire as a technology industry analyst a Research Fellow doing drug discovery, but we’ve always been believers that we can teach someone to be an industry analyst – we’ve been in the business for thirty years collectively, after all. What we can’t do as easily is teach the skills necessary to be a good analyst: being creatively inquisitive, being able to communicate effectively or having an understanding and ability to grasp the macro trends shaping our industry.

When we find those, then, wherever they might be and whatever the background, we’re interested.

And find those we did in Donnie Berkholz. In spite of – or was it perhaps because of? – his non-traditional industry background, Donnie hit the ground running with us. With his background in statistical and quantitative analysis, he quickly made a name for himself exploring statistical trends, making predictions and that most important of RedMonk analyst duties: buying developers beers. He’s done nothing but prove us right in our initial belief that he could do this work at a high level, which is why we’re sad to be saying goodbye.

But like his predecessor, the time has come for Donnie to graduate from RedMonk. He’ll have more to tell you about his future plans shortly I’m sure, but suffice it to say you will still be seeing him around. His last day with us will be next Friday, as he wraps up a few projects with us. In the meantime, on behalf of all of us at RedMonk: we wish you all future success, Donnie, thank you for all of your efforts in helping RedMonk keep advancing the ball downfield. We’re happy to have played a small part in helping you transition into this industry.

As with any departure, the obvious next question is: what does this mean for RedMonk?

In the short term, more travel – they’re only making more conferences, and there are only so many of us. And we’re no more looking forward to filling Donnie’s shoes than we were to filling Cote’s. But over the longer term, our mission remains the same: we’re the analyst firm that is here for – and because of – developers. We will continue to fight the good fight on behalf of that constituency, even as market awareness of their importance adds more and more allies to our ranks. As a species we have a tendency to take progress for granted, but if you stop and think it really is amazing how different the reception our developer-centric message is today versus even four or five years ago.

Who will we hire? The best fit we can find. Like the Oakland A’s, we’ll think creatively about the opening and we’re already in the process of talking to some interesting candidates. That said, we’re open to all interested parties. And given our trajectory, we might even be adding more than one new analyst, but we’ll take it one step at a time.

Fair warning to all applicants: we will be very picky. You need to be able to communicate effectively, write well and be committed to rational discourse. You should have a reasonable online presence and a passion for developers and the tools they use. Other things we’ll look for include programming skills, economics and statistics training and experience with rich media. Previous experience as an analyst is a bonus, but absolutely not required. Interested? Send a CV and anything else you believe we should consider to hiring @

You will have big shoes to fill, whoever you are. The analysts that have come before you have done some incredible work, and we expect nothing less from you.

Why work here? The most obvious reason is that RedMonk remains, in my obviously biased opinion, an amazing place to work. There aren’t many too many jobs available that allow you to influence the strategic direction and decision making process of some of the biggest and most important technology companies in the world – as well as their disruptors, that give you a pulpit to produce public research for some of the best and brightest developers on the planet. Fewer jobs still let you work on things that are important, things that improve the day to day lives of developers, and by extension, the users they service. Tim O’Reilly says to “work on stuff that matters“; we think we do, almost every day. And as you might guess from conferences like the Monktoberfest, we try and have fun doing it.

Add in the flexibility that working for a small firm offers, from the ability to define your own research agenda to good hardware to variable vacation time to the option of working from home, and it’s a damn good gig. If any of that sounds interesting to you, drop us a line.

Last, to our clients and customers: if any of you have questions about this news, feel free to contact myself (sogrady @ or James (jgovernor @ if you like, or Juliane (juliane @ as always. We’re happy to answer anything we can.

So we wish you well, Donnie, and look forward to seeing who will step up in your place.

Read the whole story
Share this story

Basic stock option questions

1 Share

It’s common for tech industry employees to be compensated with stock options. Stock options are complicated and many engineers I know are terribly naïve about how they work. But options are often the most valuable part of an employee’s compensation! This engineer’s guide to stock options is good reading.

Here are some basic questions every owner of stock options should ask their employer. With these answers an accountant can work out the value of the option package and plan a tax strategy.

  1. How many shares of the company will I own?
  2. How many shares of the company exist?
  3. What percentage of the company will I own?
  4. What is the strike price of my options?
  5. What is the fair market value of the stock today?
  6. Can I early exercise my options?

Many companies are reluctant to answer #2 and #3 (they are equivalent). Trying to keep an employee in ignorance about this is bullshit. Knowing what percentage of the company you own is the only way to evaluate your option package. Companies will generally answer this question if you press hard enough; if they refuse, it is a very bad sign.

Tax strategy is important for several reasons. Early exercising could save you ~20% in taxes later on. But even more importantly, early exercising could save you 100% should you leave the company. Most option agreements include a clause where your options disappear 30 or 90 days after you stop being an employee. If you quit and can’t afford the taxes to exercise those options, you can lose everything. Planning ahead matters.

Read the whole story
Share this story
Next Page of Stories