End-to-end encryption: How it works, and why it breaks (Part 1 of 2)

Encryption at rest

Encryption has been an important part of our digital lives for many years now.

All Apple iPhones, for instance, and most Android devices, are shipped with what’s known in the trade as FDE, short for full-disk encryption, already activated.

The idea is simple and useful: anything, or more precisely almost anything, that you write to your device gets automatically encrypted on the way from the operating system to storage, and decrypted when it’s read back in.

Laptops, too, are often protected by FDE so that most of the data on their hard disks (a term we still use even though they generally aren’t disk-shaped any more) is stored in encrypted form.

Windows has BitLocker, Apple Macs have FileVault, Linux has LUKS, and the various BSD flavors have full-disk encryption software of their own.

Assuming you have a sensible lock code on your phone (please don’t use 000000, 123456, or anything obvious), a decently-chosen password on your laptop, or something of similar strength, FDE has two simple and useful benefits:

• Everything gets encrypted, whether it’s a cat video, your latest tax return, or a log file that you didn’t know was there that inadvertently included confidential data. If your laptop or phone gets stolen, you don’t have to fret about whether there was anything important that you forgot to encrypt.
• Wiping the decryption key alone is equivalent to overwriting the entire contents with pseduo-random garbage, because only encrypted data will be left behind once the key is destroyed. This gives you additional confidence when you want to re-use or recycle a device, and less to worry about when you trigger a remote wipe.

Something’s missing

But there’s something obvious missing here.

Once you’ve typed in your lock code or FDE password and started using your computer, all the data that you or the operating itself happen to access…

…is automatically decrypted the moment it’s needed, which makes the encryption as good as invisible.

If you attach a document to an email, for example, the data that your email program reads in will automatically be decrypted before it’s packaged for transmission.

If you upload a file to the cloud, it’s sent out in plaintext form, and if you plug in a USB drive and back up all your data (on Windows, the 40-year-old XCOPY program can do that with a single short command), the data will be transparently unscrambled as it’s read in, and written out unencrypted into the backup.

Clearly, it’s a good idea to add an extra layer of encryption at the file or folder level for data that needs protection of its own, but that still only provides what’s known in the jargon as encryption at rest.

Encryption in flight

End-to-end encryption, rather than directly protecting files, folders and disks, aims to deal with the problem of data that you need to exchange with someone else, whether it was encrypted or not to start with.

You’ll hear this sort of data protection referred to as encryption in transit, or perhaps a little more poetically as encryption in flight.

Whether we realize it or not, many of us rely on two types of end-to-end encryption every day:

• HTTPS: Secure HTTP

This is the encryption protocol that puts the padlock into your browser’s address bar, and protects against rogue or modified downloads.

It’s based on a system called transport layer security, or TLS for short.

(TLS used to be called SSL, so you will still see software names such as OpenSSL and wolfSSL, and hear people talking about SSL certificates, even though the old SSL version of the protocol is insecure and no longer used.)

In theory, and for the most part in practice, HTTPS means that data travelling to and from your browser, including sensitive information such as passwords and payment card details, is encrypted inside your browser using one-off encryption keys agreed when you connect to the server.

The data isn’t decrypted until it reaches the server at the other end.

This means not only that it can’t be snooped on, because potential eavesdroppers don’t know how to decrypt it, but also that it can’t be modified in transit, because they don’t know how to produce correctly authenticated data either.

• VPNs: Virtual private networks

As we explained in an earlier article, most VPNs act as if they were a pair of physical network adapters that were plugged directly into each other, with no other wires or monitoring devices along the way.

As long as the encryption keys used at each end are unique to that connection, and aren’t shared with anyone else, the data exchanged between them is meaningless to any eavesdropper or interceptor in the path, even if the encrypted traffic itself goes over the public internet.

Lest you forget

As with full-disk encryption, which ensures that everything written to disk is encrypted and neatly avoids the problem of files that you intended to encrypt but didn’t, VPNs provide protection even for apps that you forgot to set up (or that can’t be configured) to encrypt data across the network.

HTTPS, for example, is (almost) universally supported by every browser and web server on the market, but some applications that really ought to be using it, such as software updaters or network searching tools, might not do so, or might not do so all the time.

A good example of a snoopable network service that isn’t always protected by encryption is DNS, or the domain name service, whereby your computer turns a human-friendly domain or server name such as solcyber.com into a network number (IP address) that your operating system can use, such as 198.51.100.17.

Even if your browser is configured to shield your DNS queries by encrypting them, which many browsers do by default these days, other apps on your computer may be relying on unencrypted DNS replies provided by your ISP.

This can leak a record of every site you were interested in, albeit without recording what you did there if you then paid the site af visit.

This DNS request, for instance, was captured on my own router, situated one hop away from my laptop on the way to Google’s 8.8.8.8 DNS server, but any other router at any other ISP along the network path could have snagged the same data:

A VPN would shroud the data in all my network packets, including this one, something that feels as though it would improve security.

But a VPN alone doesn’t provide true end-to-end encryption, because the VPN’s layer of encryption gets stripped off at the VPN provider’s own servers, delivering the plaintext DNS packet back onto the internet at that point.

That leaves the packet snoopable not only for the rest of its journey, but also snoopable directly by any VPN service that is incompetent or goes rogue.

In contrast, DNS-over-HTTPS (DoH), made popular by many modern browsers that activate it by default if they can, extends the “ends” of the end-to-end encryption.

With DoH, your DNS requests are scrambled even before they enter any VPN (which scrambles them again, of course); remain scrambled while they’re transiting the VPN; and are only decrypted for processing when they reach the DoH server, which in our example below is at Google:

Between your computer and a DoH server, however, even snoops and cybercriminals in a position to intercept all your network traffic (for example, by monitoring routers at your ISP or at a VPN service acting as a virtual ISP) can’t make sense of what you asked for.

Just as importantly, they can’t modify the reply to feed you with fake information, for example to misdirect your computer to an imposter site in order to land your browser on a phishing page, or to feed an automatic software update program with a malware-infested substitute for the real thing.

And it’s not only browsers and auto-update tools that support HTTPS these days.

Other applications that support TLS and HTTPS, and indeed often insist on it, include dedicated banking apps and video streaming services.

Other software, notably instant messaging apps such as WhatsApp, Signal and Telegram, also rely on end-to-end encryption, using cryptographic protocols that are similar in purpose to TLS but designed to provide functionality that isn’t needed by web browsers or online downloaders.

You can only browse to a website when it is online, so it’s always possible to negotiate an end-to-end cryptographic connection every time you visit. But you may want to send an instant message to a user whose device is offline and who will retrieve and read it later, so the message will need to be stored temporarily on an intermediate server, which mustn’t be able to decrypt it. Instant messaging protocols like Signal’s Double Ratchet take this and more into account.

Why encrypt end-to-end?

The primary protections that end-to-end encryption should provide are:

• Data in flight must be shielded against eavesdropping. No one listening in should be able to decode any of the data as it passes through the network. Even if all the setup packets used to establish a secure connection are intercepted, including those used to agree on a cryptographic key in the first place, the connection should remain secure.
• Data in flight must be shielded against tampering. Note that encryption alone generally doesn’t provide this guarantee. An attacker might not be able to adjust the encrypted data so that a bank transfer that should decrypt as RECEIVE $145 comes out as PAY $279 instead. But even if the attackers can’t predict what their tampering might do to the decrypted output, they might be able to provoke bugs or software misbehavior at your end, for example if a product code that is supposed to come out as ARXC89AQ emerges as Z55!W87A instead. There should be a reliable cryptographic method to detect tampering along the way, so that any output based on modified input can be considered untrusted and discarded.
• Data in flight must be shielded against decryption in the future. Encrypted payment card details, contract information and intellectual property are all examples of data that needs to be secure tomorrow, even if it’s intercepted and stored today. Importantly, end-to-end encryption should not depend only on long-term cryptographic keys that could be stolen or leaked later. Ephemeral, or one-off, key material that is never saved at either end must be incorporated into the protocol to provide what’s known as forward secrecy, to ensure that a future data breach doesn’t compromise data intercepted in the past.

What to do?

With all this in mind, and with end-to-end encryption widely implemented and used, it sounds as we though can stop worrying about data-grabbing rogues on the internet.

Yet we we keep hearing about breaches and busts based on data recovered from individuals and businesses, even when they thought they were taking great care to use end-to-end encryption for all their communications.

Unfortunately, there are lots of reasons why this happens, and you can read about those problems, how they can be exploited, and how to protect against them, in Part 2.

Until next time:

• Build a strong, human-centric security culture in which cybercriminality of any sort is more likely to be spotted and reported. Signing up with SolCyber will actively help you to do this.
• Aim for a security operations center (SOC) that will proactively monitor for intrusions and data exposures, and ensure that you really are getting the protection that your data encryption choices are supposed to provide. Signing up with SolCyber means you don’t need to build and then run a SOC of your own.
• Add an extra layer of security to your mobile devices, which are typically protected only by basic MDM (mobile device management) tools that assume apps are working correctly and securely because they came from the App Store or Google Play. Signing up for SolCyber Mobile Protection brings your mobile threat response to a new level.
• Now read 🔗 Part 2. Find out how attackers subvert end-to-end encryption, and learn how to stop them.

Learn more about our mobile security solution that goes beyond traditional MDM (mobile device management) software, and offers active on-device protection that’s more like the EDR (endpoint detection and response) tools you are used to on laptops, desktops and servers:

More About Duck

Paul Ducklin is a respected expert with more than 30 years of experience as a programmer, reverser, researcher and educator in the cybersecurity industry. Duck, as he is known, is also a globally respected writer, presenter and podcaster with an unmatched knack for explaining even the most complex technical issues in plain English. Read, learn, enjoy!

Featured image of old phone by Andy Song via Unsplash.