This is an introductory post which describes how a secure communication channel works on top of HTTP. If you already know what HTTPS, SSL/TLS, Certificate or Public/Private keys stand for and how they work - skip this post as you won’t find anything new. Of course, you can still skim through and notify me of any errors or omissions. That would be very welcome!
The purpose of this post isn’t to repeat the Wikipedia or re-state obvious definitions, but to tie together some of the most important concepts underlying security of the web applications based on HTTP. The concepts will be explored through a mutual authentication problem stated in the next chapter. There will also be a follow-up post on the concrete implemenatation of the described security scheme on the JVM.
This post is targeted at people who are already familiar with HTTP, the network stack and how HTTP based applications work in general but aren’t quite sure what a certificate or a trusted CA means and what exactly goes into a HTTPS handshake.
Suppose you have a server providing an API through the HTTP protocol and a known set of clients which should be allowed to access the server. What is the best way of authenticating clients and securing the API at the same time?
The standard way to secure HTTP is through upgrading the protocol to HTTPS (HTTP Secure). HTTPS is widely used in the internet (mainly by browsers) to verify sites identities and to encrypt all of the communication happening between the server and the client. Less known fact is that HTTPS allows for client authentication by the server as well as server authentication by the client - also called mutual auth.
HTTPS is the same old HTTP transmitted in a secure way through the encrypted channel. Encryption in this case is achieved by the means of TLS1 or Transport Layer Security which specifies how clients and servers interact together in order to establish and maintain a secure connection. In the case of HTTP clients and servers, TCP is usually used as the transport layer protocol in order to guarantee reliability. However, TLS has also been implemented over other transport-level protocols like UDP or DCCP2.
To give a better high-level view, the following diagram shows how all of the protocols come together in the network stack:
When a client expresses an intent to communicate with a server over TLS, the following steps (also known as a handshake) happen3:
That’s quite a bunch of new terms! To figure out what’s actually happening during a handshake, we ought to have a grip on the most significant terms used in the client-server conversation: public and private keys, certificates and cipher suites. Each of these terms is a huge topic by itself. In this post we will cover the absolute minimum needed to understand how these elements work together in order to provide a secure environment.
Public/private key technology is what underlines the cryptosystem of HTTPS4.
The main idea of a public/private key pair is that any message encrypted using a public key can later be decrypted only using the private key of the same key pair. Public keys, as stems from their name, are distributed publicly, while private keys must always be kept in secret. Exposure of a private key always leads to security breaches and major headaches for people supporting the security infrastructure.
From a practical point of view a public/private key is just a string of bytes kept in a file. Usually the bytes are encoded in human-readable characters (Base64) to facilitate easier sharing. For example, a public key generated with
ssh-keygen, looks like this:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjPgT2BBP0pCS5fZ4geP8rgbs0hxJq4bV55aCs Sfp6OJP9DaRZchq3Fmp7BtAG/MMDwWvD/Wrhf7Wl7gsyTAzFY8rHjIAAY5mUQfOjVX2gLRsc5KNJ nKhOKiqTbm2s3Du0Fi9qrxu1LEG74LWNyRbW2T08NusFcBr4lCCHsZZSP74n2ju9aj1f5RBcEQtW 0/GG72cawptb//ZMvyPEjoUY0zhnp91fTrl33ckHU9e/omR3AtlCh2BX8x3+DhEG9dY3uEK7C1Mn OOHRfu7X+R9w2ZY6au2upnUoyFkN89ELoIevsUDtUeG0i+d59qP9sVgHUTIFNC60wBwswSe/56Hz dm3@dm3
The corresponding private key is longer but just as confusing for human eyes. You can generate keypairs any time you want, but once you start using a pair, remember - a private key must always be kept in secret.
If you ever used
Github or any other developer tool which requires security, you’ve probably bumped into the instructions on how to generate a public/private key pair. Without understanding how keys work you’ll see that you keep coming back and re-reading instructions every time you have to use a secure mechanism. I encourage you to familiarize yourself with tools used to generate and manage public/private keys on your platform of choice - thankfully, there are heaps of tutorials on the internet.
In layman terms a digital certificate is just a public key associated with an identity - information such as a name or email. The association is performed by signing the public key and the identity with a digital signature.
Now you might wonder, who has the authority to create digital signatures? The answer is - anyone. The same way anyone with a spare pen could sign a paper document, anyone with a spare private key could sign a digital document. However, a certificate signed by a malicious party doesn’t mean much as the receiver of the certificate is capable of verifying whether the signer is trusted.
Verification is possible because the signature of the certificate is extracted from the private key of the signer. This way we can always know whether the certificate was signed by the authority we already trust (as we can access the public key of this trusted authority). Therefore, certificates signed by trusted parties can be used to perform authentification in security schemes like HTTPS.
Certificates can come in different encodings stored in files with varying file extensions, however, when decoded they all follow the X.509 specification which includes the following fields5:
Certificate: Data: Version: 1 (0x0) Serial Number: 7829 (0x1e95) Signature Algorithm: md5WithRSAEncryption Issuer: C=ZA, ST=Western Cape, L=Cape Town, O=Thawte Consulting cc, OU=Certification Services Division, CN=Thawte Server CA/emailAddressemail@example.com Validity Not Before: Jul 9 16:04:02 1998 GMT Not After : Jul 9 16:04:02 1999 GMT Subject: C=US, ST=Maryland, L=Pasadena, O=Brent Baccala, OU=FreeSoft, CN=www.freesoft.org/emailAddressfirstname.lastname@example.org Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (1024 bit) Modulus (1024 bit): <...public key bytes...> Exponent: 65537 (0x10001) Signature Algorithm: md5WithRSAEncryption <...signature bytes...>
As you can see, the example certificate includes the identity in the
Subject field together with the
Subject Public Key Info.
A party which issues certificates is called a Certificate Authority, or just a CA. In some cases you can be your own CA if you sign the certificate yourself. However, such a certificate will only be trusted if your client already believes in your trustworthiness through some external channel (e.g. knows you in person).
Self-signed certificates are useful outside of the public internet - in private networks or custom security schemes where clients implicitly trust the signer (like our mutual authentication problem). For example, some companies provide computers with pre-installed Certificate Authority bundles which list the company as a trusted CA. This allows the company to launch secure internal web applications without triggering any browser safeguards (e.g. ‘Continue at your own risk…’ screens).
On the other hand, public internet depends on commercial CAs which are trusted by all of the major browsers. You can obtain a signed certificate from such a CA if you can prove your identity and pay the requested amount of money - that’s exactly how owners of secure domains get their certificates. In the following diagram you can see an example of how a certificate is requested and used in the public internet.
A Cipher Suite is a combination of algorithms used to provide authentication, encryption and message authentication code (MAC) algorithms in the first steps of the TLS handshake. It’s a way to succinctly negotiate the kernel of the actual security mechanism. I won’t go into any of the specific algorithms and won’t even try to explain how they work as it is a job best left to security professionals.
The only thing worth noting is that Cipher Suites usually look like the following:
ECDHE-RSA-RC4-SHA ECDH_ECDSA_WITH_AES_256_CBC_SHA RSA_PSK_WITH_AES_256_GCM_SHA384
Which is a concatenation of separate algorithm names. The first part identifies the authentication algorithm (
RSA, …), the second part - encryption (
AES, …) and the last - MAC algorithm (
SHA, …). You can see the complete list of Cipher Suites used in TLS on the IANA site.
Now we have all of the pieces for the mutual authentication puzzle:
In the next post we will bring them together in a working implementation.
All of the diagrams were done in the free Diagram.ly tool.