This is an introductory post which describes how a secure communication channel works on top of HTTP. If you already know what HTTPS, SSL/TLS, Certificate or Public/Private keys stand for and how they work - skip this post as you won’t find anything new. Of course, you can still skim through and notify me of any errors or omissions. That would be very welcome!

The purpose of this post isn’t to repeat the Wikipedia or re-state obvious definitions, but to tie together some of the most important concepts underlying security of the web applications based on HTTP. The concepts will be explored through a mutual authentication problem stated in the next chapter. There will also be a follow-up post on the concrete implemenatation of the described security scheme on the JVM.

This post is targeted at people who are already familiar with HTTP, the network stack and how HTTP based applications work in general but aren’t quite sure what a certificate or a trusted CA means and what exactly goes into a HTTPS handshake.

The Problem

Suppose you have a server providing an API through the HTTP protocol and a known set of clients which should be allowed to access the server. What is the best way of authenticating clients and securing the API at the same time?

The standard way to secure HTTP is through upgrading the protocol to HTTPS (HTTP Secure). HTTPS is widely used in the internet (mainly by browsers) to verify sites identities and to encrypt all of the communication happening between the server and the client. Less known fact is that HTTPS allows for client authentication by the server as well as server authentication by the client - also called mutual auth.

HTTPS in detail

HTTPS is the same old HTTP transmitted in a secure way through the encrypted channel. Encryption in this case is achieved by the means of TLS¹ or Transport Layer Security which specifies how clients and servers interact together in order to establish and maintain a secure connection. In the case of HTTP clients and servers, TCP is usually used as the transport layer protocol in order to guarantee reliability. However, TLS has also been implemented over other transport-level protocols like UDP or DCCP².

To give a better high-level view, the following diagram shows how all of the protocols come together in the network stack:

When a client expresses an intent to communicate with a server over TLS, the following steps (also known as a handshake) happen³:

That’s quite a bunch of new terms! To figure out what’s actually happening during a handshake, we ought to have a grip on the most significant terms used in the client-server conversation: public and private keys, certificates and cipher suites. Each of these terms is a huge topic by itself. In this post we will cover the absolute minimum needed to understand how these elements work together in order to provide a secure environment.

Public and private keys

Public/private key technology is what underlines the cryptosystem of HTTPS⁴.

The main idea of a public/private key pair is that any message encrypted using a public key can later be decrypted only using the private key of the same key pair. Public keys, as stems from their name, are distributed publicly, while private keys must always be kept in secret. Exposure of a private key always leads to security breaches and major headaches for people supporting the security infrastructure.

From a practical point of view a public/private key is just a string of bytes kept in a file. Usually the bytes are encoded in human-readable characters (Base64) to facilitate easier sharing. For example, a public key generated with ssh-keygen, looks like this:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjPgT2BBP0pCS5fZ4geP8rgbs0hxJq4bV55aCs
Sfp6OJP9DaRZchq3Fmp7BtAG/MMDwWvD/Wrhf7Wl7gsyTAzFY8rHjIAAY5mUQfOjVX2gLRsc5KNJ
nKhOKiqTbm2s3Du0Fi9qrxu1LEG74LWNyRbW2T08NusFcBr4lCCHsZZSP74n2ju9aj1f5RBcEQtW
0/GG72cawptb//ZMvyPEjoUY0zhnp91fTrl33ckHU9e/omR3AtlCh2BX8x3+DhEG9dY3uEK7C1Mn
OOHRfu7X+R9w2ZY6au2upnUoyFkN89ELoIevsUDtUeG0i+d59qP9sVgHUTIFNC60wBwswSe/56Hz
dm3@dm3

The corresponding private key is longer but just as confusing for human eyes. You can generate keypairs any time you want, but once you start using a pair, remember - a private key must always be kept in secret.

If you ever used SSH or Github or any other developer tool which requires security, you’ve probably bumped into the instructions on how to generate a public/private key pair. Without understanding how keys work you’ll see that you keep coming back and re-reading instructions every time you have to use a secure mechanism. I encourage you to familiarize yourself with tools used to generate and manage public/private keys on your platform of choice - thankfully, there are heaps of tutorials on the internet.

Certificates

In layman terms a digital certificate is just a public key associated with an identity - information such as a name or email. The association is performed by signing the public key and the identity with a digital signature.

Now you might wonder, who has the authority to create digital signatures? The answer is - anyone. The same way anyone with a spare pen could sign a paper document, anyone with a spare private key could sign a digital document. However, a certificate signed by a malicious party doesn’t mean much as the receiver of the certificate is capable of verifying whether the signer is trusted.

Verification is possible because the signature of the certificate is extracted from the private key of the signer. This way we can always know whether the certificate was signed by the authority we already trust (as we can access the public key of this trusted authority). Therefore, certificates signed by trusted parties can be used to perform authentification in security schemes like HTTPS.

Certificates can come in different encodings stored in files with varying file extensions, however, when decoded they all follow the X.509 specification which includes the following fields⁵:

Certificate:
    Data:
        Version: 1 (0x0)
        Serial Number: 7829 (0x1e95)
        Signature Algorithm: md5WithRSAEncryption
        Issuer: C=ZA, ST=Western Cape, L=Cape Town, O=Thawte Consulting cc,
                OU=Certification Services Division,
                CN=Thawte Server CA/emailAddress=server-certs@thawte.com
        Validity
            Not Before: Jul  9 16:04:02 1998 GMT
            Not After : Jul  9 16:04:02 1999 GMT
        Subject: C=US, ST=Maryland, L=Pasadena, O=Brent Baccala,
                 OU=FreeSoft, CN=www.freesoft.org/emailAddress=baccala@freesoft.org
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
            RSA Public Key: (1024 bit)
                Modulus (1024 bit):
                    <...public key bytes...>
                Exponent: 65537 (0x10001)
    Signature Algorithm: md5WithRSAEncryption
        <...signature bytes...>

As you can see, the example certificate includes the identity in the Subject field together with the Subject Public Key Info.

Certificate Authorities

A party which issues certificates is called a Certificate Authority, or just a CA. In some cases you can be your own CA if you sign the certificate yourself. However, such a certificate will only be trusted if your client already believes in your trustworthiness through some external channel (e.g. knows you in person).

Self-signed certificates are useful outside of the public internet - in private networks or custom security schemes where clients implicitly trust the signer (like our mutual authentication problem). For example, some companies provide computers with pre-installed Certificate Authority bundles which list the company as a trusted CA. This allows the company to launch secure internal web applications without triggering any browser safeguards (e.g. ‘Continue at your own risk…’ screens).

On the other hand, public internet depends on commercial CAs which are trusted by all of the major browsers. You can obtain a signed certificate from such a CA if you can prove your identity and pay the requested amount of money - that’s exactly how owners of secure domains get their certificates. In the following diagram you can see an example of how a certificate is requested and used in the public internet.

Cipher Suites

A Cipher Suite is a combination of algorithms used to provide authentication, encryption and message authentication code (MAC) algorithms in the first steps of the TLS handshake. It’s a way to succinctly negotiate the kernel of the actual security mechanism. I won’t go into any of the specific algorithms and won’t even try to explain how they work as it is a job best left to security professionals.

The only thing worth noting is that Cipher Suites usually look like the following:

ECDHE-RSA-RC4-SHA
ECDH_ECDSA_WITH_AES_256_CBC_SHA
RSA_PSK_WITH_AES_256_GCM_SHA384

Which is a concatenation of separate algorithm names. The first part identifies the authentication algorithm (ECDHE, RSA, …), the second part - encryption (RC4, AES, …) and the last - MAC algorithm (SHA, …). You can see the complete list of Cipher Suites used in TLS on the IANA site.

The Complete Picture

Now we have all of the pieces for the mutual authentication puzzle:

• A Keypair and a Certificate for the Server
• A Keypair and a Certificate for each of the Clients
• A Certificate Authority which signs Certificates for the Server and each of the Clients

In the next post we will bring them together in a working implementation.

All of the diagrams were done in the free Diagram.ly tool.

---

1.  I will use TLS instead of SSL/TLS everywhere because TLS has mostly superseded SSL on the web. You may see TLS and SSL used interchangeably or in a union (like SSL/TLS). The truth is - both of these protocols are still in use with their latest versions being insignificantly (for this post) different. For a complete treatment you can refer to this serverfault post and Eric Rescorla’s book - SSL and TLS: Designing and Building Secure Systems, Addison-Wesley, 2001 ISBN 0-201-61598-3
2.  TLS Applications
3.  This is also a simplification. Refer to the Wikipedia article on TLS for the full handshake protocol in all of the flavours. Also I encourage you to read the post on Moserware for an in-depth look at the communication which occurs during a handshake.
4.  Wikipedia on Public Key Cryptography
5.  Wikipedia on X.509