SSL Certificates on ELabs VMs SSL basics Obtaining a certificate Certificate files www.i2u2.org Certificates Checking certificate status Certificate history and renewal i2u2-dev

SSL Certificates on ELabs VMs

We maintain an SSL certificate for the e-Labs website www.i2u2.org on i2u2-prod. Since we don't exchange sensitive data with our users, we use SSL primarily for convenience: many modern browsers look for https connections automatically, and general support for network encryption is a worthwhile goal.

SSL basics

TLS/SSL (Transport Layer Security/Secure Socket Layer) certificates are used to verify a website's identity to a user and to allow secure communication between the user and website server. More precisely, the server's public RSA key allows for secure communication, while the SSL certificate confirms that that public key does, in fact, belong to i2u2.org (for example).

Conceptually, a certificate verifies that the webserver you are connected to is in fact a physical webserver associated with the domain name that appears in the address bar of your browser, and not a man-in-the-middle server attempting to eavesdrop on or hijack your connection - to collect the your password or credit card number, for instance.

The TLS standard has replaced the SSL standard almost completely. TLS certificates are still referred to as SSL certificates, though, out of habit. Certificates are issued by private companies called Certificate Authorities (CAs), such as Symantec, Comodo, GoDaddy, and StartSSL. Their own root certificates, which verify their ability to verify other websites, are built into all modern web browsers.

There are several tiers or classes of SSL certificates, based on how thoroughly the CA verifies the identity of the requester. StartSSL, for example, offers free (Class 1) certs to individuals for noncommercial purposes, lower-cost (Class 2) certs to individuals for commercial purposes, and higher-cost (Class 3 and 4) certs to enterprises. Being a noncommercial entity whose users don't submit high-security data, QuarkNet uses Class 1 certs for the i2u2.org VMs.

Obtaining a certificate

Certificate providers typically provide installation instructions on their websites. StartSSL, the one we use (as of 2016), provides a convenient pdf. DigiCert also provides good information. A general description of the process is provided here.

In practice, you must submit a Certificate Signing Request (CSR) to the CA to be issued an SSL certificate. This request contains information on key bit length and encryption algorithm. In our case, these are constrained by the CRC's network, through which all www.i2u2.org traffic travels. This means that generating a CSR on the VMs, including i2u2-prod, is best left to the CRC engineers, who will happily do so on request (it takes just a couple of minutes).

The CSR must be generated on the server that serves the pages to be certified. The generation process establishes a public/private key pair; the private key remains on the server, while the public key will be included with the certificate once it is issued. This allows a user to verify that the physical webserver is the one to whom the SSL certificate was issued.

Once the CSR is submitted to the CA, the CA will provide a zipped file containing certificates tailored for several of the major server programs. We'll be interested in the one for Apache, of course.

Certificate files

In Apache, using SSL requires enabling the mod_ssl module and including the directive

SSLEngine    On

in the site's configuration file. In the case of i2u2-prod, this is /etc/apache2/sites-available/i2u2-ssl.

The certificate files are indicated using the directives

SSLCertificateFile
SSLCertificateKeyFile
SSLCertificateChainFile
SLCACertificateFile

SSLCertificateFile

This is the certificate file itself, which includes the public key of the public/private key pair. Apache provides it to any browser that requests an https connection to the site.

SSLCertificateKeyFile

This is the site's private key. When the Certificate Signing Request is executed on the site's web server (in our case, i2u2-prod), it generates the private key on that server. This file must be kept private and it should never be removed from the server it was generated on, since it is vulnerable to interception during transit. Even the CA that provides the certificate does not (and should not) know the private key.

SSLCertificateChainFile

SSL certificates can be "chained," meaning that a cert can verify the authenticity of another cert, which verifies another cert, etc., which verifies a webserver. This file contains any "chain" of certificates required to connect the site's certificate (that given by SSLCertificateFile) with the root certificate of the CA embedded in all users' browsers. It's not always necessary, and in fact it became obsolete in Apache 2.4.8, when CertificateFiles were allowed to contain their own chains.

SLCACertificateFile

This file is used by the server to verify the identity of clients, if such utility is desired. Like the SSLCertificateChainFile file, its use is optional.

Identifying which file provided by the CA belongs to which Apache directive is not always clear. The file extensions, like .crt, .cert, .key, .pem, .ca, etc., are not standardized. Rely on the CA's instructions to figure out how to map the files to the directives.

www.i2u2.org Certificates

On i2u2-prod, these Apache directives are defined as:

        # SSL
        SSLEngine On
        SSLCertificateFile      /home/quarkcat/sw/www-apache/ssl/server.crt
        SSLCertificateKeyFile   /home/quarkcat/sw/www-apache/ssl/server.key
        SSLCertificateChainFile /home/quarkcat/sw/www-apache/ssl/server-intermediate.ca
        SSLCACertificateFile    /home/quarkcat/sw/www-apache/ssl/server.ca

The files specified by these directives are symlinks to the real certification files, which are typically stored in additional directories within ssl/, like ssl/startssl/ or ssl/godaddy/. To change the SSL certificates on i2u2-prod, then, you just have to place the new certificate files somewhere safe and organized, and then change the symlinks above to point to them. This method avoids altering the Apache configuration files directly, so we can change SSL certs without having to restart the server.

The following table summarizes the arrangement as of early 2017, using StartSSL certificates. All paths are relative to i2u2-prod:/home/quarkcat/sw/www-apache/ssl/.

Directive	symlink	target
SSLCertificateFile	server.crt	startssl/2_www.i2u2.org.crt
SSLCertificateKeyFile	server.key	startssl/www_i2u2_org.key
SSLCertificateChainFile	server-intermediate.ca	startssl/1_root_bundle.crt
SLCACertificateFile	server.ca	startssl/ca.pem

Checking certificate status

You can examine the site's certificate through your browser. For example, while on www.i2u2.org,

• Firefox 50.0 Tools > Page Info > "Security" tab > View Certificate

You can also get information by running the public certificate file on i2u2-prod through openssl:

$ openssl x509 -in server.crt -text

This'll spit out a bunch of stuff, but the validity dates are in there in the format

        Validity
            Not Before: Mar 14 20:06:49 2016 GMT
            Not After : Mar 14 20:06:49 2017 GMT

Certificate history and renewal

Certificates expire after a fixed time, a year for most CAs but 3 months for Let's Encrypt.

The 2014 SSL cert for www.i2u2.org was issued by StartSSL directly to Phong, who renewed it in 2015 for use on www18 at Argonne. After moving to servers at Notre Dame, Joel obtained a new SSL cert in 2016 from StartSSL in QuarkNet's name to put on i2u2-prod. We used StartSSL because they offered free certs.

In late 2016, StartCom/StartSSL was purchased by WoSign, who was subsequently caught doing some shady crap with backdating certificates. Most major browsers revoked trust in WoSign/StartSSL certificates in their Q1 2017 releases, so when the www.i2u2.org cert expired in March 2017, Joel moved to Let's Encrypt as a CA.

The Let's Encrypt SSL certificate is set to auto-renew via a cron job on quarkcat's i2u2-prod crontab. The auto-renew command is run twice daily at Let's Encrypt's own suggestion; it does nothing if the cert isn't about to expire, and it alerts you to potential problems. Ideally, we'll never have to worry about renewing the SSL certificate for i2u2-prod ever again.

i2u2-dev

In March of 2017 we installed an SSL cert on i2u2-dev to get rid of annoying error messages when accessing it. Since the CRC controls the crc.nd.edu subdomain, SSL management on this VM must be coordinated with the CRC engineers.

The CRC obtains SSL certs from InCommon, a subsidiary of Comodo. The CSR on i2u2-dev can be generated with the command

$ sudo openssl req -new -newkey rsa:2048 -sha512 -nodes -keyout i2u2-dev.crc.nd.edu.key -out i2u2-dev.crc.nd.edu.csr

This will present a series of prompts (the most important is "Common Name" which must be "i2u2-dev.crc.nd.edu") and generate the private key file and the CSR file. The CRC engineers will need the CSR file; the easiest way is to copy it from $ cat i2u2-dev.crc.nd.edu.csr into an email (everything in this file is public, so this is perfectly safe).

The CRC will submit the request to InCommon, which will send you a couple of automated emails including one with links to download the certificate file containing the public key. From there it works much like i2u2-prod in terms of the Apache directives and symlinks to the cert files in /home/quarkcat/sw/www-apache/ssl/incommon/.

The InCommon certs are good for three years. The one on i2u2-dev will expire on March 1, 2020.

-- Main.jgriffith - 2016-03-10