Drupal with Varnish caching and HTTPS using Nginx and Let's Encrypt

Varnish is great at serving pages to anonymous users at high speed, but it doesn't speak HTTPS so if you want to use Varnish and provide your site over HTTPS then you need to use something to do HTTPS and proxy the traffic to Varnish. In this blog post I will detail one way to do this by using Nginx to do HTTPS termination and proxy the requests to Varnish.

Server overview

In the following setup Varnish listens for HTTP requests on port 80. Apache with mod_php handles the Drupal stuff, listening on port 8080. Varnish proxy requests from port 80 to Apache on port 8080. To handle HTTPS, Nginx listens on port 443 and proxies requests to Varnish on port 80. Let's Encrypt provides a free SSL certificate for use by Nginx.

The following commands work on Ubuntu 18.04. The configuration will generally work for different versions of Ubuntu or Debian, although the versions of some software packages will vary; PHP and Varnish in particular have changed from previous long term support versions of Ubuntu.

Apache setup

We start by installing Apache and PHP and configuring Apache to listen on port 8080.

sudo apt-get install -y apache2 apache2-utils php7.2 php7.2-cli php7.2-common php7.2-curl php7.2-json php7.2-gd php7.2-mysql php7.2-opcache php7.2-xml libapache2-mod-php7.2
sudo a2enmod expires headers rewrite php7.2

Create an Apache virtual host file in /etc/apache2/sites-available/example.conf

<VirtualHost *:8080>

    ServerName example.com
    ServerAdmin support@example.com
    DocumentRoot /var/www/html

    ErrorLog /var/log/apache2/example-error.log
    CustomLog /var/log/apache2/example-access.log combined

    <Directory /var/www/html>
        Allow from all
        Options +Includes -Indexes +FollowSymLinks
        AllowOverride all
    </Directory>

</VirtualHost>

Edit the /etc/apache2/ports.conf so that it contains

Listen 8080

Then reconfigure Apache

sudo a2ensite example.conf
sudo systemctl restart apache2

Assuming Drupal is installed in /var/www/html, then it will be available at http://example.com:8080/.

Varnish setup

We now install Varnish and configure it to handle HTTP requests on port 80.

sudo apt-get install -y varnish

If the varnish software package isn't found then you will need to enable the Ubuntu universe repository.

sudo add-apt-repository universe

Create a file called /etc/systemd/system/varnish.service with the following content; the arguments for varnishd can be tweaked according to your requirements.

[Unit]
Description=Varnish HTTP accelerator
Documentation=https://www.varnish-cache.org/docs/5.2/ man:varnishd

[Service]
Type=simple
LimitNOFILE=131072
LimitMEMLOCK=82000
ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T :6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m -p vcc_allow_inline_c=on
ExecReload=/usr/share/varnish/reload-vcl
ProtectSystem=full
ProtectHome=true
PrivateTmp=true
PrivateDevices=true

[Install]
WantedBy=multi-user.target

Edit the /etc/varnish/default.vcl file for your specific requirements. The key thing is to add a backend that points to port 8080, like:

# Default backend definition. Points to Apache on port 8080.
backend default {
  .host = "127.0.0.1";
  .port = "8080";
  .first_byte_timeout     = 300s;
  .connect_timeout        = 5s;
  .between_bytes_timeout  = 2s;
}

An example Varnish 5 configuration file for Drupal can be seen here: https://gitlab.agile.coop/snippets/27

Then reconfigure varnish

sudo systemctl daemon-reload
sudo systemctl restart varnish.service

The site will now be cached by Varnish and be available at http://example.com/.

Let's Encrypt setup

For HTTPS to work properly we need a SSL certificate. For this we can use certbot to get a certificate from Let's Encrypt using the webroot plugin.

sudo apt-get install -y certbot
sudo certbot -m suport@example.com --agree-tos -n certonly --webroot -w /var/www/html -d example.com

This is a very simple example, but you can use the webroot plugin with as many domains as required and with more sites by specifying extra -w and -d options. One thing to note is that the DNS must be in place for this to work, each domain will be checked so if a domain doesn't resolve to the server's IP address then the command will fail. More information on the webroot plugin can be found here: https://certbot.eff.org/docs/using.html#webroot

Nginx setup

Now we have a SSL certificate we can install and configure Nginx to handle HTTPS requests on port 443.

sudo apt-get install -y nginx
sudo rm /etc/nginx/sites-enabled/default

Create a Nginx configuration file to proxy HTTPS requests to Varnish at /etc/nginx/sites-available/https-proxy with the following configuration

# Proxy all HTTPS requests to Varnish
server {
    listen 443 ssl default_server;

    ssl  on;
    ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

   client_max_body_size 64M;

    location / {
        proxy_pass       http://127.0.0.1:80/;
        proxy_redirect   off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-Port 443;
    }
}

Then reconfigure nginx

sudo ln -s /etc/nginx/sites-available/https-proxy /etc/nginx/sites-enabled/
sudo systemctl restart nginx.service

The site will now be available at https://example.com/

Drupal configuration

The site is now available using HTTPS and is being cached by Varnish. There are a few configuration bits and pieces that Drupal needs. One is to enable and and configure the Drupal 8 Purge and Varnish purger modules. For Drupal 7 there is the Varnish module.

Drupal also needs to be told it's behind a reverse proxy. For Drupal 8 add the following to the site's settings.php or settings.local.php file.

$settings['reverse_proxy'] = TRUE;
$settings['reverse_proxy_addresses'] = array('127.0.0.1');

if (getenv('HTTPS') !== 'on' && getenv('HTTP_X_FORWARDED_PROTO') === 'https') {
  $_SERVER['HTTPS'] = 'on';
  $_SERVER['SERVER_PORT'] = 443;
}

For Drupal 7 the configuration is

$conf['reverse_proxy'] = TRUE;
$conf['reverse_proxy_addresses'] = array('127.0.0.1');

if (getenv('HTTPS') !== 'on' && getenv('HTTP_X_FORWARDED_PROTO') === 'https') {
  $_SERVER['HTTPS'] = 'on';
  $_SERVER['SERVER_PORT'] = 443;
}

Finally, for maximum security, all HTTP traffic should be redirected to HTTPS by adding the following to the site's .htaccess file after the RewriteEngine on line.

# Redirect HTTP to HTTPS
RewriteCond %{HTTPS} off
RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]