High Performance Craft Caching with Fastly

Recently One Design Company had the chance to work on a brand new website for a certain high-profile design competition that was expected to experience intense bursts of traffic on a regular basis. Early on in the project we made the decision to build the new site with Craft, a wonderful content management system that takes a lot of the pain out of building content-heavy sites. Craft does have built-in caching, its approach being to cache the results of common queries to the database, and make requests for those results on subsequent page loads.

While this built-in caching brings a huge performance boost and is incredibly easy to configure, those database requests still result in a less-than-ideal server response time for a site like Layer Tennis that receives swarms of simultaneous traffic. What we'd ideally like to do is serve static pages that only get updated when our content changes.

Another Layer of Caching

This is usually where full-page caching solutions like Varnish come into the picture. Think of Varnish as a layer on top of your site's backend. When a request is made to your domain, that request first goes through Varnish. If Varnish has a cached copy of the requested resource, it will serve it incredibly quickly and your server won't be touched. If Varnish can't find a cached copy, it will request the resource from your server and create a cached version to use next time.

This can result in incredibly speedy server response times. Unfortunately, configuring Varnish and ensuring that its caches are always up-to-date can require a lot of time and effort.

Meet Fastly

Fastly is a service built on top of Varnish that removes many of the hurdles and makes full static caching of your site incredibly easy. Figuring out how to get Craft and Fastly to play nice together initially takes some work, but once you've been through the process once it is very easy to repeat for additional Craft sites in the future, and the performance difference is impressive.

Step by Step

We'll walk through each of the steps you need to take one by one, but first here's a summary of what we need to accomplish:

  • Sign up for Fastly and create a new Fastly service.
  • Create a CNAME record pointing from your domain to Fastly.
  • Tell your server to cache all static assets with an nginx config or .htaccess file.
  • Add the proper cache headers to your Craft templates.
  • Add a Fastly rule to remove Craft's default Set-Cookie header.
  • Add a Fastly rule to ignore all requests to /admin.
  • Increase Fastly's First Byte timeout.

Sign up for Fastly and Create a New Service

To cache your site using Fastly, you're going to need an account. Visit fastly.com and sign up for a new account.

Next, you'll need to create a Fastly service. While signed into Fastly, click on Configure, then click New Service.

The Name can be anything you want. The Origin Server Address will be the IP address of your existing server. The Domain Name should be the domain you want your live site to be available at. When you're done, click Create.

Point your Domain to Fastly

When someone makes a request to www.yoursite.com, we want that request to be handled by Fastly before your server even knows about it. You'll configure this by adding a CNAME DNS record for your domain that points to global.prod.fastly.net

The specifics of creating this vary depending on who your DNS provider is, but Fastly provides instructions for the major DNS providers that may help.

We'll tell Fastly to cache the pages of our site in a minute, but first we need to tell it to cache all non-html static assets, such as CSS, Javascript, and images.

If you're using an nginx server, you'll do this using a location block:

Ini

location ~ ^(?!(/admin/))(.*)\.(js|css|png|jpg|jpeg|gif|ico|svg)$ { expires 24h; }

This tells the server to set a 24 hour expiration on all of the listed file types that are not located in Craft's Control Panel.

Add cache headers to Craft templates

Next we need to tell Fastly to cache the actual pages of our site. We do this by adding a header to our templates. The easiest way to do this is add the following to the very top of /craft/templates/_layout.html:

Ini

{% set expiry = now|date_modify('+30 days') %} {% header "Cache-Control: max-age=" ~ expiry.timestamp %} {% header "Pragma: cache" %} {% header "Expires: " ~ expiry|date('D, d M Y H:i:s', 'GMT') ~ " GMT" %}

Tell Fastly to remove the default Craft Cookie Header

By default, Craft delivers all pages with a Set-Cookie header that is use for storing whether a user is currently signed in. Unfortunately, this header interferes with caching and needs to be removed for all non-control panel pages. Leaving this header in place is important for control panel pages to continue to work, so we'll take extra steps to ensure it exists when needed.

  1. While viewing your Fastly service, click Configure.
  2. Click Content in the sidebar.
  3. Next to Headers, click New.
  4. Add the following settings, then click Update.

Ini

Name: Remove Craft Cookie Header Type / Action: Cache / Delete Destination: http.Set-Cookie Ignore if Set: No Priority: 10

Now the header is being removed everywhere, but like I said, we need to make sure we don't touch the header for control panel pages. Fastly makes it possible to create rules for this type of exception using Conditions. Let's create a condition for the header change we just made.

  1. Click the settings icon (looks like a gear) next to the header you just created.
  2. Choose Conditions from the dropdown.
  3. Set the following, then Click Update.

Ini

Name: Not Admin Apply If... !req.url ~ "/admin" Priority: 10

Tell Fastly to ignore all requests for the Craft control panel

As things are set up so far, all requests will route through Fastly, including requests for the Craft control panel. Routing control panel pages through Fastly not only makes little sense, but it actually interferes with the ability to log into the control panel. That's obviously not good.

As you might expect, we can tell Fastly to pass any requests it receives for Craft control panel pages on to the original server. Let's do that.

  1. While viewing your Fastly service, click Configure.
  2. Click Settings in the sidebar.
  3. Next to Cache Settings, Click New.
  4. Add the following settings, then click Create.

Ini

Name: Force Pass on Admin TTL: 0 Stale TTL: 0 Action: Pass

By itself, this won't do anything. First, we need to tell it what conditions should trigger it.

  1. Click the Settings icon next to the cache setting you just made.
  2. From the dropdown choose Conditions.
  3. Set the following, then click Update.

Ini

Name: Admin Apply If... req.url ~ "/admin" Priority: 10
  1. Click the Settings icon next to the cache setting again.
  2. Select your new Admin setting from the Name select, then Click Apply.

Increase the Fastly First-Byte timeout

Craft's updating process can sometimes take awhile, so we need to make sure Fastly gives it enough time to finish before timing out.

  1. While viewing your Fastly service, click Configure.
  2. Click Hosts in the sidebar.
  3. Under Backends, you should see one item. Click the settings icon next to it.
  4. Choose Advanced Configuration from the Name dropdown.
  5. Click Assign.

Activate your New Fastly Changes

Before any of the Fastly changes you just made take effect, you need to activate the new Fastly version. Near the top of Fastly, click the Activate button.

Cache Busting

There's one important thing we haven't discussed: how does Fastly know when your content is changed and that it should refresh its caches? Well, the short answer is, it doesn't. Even if you update your content in Craft, Fastly will continue to serve its old, existing cached version until you tell it not to or the cache expires.

Of course, you can manually clear the caches for individual URLs or all pages at once through the Configure section of your Fastly service, but unless your content updates very infrequently you probably won't want to do this every time you make an update in Craft.

We solved this problem by creating a custom plugin Fastly plugin for Craft. This plugin takes a brute force approach to clearing Fastly's cache. It listens for any Craft Entry to be saved, and makes a call to the Fastly API to clear the entire cache for your server. This will result in some URLs having their cache cleared unnecessarily, but it's far simpler than determining all of the URLs affected by the changed entry and telling Fastly to clear each of them individually and ensures that the latest content is being served even when your Craft entries have complex relationships.

View the Fastly plugin for Craft

Wrapping Up

Setting up Fastly to work with Craft takes some time the first go-round, but the performance difference is more than worth the trouble. Layer Tennis went from an average HTML load time of 2.1 seconds without caching to less than 120 milliseconds. That's a reduction of 1,750%. Not too shabby.

With Fastly caching of a Craft-backed site, you get the best of both worlds: high-performance delivery with the world's best content management system.

Most Recent Issue