Page speed is an incredibly important factor for UX, conversion rates and SEO. This case study describes in great detail how we used a basic implementation of Cloudflare’s HTML caching feature on an e-commerce website to improve average page load times by 28%.
Are you in a hurry? Jump straight to the TL;DR summary!
- The result: 28% better page load times with Cloudflare’s HTML caching
- Context: HTML caching vs. standard caching in Cloudflare
- Challenges: Complications with HTML caching on e-commerce sites
- Solutions: Cloudflare’s “Bypass Cache on Cookie” and “Edge Cache TTL” features
- The implementation: Page rules for HTML caching in Cloudflare
- Optional: Setting cookies with Google Tag Manager
- Limitations, further optimisation potential & next steps
- TL;DR
The result: 28% better page load times with Cloudflare’s HTML caching
Let’s start with the results of this case study, before we dive into the details of the implementation. One way of measuring the impact of page speed optimisations is by looking at the “Site Speed Page Timings” report in Google Analytics:
In the above graphic, you can clearly see the performance improvement starting on the day we activated HTML caching in Cloudflare (20th of March 2019) on reifen24.de, a German online shop for tyres. The average server response time went down from around 1,5 seconds to under 1 second, and the average page load time also benefitted from the optimisation.
As you can see in the following screenshot from Google Analytics, which compares the average page load time since the 21st of March (the day after the implementation) with the previous period, page speed was improved by 27,56% (from 4,49 seconds average page load time down to 3,25 seconds):
During the time frame we are looking at here, no other changes were made that could have had a significant impact (positive or negative) on page load or server response times.
Continue reading to learn everything you need to know about HTML caching for e-commerce sites with Cloudflare and how the improvements we were able to generate in this case were only the tip of the iceberg with a lot of room for further optimisations.
Context: HTML caching vs. standard caching in Cloudflare
In order to understand how Cloudflare’s HTML caching can be used to improve the performance of websites (and e-commerce sites in particular), we first need some context around what Cloudflare does and how its different levels of caching work.
To put it simply, Cloudflare offers a network of very powerful, fast and reliable servers that save parts of your website and serve them directly to your visitors, so that your own web server does not have to communicate directly with every visitor every time a resource is requested. This setup helps improve the load times of your website for users and crawlers, especially if your own web server does not have a very good performance (or at least not as good as Cloudflare’s network of servers).
Cloudflare’s standard level of caching (i.e. saving parts of your website on their network of servers) applies to static files on your website, such as images, PDFs, CSS or JavaScript files. These files normally have the same content every time they are requested (a certain image, for example, looks the same for all users every time they access it), so Cloudflare can save them on their servers (i.e. cache them) and serve them directly to your visitors, without the need to fetch them again and again from your own web server.
HTML pages (normal web pages) are different, as their content often changes over time and between users. The content of your home page might change several times every day and some of its elements might even look different for every user. This is why Cloudflare does not save a version of HTML pages to serve to all users by default. Instead, with Cloudflare’s standard caching level, HTML files are fetched from the origin server (your web server) every time a user requests the page. The static files (CSS, JS, images, etc.) that are referenced within the HTML, on the other hand, are served directly from Cloudflare’s network of fast and reliable servers.
HTML caching in Cloudflare is an additional feature that can be activated for certain pages under certain circumstances and should be handled with care.
Challenges: Complications with HTML caching on e-commerce sites
Most e-commerce websites have dynamic elements within their HTML pages, which means that the content of their pages does not look the same for all users. The content of e-commerce sites can also change frequently over time, for example when products go out of stock or when prices and offers change. In the case of Reifen24, we were facing three main challenges that prevented us from simply activating Cloudflare’s HTML caching on all pages:
First challenge: Dynamic shopping cart preview
On reifen24.de, as on many other e-commerce sites and online shops, the shopping cart preview changes every time a product is added or removed from the shopping cart. The preview is included on all pages, so all pages have this dynamic content element (marked with a red box here):
With HTML caching activated, Cloudflare always saves a version of a page in its cache that has been served to a real user. If we simply activated HTML caching on all pages for all users, then Cloudflare would cache versions of pages that include a value in the shopping cart preview. This would result in lots of users seeing shopping cart previews that do not represent the content of their own shopping cart.
For instance, if you were the first user to request the winter tyres category page on Reifen24 after HTML caching was activated, and you had 4 products worth EUR 468,72 in your shopping cart, like in the screenshot above, then Cloudflare would cache this version of the page, including your shopping cart preview, and serve this version to all following users that request the same page. This obviously has to be avoided by all means.
Second challenge: Dynamic content for logged-in users
A similar problem to the one described above occurs when a user logs in on Reifen24. The user’s name is displayed at the top of the page (marked with a red box here):
If I, while logged in, was the first user to request a certain page after HTML caching was activated, then Cloudflare would cache this version of the page, including my name, and show it to all users that request the page from that moment on. This is not something we want users to experience on our website.
Third challenge: Prices on reifen24.de are updated twice every hour
The prices for tyres and other products on Reifen24 change frequently, and the displayed prices are updated every 30 minutes. Not all prices change every time the data is updated, but it is impossible to predict which prices will change and how many prices will change every time the update runs.
HTML caching would lead to one version of every product and category page being cached, displaying the prices that are shown at the moment of caching. The third big challenge we faced when implementing Cloudflare’s HTML caching on Reifen24 was making sure that correct prices are shown at all times.
Solutions: Cloudflare’s “Bypass Cache on Cookie” and “Edge Cache TTL” features
There are several ways of tackling the problems described above, but in this case, we needed a simple solution that wouldn’t require too many dev resources.
Hamlet Batista, in this great article about HTML caching for e-commerce websites, suggests that one solution for dynamic content elements like the name of a logged-in user would be to add this information to the page via JavaScript, instead of including it in the original HTML. This way, HTML files without dynamic elements that work for all users could be cached and dynamic elements could be added individually for every user on the client side. Our problem with this approach, although it sounds like a great solution, is that it would require a significant change to the way certain elements of the shop are programmed.
An alternative that Hamlet suggests is to only activate HTML caching for users that are not logged in. If we apply this idea to our case, with dynamic content elements for logged-in users and for users that have added products to their shopping carts, this would mean that we would have to find a way of activating HTML caching only for users that are not logged in and that haven’t added anything to their shopping carts.
Cloudflare offers a solution for caching HTML pages for certain users only, called “Bypass Cache on Cookie” (available for Business & Enterprise plan users). The only additional element you need on your website in order to use this feature is a cookie that is set whenever you want to exclude a user from HTML caching. In our case, we needed a cookie that would be set whenever a user added a product to the cart or logged in and that would be deleted whenever the cart was empty or the user logged out.
Depending on which CMS or shop system you are using, there are different ways to set cookies – you might even be lucky and the cookies you need already exist. If you don’t know how to set up the cookies you need to implement HTML caching on your website, a later part of this article (where we explain how we set up cookies with Google Tag Manager for this particular implementation) might help.
Before we move on to how we set up the cookies we needed with GTM, let’s have a quick look at the last challenge we faced (frequently changing prices) and what our HTML caching implementation in Cloudflare looks like.
Cloudflare’s Edge Cache TTL is a feature that allows us to set a time frame after which the cache is emptied and Cloudflare fetches a new version of the page from the origin server the next time the page is requested by a user. We decided to use this feature to make Cloudflare’s cache expire just as often as the prices on the website are updated – every 30 minutes.
The use of the features “Bypass Cache on Cookie” (for excluding logged-in users and users that have products in their shopping cart from HTML caching) and “Edge Cache TTL” (for making the cache expire after a certain time frame) results in the following implementation in Cloudflare:
The implementation: Page rules for HTML caching in Cloudflare
In the “Page Rules” section in Cloudflare, we added the following rules to activate HTML caching for users that are not logged in and that do not have any products in their shopping carts:
The first rule sets the caching level for all URLs that end in “.php” to “Standard”. Remember how we talked about the “Standard” caching level in Cloudflare meaning that only static files, such as images or stylesheets, are cached, and the HTML is fetched from the origin server every time the page is requested. We chose this setting for all URLs ending in “.php” because, on this particular website, URLs that end in “.php” are completely dynamic pages, like the shopping cart or the checkout process. We absolutely do not want HTML caching on pages that mainly consist of content that changes for every user.
The second rule then sets the cache level to “Cache Everything” for all pages in the directory “/info/”. The cache level “Cache Everything” means that HTML caching is activated for the pages the rule applies to.
We set the value for “Edge Cache TTL” to “a day” for all pages in the “/info/” directory, as this content might change on a daily basis, but the pages do not include any prices, so there’s no need to empty the cache every 30 minutes. After a HTML page from this directory is cached on Cloudflare’s server, all users will now be served this version of the page for 24 hours. After the 24 hours are over, the cached version of the page is deleted. Once it’s deleted, when the page is requested again, Cloudflare will fetch a fresh version of the page from the origin server and cache this version for the next 24 hours.
Finally, the value “bypassCache” of the setting “Bypass Cache on Cookie” is the name of the cookie that our shop sets whenever a user logs in or adds a product to the shopping cart. Whenever this cookie is present, the cache level goes back to the universal setting for the website (“Standard” in this case – not defined via a page rule, but in the “Caching” section of the Cloudflare interface).
Please note that Cloudflare page rules are applied from top to bottom and that later rules will not apply to pages that an earlier rule has already applied to. This means that the second rule will not apply to the pages the first rule applies to and the third rule will not apply to the pages the first and second rule apply to.
The third rule in our implementation applies to all remaining URLs (everything except the pages that are covered by the first two rules) and sets the caching level to HTML caching whenever the “bypassCache” cookie is not present. The only difference to the second rule is that the Edge Cache TTL is 30 minutes here, instead of a day.
You probably remember that one of the challenges we were facing with Cloudflare’s HTML caching on Reifen24 was that the prices on the website are updated twice every hour. Setting the Edge Cache TTL to 30 minutes was the easiest solution we could come up with here, even if it’s not perfect:
Unfortunately, Cloudflare does not offer the possibility to empty the cache regularly at a fixed time. Setting the Edge Cache TTL to 30 minutes results in Cloudflare caching each page the moment it is first requested by a user and then serving this cached version to all users for the next 30 minutes. When the 30 minutes are over, Cloudflare deletes the cached version and waits until the page is requested by a user again, before it caches a new version for another 30 minutes.
This means that, in the worst case, users might still see the wrong prices for up to 30 minutes with our current implementation, if the caching of a certain page happens just before a price on the page is updated – A risk that we were willing to take in order to test the impact of HTML caching. So far, no complaints by users about displayed prices changing after adding a product to the shopping cart have been reported. Nevertheless, we are working on a more sophisticated solution to this challenge, involving the Cloudflare API, which we will discuss later on (when we talk about further optimisation potentials).
The three page rules you see above are the exact implementation in Cloudflare that resulted in a 28% page speed increase for Reifen24. I hope that with the help of this case study, you will be able to adapt this setup to any website you are working on. The last piece that might be missing from your puzzle at this stage is how to set the cookies you need for users that you want to exclude from HTML caching (e.g. logged-in users or users who have added products to their shopping carts). There is no one-size-fits-all solution for this, but in the next part of this article, we will show you how we set up the cookies in this particular case, using Google Tag Manager.
Optional: Setting cookies with Google Tag Manager
When you are looking for ways to make changes on a website without having to bother the dev team, Google Tag Manager is often a tool that can help. This being said, please keep in mind that there is usually a better solution than using Google Tag Manager. In the case of Reifen24, we needed solutions that did not require too many dev resources, so we decided to set the “bypassCache” cookie with GTM.
Whenever I need to set or delete a cookie with Google Tag Manager, I refer to this brilliant guide by Julius Fedorovicius. By copying the scripts Julius provides in his article and adapting them a tiny bit to my needs, I came up with the following tag configuration in Google Tag Manager to set the “bypassCache” cookie whenever a page is loaded while the user is logged in or has products in the cart:
In order to detect whether a user is logged in or has products in the cart, I created two “DOM Element” variables in Google Tag Manager. This variable type allows us to extract values from HTML elements, using CSS selectors or HTML IDs. Julius also has a great guide on “DOM Element” variables in GTM, in case you need some more information on this topic.
Using “DOM Element” variables, we can extract the text of the login link and the value of the shopping cart preview from our website. With the help of these variables, we can then create triggers for when a page is loaded with or without products in the cart or while the user is logged in or logged out:
The triggers “Product in cart” and “User logged in” are used for the tag you saw above, which sets the “bypassCache” cookie, and the triggers “No product in cart” and “User logged out” are used for the following tag, which deletes the cookie for logged-out users with empty carts, if it is present:
If you don’t have a lot of experience with Google Tag Manager, this might seem like pretty advanced stuff to you. There’s not much point in explaining the GTM part in more detail, as the solution would be very different on every website, depending on what the source code looks like exactly. If you need any support with this part, or with anything else that has been mentioned in this article, please just leave a comment, and I’ll be happy to help.
Limitations, further optimisation potential & next steps
Let’s sum up what we’ve achieved with the current implementation and where there is still optimisation potential left.
HTML caching on Reifen24 is currently active whenever a user is not logged in and doesn’t have any products in the shopping cart. We do not have very reliable tracking data about which percentage of page views happen under these conditions (shame on us), but from analysing the Enhanced E-commerce Tracking data we have in Google Analytics, we can conclude that about 95% of all sessions do not involve any products being added to the cart. The percentage of users that are not logged in is even higher.
This means that most visitors already benefit from the HTML caching implementation, but we would obviously also like users to experience super fast page load times after they have added products to their carts and are logged in. A solution like the one discussed earlier in this article, where dynamic elements like the shopping cart preview or personalised areas on otherwise static pages are added via JavaScript on the client side, is on our list of planned optimisations.
Another strong limitation of our current implementation is that we have to let the Cloudflare cache expire after 30 minutes on most pages, due to the frequently changing prices. There might not often be a cached version for pages that do not have a lot of traffic (whenever a page is requested that hasn’t already been requested in the past 30 minutes). And even pages with lots of visits currently have to be fetched from the origin server up to 24 times a day (twice every hour, according to our current cache expiry settings).
We are currently working on an automatic solution where we check which prices have changed after every update, and then use the Cloudflare API to only remove exactly those URLs from the cache that contain prices that have changed. This way, we could remove the Edge Cache TTL setting or change it to a significantly longer period (e.g. one week). We hope that this optimisation will help further speed up pages that do not often have a cached version available. It should also have a significant impact on page load times for crawlers, as they are even more likely than users to access pages that don’t receive visits very regularly.
HTML caching does not really make sense on pages that contain mainly dynamic content, such as the shopping cart or the checkout process. For these pages, and also for dynamic content elements on other pages that can’t be cached, Cloudflare offers an additional feature called Railgun. In a nutshell, and without entering into too much technical detail, Railgun is like HTML caching for dynamic content, and it can be used together with HTML caching, as an addition to HTML caching, or completely independently. Once you have exhausted the potential of HTML caching on your website, or come to the conclusion that you cannot use it at all because there are too many technical limitations, implementing Railgun should be the next logical step.
As you can see, the 28% improvement in average page load times we achieved with this basic implementation (only for certain users and only caching HTML for 30 minutes on most pages) is only the beginning. With the next steps we have planned, we should be able to generate even better results.
TL;DR
- We achieved an 28% increase in average page load times by implementing Cloudflare’s HTML caching on reifen24.de, a German online shop for tyres.
- When HTML caching is activated, entire HTML documents (instead of just static files) are saved on Cloudflare’s servers and served directly to website visitors (instead of being fetched from the origin server every time they are requested).
- HTML caching on e-commerce websites can be complicated, as online shops often have dynamic content elements that look different for every user (e.g. shopping cart preview) or elements that change frequently over time (e.g. prices).
- Cloudflare can be instructed to deactivate HTML caching when a certain cookie is present (e.g. a cookie that is set when a product is added to the cart) and the cache can be emptied automatically after a certain time.
- The cookies that are needed for this setup can be implemented with Google Tag Manager, if there is no better solution available.
- Although this basic implementation of Cloudflare’s HTML caching already resulted in a 28% average page speed improvement, there is still further optimisation potential by inserting dynamic content elements on the client-side (instead of the raw HTML), by leveraging the Cloudflare API to empty the cache when needed (instead of automatically emptying it after short periods of time), and by activating additional Cloudflare features that are similar to HTML caching, but work for dynamic content.
I hope you enjoyed reading this case study and that it helps you with the implementation of HTML caching on the website(s) you are working on. If anything remains unclear, or if you have any questions or comments whatsoever, please don’t hesitate to leave a comment.
And in case you still need some extra motivation: The page speed optimisation in this particular case also resulted in a significant improvement in conversion rates and a six-digit rise in revenue within a month, so the work we put into this has certainly paid off.
