One of my obsessions as of late has been page speed. A more technical client early last year made me realize I’d phoned it in for the page speed recommendations that I’d been including in my SEO Site Audits. You know the ones, minifying code, removing inline styles and JavaScript, specifying image dimensions, leveraging compression, etc. So in the last 9 months I’ve taken a dive in to improve my understanding of networking, the critical rendering path, what every single thing in PageSpeed Insights actually means and generally acclimating with the latest in front end development best practices. It’s safe to say I’d spent too much time with my marketer hat on and not enough time staying on top of my original skillset.
In coming across the phenomenal work done on the subject by Ilya Grigorik, I learned about a directive called rel-prerender which has effectively allowed me to speed up this very site by over 68% with one line of code.
That’s right, the title is not clickbait.
What is Rel-Prerender?
You should already know Google is even more obsessed with page speed than I am, as they’ve continued to rollout things that make the web a faster and better place. At the center of that is Chrome, a browser with components from Webkit, the open source code that Safari, Firefox for iOS and number of other browsers are built from. Chrome’s Chromium project has obviously matured and along the way has developed a whole lot of tricks of its own.
For example, Chrome predicts a lot of things that you do. As soon as you type in “ww” it’s going to pick the URL you visit the most. Funnily enough, we recently had someone present off their personal laptop in an interview, and that prediction led to some memorable results. If you want to see your list of predicted results, you can find them at chrome://predictors. Here are some of mine:
Buried deep in the HTML5 specification is a series of resource hints or directives that Ilya Gregorik calls the “pre-* party” in his Preconnect, Prefetch, Prerender deck. They have a variety of different directives that allow you to inform your browser that you want to load things before they are required.
As a quick overview, those directives are as follows:
- Preresolve – This directive performs a DNS lookup before it’s needed. The roundtrip network request to turn a domain name into an IP address can add hundreds of milliseconds to load time. For example, I’ve seen the chartbeat.js file take several seconds to do the DNS Lookup, and using preresolve can shave that time off considerably. You can specify this directive as follows:<link rel=”dns-prefetch” href=”hostname.com”>
- Preconnect – This directive opens a socket connection to a resource before it’s required in order to speed up the downloading process. You can specify this directive as follows:<link rel=”preconnect” href=”https://www.domain.com”>
- Prefetch – This directive downloads a resource before it’s needed so that when it is time to use it, it appears instantly. Each link applies to one specific file, but you can specify many of them if you’d like. However, this is where the pre-party ends for Firefox. You can specify this resource hint as follows:<link rel=”prefetch” href=”filename.ext”>
Prerender – This directive allows you to completely load one page in an invisible tab in the background once the initial page completes its loading. If the user then navigates to that URL, it will appear instantly. If they don’t, the page is then unloaded. This is the one line of code that we’ll be focusing on and the one that yielded the 68.35% speed improvement. Somehow Firefox missed the memo on rel-prerender so some developers have leveraged rel-prefetch in its place. We’ll talk about the syntax in a few paragraphs.
To be clear, in some cases Chrome is smart enough to do these things on its own, but you can use the specific directives to improve the chances that they will execute. Keep in mind that there is an option for users to turn off their prerender in Chrome.
Not to be Confused with Prerender.io
In interviewing SEOs recently for positions here at iPullRank, I’ve recognized that very few people understand how JavaScript MVW frameworks operate or that they need special considerations when being optimized. One of those special considerations is a SaaS PhantomJS wrapper called Prerender.io. The platform, for example can be used to render pages written in AngularJS for caching and serving as processed HTML. This allows search engine crawlers to crawl and index the content as they normally would if AngularJS was not in place. I encourage you to read this post from the BuiltVisible team to learn more about SEO for JavaScript frameworks, especially if you apply for an SEO specific position with us.
This is effective because JavaScript causes content to load after the page loads and there is no visibility for a text-based crawler. As we’ve previously discussed, Googlebot is a browser-based crawler these days and they have continued to improve their ability to crawl JavaScript-driven content. As that continues to improve, we’ve found Prerender.io and its competitors to be good ways to improve the speed at which Google crawls this type of content, but we’ve also seen evidence that it’s not an absolute requirement for it to be crawled and indexed.
For those of you that are aware of this platform, it’s an entirely different thing from rel-prerender.
Google Uses Rel-prerender in the SERPs
Google themselves uses rel-prerender in the SERPs for results that they are reasonably sure of what you’re looking for. For example, this SERP for “cnn” features rel-prerender because moreoften than not a user is going to click on the result that drives them to CNN.com.
I can see the wheels turning for those of you on the darker side of the force. Yes, this does prove that cross-domain rel-prerender is possible. I’m not sayin’, I’m just sayin’. (#brotherhood, I see you).
How do I use Rel-Prerender?
For the SEO folks reading this, rel-prerender works just like the rel-canonical tag, or most <link> tags for that matter.
<link rel=”prerender” href=”https://www.example.com/page”>
As I mentioned before, Firefox does not support rel-prerender, so you’re limited to rel-prefetch. In those cases, you’ll still be loading the base HTML file, so to support both use cases in one line of code you can specify:
<link rel=”prerender prefetch” href=”https://www.example.com/page”>
Also, you can only set one rel-prerender at a time. This page loads in an invisible tab and there is only one to go around all browser processes. So you want to be reasonably sure that the user is going to go to that URL in order to maximize the bandwidth and CPU usage. However you can also inject the rel-prerender to the code at any time to have it preload a page whenever you’re sure. That is to say, if a user hovers over a link for a long period of time, you could potentially insert rel-prerender and have it load that page in efforts to determine where the user is going next.
Scaling Rel-Prerender Definitions with Google Analytics API
In playing around with rel-prerender I quickly realized how unrealistic it is to specify a URL for every page. Even moreso, I realized how problematic it would be to do that and have the specified URL be wrong. So as with anything else I chose to leverage data to find a solution. Enter: the Google Analytics API.
Leveraging the GA API, I’m able to make an educated guess as to what the next page the user is most likely to visit is based on the pages other users visit. Aside from the fact that building Rome in a day is often easier than that authenticating into Google APIs, getting the data we need is actually quite simple. I’ll leave those details to your development team to work out, but in terms of the data we need to make our rel-prerender decision, you want to get the pageviews of the ga:nextPagePath based on setting the ga:previousPagePath to the page that the user is currently on.
You can see what this looks like if you login to the Google API Analytics Query Explorer and set the metrics to ga:pageviews, dimensions to ga:nextPagePath and set the filters to ga:previousPagePath which is set equal to a URL path to a page on your site. Here it is for this site:
The result of that will be a list of URLs in order of their page views for the past 30 days. For e-commerce sites, I prefer to use the ga:pageValue as the metric so we can focus on the literal money pages.
You’ll notice that almost always the homepage is the #1 page, so you’ll want to develop logic to ignore it in your code. You’ll also want to check for the page that you’re already on.
To programmatically set the rel-prerender tag once its returned from the API, you can use the following JavaScript code:
[cc lang=”javascript”]
var relPreRender = document.createElement (“link”);
relPreRender.setAttribute(“rel”, “prerender”);
relPreRender.setAttribute(“href”,relPreRenderPage);
document.getElementsByTagName(“head”)[0].appendChild(relPreRender);
[/cc]
or the following jQuery code:
[cc lang=”javascript”]
$(“head”).append(”);
[/cc]
Keep in mind that this site is small and low traffic, so you’ll want to cache these URLs if you get a lot of traffic because the GA API gives you a limited amount of pings per month.
What Type of Impact Does Rel-Prerender Have?
Upon discovering this, like most things, I wanted to put it to the test. So I set up rel-prerender between my pages and then did a before and after. If the UA was Firefox, the user got the rel-prefetch, otherwise they got rel-prerender and the results are as advertised in Chrome.
Due to the lack of content in last few years (what can I say? I’m focused building a business and getting results for our clients!) this site wasn’t generating much in the way of traffic. In fact, I had hoped to get this placed with one of our bigger media clients to get a case study, but they had concerns about CPU usage and the amount of work required to update how ads are handled in the code, so instead I set up a series of pages on an AWS box that I could blast with pageviews as a test.
I made two sets of the same 3 pages. The pages featured content with rich media. I built one control group with no rel-prerender and another with rel-prerender from page one to page two and page two to page three.
Then I ran thousands of visits at the page across a few days using a headless browser. Here’s what that looked like in Google Analytics real-time.
After that I reviewed the site speed page timings in Google Analytics.
My prerender group of pages generally had a much faster page load time than my control group. The anomaly is that the third page in the control set performed better than the other two pages in the control. This may be a function of the headless browser potentially retaining some of the cached page. My hypothesis is that the video on the page on the third page actually stopped the resource hint from firing. See below for why that might happen.
So What’s the Catch?
There are a number of catches, but the most important to marketers will be the impact that rel-prerender can have on analytics and advertisements.
While Google Analytics automatically accounts for it, most analytics packages will consider all prerendered pageviews as valid. Similarly, a prerendered pageview is most likely counted as an impression to your ad server.
The way to account for this is by using the HTML5 PageVisibility API which allows you to account for the prerender visibility state. Effectively, you can wrap your code in callbacks that only allow certain components to fire once the page is actually visible.
Also, as Ilya discusses in High Performance Networking in Chrome and the outline specifies in the design documents for prerender, the following are other technical caveats that may cause rel-prerender to not execute:
- At most one prerender tab is allowed across all processes
- Prerendering is abandoned if the requested resource, or any of its subresources need to make a non-idempotent request (only GET requests allowed)
- All resources are fetched with lowest network priority
- The page is rendered with lowest CPU priority
- The page is abandoned if memory requirements exceed 100MB
- Plugin initialization is deferred, and pre-rendering is abandoned if an HTML5 media element is present
- The top-level page is not an HTTP/HTTPS scheme, either on the initial link or during any server-side or client-side redirects. For example, both ftp are canceled. Content scripts are allowed to run on prerendered pages.
- window.opener would be non-null when the page is navigated to.
- A download is triggered. The download is cancelled before it starts.
- A request is issued which is not a GET, HEAD, POST, OPTIONS, or TRACE.
- An authentication prompt would appear.
- An SSL Client Certificate is requested and requires the user to select a certificate.
- A script tries to open a new window.
- alert() is called.
- window.print() is called.
- Any of the resources on the page are flagged by Safe Browsing as malware or phishing.
- The fragment on the page does not match the navigated-to location.
Long list, but I haven’t run into any of these aside from potentially the HTML5 media element issue. Finally, although it’s available for some mobile browsers, I wouldn’t use it there. LTE downloads are slow enough.
Other Ways to Improve It
Naturally, rel-prerender makes a lot of sense in situations when users are going through a series of pages like slideshows or multi-page lead flows. To further improve the accuracy, I’ve tried to find a way to pull URLs from the browser history, but I was unsuccessful. Instead what we’ve begun to do is track usage history of a user and determine if they’ve visited a given URL in the past before we set the tag. That way we ensure that only pages that haven’t been cached previously are not specified as rel-prerender.
Either way, not bad for one line of code.
How about you? Have you used rel-prerender before? How do you see yourself using it to improve your site? I’d love to hear from you in the comments below.
52 Comments
Thanks for making us aware of another valuable tool in our arsenal. I would love to see a follow-up post with several use-cases. I’m thinking any time you’re pretty sure of the path, like in a tight funnel, or just for speeding up the top pages, as you discussed.
Real cool. I have been struggling with getting developers to understand the importance of prerendering the single page frameworks like angular but have shown massive improvements when we do. So good to know something similar is in the arsenal for all pages regardless of how they are coded, though for separate purposes. We have also seen some huge gains in traffic from increasing page load times. So, I am starting to think that Google has moved from just pushing slow sites down to lower rankings.
Thanks Mike – possible on pages in the checkout process? or would that be a nightmare
Yep, but you’d have to make sure any part of the page that requires a variable wouldn’t fire until the page is actually in view. That’s where the PageVisibility API comes in.
That’s a smart solution to the page speed issue.
Testing this on our landing pages I’m confident it should increase conversion rate.
Can you explain to me how this would have any benefit on a landing page?
Landing Pages are 50% design, 50% speed. You only have a few seconds to truly grab a visitors attention, so would it not be smart to have such pages load as quickly as possible?
rel-prerender is for speeding up pages not yet navigated to by the user. The user is on a page and it’s likely they’ll navigate to another page so you setup rel-prerender on the page the user is on so that the page the user is likely to go to will load faster. Landing pages are the kind of page that would most benefit from this but can’t because they’re the first page the user navigates to on a site. There are no other page before it that you can setup rel-prerender on.
I think he’s referring more so to his conversion page where the actual sale is made so this would work nicely for that purpose.
Exactly
@disqus_SjQWzrrlFi:disqus Correct me if I’m mistaken (no, really! I’m not one of those folks that’s afraid to be wrong!) But this actually DOES apply to landing pages, right?
When a person clicks on a SERP and the click yields a “landing page”, then prefetch and prerender & preconnect will ~INDEED~ create a faster loading landing page.
…again, not afraid to be wrong here 😉
I wondered the same thing. 🙂
Great idea as a starter and very quick. Thanks 🙂
Great article, love the Analyitics API idea, very clever!
So just to be clear, to add prerender for Chrome and prefetch for Firefox I just add
(update the URL to the URL of the page you’re coding for)
To the head of the page, or can it be anywhere in the page (Should it be in the head ideally?)
great question, I’m interested in THAT answer too~
It can be anywhere in the page ideally, you could also create link tags manually in JavaScript and append it in the page later.
Nice write-up Mike! Tons of interesting results, saving me the trouble of figuring each and every one myself.
Thanks man. .i tried this.. really amazing.. my site comedymood.com is little faster then previous.
Quick Question here Mike: (update the URL to the URL of the page you’re coding for)
To the head of the page, or can it be anywhere in the page (Should it be in the head ideally?)
I believe the ‘link rel’ attribute should ideally be in the head section of a page. You can refer to this page(s) https://docs.google.com/presentation/d/18zlAdKAxnc51y_kj-6sWLmnjl6TLnaru_WH0LJTjP-o/present?slide=id.g120f70e9a_0148 for more information. Moreover, anything in the head has a faster execution speed rather than being in the body.
For easy wins, I can see this being super useful for app onboarding and ecommerce; pretty much anything with an obvious “next step” user flow (rather than trying to predict which page the user will go to next).
Even for urls in your navigation menu I would say.
If you are browsing with your mobile data plan this literally sucks…consuming your data for something which is not 100% sure you will visit :S Just keep your sites as small as you can please! 😀
Which is why I said “Finally, although it’s available for some mobile browsers, I wouldn’t use it there. LTE downloads are slow enough.”
What do you think about using this approach for checkout steps. Although not being always the case, I think it is reasonable to assume that people being in step 1 have the intention to go to step 2 3 and 4. In that case I would put the following line of code in the header of checkout page step1: etc.
Is that correct?
It would be , but please keep in mind that those types of pages often require variables to be passed between pages, so you’ll want to make sure those parts of the page don’t happen until the page is actually visible.
That’s a smart solution to the page speed issue. I’ve been hitting my head on optimizing the ATF content by lazy loading the images and and java scripts. But still not sure about up to what extent the rel-prerender tag can help each individual pages. May be analyzing the referral path in GA will give us a chain of pages that visitors mostly clicks. So, prioritizing the high value pages and their referral pages is important to see the rel-prerender in action. Looking forward to see more case studies and more write ups on the same. Thanks for the wonderful write up and a heads up in an entirely new direction.
One small suggestion, I wish the images are clickable to see what exactly is there in the screenshots. I believe that should be one part of user experience because not many would prefer zooming the pages out of their dimensions to see what’s really in the screenshots.
Am I right in thinking that this method will not work if the prerender URL goes through a redirect (e.g. for tracking purposes)? And if that’s the case, if you input the final URL instead (without tracking) to be prerendered, when the link is clicked will the tracking still work as expected?
As the browser creates a hidden tab in order to prerender a page, it follows the redirect.
Hi All.
Is this part, or going to be a part of AMP?
Accelerated Mobile Pages is already out there, lots of resources etc. but I’ve not seen anything that covers this, which as so many have said is a ‘Doh!” moment for capture pages/sales pages/serve it quick pages.
“At the center of that is Chrome, a browser with components from Webkit”
Google is no longer contributing to Webkit, whose main maintainer is now Apple. Google moved to using Blink, their fork of Webkit.
Is a fork of Webkit not a browser with components from Webkit? Perhaps, I am misunderstanding what a fork is, but I thought it meant you are building something new with a preexisting codebase, but diverging from it.
You’re not misunderstanding, and that’s exactly why I wanted to note this. Blink is more and more diverging from Webkit, so saying “Chrome, a browser with components from Webkit” is not necessarily true.
What you describe in this post might not work in other Webkit-based browsers if it was added by Google since the Blink fork.
Nice writeup! Quick comment: you don’t need to wrap GA code with page visibility checks, it already does this on your behalf!
Hey Ilya, nice to e-meet you and thanks for commenting. I’ve been reading a lot of your stuff as of late as I’m sure you can tell.
As far as your comment, yep, I said that as well:
“While Google Analytics automatically accounts for it, most analytics packages will consider all prerendered pageviews as valid. Similarly, a prerendered pageview is most likely counted as an impression to your ad server.”
A page that’s taking over 12 seconds to load anyway has bigger problems than lack of prerender. As has been said elsewhere here, just get the pages as light as possible in the first place. Concatenate your CSS so that doesn’t keep downloading, reduce individual HTTP requests, use a CDN, and all that other really good stuff. A technique that automatically increases bandwidth usage, often without purpose, seems wasteful at best.
Really smart solution how to speed-up some important pages, useful not only for seo, but also for user experience. Thanks all: to author for article (i found it in G+), and to all for your comments that brings to my eyes the most important things.
PENIS
) )=====D~~~~~
REDDIT SENT ME HERE :)))
Wonderful article! Incredibly in depth & loved the Google Analytics overviews as well.
Brilliant! -Love how you tested this. Thank you thank you!
Answering to your question, “Have you used rel-prerender before? ” I can say, that, I was searching mainly for this information. My site was very slow and it has a bad traffic. I start to losing interest to work with this, but your topic and this topic about 5 Leadership Strategies That Get Results return my inspiration and I start working with double force.
Thanks a lot for this piece. Really cool feature, that i’ve never considered.
I read it completely but I still dont know what to do ? Can some please tell me what change do i need to make and where, I am not a technical guy tbh !!
The article is very interesting.
Helped me understand many things.
Thanks
Nice article! I want to try the rel-prerender in a few projects I work on, but, I am looking for some automated way to predict navigation of users in order to see which pages can be prerendered. So far I came up with a simple model which I described in my blog https://nesteryuk.info/2016/09/27/prerendering-pages-in-browsers.html Anybody is welcome to contribute to the idea.
The code in the filter field is unclear ? Also i only have ga:previousPagePath for dimension and i am usign google chrome ?
how about pre* partying on hovered links? or with hover intention (around the area of a link) might be useful where you have many links out from the page, and specifically don’t want to pre-render *all* of them.
Actually, you can prerender only one page per a window (please, notice it isn’t a tab). Hence, the hover solution doesn’t work. Also, it doesn’t make any sense to prerender a few pages without a decent confidence that the user of a site will visit it. Prerendering of a wrong page will create extra load on a server without benefit. I am also interested in this subject. A few months ago I started working on an open-source project which is aimed to be more accurate in prerendering the next page. You can try it https://github.com/sirko-io/engine.
Hi Michael, have you ever tried this plugin for dynamic prerender? https://wordpress.org/plugins/eve-dynamic-prerender/
Hi Michael,
Extraordinary article Lot’s of data to Read
Cloud Services
Leave a Comment