In other words, if there’s a lot of dynamic content (such as lazy-loaded images, product reviews, live chat, etc.), bloated theme code, and additional server requests, JS rendering can add seconds to the time it takes for the page to become interactive for visitors — which hurts UX and Core Web Vitals.
Let’s look into the three key phases in a little more detail.
Googlebot fetches a URL from the crawl queue by making an HTTP request. The first thing it checks is if you allow crawling in the robots.txt file.
If the file marks the URL as disallowed, then Googlebot skips the URL. It then parses the response for other URLs in the href attribute of HTML links and adds the allowed URLs to the crawl queue.
This way, Googlebot crawls the URLs in its crawl queue, page by page.
Google decides what resources are required to render the main content of the page. In the illustration above, the renderer is where Google renders a page to see what a user sees.
The system that handles Google’s rendering process is known as Web Rendering Service (WRS). When the WRS receives URLs, it renders them and sends the rendered HTML back for processing.
There are two types of rendering processes you should be familiar with.
Server-side rendering is when the rendering of pages happens on the server before they’re sent to the client (browser or crawler), rather than relying on the client to render them.
However, it can be a complex implementation for developers, and the Time to First Byte (TTFB) could be slow because the server has to render web pages on the go.
So while client-side rendering is popular, search engine bots struggle with it and it’s not ideal for SEO.
You can implement dynamic rendering by following Google’s guidelines and using tools such as Prerender, Puppeteer, and Rendertron.
- Blocked in robots.txt
Use of hash in the URLs
When you want to make your content visible in search, it’s important to use the more static-looking URLs as Googlebot typically ignores URLs with hash when trying to index the content.
Not checking the internal link structure
1. Be persistent with your on-page SEO efforts
This includes things like:
- Unique titles and meta descriptions
- Descriptive alt attributes for images optimized for speed
- Correct URL and internal linking structure
- Keyword-optimized, well-structured, valuable content for visitors
Here’s a great actionable guide to on-page SEO from Ahrefs.
2. Avoid blocking search engines from accessing JS content
3. Use relevant HTTP status codes
Googlebot uses HTTP status codes to know if something went wrong when crawling the page.
As a general best practice, use the right status codes for your pages to avoid issues like soft 404. For example, to tell Googlebot if a page shouldn’t be crawled/indexed, use relevant codes, such as a 404 for a page that’s deleted, or a 401 code for pages that require authentication.
4. Fix lazy-loaded content and images
As you know, images can have a huge impact on page performance. To defer the loading of images for when the user scrolls to them below the fold, you can use lazy loading.
By lazy loading content via the loading attribute, search engines can extract the image URLs directly from the HTML and visitors’ browsers know how to lazy-load the image.
5. Avoid soft 404 errors in single-page apps
In the case of client-side rendered and routed single-page applications, using relevant HTTP status codes may not be possible. To avoid soft 404 errors (wherein the server returns a 200 OK status to Googlebot for the invalid “404 not found” page) when using client-side rendering and routing, use one of the following strategies:
6. Use the History API instead of fragments
Now, Googlebot only considers URLs in the href attribute of HTML for crawling links. So, don’t use fragments to load different page content, rather, use the History API to implement routing between different views of your website. This ensures that link URLs are accessible to Googlebot for crawling.
7. Use meta robots tags carefully
So, make sure to allow these files to be crawled by adding the following code in your robots.txt:
8. Use long-lived caching
Long live the cache! Essentially, caching is all about improving load speeds.
You can use file versioning or content fingerprinting (like main.2ba87581f.js) to generate new file names whenever code changes are made so that Googlebot downloads the updated version of the resource for rendering.
9. Use structured data
Using schema markup to display rich results for your products or content pages is a great way to improve your click-through rate and SERP real estate.
Either way, use Google’s Rich Results Test tool to test your structured data implementation.
10. Follow best practices for web components
Web components are a set of APIs that let you create custom, reusable, encapsulated HTML tags to use in web pages and apps. They are based on four core specifications: custom elements, shadow DOM, ES Modules, and HTML templates.
While Googlebot supports web components, when it renders a page, it flattens the shadow DOM and light DOM content and can only see content that’s visible in the rendered HTML.
To ensure Googlebot can still see your content after it’s rendered, use the Mobile-Friendly Test tool and review the rendered HTML. If the content isn’t visible in the rendered HTML, Googlebot can’t index it. In such cases, you can use the <slot> element.
11. Design for accessibility
Ultimately, SEO is about satisfying users, and not everyone has a fast internet connection or the latest mobile devices.
To make sure your site is easily accessible to all users, test your website’s accessibility by previewing it in your browser with JS turned off. You can also use a text-only web browser like Lynx to identify rich media content that may be hard for Googlebot to render, such as text embedded in images.