In this webinar, Mike King and John Murch at iPullRank sat down to discuss the challenges and gaps with modern rank tracking. Google’s systematic updates mean many rank trackers have been left behind. The usefulness of rank tracking software has declined as many providers struggle to keep up.
Today, they’ll expose the problems with modern rank tracking and provide you with a way forward.
Watch the replay here:
Here’s what we covered in this webinar:
- What is rank tracking? [0:38]
- The history of rank tracking [1:08]
- Why rank tracking matters [2:51]
- The fundamental flaw with rank tracking software [4:19]
- How Google and SEOs measure rankings [4:35]
- How rank tracking has changed [6:29]
- How should you be rank tracking? [12:47]
- The gaps in rank tracking [21:39]
- Why you should be tracking SERPs [28:05]
- The relationship between rank tracking, SERP tracking, and topic clustering [32:11]
- Rank parsing re-run to analyze historical data [39:12]
- The wall between paid and organic [40:01]
- Using Exact Science to address these rank tracking problems [41:58]
What is rank tracking? [0:38]
If you’re an experienced marketer, you probably know this one.
Rank tracking is the process of tracking a web page’s search engine position for a select set of keyword queries. That’s the thing about rank tracking — definitions are simple; however, the practice of rank tracking has become incredibly complex.
It’s an issue that is, for the most part, neglected by rank tracking tools.
How did we get here?
To answer that question, we’ll need to look at the origins of rank tracking.
The history of rank tracking [1:08]
Let’s go way back to 1995.
AOL bought Web Crawler, an old-school search engine. Web crawler would later be sold to Excite. Later, AOL would launch its own branded search engine, called AOL Search. AOL would eventually create a partnership with Google.
Why do these dinosaur search engines matter?
Many of these search engines operated on the original clickthrough rate model. Larry Kim (formerly of Wordstream) wrote:
“Clicks on the search results page are basically a zero-sum game. If there’s an increase in CTR for one part of the SERP, some other part is losing that click. There must be a decrease in CTR elsewhere. And that includes the ads.” Kim’s research showed us that the higher the ranking, the higher the clickthrough rate. The more traffic you receive, the higher you’ll rank for competitive search terms.”
Why rank tracking matters [2:51]
Traffic comes from rankings.
But there’s more to it than that. You’ll need to identify the various decreases in traffic, monitor the competitive landscape, spot various types of opportunities (and threats), and look for low-performing content or gaps in the space.
Here’s the caveat.
Great rankings don’t automatically mean you’re driving a lot of traffic. In fact, you can rank well for lots of queries but fail to drive any meaningful traffic. Rank tracking is a valuable tool, but as Mike King points out, “it’s not the end-all-be-all of organic search performance.”
Gathering rankings for keywords that do nothing to move the needle for your organization means your search campaigns are unsustainable. If there’s no improvement to revenue, no financial lift, you won’t be able to sustain your rank for very long.
It’s better than viewing rank tracking as the start of the journey.
The fundamental flaw with rank tracking software [4:19]
Rank tracking software needs to deceive Google.
Over the years, Google has introduced localization, added personalization, rich snippets, and various other helpful features. What’s common across each of these features is Google’s increasing dependence on meaning and context. Marketers know this, and they work within these parameters.
When rank tracking is done, this goes out the window.
The process begins with a clean computer — a computer that’s logged onto the internet for the first time, a browser that has no cookies, no history, and little to no location data.
That’s not an actual user.
This is a problem for obvious reasons. The SERPS and rankings this sanitized computer sees and displays to you, the marketer, isn’t the same thing actual users see. As a result, ranking data is immediately suspect.
Searchers using the query “football” will see very different things depending on their location (Chicago vs. Manchester).
How Google and SEOs measure rankings [4:35]
Google measures rankings (or they call it position) differently from SEOs. Looking at Google’s documentation, we see that they measure position in a kind of S shape, a sideways S shape; The first five positions in the organic are measured, then they start measuring the knowledge graph and other features like that are above the fold. SEOs tend to go straight down the page.
What does this mean?
If you’re comparing your position in GSC versus a rank value from an SEO rank tracking tool, you’re seeing two completely different values. Your rank tracking tools should be able to account for that.
How rank tracking has changed [6:29]
Mobile and desktop SERPs are completely different.
- Google captures more location data on the mobile side
- There’s continuous scrolling on mobile along with site links. Some brands have multiple links.
- Depending on the query, you also have personalization (a search done in New York versus Pittsburgh returns different results)
- Google’s also added indented results, filters, and pills
- Then there are rich snippets
- Time of search plays an important role as well (e.g., trending news stories vs. product listings based on open/close times of a specific store)
Then there’s also the number of features in a particular SERP.
Some SERPs include lots of features. Videos and music links, a knowledge panel, more videos, people also ask images, top stories, and links — all before the first result.
It’s a lot to keep track of.
Maybe this is why many rank tracking tools haven’t been tracking these details?
How should you be rank tracking? [12:47]
When you do rank tracking the way it’s always been done, you’re not looking at your SERPS. You don’t really understand what you’re up against. If you rank number one for coffee grounds here, you think you’re doing well, but realistically you’re probably not driving many clicks on this keyword. This may be a keyword that you’ve spent a lot of time on that wasn’t actually valuable.
You should be able to bring more comprehensive data into your rank tracking. It’s a necessity in order to be more strategic about which keywords you should be targeting based on where your true opportunities are. Looking at search volume, looking at CTR models, just looking at pure rankings — it’s not enough to give you the whole story.
Some of the data you need:
- Legacy Rankings – All platforms should continue to compute rankings the way they previously have to allow for reporting continuity. Introducing something different from how rankings are currently considered is difficult because making the switch will potentially tank all of your reporting. Legacy ranking models should still be computed and added to reports. The new rankings model could also be applied for those tools that maintain SERP archives to give marketers a good sense of what these metrics illustrate over time.
- Web Rankings – This is the metric that we’ve been using. However, we’d count the presence of a featured snippet as part of this metric. This would be the equivalent of STAT’s base rankings
- Absolute Rankings – A position complete with all features on the SERP, including ads.
- Feature Rankings – Organic Rankings that include all Organic SERP features. Presumably, this could be used to better compare GSC data with a ranking. [This includes featured snippets, video, ads, thumbnails, knowledge panels, site links, top stories, etc.]
- Offset – The pixel position of the ranking from the top of the page. This would be measured by the distance that the a tag has from the body tag.
- Page – The page of the SERP that the ranking was seen on.
The gaps in rank tracking [21:39]
This gets into kind of some of these cracks that we talked about and things that need to be seen across the board in rank tracking. This includes details like understanding the pixel height of different SERPS, your rankings, and how it compares with the other features on the page.
A Google placement of #3 organically may mean you’re actually the 25th link down because of a knowledge panel, a local pack, people also ask, and all these other featured snippets.
What does the industry need then?
At its core, we need better insights for change.
Murch paints the picture for us. “Basically, there are things on a SERP that are actually changing, but you may only be ranking or checking your rankings for one URL. So you’ll say, Hey, it went from position ten to a two. Awesome. Great. What else changed on that page? Right? Someone who was at position #83 is now at #10? There’s a new competitor who is actively going after this. Do I know about that? Are there content changes to pages that are currently ranking? We may rank within the top three positions, but someone else is actively updating their page. We should know about it or see what changes they’re making.”
Google search console gives us a lot of data, it’s true, but being able to integrate and showcase that with ranking data helps paint a clearer picture. It offers other insights showing SEOs what has increased or decreased across various SERPs.
Why you should be tracking SERPs [28:05]
The reason? There are different types of rank tracking.
You may have new links that show up and missing links that change. You’ll have different sites whose rankings improve and others that deteriorate; seeing that fluctuation on a per rank basis means you’ll be able to see what other things are happening with these SERPs.
If you’re doing this daily or hourly, every 30 or 15 minutes, you have historical data you can use to map these SERP changes over time. This is invaluable as it gives you hard data on the stability or volatility of specific keyword sets, topic clusters, and SERPs.
It’s data you can use to make good decisions.
The relationship between rank tracking, SERP tracking, and topic clustering [32:11]
Topic clustering is another essential component that’s missing from rank tracking tools.
Imagine that you’re tracking X number of URLs in the top 10.
Now, there are lots of different tools that do top three, top four, top seven, but imagine that you were able to say, “okay, here are the SERPs that are very similar between these keywords. So if we go after this cluster of keywords, we can get a better sense of how Google ranks pages and topics in this industry, niche, or category. The amazing part about all of this is the fact that this gives you insight into how Google approaches similar keywords or meanings.
What do I mean?
We can analyze different synonyms and keyword permutations; it’s you being able to provide all of that data. There’s so much data that Google basically just gives us as part of the SERP. We should be leveraging that as SEOs.
What about Google’s API?
Google Search Console offers a lot of data; their UI and tool are great, but their API provides more data. Breaking this data down on a per-day basis and overlaying that with Google Search Console data and your rankings data gives you deeper insights you can use to make (much) better decisions.
Rank parsing re-run to analyze historical data [39:12]
Then there’s historical data analysis.
Murch discusses a topic that’s near and dear to his heart — a rank parsing re-run. “I found a few issues when we pulled different SERPs from different providers and, although they can fix the issue, it doesn’t fix my historical data. So the idea of being able to re-run data or if there’s something new and it’s confirmed is an important one.
With a re-run, I go backward and say, “Hey, I can re-parse this data and state confidently, these are the changes.”
This means you can monitor historical data and go back in time with the new knowledge that you may have learned by analyzing your SERPs (both paid and organic data).
The wall between paid and organic [40:01]
Are your paid and organic search teams sharing their data?
Most aren’t.
Murch explains, ” I don’t know why, but paid and organic are always on two different teams; there’s a wall in between, and there’s a lot of PPC data that should be shared on the SEO side. If you’re targeting keywords or even just trying to rank for them, it’s a good idea to test their conversion value by throwing up a landing page and running paid ads. That should be step one.” When you have this data, you can say, “Hey, these keywords convert. Let’s now spend some time and money ranking for them.”
Is there a rank tracking tool that can do all of this?
Using ExactScience to address these rank tracking problems [41:58]
“We’re going to be launching ExactScience, which is our rank tracking software that addresses a lot of these problems because we feel like there’s a gap in the market. There are a lot of things that I just want to highlight and showcase in some of these screenshots that we’ve done.
You should be able to show pixel position, different types of ranking (whether it be branding competitive or the site), some of the features, and the different kinds of tagging and categorization you could do. There’s much more to come about this, but we wanted to have a big announcement. And for all those asking the next question, yes, sign up to become a beta tester. If you go to iPullRank.com/beta-testers, you can sign up and join our waitlist. We’ll be slowly reaching out to you and testing some of our solutions against some of your problems. If you have specific ranking challenges or ideas, please reach out and let us know.
About Mike King, Founder and Managing Director at iPullRank
Mike is the Founder and Managing Director of iPullRank. He has an extensive background in software, web development, and creative. He’s an industry leader, and his work is frequently featured on sites like Moz, SearchEngineWatch, Unbounce, and Distilled. He’s a sought-after speaker speaking internationally at SMX, SearchLove, SES, MozCon, and LinkLove.
About John Murch, Director of Software Engineering at iPullRank
John has 15+ years of experience in SEO and software development. He’s a developer with a background in online marketing, and he loves building products and services.
- The Cost of SEO Services in 2024 - January 4, 2024
- The 2024 SEO Guide To Successful Website Migration - April 4, 2023
- How Audience Research Shapes Financial Services Marketing - February 16, 2023
