You’re in trouble.
Your top five rankings across all of your web pages have skyrocketed. When searchers use your target keywords, you’re visible in Google. So why are your organic traffic counts falling?
You did everything right.
How can things be going so wrong? If rankings are up, why is your organic traffic down? This is the current state of ranking tracking today; in the past, page one rankings meant a significant amount of traffic flowed your way.
It gets worse.
Most ranking tracking tools still haven’t accounted for these changes. They’re still promoting top ten rankings without context, giving organizations the wrong idea about their performance on Google.
The purpose and value of rank tracking
Originally rankings were intended to be a kind of benchmark brands could use to evaluate their performance in Google’s organic search listings.
Those familiar with search get this.
With SEO, many of your projections and ROI predictions are based on rankings. Combined with conversion and performance data (via web analytics), brands would have the data they need to improve their search campaigns.
There are a few caveats.
- Rankings aren’t as important as core metrics: Core metrics like conversions and revenue are far more important than rankings at the end of the day. Your conversions, revenue, cost per lead, customer lifetime value, retention, and churn rates are more important than rankings.
- Rankings aren’t important on their own. Gathering rankings for keywords that do nothing to move the needle means your search campaigns are unsustainable. If there’s no improvement to revenue, no financial lift, you won’t sustain these rankings for very long.
Today, rank tracking is highly personalized and focused heavily on search indicators like intent, context, and type. But it wasn’t always this way.
How rank tracking operated historically
Originally, rank tracking was a simple affair.
Google displayed ten blue links (not including ads) in their search results. These search results were simple, direct, and easy to monitor.
Remember what they looked like?
In the beginning, Google displayed ten blue links.
The first three links received the lion’s share of visibility and traffic from searchers.
All of this changed with Google’s universal search update in 2007. Google began integrating video, images, maps, shopping, and news results into their search results. Personalization began to play a more prominent role, along with snippets, knowledge panels, the local packs, etc.
Today, we see search results that look like this.
These changes are significant.
We’ve come a long way since the early days of search. Today, Google search (and rankings) is oriented around five important factors.
- Meaning: When Google discusses meaning, they’re working to establish what you (the searcher) are looking for — the intent behind your query. For example, you might have searched for ‘change a light bulb,’ but Google knows that you mean, ‘replace a light bulb.‘“
- Relevance: This is an assessment; once Google understands the meaning of your query, they “analyze your web pages and content, looking for indicators showing whether it contains information that might be relevant to what searchers are looking for.”
- Quality: This standard looks at usefulness or value to the user. This is where inbound links and social signals play a prominent role—these votes from other trusted websites outline.
- Usability: Is your web page easy for people to use? If two pages are equal in quality, the page that is most accessible tends to be the winner. Google publishes its page experience signals (core web vitals, mobile-friendliness, HTTPS, and no intrusive interstitials) as a way to gauge usability.
- Context: Google uses factors like location, relevance, and recent search activity as a gauge to determine context. For example, ” you search for ‘Barcelona’ and recently searched for ‘Barcelona vs. Arsenal,’ that could be an important clue that you want information about the football club, not the city.”
Why does this matter?
Google ran more than 600,000 experiments and made more than 4,500+ changes/updates. They made 350 to 400 changes a year back in the 2010s.
According to their how Search works primer, they ran:
- 3,620 launches
- 17,523 live traffic experiments
- 383,605 search quality tests
- 62,937 side-by-side experiments
They’re always working to improve their algorithms and their value to searchers.
Which means what?
Their search results are always changing, and updates are always being made. That’s a problem in this current landscape because most rank tracking tools aren’t accounting for these changes.
That’s a problem.
Various factors impact the way we approach rank tracking.
Key factors that change rank tracking
Several factors have changed the rank tracking landscape. While many providers have shifted to accommodate these changes, many have not. This means the ranking data presented to clients is not as valuable to clients as it should be.
Context is missing.
That’s a big deal because a large subset of clients use ‘rankings’ as proof that results were achieved on their campaign. Rather than analyzing traffic, conversions, or revenue — which requires nuance, they focus on ‘rankings.’
It’s easy, simple, low-hanging fruit.
But it’s also incomplete, which is why it’s so important to get rank tracking right. Here are five key factors that impact rank tracking.
- Zero click results. These are searches that send zero traffic. Users stay on Google and do not click through to a third-party website. According to SparkToro and SimilarWeb, two-thirds of all search engine visits are zero-click searches; if you focus exclusively on mobile, that number balloons to 77.22 percent of all visits.
Here’s the thing.
Google claims SparkToro’s data is flat out wrong. Danny Sullivan, Search Liaison, had this to say “This claim relies on flawed methodology that misunderstands how people use Search. In reality, Google Search sends billions of clicks to websites every day, and we’ve sent more traffic to the open web every year since Google was first created. And beyond just traffic, we also connect people with businesses in a wide variety of ways through Search, such as enabling a phone call to a business.”
Some claimed that SparkToro refused to share their methodology but Rand corrected that assertion and pointed us to an update where he shared his methodology and provided several updates. Google (obviously) refuses to share its data so we won’t be able to get their point of view. Your best option? Plan for the worst, use SparkToro’s research and assume that zero-click results are an issue you’ll have to deal with.
Better to be safe than sorry.
- Rich snippets. According to Google, rich Snippets are a visually enhanced search result that uses structured data (pulled from the HTML in your web pages) to provide search engines with important details about specific web pages. These rich snippets rely on the standards set by Schema.org.
There are various types of rich snippets.
Article excerpts, Google’s knowledge panels, product carousels, book listings, recipes, datasets, and more. The key point with each of these is these results are visual. Some people believed that structured data would be used as a ranking signal, but John Mueller, Search Advocate at Google, said they are not used as a ranking signal.
They do provide direct and indirect benefits.
Direct benefits:
- Ben Goodsell found that the clickthrough rates on Featured Snippets jumped from 2 to 8 percent and revenue from organic traffic grew by 677 percent
- In a featured case study, he found that organic sessions grew from 139,758 to 861,385 in just four months, a 516.34 percent increase
Indirect benefit: Structured data gives Google a deeper understanding of the content on your pages. The easier it is for Google to understand The theme and overall message behind your content, the easier it is for them to rank and display your pages where they’re most relevant.
Here’s the caveat.
Rich snippets are considered zero clicks, yet they take up search results. What’s worse, if your web page receives the rich snippet treatment, that same page won’t be visible on the first page of Google’s search results.
Google decides which pages are displayed as snippets. You can optimize your web pages using Schema.org, but there’s no guarantee they’ll agree. Rich snippets have decreased the value of #1 rankings. This is a problem for a variety of reasons, one being…
- Ranking below the fold. Dr. Pete Myers discussed this recently, stating: “Being #1 on Google isn’t what it used to be. Back in 2013, we analyzed 10,000 searches and found out that the average #1 ranking began at 375 pixels (px) down the page… Here’s our big “winner” for 2020, a search for “lollipop” — the #1 ranking came in at an incredible 2,938px down.”
That’s outrageous!
From 375 px in 2010 to 2,938 px down in 2020! Yeah, yeah, not every search result is this extreme, but it’s still far worse than it used to be. A video, music links, knowledge panel, more videos, people also ask, images, and top stories links all came first before the first result from Wikipedia, a trusted site.
This is sobering when you realize “Wikipedia is the No. 1 result on Google for 56 percent of searches, while 96 percent of searches saw Wikipedia in one of the top five positions.” Wikipedia is doing everything right, and it’s still not enough!
- Location/personalized search results. Google has stated that personalization is light and that their changes to personalize their search results are subtle. That may be true, but it’s also true that these ‘minor’ changes impact search results.
Location plays a prominent role in Google. The three local factors they state are disproportionately important are:
- Relevance: How will your business matches a searcher’s query.
- Distance: Proximity, e.g., how far each business is from the searcher’s location.
- Prominence: How well known a business is.
This is in addition to the criteria listed above.
What about personalized results?
Google personalizes searches based on a variety of ranking factors. The biggest four are as follows:
- Location: The search query “football” in Chicago may bring up search results for the Chicago Bears, while “football” search queries in London may bring up search results for Manchester United. Location plays a prominent role in addition to the other factors mentioned.
- Search history: Google uses your search history (past and present) to determine intent, relevance, and context. Your search history provides Google with the important clues they need to serve up results.
- Browsing history: If you repeatedly visit specific web pages, Google assumes you like those web pages and will rank them higher for you. Google uses a variety of sources (including Gmail) to personalize data for searchers.
- Language preferences: Google takes note of the language used in your query. If you enter Spanish queries, visit Spanish-speaking websites, and select Spanish as your preferred language on sites you visit, Google notices. What about Google products? If you have specific language preferences, these behaviors impact the web pages Google displays in your search results.
- Advertising. iPullRank CEO Mike King made a very important point.
“Ads. Featured Snippet. People Also Ask. Then, finally, the first organic (web) result. This is a clear indication that Organic Search rankings need to be presented with context of the visual position, not just its numerical position in terms of other Organic Search results.” (emphasis added)
Why is this important?
Searchers don’t care about the numerical position! They care about the visual position!
How do I know?
The data says so! Click-through rates fall below average for all positions when the top of the SERP is full of Google Ads.
This isn’t new at all.
This was still the case way back in 2014.
Search Engine Land found that “On average, the presence of ads on a search results page caused the organic CTR of the first position to drop by 30% — from 25.7% organic CTR in the absence of ads to 17.9% CTR when ads are displayed.”
Larry Kim, Wordstream wrote:
“Clicks on the search results page are basically a zero-sum game. If there’s an increase in CTR for one part of the SERP, some other part is losing that click. There must be a decrease in CTR elsewhere. And that includes the ads.”
Again, this was in 2014.
If you’re relying on a rank tracking tool, that tool should be able to provide you with data on both the visual and numerical rankings. This is incredibly important. This data (combined with your conversion data and data from search console) tells you whether it’s worthwhile to spend to dominate the search results.
If it’s a revenue-driving keyword, yes, it’s worth it.
A vanity keyword? Not so much.
Okay, we’ve seen the current state of rank tracking on Google’s end. How are current rank tracking tools doing? Are they able to account for all of these changes (e.g., rich snippets, knowledge panels, product carousels, ads, people ask boxes, and more?
It’s time to find out.
Rank tracking tools and the value they provide
The current ranking paradigm is obsolete. The value of position #1 is in freefall. If we’re going to rely on tools for rank tracking, we need a tool that accounts for all of the changes in search. What sort of analysis do we need for rank tracking today?
King recommends that rank tracking include the following data:
- Legacy Rankings – All platforms should continue to compute rankings the way they previously have to allow for reporting continuity. Introducing something different from the way that rankings are currently considered is difficult because making the switch will potentially tank all of your reporting. Legacy ranking models should still be computed and added to reports. The new rankings model could also be applied for those tools that maintain SERP archives to give marketers a good sense of what these metrics illustrate over time.
- Web Rankings – This is the metric that we’ve been using. However, we’d count the presence of a featured snippet as part of this metric. This would be the equivalent of STAT’s base rankings
- Absolute Rankings – A position complete with all features on the SERP, including ads.
- Feature Rankings – Organic Rankings that include all Organic SERP features. Presumably, this could be used to better compare GSC data with a ranking. [This includes featured snippets, video, ads, thumbnails, knowledge panels, site links, top stories, etc.]
- Offset – The pixel position of the ranking from the top of the page. This would be measured by the distance that the a tag has from the body tag.
- Page – The page of the SERP that the ranking was seen on.
So reports would present the following data (in a visual format, of course).
For the keyword [car insurance] that yields:
- Web Ranking – 7
- Legacy Ranking – 7
- Absolute Rankings – 19
- Feature Rankings – 15
- Offset – 2830px
- Page – 1
Which tools provide this kind of comprehensive view of web page rank tracking?
Let’s take a look.
NOTE: This data is above and beyond the industry’s normal scope of rank tracking. The features we’re discussing are necessary additions but don’t negate the existing features list customers expect (e.g., local search + organic search) from these tools.
| Ahrefs | AWR | Stat | RankTracker | Semrush | Whatsmyserp | Cognitiveseo | SEOmonitor | |
| Web rank | X | X | X | |||||
| Legacy rank | X | X | X | X | X | X | X | X |
| Absolute rank | X | X | X | |||||
| Feature rank | X | X | ||||||
| Offset | X | |||||||
| Page | X |
See the problem?
The vast majority of tools don’t provide the kind of data you need to make good decisions.
Let’s visualize this.
Suppose you ranked #1 for the query “car insurance.” That sounds awesome, right? Your rank tracker tells you that’s awesome, and you believe them. But you’re not generating the kind of traffic your keyword research says you should be generating.
If you’re following your rank tracker, you may decide to divert resources away from this keyword.
This would be a serious mistake.
Why?
Because your rank tracking tool doesn’t have the know-how, it needs to tell you that visually, you’re actually #8.
Remember, visual rankings matter more to searchers.
Yikes.
This leads to all sorts of horrible things:
- Paying for vanity keywords that provide little to no value
- Abandoning keywords with significant value due to a lack of intel
- Neglecting web pages that rank well but fail to perform
Rank tracking, as it is, is fundamentally broken. Google made significant changes to their search results, but rank tracking tools haven’t caught up yet. Best case scenario, customers are getting inaccurate data. Worst case scenario, customers are making bad decisions using false data.
It’s a disaster.
What does the future hold for rank tracking?
As an industry, we’re just not set up to handle rank tracking in a way that’s comprehensive and makes sense for SEOs. As an industry, we can’t provide customers with the detail they need in their reports right now. There are just too many gaps.
Gaps? What gaps?
- Tracking changes for the entire SERP, not just what you rank for. (e.g., new – ae.com, missing – macys.com, improved- nordstroms.com, decreased – gq.com)
- Tracking changes to the SERP over time with increased granularity ( e.g., daily, hourly, 30 min., 15 min., etc.)
- Tracking SERP features over time (rich snippets, local packs, knowledge panels, carousels, etc.)
- Track changes to SERP pages
- Topic clustering
- Backlinks per page
It’s sink or swim time for the industry; if we’re going to continue to add value, our industry needs to be able to fill the gaps marketers are currently dealing with.
What’s the current state of rank tracking?
Rank tracking is broken.
Marketers are seeing top ten rankings increase while organic traffic counts are falling; This is the current state of ranking tracking today. The rules have changed; in the past, page one rankings meant a significant amount of traffic flowed your way.
Today, feature rankings and blended search elements have changed the game. Most ranking tracking tools still haven’t accounted for these changes. They’re still promoting top ten rankings without context, giving organizations the wrong idea about their performance on Google.
With SEO, many of your projections and ROI predictions are based on rankings. With the right amount of granularity and detailed analysis, rank tracking tools move in lockstep with Google, no confusion necessary.
NOTE: An earlier version of this post stated that SparkToro failed to share this methodology. Rand Fishkin reached out to us and shared that data. He also provided important updates and context for this study.
- The Cost of SEO Services in 2024 - January 4, 2024
- The 2024 SEO Guide To Successful Website Migration - April 4, 2023
- How Audience Research Shapes Financial Services Marketing - February 16, 2023
