Google Search Console is a gold mine for data.
Launched in 2001 as Google Webmaster Tools this was a way for marketers to submit sitemaps and check crawling/indexability. Google rebranded Webmaster Tools as Search Console in 2015 and has been slowly making changes and updates along the way. Most recently, they updated the look and feel as part of a UI redesign in November 2021. Although these feature updates are useful, like the addition of RegEx filtering in June 2021, These cosmetic and functionality changes lack the bite that many data analysts crave: a firehose of data or the flexibility to mashup 3rd party data – until now.
In December of 2021, iPullRank and GrepWords offered a practical and tactical webinar that showcases how you can leverage keyword intelligence at scale with the GrepWords database. One of the most popular use cases demonstrated how you can now dynamically alter the Google Search Console display and insert search volume and CPC data directly, without having to export the data.
How do we make this magic? It’s done using a proxy with GrepWords API integration and a bookmarklet. The proxy acts as a way to get around CORS and provides the trust required to inject and update the DOM on the GSC page. The bookmarklet parses the GSC page and provides the keywords to the proxy, which fetches data from GrepWords and parses the search volume and CPC of the keyword.
Pro-tip: This process is done in real-time, so as suggested in the webinar, you might want to create multiple bookmarklets to limit the keywords and time it takes to fetch and update.
If you’re an SEO who’s interested in implementing this for your business, you may want to work with a developer for the next steps of this process. This might get a bit more technical, but I’m trying to make it as simple as possible by offering the entire process and codebase completely open sourced on Github
Let’s dive in.
The proxy is just a node.js (express.js) application that makes the POST API for the Keywords endpoint with GrepWords and provides the response to the client, in this case, the bookmarklet code. This could easily be swapped for a cloud function or even a Cloudflare worker. You just need to make sure to allow the origin for CORS, as the request will be coming from “search.google.com”.
The bookmarklet has 3 main constants that need to be adjusted before using:
- APIKEY
- KEYWORDLIMIT
- BASEURL
The APIKEY is the GrepWords API. Even though you could put this logic in the proxy, I wanted to allow for the most flexibility, as you might already have a CORS proxy and this simply allows you to define the BASEURL.
The KEYWORDLIMIT is how many keywords from GSC should be fetched to check the search volume and CPC. As mentioned in the webinar, I would create one for 25, 100, and 1000, as this data is fetched in real-time and 1000 takes a while. You do not need to always run 1000 keywords.
For example, name them as follows:
GSC – 25
GSC – 100
GSC – 1000
Adjust the KEYWORDLIMIT value as 25, 100, and 1000 respectively for each of the bookmarklets. This way you can easily limit the keywords that are processed and throttle the number of API requests you are making so as not to waste API credits.
The BASEURL is the endpoint for the proxy, which in my demo case, I used Heroku so it would be something like https://xxxx.herokuapp.com.
After you update these 3 constant values, I would suggest using a tool like Bookmarkleter which allows you to copy/paste and minify the bookmarklet code.
In your Chrome browser, just open chrome://bookmarks to view your bookmarks, since you will then need to add a new Page to the Bookmarks Bar. Once you add the new bookmark, you can easily edit it just by right-clicking and selecting edit.
Under URL, you will be pasting the javascript code which allows you to kick off the process for adding the search volume and CPC data to GSC.
This workflow allows for easy development as I would just paste the changes into the Bookmarkleter, then copy/paste and edit the new Bookmark and open GSC and click the GSC – 25 Bookmarklet.
So now that the proxy is set up and the bookmarklets are created, I just open GSC on my favorite property and click the Performance tab as it should default to the Queries listing. Then click the bookmarklet and wait, as it fetches and updates the Search Volume and CPC data.
Please note that if you want to force a specific list of keywords, you could sort by Clicks or Impressions BEFORE pressing the bookmarklet, as it starts from the top of the list. Also as an FYI, this data is just injected in the DOM, so if you click to somewhere else on GSC or reload the page, it is removed.
Armed with search volume and CPC data directly in GSC, you can now analyze your search performance in the context of opportunity and importance. With search volume, you can visualize traffic performance potential at a glance. CPC can provide insight into the value of keywords.
Take this information, go forth, and make SEO magic.
- Puppeteer on The Edge: SEO Use Cases with Cloudflare’s New Rendering API (beta) - November 23, 2022
- What To Do When Rich Snippets Steal Your Clicks - February 22, 2022
- Create a Slack App: Lookup Search Volume and CPC for a Keyword - February 8, 2022
