A tutorial on estimating your share of search for different keywords using Google’s Webmaster tools search queries data
Since search engines were first created, they have offered great options for businesses to grow awareness and sales at a relatively low-cost. Given the potential of search to drive new business, being able to benchmark how effective your SEO is against your competitors is essential to get a baseline from which to improve.
While marketers have and still do use rank checking tools for the keywords they are targeting, what you really want to know is how many visits (and so potential sales) you are attracting to your site as a share of the total demand for products and services based on the consumer intent, i.e. the number of searches by those looking for new products and services. The Google Keyword Planner is great for gaining an idea of the number of searches for different keywords, but it doesn’t help you readily compare to the number of actual visits you’re receiving to your site. This is where you need a SEO gap analysis.
What is an SEO gap analysis?
I would define the purpose of a gap analysis as:
“An SEO gap analysis helps you review your ”market share of consumer searches“ for products and services you offer by comparing the number of searches performed for the keywords you are targeting against the number of visitors you get to your site for these keywords through natural search in the organic search engine listings”.
Where there is high demand for products or searches indicated by the number of searches, but you have a relatively low share of visits to your site from search this is a ‘gap’ that you need to understand and review techniques to improve the number of SEO visits. Dan Barker explained the approach in more depth in this 2010 post on using Google Analytics to audit and improve SEO.
Going further back, I first talked about this approach when I wrote the first edition of the Econsultancy SEO Best practice guide in 2006, explaining that it was helpful to perform an analysis of demand or intent for different keyphrases compared to how many visits and conversions SEO was delivering for these phrases.
The challenge of gap analysis in 2014
Back in 2014 we have a big challenge with gap analysis since Google has introduced secure search leading to the growth in Not Provided Keywords in analytics where over 80% of the search terms entered by searchers are unknown.
However, Google has provided a new tool to help SEO analysis within Google Webmaster Tools which can still be used for a gap analysis for marketers looking to assess their SEO performance.
How to perform a gap analysis using the Google Webmaster Tools data
Gavin Llewellyn has explained the insight this gives for SEO in his recent tutorial. For example, here you can see the number of searches for different types of content and assets as shown below and review the keywords too.
Gavin also described the process, but I’ll give a quick summary here to help members who have downloaded our SEO gap analysis template which makes it quicker and easier for gap analysis if you’re not an Excel whizz!
Step 1. Check you have access to the Google Webmaster Tools data in your Google Analytics
Step 2. Access and export the queries data into Excel format
Use the Export menu to create a spreadsheet .xls import when viewing the keyword queries, this screen grab shows a summary of our keywords related to our race planning framework.
Step 3. Import the Excel data into the spreadsheet you are using for gap analysis
You will want to paste the entire worksheet labelled “Dataset1” by Google Analytics on export into your spreadsheet.
Step 4. Perform an “exact match” gap analysis
This involves comparing the actual number of visits to your site for different keywords you are targeting (column B) against the total number of searches (impressions) measured as an exact match (column C). Conditional formatting enables you to quickly review where there is an opportunity to improve position (column E) or clickthrough rate (share of search in column F).
Step 5. Perform a “keyword includes” gap analysis
I also recommend that a gap analysis can involve a review of individual keywords you are attracting visits for across a series of key phrases. I think this is powerful since you can quickly analyse how good your visibility is for head terms (such as product categories) or long-tail search terms (such as reviews or detailed product features) you are targeting as shown in this example for different products:
Warning - remember the data limitations
As Gavin notes, although the Google Webmaster Tools data has been roundly criticised in the past, I believe that the data should be used since some of the restrictions have been lifted with more improvements to come. That said you do need to be aware of the limitations compared to previously and should be careful to communicate these when reviewing and sharing the data:
The data goes back to 90 days (expected to soon be extended to one year).
You can only see the top 1,000 terms for a single day usually up to a maximum of several thousand keywords (it’s around 8,000 for us)
You can’t review bounce rate, conversion rate or revenue per visit as for the original keyword data which show the degree of engagement and value generated from each keyword. Our previous spreadsheet showed all of this and compared to paid search too; much more helpful to see the commercial impact of SEO.
You can’t readily compare with Google AdWords data which was one of the main benefits of the form of gap analysis we developed. But Google has recently launched a feature in its AdWords accounts that does give a comparison.
I hope you find this article prompts you to perform a gap analysis or ask your SEO provider about it. If you know your way around Excel it’s quick to create and I know many agencies and consultants have their owns versions. Please let us know in the comments if you have any questions or observations based on how you tackle gap analyses! Our spreadsheet is available for download by members here.
By Dave Chaffey
Digital strategist Dr Dave Chaffey is co-founder and Content Director of marketing publisher and learning platform Smart Insights.
Dave is editor of the 100+ templates, ebooks and courses in the digital marketing resource library created by our team of 25+ digital marketing experts. Our resources are used by our Premium members in more than 100 countries to Plan, Manage and Optimize their digital marketing. Free members can access our free sample templates here.
Dave is a keynote speaker, trainer and consultant who is author of 5 bestselling books on digital marketing including Digital Marketing Excellence and Digital Marketing: Strategy, Implementation and Practice. To learn about my books, see my personal site Digital marketing books by Dr. Dave Chaffey.
In 2004 he was recognised by the Chartered Institute of Marketing as one of 50 marketing ‘gurus’ worldwide who have helped shape the future of marketing.
Please connect on LinkedIn to receive updates or ask me a question.
Start the discussion on our community and social networks