Dealing with the "Keyword not provided" problem in your statistics

This article explains why the proportion of "keyword not provided" visits to most websites is increasing, and gives you options for finding out what keywords people are searching for when they reach your blog.

Why the percentage of not-provided search visits to your blog has increased

If SEO is important for your blog, and if you therefore watch the Stats > Traffic Sources tab in your Blogger dashboard or your Google Analytics results, you'll probably have seen that proportion of your search-visitors whose keyword is "not provided" has gone up a lot recently, to be more-or-less 100% of your Google search traffic.   (In the Blogger Stats tab "not provided" isn't shown - but the number of visits per keyword is now massively less than the vists from Google.)

This is no accident: Google is now witholding the keywords that people use, and (says that) this to protect your visitor's privacy. The issue has been widely discussed in sites like SearchEngineLand.

Opinions vary, but many people believe that

"Not knowing keywords has big implications if you use data about what people search for to decide how to develop your blog." [tweet this quote].

For example,   I publish listings of the contents of old (ie graphical copyright expired) song-books in a particular niche on one of my blogs.  There are far too many songs for me to load the full text or sheet music of all of them. And this is a niche with lots of competition:   there are a zillion websites distributing song-lyrics (most illegally).  But by watching the search-terms that led people to arrive at certain pages, I can identify particular songs that people were looking for and not finding anywhere else (the so called "long tail" of search keywords). If these songs are now in the public domain, I can make a dedicated page for them, and share what I know - in many cases after doing more research and pulling together information from a range of different sources.    Not knowing the keywords that people use to get to the book-listing pages would totally destroy this approach.

What you can do about it

So far I've identified three alternative options for getting data about what my visitors are searching for.

Ask for user-provided information

I've used Google Docs to make a data-collection form, and invited my visitors to use it to tell me about songs they are looking for.

The advantage is that I can ask them for richer information than just the keywords, eg where / when they remembmer it from, multiple snatches of the lyrics, what style the music is, etc.

But the disadvantage - and it's a big one - is that it only works for people who actually get to my site and then go into the other page where this form is kept, and fill in the form. I don't want to go into details - but let's just say that I haven't been run off my feet!

Get data from WebMaster Central

If you have verified your blog in Google Webmaster Tools, then the Search traffic > Search Queries tab shows the queries that have caused your blog to show up in search results pages, as well as how many times this has happened and what position, on average, you had in these pages.

This is richer information than you get from Analytics or Blogger-Stats, which only tell you about people who actually visited your blog.

But the disadvantages are that data is only kept for 90 days, and it only shows the top 2000 keywords.   Both of these are issues for me - some of my song-book contents are seasonal - if something is being looked for now, then the moment (week, month) may have passed by the time that I've noticed the trend, researched the song and written it up to a standard that I'm happy to publish. So really I want to checking the logs for nine months ago, so I can research things that are likely to be popular again next year.

Get data from AdWords

Advertising campaigns are the one place where Google is passing the search-keywords through to back-end systems. And because of this, Adwords does have data about what your visitors are searching for - provided you've set it up to collect this data. To get it up:

Firstly, sign up for an AdWords account. You probably have to deposit $10 into the account to get started - but you don't actually need to set up any advertising campaigns or spend any money after that.

Then link your AdWords account to your Google Webmaster Central account.

Once this is done, Adwords will start collecting the search-keywords for your blog. To get at the data:

• Log in to AdWords
• Select "All Online Campaigns,"
• Make an empty campaign (if you haven't got one already)
• Go to the "Dimensions" tab
• Change "View" to "Paid & organic".

AdWords will display your stats, since you signed up and linked your account. This includes the top search terms that users got to your site with, number of clicks, number of queries and some other measures too.

I'm only just starting to assess how well this will for for my song-listing site - will update this post when I have more specific information about how well it works and whether I can get actionable results from it.

What other alternatives have you found?

Leave a comment below, and I'll expand this list as we find out more options for accessing keyword-based search traffic information.

---

Related Articles:

Using Google Docs to put a survey questionnaire into Blogger

Six reasons why SEO doesn't matter for your blog

Webmaster tools Structured Data Testing Tool - helping bloggers who care about SEO

This quick-tip introduces the Google Webmaster Tools structured data testing tool, which gives you a view of how your site looks to the search-engines.

Today I discovered that Google Webmaster Tools offers tools for testing the structured data on your website.

I haven't seen any announcements about it, just noticed it there when I was looking for something else - so I'm not sure if it's really new, just new-to-me, or I've been lucky enough to get a it before most people do.

You can find it here (or at least that's where I'm finding it):    http://www.google.com/webmasters/tools/richsnippets

Once you're at that page, you can paste in either an URL (your own, or someone else) or some HTML, press preview - and the system will show you how the meta-tags, open-graph tags and other Google-supported structured data on your site looks to Google.   This isn't important for many bloggers - but is very useful information if SEO matters for your blog.

And you can also "Select the HTML tab to view the retrieved HTML and experiment with adjusting it." - and so test out the effect of making changes to your template etc.

The results show you:

• A preview of how the URL looks in a Google search-engine-results-page.
• Authorship testing results - whether have a Google+ page or profile associated with the site
• Authorship email verifications results
• Publisher markup verification status
• An extract of the structured data

I don't even begin to understand what all the results mean.    And I'm not sure if we can do something about all results that are shown - eg   checking Blogger-hints-and-tips currently tells me that there are values for properties that I've never set (eg blogid and postid)  and also properties that aren't part of the schema, eg:

• Error: Page contains property "image_url" which is not part of the schema.
• Error: Page contains property "blogid" which is not part of the schema.
• Error: Page contains property "postid" which is not part of the schema.

But, much like the syntax-checker provided by Facebook for checking how successful you were at  installing Open Graph tags I'm sure that this will be a useful SEO diagnostic tool.

Tell Google more about the posts in your blog with Webmaster Central's Data Highlighter - now supporting more data types

This QuickTip is about the new types of "things" which the Data Highlighter tool knows about.

In December 2012, Google introduced the Data Highlighter (DH) - this is a tool that lets you point to items in your blog posts LINK and use them to teach Google how you write about certain things.

Now they have announced that this tool has been extended to cover schema.org definitions of seven new types of things:

• Products
• Local businesses
• Articles
• Software applications
• Movies
• Restaurants
• TV episodes

The basic instructions for using the DH tool have not changed: visit Webmaster Tools, select your site, click the "Optimization" link in the left sidebar, and click "Data Highlighter".

Google now advise that "The tagging process takes about 5 minutes for a single page, or about 15 minutes for a pattern of consistently formatted pages."

After you have done some tagging, you can verify Google's understanding of your structured data. If Google has understood your work correctly, and you "publish" it to Google, then, when your site is displayed in search results, Google will use enhanced displays of information "like prices, reviews, and ratings".

It will take time for this to happen: Google needs to re-index your blog-posts before it takes effect.

Note:  schema.org is a markup language that this tool is based on.    However not all item types are covered by the tool at the moment.   So, for example, we can only use the Data Highlighter tool to show Google how articles, products, TV-show episodes etc are represented on our blogs.    It doesn't go into more detail for things like blog-posts which schema.org does have definitions for.

Getting started with SEO, for Bloggers

This article introduces Google Webmaster central's first-steps-SEO cheatsheat, and explains how the points in it apply to Blogger.

Google have produced a "first steps cheat sheet" for people who've got a blog or website, "but never gave search much thought". They describe it as a "short how-to list with basic tips on search engine-friendly design".

It lists things you can do which may "help Google and others better understand the content and increase your site’s visibility".

You can find it here. (It's a one-page PDF file).

The Webmaster Central post where it was announced said to "read it, print it, share it, copy and distribute it" - so I'm going to tell you how the points listed apply to Blogger users.

One caveat:  As I've explained previously, SEO (aka search engine-friendly design) is totally irrelevant for some bloggers.  Unless you know that being found in search-engines is important for your blog, then  please don't waste any more time on this.

But if you do depend on Google (or Yahoo, Bing, etc) to bring visitors to your blog, read on, then these are the first things that Google suggest you should think about.

I've divided them into three sections:

1. Looking good in the search results
2. Helping Google to understand your pictures
3. Update and keep going.

Looking good in the search results

Blog address

Make sure that your blog's URL is "descriptive and easy-to-read".

The URL is the blog's website address:  you choose it when you first set the blog up, but you can change it again later if you need to, using  Settings > Basic > Publishing in the Blogger dashboard

Many people recommend using a custom domain because it look more professional and like a real website.   But the same "descriptive and easy to read" guideline matters even if you use a blogspot.com address.

An example search-engine results page - if your blog looks good here, then it will get more visitors.

Overall blog title and description

Your blog Title should "Describe your ... [blog very] concise[ly]."

Your blog's Description should "Describe your ... [blog] in a concise, informative phrase."

You can edit these under Settings > Basic > Basic in the Blogger dashboard.

I usually make the title match the blog's URL  (viz http://blogger-hints-and-tips.blogspot.com/ /  Blogger-hints-and-tips) - but it can be a little longer, maybe about five words.

Blogger lets you make the description up to 500 characters long, but I usually recommend less than this - a maximum of 160 is better.

Post title and descriptions

Each post and page needs a title that is concise and informative. 

You set the title in the Post-editor, when you are first writing a post.   And you can change then later by editing the post again after it has been published.

Some people suggest using a catchy phrase that people are likely to share on Facebook, Twitter etc for the first week after you publish a post, and then changing it to a more descriptive phrase (which looks better in search results) after that.

Post descriptions

Meta descriptions are page summaries which are often used by search engines to describe your blog on the search-result listings.

Turn on descriptions in Blogger by

• Going to Settings > Search preferences > Meta-tags in the blogger dashboard.    
• Choosing the choosing "yes" radio button
• Putting the overall summary for the whole blog into the text field.   (I recommend the same one from Settings > Basic > Basic) into the text field
• Clicking Save changes

Once Descriptions are turned on, there will be a Search Description entry in the right-hand Post-settings area each time you edit a post.  Write a short (160 characters or less) unique description for each post into this.

Help Google understand images

File names

Before you load a picture to your blog (or to a photo-hosting service like Picasa-web-albums or Google+ Photos), give it a short, descriptive file name.  

For example, I just made the screen shot that I used in the previous section, and called the file  "seo-basics-meta-description-field-blogger-post-editor.png"

Tagging

Google's sheet just says to Use an “alt” and "title" tag to describes the picture.   I've described this in detail previously - see Telling Google, and visually impaired people, about your pictures

You can also use the "add caption" feature (on the tool-bar when you hover over the picture in the post-editor) to add a short caption describing the picture.

Put information in text, not just pictures

Look again at the picture that I used in the previous section.   When I was making it, I put the 160-characters-long suggestion right inside the picture.   This is useful for readers who see more detail in the photos than the text.   But it's no good for search engines:   Google is clever, but it's still not clever enough to extract reliably the meaning from text you have photographed.

So I made sure that the 160-character advice was in the text, as well as in the picture.

Update and keep going

Lots of people start a blog, and then get discouraged because it's not successful straight away.

But this is a big mistake.  Domain age, ie how long you have had the website address for, is one factor that search-engines take account of. Even if you don't get many visitors in the first year, the fact that your blog has been going for a year and you are still posting to it makes it attractive to Google.

So Google's final piece of advice is to keep going, and to publish new posts on your blog on an on-going basis.  

A few final words from me

Google's advice is a good starting point.   The bottom of their cheat-sheet links to various other good sources too:

• Google SEO Getting Started guide   (a much longer and more detailed version of the cheatsheet)
• Webmaster Academy (a series of instructional videos)
• Webmaster Tools - an online utility that you can use to measure and diagnose SEO issues with your site.

These resources are all good - but they aren't targeted to Blogger users, so sometimes they recommend changes to things we cannot change (eg URL-structrue), or they simply explain things in non-Blogger ways.   So read them, but don't get to worried if you cannot put all their advice into practise.   SEO is one of those areas where even doing some of what is recommended can help a lot.

There are about a zillion websites offering SEO advice too - just google and you will find them.    Look for ones that are up-to-date (the SEO "rules of the game" change often), and have lots of positive comments.

Keyword:  You will see lots of SEO advice about keyword research.   Google do have a free keywords tool - but it looks like it's going away - so don't get hooked on using it.   Overall, my advice is to ignore this for a very long time in your blogging:  focus on writing content that your blog-visitors will want to read, describe it in interesting ways, and the keywords will look after themselves.    

---

Related Articles:

Telling Google, and visually impaired people, about your pictures

Using Blogger to make a real website

Editing a blog-post that you have already published

How to put a picture into a blog-post

Introduction to Picasa and Picasa-web-albums

Teach Google-search about your blog with the Data Validator

This quick-tip is about the Webmaster Tools Tool Data Validator tool - which is an easier way to "teach" Google about the way things are described on your blog or website.

Recently, Google announced a new feature in Webmaster Tools called the Data Validator.

This is like a simpler way for website-owners (which is what bloggers really are) to tell Blogger about structured information that they put onto their sites - by point-and-clicking at the information on screen, rather than by learning the intricacies of Rich Snippets and the snippet testing tool.

What they've released so far is just an English-language tool for events - "such as concerts, sporting events, exhibitions, shows, and festivals".

To use it you need to

visit Webmaster Tools, select your site, click the "Optimization" link in the left sidebar, and click "Data Highlighter".

And obviously you need to have verified the blog in Webmaster Tools before you start, and to do the "learning" with the same account.   NB   Some blogs will be automatically verified, but some won't:  if yours isn't, then the best way to do this is to choose Alternate Methods and choose HTML Tag - then add the meta-tag to your blog.

The bottom line is that for this to work, you need to put the information into your blog the same way every time (aka in a "consistent format"). What will be interesting to see is exactly how "consistent" it needs to be - I suspect that the tool will probably get smarter over time.

I haven't had a chance to try it yest,  /But I am using a Blogger to manage the website for a small choir which has a concert in January.   So I think I'll start by structuring the announcement of the last concert in June 2012, and then see how well it applies the learning to the next one.

Stop spammy links from hurting your blog

This article describes the Disavow tool, which you can use to ask Google to stop taking notice of links to your website that you consider to be bad.

Google's Webmaster tools now include a way of telling Google to ignore certain links when it is deciding what reputation your website or blog should have.

This can be important if SEO is important to you, ie if your blog relies on Search to get visitors, because your "reputation" helps Google to decide where to list your site in the search results. And there is plenty of evidence that the higher your listing, the more "clicks" you will get.

Sometimes you can get links to your blog removed by simply by asking the person who made the link in the first place. Maybe they made a mistake, maybe they've reconsidered their own linking strategy, etc.   To ask them, you need to find them.  Options for this include:

• Looking at their website to find contact details
• Finding them on Google+ or via their Blogger profile
• Tracing them on other tools like Facebook or LinkedIn. 

Worst case, your only option may be to leave a comment on their blog, and hope they're still looking at new comments.

But often enough, you won't be able to get "bad" links removed: the person who made them can't be contacted, or won't co-operate. Worst case, they may be a competitor who is trying to make Google think that your site is "bad" by doing "negative SEO", ie creating lots of spam-links to it.

Telling Google about bad links

The Disavow tool is WebMasterTools new approach to dealing with problems like this.

It lets you tell Google that you think they should ignore certain links. To do this, make a text file (using NotePad, etc).    The file should have

• One line for each link that you want to fix.
• The phrase "domain:" at the start of lines listing websites that you don't want any links from
• A hash (# - AKA a pound-sign in the USA) at the start of any lines containing comments (eg your own notes about what's happened) that you want the Disavow tool to ignore.

An example file might look like:

# Left comment on SammySpammy's blog on 4/5/2012. Asked him to
# remove links, but he said "No"
# Fred from BuildYourBlogNetwork.com removed most links, but missed these
http://www.BuildYourBlogNetwork.com/firstPageTheyMissed.html
http://www.BuildYourBlogNetwork.com/another-page-they-missed.html
http://www.BuildYourBlogNetwork.com/and_Another.html

Once you have made your file, go to the Webmaster Tools Disavow page, choose the correct blog from the list (if you have several), and then upload the file.

Note: you need to be verified in WebMaster tools as the owner of your bog to access it on that page.   Blogger-administrators are "supposed" to be automatically verified, but sometimes this has not happened - if this has happened to you, then your blog won't be in the drop-down list. If necessary, you may need to manually verify your ownership by adding a meta-tag to your blog. LINK (Webmaster tools will give you the tag to add.)

Also, notice the warning that they give:

This is an advanced feature and should only be used with caution. If used incorrectly, this feature can potentially harm your site's performance in Google's search results. We recommend that you only disavow backlinks if you believe you have a considerable number of spammy, artificial, or low-quality links pointing to your site, and if you are confident that the links are causing issues for you.

What they are saying is that if you ask them not to take notice of a link which isn't bad, they might well do so - and this could have quite a bad affect on your search results.   In short, if you are going to disclaim all knowledge and validity of a link, be absolutely certain that this is the right thing to do.

What does the Disavow-links action do?  And how fast does it happen?

Effectively, the disavow file lets you suggest that Google should ignore the links you have listed. If Google accepts your suggestions (there are no promises), they will be applied the next time that Google re-indexes the site(s) that you have suggested.

This can take weeks to happen - because if they really are spammy sites, then Google probably isn't that keen on them anyway! So you won't see an immediate effect. But over time, they should improve your standing in Google's eyes if the links really were causing a problem.

The FAQs in Google's announcement of this tool say that it's really aimed at people who know they have done silly things in their own link-building, and who want to fix the problems they have caused - and which Google have told them about. Specifically, they say "If you haven’t gotten [notification of a manual spam action based on “unnatural links” pointing to your site], this tool generally isn’t something you need to worry about".

But my guess is that this approach will be interesting to many blog owners who do worry about inwards links that they aren't happy to be associated with anyway. It won't remove the links from the internet totally (only the other site owner can do that), but it will stop Google from penalizing you because of them when it decides how to show search results.

What your readers see

Visitors to your blog see absolutely nothing different - using this tool has no immediate effect on your layout or content.

But if your suggestions have an impact on how Google ranks your site, they will hopefully see your site at an earlier position in their search-results pages in future.

---

Related Articles:

5 reasons why SEO doesn't matter for your blog

Adding a meta-tag to your blog

Taking action when someone has used your copyright material without permission

Get notified about changes in popularity of pages on your blog/site.

This Quick-Tip describes the alerts that Webmaster Tools are now offering for changes in search popularity of particular pages.

Google Webmaster Tools have added to their alerts features.

Basically, this means that they try to notice the changes "that could be most interesting for you", and send the website owner a message when they spot something.

You can view these messages in the Webmaster tools site, or by signing up inside Wesmaster Tools to have alerts forwarded to your email.

The newest(*) alert they added is for Search Queries.   ref:   http://googlewebmastercentral.blogspot.ie/2012/08/search-queries-alerts-in-webmaster-tools.html

This features shows "impressions and clicks for your top pages over time."

They say

"For most sites, these numbers follow regular patterns, so when sudden spikes or drops occur, it can make sense to look into what caused them. Some changes are due to differing demand for your content, other times they may be due to technical issues that need to be resolved, such as broken redirects. For example, a steady stream of clicks which suddenly drops to zero is probably worth investigating."

They're still working on what levels of change to report: it should be interesting to see how they cope with the wide variety of seasonal patterns.  For example, I have one site that has weekly content: each week's post builds interest in the preceeding three weeks, then has strong traffic until a certain day and then drops to pretty close to zero.   It will get popular again, but not until almost three years have passed - simply due to the niche that this blog is in.    Now, I don't want to be alerted to this, because it's pattern I know very well! But I wouldl be interested if they could tell me that the expected "strong growth in the last week" isn't happening.



(*)   I originally wrote this in late August, but somehow forgot to publish it.    Found it again just now, and decided to share it, even though it's not as fresh as it was then.    Hopfully it will still help someone.

Have all your posts been added to Google search results yet?

How can you tell if Google has indexed your blog - or if it's indexed all your posts.

The quick way is to search for site:YOUR-BLOG-NAME and see how many results are returned.   If it's roughly the same as your number of published posts, then you're about right.   But what if it's not?

For example, if I google

site:blogger-hints-and-tips.blogspot.com

I get

About 385 results (0.26 seconds)

However I've only 226 published posts in blogger-hints-and-tips.   That tells me there's a problem with content being indexed more than once.   Looking at the results, I can see the problem immediately:  my dynamic-views have been indexed (goodness only knows why) - but also, my archive pages appear to have been indexed.  

For a more in-depth analysis, though, Webmaster Tools now has an Index Status feature, which is particularly useful if there are less pages indexed than published posts, because it shows you if it's always been like that, or if there has been a change at some time which led to a lot of your posts being removed from Google's index.

To use it:

• Log in to Webmaster tools with the Google account that has administrator rights to the blog. 

• If you haven't verified the account before, you may need to do this.
  (Google have said before that blog-owners are automatically verified, but I've found a few times that this hasn't quite worked - I've had to manually verify by installing a meta-tag that they provide)

• Look under the Health tab for Index Status. It shows a graph of the number of pages indexed vs date for (up to) the last year.

Looking at the graph, if:

• the line is flat and you havn't made posts at that time, or
• you have been posting and the line is going upwards,

then it probably means that all your posts are being crawled and indexed (and so can be found in google searches).  

But if there's a different pattern, the advanced tab might be helpful.   It shows:

• the number of pages indexed
• the cumulative number of pages crawled,
• the number of pages that Google knows that it couldn't look at because they are blocked by robots.txt, and
• the number of pages that were not selected for inclusion in search results.

These numbers are point-in-time totals, not numbers of additions.

Note the important difference: the first graph shows pages indexed (ie added to Google). The advanced one shows pages crawled. If Google crawls a page and finds it hasn't changed since last time they looked at it, then they don't bother indexing it again.

If some of your posts aren't in Google anymore, and the graph of pages indexed has a sudden drop, then most likely a change that you made at that time has caused a problem with indexing or includign your results.

Of course just knowing when things happened doesn't tell you why they happened.  In my case, I remember adding the Archive gadget into Blogger-HAT's layout.   Obviously I didn't stop the pages from getting indexed (though it does make me bitter and twisted that Google doesn't just handle this for us!)

What this does point out, though, is that if you're serious about managing your site for maximum searchability (SEO-friendliness), you should keep a note of the date and time of any structural changes you make, so that you can compare this with diagnostic tools and see if there's a link between the changes and changes in your visits, indexing etc.

Test HTML rich-text post snippets before publishing them

Rich snippets. are a tool that Google uses to give website-managers some control over how their sites appear in search-results (while enhancing the quality of search results, of course).

Blogger users are website-managers - the process that we follow to add meta-tags to our blogs is similar to what other web-developers, who don't have simplified software like Blogger, have to do.

Google have added a HTLM rich-text testing facility to their Webmaster-tools: this lets you see if you've got rich-snippet code (eg XX) right before hitting publish - because of course the effect on Google is one thing that you cannot test in a private pre-publication blog.