November 6, 2009 by Simon King
SEOs spend most of their working lives – and, let’s be honest, some of their private lives – trying to think like the Google Bot, in order to make their clients sites as welcoming and comfortable for it to visit and return to as possible. For the most part this is a pretty difficult thing to do, the Google Bot is, after all, a computer programme – not a living thing and, as such, is pretty hard to empathise with.
Until now, that is…
That’s right, the Google Labs section in webmaster tools has just played another blinder by introducing a tool that at last lets you see what the Bot actually sees when it crawls the internet, viewing and judging sites. This allows you to get an idea of how your optimisation is going without having to wait until your site is crawled again. Pretty useful, I think you’ll agree.
The experimental app is called ‘Fetch as Googlebot,’ and it works like so…
When you put your site through ‘Fetch’, it gives you a real-time representation of what the bot actually sees when crawling a site. It returns HTML code exactly as a Googlebot would encounter it.
The plan is to use this as a tool to allow webmasters to view their own sites after they have made major changes and it’s an invaluable resource as it allows you to get an invaluable insight into potential trouble spots, diagnose why you aren’t being indexed in the way you’d like to be etc. In fact, the list of things you could use this for is probably way too long to go into here, though I will let you in on what the Google blog itself has to say about the new tool…
“This feature will help users a great deal when they re-implement their site with a new technology stack, find out that some of their pages have been hacked, or want to understand why they’re not ranking for specific keywords.”
Well, there you have it that’s the latest Google Labs innovation for you, and it gets an a-star from me, literally the only limitation is that it only returns 100kb of data. Don’t worry, we think this is just how the app works, we have no reason to believe that the actual bot only views this much data. However, that said it is a good idea to keep the 100kb mark in mind when designing pages, purely because smaller pages load faster, and load times are always a factor in ranking.
Related posts:
The new function in google webmaster tools seems like a nice addon. I’ll try it in a couple of days
Comment by arnac — November 21, 2009 @ 1:28 am