How to increase your crawl rate (Part two)
4. Submitting an XML sitemap will result in increased crawling by Google simply because it makes it easier for it to do so.
5. Monitor the server you use to make sure it doesn’t regularly experience outage or downtime. The spiders cant visit if the server is down.
6. Make sure the site is running well and that pages are quick to load. Make Google’s job as easy as you can, their ability to crawl sites is not infinite, those spiders are rushed off their feet.
7. Links are good in general, but each new link you get also provides another access point for Google. Get as many as you can especially from sites that are regularly crawled themselves… Kind of goes without saying this one, doesn’t it?
8. Uniqueness is good. Try to use one of a kind descriptions and Meta titles on your pages. Combined with interesting content that nobody else has.
9. Don’t have duplicate URLs. Google will probably only index one of them, it mightn’t be the one you want.
10. Be W3C compliant, and validate the site’s CSS and XHTML. Your code will be easier to maintain, and you’ll be compatible across more browsers which will make life easier for the spiders.
11. Social media sites have brand new content crawled every day, Content bookmarked on something like Delicious or StumbleUpon could well trigger a Googlebot visit.
Well, that should be enough to get you started and on your way to being inundated with Spider visits. Happy hunting.
Related posts:
