Even if you’re arachnophobic, there’s one kind of spider you want on your website. Search bots. These ‘spiders’ crawl through your pages looking for what to show search engine users and save a cached image of what they find.
http://www.smartinsights.com/search-engine-optimisation-seo/seo-content-strategy/6-reasons-google-bots-not-crawling-blog-post/
6 Reasons why Google Bots are not crawling your blog post
1980