Archive | Search engines RSS feed for this section
Prevent search engines from crawling and indexing your website

Prevent search engines from crawling and indexing your website

by on 29. Sep, 2008 in Html, Search engines, Web

All search engines have automatic web crawlers that are constantly browsing the internet in order to keep their search results up to date.  Unless you specifically configure your website to disallow them, they will eventually find you and list you in their search results. To disallow these crawlers(robots) from crawling your website, just create a [...]

Read full story • Comments { 0 }