Main menu:


Site search

Categories

Google Shows URLs Blocked by Robots.txt

Yes, Google will still show pages even if you have set them up to be blocked in your robots.txt file.  Why exactly, you’re wondering, I’m sure.

Google is actually not crawling any of the URLs you have listed as blocked in your robots.txt file.  What they are doing is displaying the page, just not showing your description meta tag or any information from inside that page.  Matt Cutts gets into more detail about that here:

So to prevent Google from displaying your pages at all, you need to remove the page inside your Webmaster Tools or else use a robots meta tag with a noindex value in it for each page.  Then your page will not show up in Google’s results at all.

You can get more details on this information from Quick Online Tips.

Related Posts

  1. Google “link:” Command is Overrated
  2. Google Does SEO?
  3. Google’s SEO Rules
  4. Google Hits Back Against Scammers
  5. Web Site Speed is New SEO Element

Write a comment





Spam protection by WP Captcha-Free