Remove URL's From GoogleBot's Crawl List Using Google Webmaster Tools

Wednesday, October 3rd, 2007

I recently had to remove some development pages from Google‘s search results pages and I used Google Webmaster Tools to do so. Google Webmaster Tools doesn’t automatically stop GoogleBot from crawling your site, it does however provide options for alerting Google that you no longer want to have a page indexed for whatever reason.

1: Choose your website.
Once you logged into Google Webmaster Tools using your Google account info you’ll be asked to choose a website you have associated with your Google account.

2: Website Dashboard
Here you get a quick rundown of crawl errors Google ran into. From this page select the tools tab on the left hand bottom side.

3: URL removal tool (Requests Google Stops Crawling Pages You Assign)
Here’s where you’ll choose which page or pages you want GoogleBot to pass by on it’s next crawl.

4: Removal Tool Options
Choose an individual URL, a directory, a cached page or your entire site here.

5: Removal Request Page
Here’s where you leave the specifics. Is this a dead page or a dead image link?

6: Not What You Needed? Try Altering Your Robot.txt
Are your results still showing up? You may need to alter your robot.txt file. You can review your robot.txt from Google Webmaster Tools, but you can not alter it here. This way you can tell which pages/ directories are being left out. Get more information about Robots.txt files here.

Rating: 0.0/10 (0 votes cast)

Rating: 0 (from 0 votes)

Incoming search terms for the article:

About Me: Nate Balcom is a web designer and social media guru with over 11 years of experience in the web industry. He specializes in web design and development, SEO and social networking. You can follow him on Twitter here.