You may not be aware of it, but there are a lot of ways you can control the Google Bot and the effect it has on your search listings. Two important methods of control you have is the crawl rate and parameter handling.
The crawl rate of the Google Bot can be changed once you have access to Webmaster Tools. If you have made lots of recent changes to pages, added new pages or simply find that a lot of your pages haven’t been indexed yet, you can tell Google Bot to crawl your site faster. Any changes you make to the crawl speed will be effective for 3 months from the time of change. This allows the Google Bot to get up to date with your site and then return to its normal speed.
Parameter Handling allows you to tell the Google Bot which variables in URL’s to ignore. This can be useful if you have variables that allow you to have a printing view, or one that simply changes the direction of post listings. The reason we would want to tell Google to ignore these parameters, is because of duplicate content. A page that has a variable for a print view will generate what looks like a new page to the search engine and in doing so creates duplicate content on the site.
For example, a normal page and print view:
www.domain.com/view.php?topic=1
www.domain.com/view.php?topic=1&printview=1
Or a forum entered by forum and topic number or directly by post number:
www.domain.com/viewtopic.php?forum=1&topic=1
www.domain.com/viewtopic.php?post=1
These two links could point to the same content but under a different URL. This is bad for search engine ranking and needs to be prevented. You have to decide which is more useful to the search engine, and block the others.
