MediaBot vs Googlebot – what you should know
Posted on 21st April, 2006
Mediabot is the web crawler that Google uses to crawl those pages running Adsense with the purpose of analyzing the content so that Google can server contextually relevant advertisement
Matt Cutts confirms AdSense Mediabot results will be added into the natural search index
At the Lunch sponsored by Google today Matt Cutts confirmed the recent rumors about media bot results getting into Big Daddy. Matt said it is a bandwidth saving feature to have GoogleBot and MediaBot both contributing to big daddy. Matt also stated that you will gain zero advantage in search listings however if you are serving different content to MediaBot then to Googlebot then you could be in trouble.
Later, there’s a concern that pages running Adsense will be given preferential treatment with more frequent indexing. A concern which was dismissed by Matt Cutts recently:
Matt Cutts confirms AdSense publishers not crawled more frequently
Eric, I talked about mediabot more today and even made a couple PowerPoint slides. I may post about this more when I get back from WMW, but: pages with AdSense will not be indexed more frequently. It’s literally just a crawl cache, so if e.g. our news crawl fetched a page and then Googlebot wanted the same page, we’d retrieve the page from the crawl cache. But there’s no boost at all in rankings if you’re in AdSense or Google News. You don’t get any more pages crawled either.