MediaBot vs Googlebot – what you should know

Posted on 21st April, 2006

Mediabot is the web crawler that Google uses to crawl those pages running Adsense with the purpose of analyzing the content so that Google can server contextually relevant advertisement

Matt Cutts confirms AdSense Mediabot results will be added into the natural search index

Matt Cutts confirms AdSense media bot in natural search index

At the Lunch sponsored by Google today Matt Cutts confirmed the recent rumors about media bot results getting into Big Daddy. Matt said it is a bandwidth saving feature to have GoogleBot and MediaBot both contributing to big daddy. Matt also stated that you will gain zero advantage in search listings however if you are serving different content to MediaBot then to Googlebot then you could be in trouble.

Later, there’s a concern that pages running Adsense will be given preferential treatment with more frequent indexing. A concern which was dismissed by Matt Cutts recently:

Matt Cutts confirms AdSense publishers not crawled more frequently

Eric, I talked about mediabot more today and even made a couple PowerPoint slides. I may post about this more when I get back from WMW, but: pages with AdSense will not be indexed more frequently. It’s literally just a crawl cache, so if e.g. our news crawl fetched a page and then Googlebot wanted the same page, we’d retrieve the page from the crawl cache. But there’s no boost at all in rankings if you’re in AdSense or Google News. You don’t get any more pages crawled either.

Related Posts
No related posts for this content

  • […] My experience from managing several of my forums with their content written mostly in Malay shows that section targeting start working after Googlebot has crawled my pages or Mediabot (crawler that Google uses to crawl those pages running Adsense) crawl results are added into Google natural search index. […]

  • >
    Scroll Up