# www.robotstxt.org/ # www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449 User-agent: MJ12bot Disallow: /ar/ User-agent: * Disallow: /preview/ Disallow: /redirect/ Disallow: /redirect/* Disallow: /mailtofriend/ Disallow: /ads/ftp* Disallow: /fr/profile/ Disallow: /ar/profile/ Disallow: /en/profile/ Disallow: /fr/recherche/?Search Disallow: /en/search/?Search Disallow: /ar/search/?Search User-agent: bitlybot Disallow: /