User-agent: * Disallow: /toto/icons/ # HotBot-robot get's confused when multiple # directories on your site have the same name. # Result: a lot of errors in the logs # 31-01-2004 User-agent: slurp Disallow: / # Covera-robot fucks up with upper/lower case. # Result: a lot of errors in the logs # 03-08-2005 User-agent: ConveraCrawler Disallow: / User-agent: ConveraMultiMediaCrawler Disallow: / # Tries to activate scripts # 25-01-2009 User-agent: panscient.com Disallow: /