User-Agent: MJ12bot Disallow: / User-agent: ia_archiver Disallow: / User-agent: BaiDuSpider Disallow: / User-agent: Baiduspider+ Disallow: / User-agent: noxtrumbot Disallow: / User-agent: LinkWalker Disallow: / User-agent: slurp Disallow: / User-agent: ilial/Nutch-0.9-dev Disallow: / User-agent: ZyBorg Disallow: / User-agent: ZyBorg/1.0 Disallow: / User-agent: TurnitinBot Disallow: / User-agent: Speedy Disallow: / User-agent: Shim-Crawler Disallow: / User-agent: ConveraCrawler/0.9d Disallow: / User-agent: Jyxobot/1 Disallow: / User-agent: MQBOT/Nutch-0.9-dev Disallow: / User-agent: Teoma Disallow: / User-agent: ConveraCrawler/0.9e Disallow: / User-agent: bot/1.0 Disallow: / User-agent: Sproose Disallow: / User-agent: complex_network_group Disallow: / User-agent: findfiles.org/0.91 Disallow: / User-agent: MQBOT Disallow: / User-agent: Nutch-0.9-dev Disallow: / User-agent: CazoodleBot/CazoodleBot-0.1 Disallow: / User-agent: CazoodleBot-0.1 Disallow: / User-agent: CazoodleBot Disallow: / User-agent: psbot Disallow: / User-agent: TwengaBot/1.1 Disallow: / User-agent: TwengaBot Disallow: / User-agent: Nutch Disallow: / User-agent: Snapbot/1.0 Disallow: / User-agent: Snapbot Disallow: / User-agent: WebAlta Crawler/2.0 Disallow: / User-agent: WebAlta Crawler Disallow: / User-agent: Yandex/1.01.001 Disallow: / User-agent: twiceler Disallow: / User-agent: Yanga WorldSearch Bot v1.1/beta Disallow: / User-agent: Bot v1.1/beta Disallow: / User-agent: WorldSearch Bot v1.1/beta Disallow: /