# Adult Spider User-agent: EroCrawler Disallow: // # eMail Collector User-agent: ExtractorPro Disallow: / # faked user agent (for email harvesting, etc.) User-agent: Mozilla/3.0 (compatible) Disallow: / # often used as email collector User-agent: Mozilla/3.0 (compatible; Indy Library) Disallow: / # often used as email collector User-agent: Mozilla/4.0 (compatible; Advanced Email Extractor v2.xx) Disallow: / # often used as email collector User-agent: Mozilla/4.0 (compatible; Advanced Email Extractor v2.xx) Disallow: / #SingingFish Media Spider. I'm sick of the 404's from this media spider User-agent: asterias Disallow: / User-agent: * Disallow: /library Disallow: /slide_shows User-agent: Googlebot Disallow: /slide_shows Disallow: /library User-agent: Slurp Disallow: /slide_shows Disallow: /library User-agent: YahooSeeker/M1A1-R2D2 Disallow: /slide_shows Disallow: /library User-agent: Teoma Disallow: /slide_shows Disallow: /library User-agent: msnbot Disallow: /slide_shows Disallow: /library User-agent: Googlebot-Image Disallow: /slide_shows Disallow: /library User-agent: Yahoo-MMCrawler Disallow: /slide_shows Disallow: /library User-agent: msnbot-media Disallow: /slide_shows Disallow: /library User-agent: psbot Disallow: /slide_shows Disallow: /library User-agent: Mediapartners-Google Disallow: /library Disallow: /slide_shows User-agent: YahooYSMcm Disallow: /library Disallow: /slide_shows User-agent: MSNPTC Disallow: /library Disallow: /slide_shows User-agent: ia_archiver Disallow: /library Disallow: /slide_shows