User-agent: Cerberian User-agent: ConveraCrawler User-agent: HTTrack User-agent: Xaldon WebSpider Disallow: / User-agent: * Disallow: /locations/ Disallow: /img/ Disallow: /misc/ Disallow: /labelfiles/