User-agent: HTTrack Disallow: / User-agent: Python-urllib Disallow: / User-agent: Web Downloader Disallow: / User-agent: WebCopier Disallow: / User-agent: WebReaper Disallow: / User-agent: WebSauger Disallow: / User-agent: WebStripper Disallow: / User-agent: Wget Disallow: / User-agent: libwww-perl Disallow: / User-agent: * Crawl-delay: 1.00 Disallow: /uploads # BEGIN XML-SITEMAP-PLUGIN Sitemap: http://www.feetmanseoul.com/sitemap.xml.gz # END XML-SITEMAP-PLUGIN