#=== antistatic.com robots exclusion file User-agent: * #-- Webpoison: Disallow: /email #-- Crawlers shouldn't be here: Disallow: /c-cgi Disallow: /cgi-bin Disallow: /scripts #-- Don't want my FTP tree indexed yet. Disallow: /pub #-- Danger: Infinite recursion ahead! Disallow: /edifices/changing/ #-- These shouldn't be searchable. Disallow: /database #-- Don't let anyone see old versions at archive.org User-agent: ia_archiver Disallow: / User-agent: Roverbot Disallow: / User-agent: Sqworm Disallow: /