User-agent: * Disallow: /arch/ Disallow: /cgi-local/ Disallow: /docs/ Disallow: /logs/ Disallow: /online/ Disallow: /redirects/ Disallow: /sites/