# contact sourcemaster@sourceware.org for questions. # see http://info.webcrawler.com/mak/projects/robots/norobots.html # for information about the file format. User-agent: Scooter Disallow: /ml/ User-agent: Test Crawler Disallow: / User-agent: DigExt Disallow: / User-agent: CW Crawler Disallow: / User-agent: Sosospider Disallow: / User-agent: * Disallow: setup.exe Disallow: setup-legacy.exe Disallow: /snapshots/ Disallow: /cgi-bin/ Disallow: /cgi2-bin/ Disallow: /cgi/ Disallow: /viewvc/ Disallow: /viewcvs/ Disallow: /bugzilla/ Disallow: /ml/overseers/ Crawl-Delay: 60