# this file is scanned by robots (case insensitiv) # explicitely specified user agents just follow the instructions of their section # non explicitely specified UA's follow the UA * (if a section * is defined). Default: everything allowed # permission may be "allow" or "disallow" # examples of record paths: "/" means everything, "" means nothing # "/serv" means every file or dir starting with serv, "/~serv" means every file or dir with serv in between # User-agent: scooter.ethz.ch Disallow: / User-agent: scooter Disallow: / User-agent: 129.132.200.5 Disallow: / User-agent: * Disallow: /www/search/ # OK, aber zu streng: User-agent: * # OK, aber zu streng: Disallow: /