# Robots.txt file created by http://www.webtoolcentral.com # For domain: http://www.frayd.com # All robots will spider the domain User-agent: * Disallow: # but, Disallow the following directories Disallow: /css/ Disallow: /img/ Disallow: /img/logo/ Disallow: /media/ Disallow: /sb/ Disallow: /sb/br2003/ Disallow: /sb/030503/ Disallow: /sb/040228/ Disallow: /sb/040508/ Disallow: /sb/040709/ Disallow: /stats/ Disallow: /test/ Disallow: /test/css/ Disallow: /test/img/ Disallow: /promo/ Disallow: /demos/ Disallow: /demos/live/ Disallow: /demos/pilotpen/ Disallow: /demos/rehersal/ Disallow: /demos/studio/ Disallow: /tunes/ # Now you just copy the text above, create a new .txt file called robots.txt, # paste the above text in it and upload it to the root your server ! # That's all there is, be careful with editing, because a mistake can ban your # site to the search engines for a long time. # (gotta love this disclaimer) # Last but not least, we are never responsible for the results of our tool, # we do our best to make quality tools, but after all it's up to you to use # our generated robots.txt file # Change History # 08.06.04 - created using tool. # 08.06.04 - concatenated the disallows under one user-agent. # 03.13.04 - added /promo/ to the disallow list. # 06.26.07 - added /tunes/ to the disallow list. # << end of file >>