Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

Free Robots.txt Generator

Robots.Txt is a document that may be located within the root folder of your hosting website, to assist SERPS index your site very quickly as it should be. Serps which include Google, Bing use website crawlers, or robots that evaluation all of the content material on your website. There can be elements of your internet site that you do no longer need to crawl to encompass in google search results, such as admin page. You could upload these pages to the document to be explicitly not included. Robots.Txt documents use something known as the Robots Exclusion Protocol. This website will easily generate the report for you with inputs of pages to be excluded