The robots.txt is a very important file that is placed in your root directory, the example would be yourdomain.com/robots.txt. This Robots.txt file Generator Help the search engines and other robots which areas of your site they are allowed to visit and index. When search engine spiders bots start crawling a website, they start by identifying a robots.txt file at the root domain level. You can add only one robots.txt on your site and ONLY in the root directory (where your home page is): It is a very useful tool that has made the lives of many webmasters easier by helping them making their websites Googlebot friendly. It is a robot.txt file generator tool that can generate the required file by performing the difficult task within no time and for absolutely free. Our tool comes with a user-friendly interface that offers you the options to include or exclude the things in the robots.txt file.