New Topic
Post Reply
Follow Topic
Basic purpose of robot.txt is to stop the search engine bots or crawlers to visit certain pages or areas of your website. In other words you can instruct the SE bots how to crawl your website.
You need to create a separate robots.txt file for each subdomain you want to control and put the above code in that. Be sure it should not be accessed by the main domain or your main site will be blocked. Its not that easy but you need some programing skill to get this effectively.
Did you put the robots.txt file in your subdomain folder, or your domain root folder? Each subdomain should have it's own robots.txt file.
Post Reply
