Robots.txt

SearchRobots.txt: This is a small file that points and guides spiders as to what they can index and what they should keep away from.

Read On…