Search Dirs in robots.txt: Searches for robots.txt on the target host, and then searches the directories found in the file.