25 February 2006 16:30
somesound.org
robots.txt is a PHP script that acts like a normal robots.txt file, but with a few differences. When a Spider attempts to access robots.txt, the script will "disallow" access to a list of pre-defined directories. When a normal user attempts to access robo
1
(1 marks)