# Robots Exclusion Standard # Robots.txt # # All robots will spider the domain except /magnus/protected/ User-agent: * Disallow: /magnus/protected/