# robots.txt User-Agent: * Disallow: User-Agent: Mediapartners-Google Disallow: /images/ User-Agent: grapeshot Disallow: /images/ User-agent: Libwww-perl Disallow: / User-agent: GPTBot Disallow: / #Allow: WebSurge