User-agent: * Disallow: /redirect.php Disallow: /.pdf Disallow: /.doc Disallow: /?