Robots.txt Template for Community Server
I was getting annoyed by MSNBot trying to index /reader/default.aspx. So, I decided to deploy a Robots.txt file. In case anyone wants a template for a Community Server Robots.txt file, here it is. I don't know what the Downloads folder is used for (other than what I found at this thread), so I've included it anyway.
# Robots.txt file for http://domain
# User-agent: * Disallow: /aspnet_client/ Disallow: /ControlPanel/ Disallow: /Downloads/ Disallow: /Emoticons/ Disallow: /FreeTextBox3/ Disallow: /Ink/ Disallow: /Installer/ Disallow: /Languages/ Disallow: /Messages/ Disallow: /reader/ Disallow: /Themes/ Disallow: /User/ Disallow: /Utility/




