Sites Robot.txt and Favicon Support

Improve Search Engine Marketing of Your Force.com Sites
  • Use robot.txt to tell search engines how to index your site
  • Make your site look professional by uploading your personal icon to your site

With Spring ’09, you can easily assign a robot.txt and favorite icon files for your Force.com sites.

Your Web site’s robots.txt file is used to tell search engine spiders which Web pages of your site to crawl and index as well as which Web pages to ignore. Use the robots.txt file in Force.com sites to prevent search engines from indexing directories that contain duplicate content, files not relevant for search engines (such as images, download links, and admin files), or sensitive information such as customer info, emails, and phone numbers.

Note: Not automatically visible. Feature is enabled, but requires some setup.
(Available in Developer edition)



Additional Resources
: