I think you need to have a bit of confidence in yourself rather than have to feel you need to check everything.
If you look at Brogan's earlier post, this should help you to write the correct paths for your robots.txt file based on wherever you have yours installed.
If the robots/txt is installed in the /community directory as opposed to the root, then why is /community/ included in the disallow. I thought that should be there if the file is i the root, but not the folder itself ???
i find this tutorial
1) Edit this file to change "/forum/" to the correct relative path from your base URL, for example if your forum was at "domain.com/sites/community", then you'd use "/sites/community/"
# 2) Rename the file to 'robots.txt' and move it to your web root (public_html, www, or htdocs)
# 3) Edit the file to remove this comment (anything above the dashed line, including the dashed line
#
# NOTES:
# Even though wild cards and pattern matching are not part of the robots.txt specification, many search bots understand and make use of them
#------------------------ REMOVE THIS LINE AND EVERYTHING ABOVE SO THAT User-agent: * IS THE FIRST LINE ------------------------------------------
Quick question ... is this information found in this thread still relevant for version 1.4.x of Xenforo? I am trying to find a way for search engines to stop searching member profiles. I have already set the permissions for the user group Unregistered / Unconfirmed to Not Set (No) ... do I need to do more than that to stop search engines from searching member profiles?