The Robots META Tag for MSNBot is meant to provide users who cannot upload or control the /robots.txt file at their websites, with a last chance to keep their content out of MSN's indexes and services.
The "robots" tag is obeyed by many different web robots. If you'd like to specify indexing restrictions just for MSNBot, you may use "msnbot" in place of "robots".
<meta name="msnbot" content="robots-terms">
MSNBot obeys the
nofollow Robots META Tag. Placing these tags in the heading of your HTML document prevents MSNBot from indexing or following specific documents.
content="robots-terms" is a comma separated list used in the Robots META Tag for MSN that may contain one or more of the following keywords without regard to case: noindex and/or nofollow.
MSNBot will retrieve the document, but it will not index it.
MSNBot will not follow links to other documents.
If this Robots META Tag is missing, or if there is no content, or the robot terms are not specified, then the robot terms will be assumed to be
"index, follow" (e.g.
"all") which is the default indexing behavior for most major search engine spiders.
The tags to include and their effects are:
noindexwill produce the following effect; MSNBot will retrieve the document, but it will not index it.
<meta name="msnbot" content="noindex">
nofollowwill produce the following effect; MSNBot will not follow links to other documents.
<meta name="msnbot" content="nofollow">
You can also combine any or all of the above
robots-terms into a single META Robots Tag for MSN. For example:
<meta name="msnbot" content="noindex, nofollow">
Additional information on specific MSNBot Robots META Tags can be found at the MSNBot FAQs page.
MSNBot's default indexing behavior is to
index, follow or
all. The below Robots META Tag is not required nor is it suggested in the MSNBot guidelines which clearly state that the use of the Robots META Tag is for restricting the indexing of content.
<meta name="msnbot" content="index, follow">
When utilizing the above Robots META Tag, you are adding page weight (HTML bloat) that is not required. You shift the text to html ratio when inserting the additional code within your documents. Utilizing metadata elements like the example shown above may not present a professional image to both your peers and potential clients.