???? ID: 217103 - ????? ???????: 02 ?????? 2010 - ??????: 2.0

???? Robots.txt ????? ?????

???? ?????? ???????????? ?????? ???????? KB ?? ???????? ????? ?? ??? ???? ????? ????
?? ???????? ?? ?????? ??? ?? ?? ???? ???? ???? ??.
?????? ??????This article applies to a different operating system than the one you are using. Article content that may not be relevant to you is disabled.
?? ?????? ??????? ???? ??? ?? ??? ???????????? ?? ??? Microsoft ??????? ?????????? ???????? (IIS) ??????? 7.0 ???????? ?? Microsoft Windows Server 2008 ??? ??? ??? IIS 7.0 ???????????? ??? ?????? ??????? ?????? ??? IIS ???????-?????? ?????? ?? ???? ??? ???? ??????? ?? ??? ????? Microsoft ??? ???? ?? ????:
HTTP://www.Microsoft.com/technet/Security/prodtech/IIS.mspx (http://www.microsoft.com/technet/security/prodtech/IIS.mspx)
IIS 7.0 ?? ???? ??? ???? ??????? ?? ??? ????? Microsoft ??? ???? ?? ????:
HTTP://www.IIS.NET/default.aspx?tabid=1 (http://www.iis.net/default.aspx?tabid=1)
??? ?? ??????? ???? | ??? ?? ??????? ????

??????

??? Spiders, Robots, ???? ????? ??? ???? ??, WWW ??? ????? ?? "?????" ??????? ?? ??????????? ??????? ??? ??? ????? ?? ???? ??? ??? ??????? ???? ???? ??? ?? ??????? ?????? ?? ??? ?????? ????? ??? ??? ???? ?? ??? ???????? ?? ??? ????? ?? ??????? WWW ???? ?? ?????? ?? "?? ????? shopping" ?????? ???? ?? ??????? ??????? ??????? ?? ????? ?? ??? ???????????? ?? ??? ???? ??? ??????? Robots prioritize ?? "scored" ?? ??? ?? ???? ???? ?? ?? ??? ?? ???? ??? ????????? ???? ?? ??? ??? ?????? ?? ??? ??, ??????? ?? ????????? ???

??? Robots.txt ????? ?? ????? ???? ??? ????? ?? ??? ??????????? ??? ????? ?? ?? ????? ??? ????? ??? ?? ????? ??? ?????? telling ???? ?????? ????? ?? ?????? ??, ??? Spiders ?? ??? ???????? ??? ???? ??? ???? ?? ????? ?? ??? ?? ??? Robots Robots.txt ??????? respect ???? ?? ??? ?????? ???? ??, ????? ???? well-written ??? Spiders ?? ????????? ?????? ?? ???? ?????

???? ???????

??? ????? ??????? ????? ?? ??? browses ???? ????, ?? "??????????-?????" ?? ??? ??? ???? ???? ?? ?? IIS ?? ??? ??? ??? ????? ??? ??????????, ?? ???? ??? ??????? crawls ???? ???? ??? ????? ?? ?????? ????? ?? ???? ???? ??:
  1. ????? ???? /robots.txt ????? ?? ??? ???? ??, ?? ????? ?? "?????????? ?????:" ?????? ?? ?? ???? ?? ??? ????? ??? ?? ??????? ???
    • ??? ?? ?????? ????????? ?? ??? ?????, ???? "?????????? ?????: WebRobot," ??? ??? ??? ???????? ?????? ?? ??? ??? pertain.
    • ??? ?? ????????? ???? ????? ?????, ?? ??? ??? ????? ?? ????, ??????? ??? ???? ?? ??? ???? "?????????? ?????: *," ?? ???? obeys ???
    • ??? ????? ????? ?? ??? ?? ????????? ??, ?? ?????? ??? ?? ?? ??????? ??? ?? ????? ??, ????? ????????? ?????? ?? ??????? ???? supersede ??????
  2. ??? ??????????-????? ?? ??? ???? ??? ??? ?? ??? ??? "Disallow:" ????? ????? ?? ????? ???? ?? ????? ???? ?? ????? ??? disallow ??? ????? ?? ?? ???? ?? ?????? ???? ??? ?? ?? ???? ?? ??? ???? ?? ??? ???? ??? ?????? ?? ???::
    • "????????: / ???????" /test/index.htm ?? ????? ? ???, ?? on ??? ???? ?? ??? ???? ??? ??????? ?? ???? ???? ???
    • "????????: /" ????? ????; ?? ????? ? ???? ?? ??? ???? ??? ??????? ?? ???? ???-??? ?? ?? desirable.
    • "????????:" ?? ??? ??????? ????? ???? ?? ????? ???? ?? ?????? ???? ???
  3. ????????? pound ?????? (#) ?? ??? ??????? denote ??????????, ?? ?????? ?? ???? ?? ?????? ?? ??? ???? ????? ????? ?????????

    • ?? ?????? disallows ???? ???? ?? ??? ??? ??? Spiders ??:
      # Make changes for all web spiders
User-agent: *
Disallow: /
    • ????? ?????? disallows ?? ??????? ?? "WebSpider" ?? ??? ?? ?? ????? "/ ?????" ?? "/ ??????":
      # Tell "WebSpider" where it can't go
User-agent: WebSpider
Disallow: /marketing
Disallow: /sales

# Allow all other robots to browse everywhere
User-agent: *
Disallow:
    • ?? ?????? ???? ???? ??? ??????? ???? ??? Spiders denying ???? ??? ??? ???? ??? "SpiderOne" ?? ??? ?? ?????? ???? ??:
      # Allow "SpiderOne" in the site
User-agent: SpiderOne
Disallow:

# Deny all other spiders
User-agent: *
Disallow: /
    • ?? ????? ?????? disallows ??? ???? ??? ???? ?? ??? FrontPage-??????? ?? ??:
      # Ignore FrontPage files
User-agent: *
Disallow: /_borders
Disallow: /_derived
Disallow: /_fpclass
Disallow: /_overlay
Disallow: /_private
Disallow: /_themes
Disallow: /_vti_bin
Disallow: /_vti_cnf
Disallow: /_vti_log
Disallow: /_vti_map
Disallow: /_vti_pvt
Disallow: /_vti_txt
Robots.txt ?????? ??? ??? ?? ???? ??????? ?? ??? ????? ??? ???? ?????:
HTTP://www.robotstxt.org/orig.HTML (http://www.robotstxt.org/orig.html)

HTTP://www.robotstxt.org/wc/FAQ.HTML (http://www.robotstxt.org/wc/faq.html)

HTTP://www.robotstxt.org/ (http://www.robotstxt.org/)
???? ???? ???? ??:
  • Microsoft Internet Information Services 6.0
  • Microsoft Internet Information Server 1.0
  • Microsoft Internet Information Server 2.0
  • Microsoft Internet Information Server 3.0
  • Microsoft Internet Information Services 5.0
  • Microsoft Internet Information Services 7.0
????? ?? ???? ????
??????: 
kbhowto kbmt KB217103 KbMthi
???? ?????? ???????????? ?????? ????????
??????????: ?? ???? ?? ???? ??????? ?? ????? ?? Microsoft ????-?????? ?????????? ?????? ?????? ???? ??? ??. Microsoft ???? ??? ????-???????? ?? ????-???????? ????? ?????? ?? ???? ???????? ???? ?? ???? ????? ????? ??? ?? ??? ?????? ?? ???? ???? ???? ??? ????? ??. ???????, ????-???????? ???? ????? ???? ???? ???? ???. ?????, ????????, ?????-???? ?? ??????? ?? ???????? ?? ???? ???, ???? ?? ??? ?????? ???? ???? ??? ????? ??? ?? ???? ??. Microsoft ??????? ??? ???? ?? ?????? ?? ??????????, ????????? ?? ??? ?????? ?? ???? ????? ?? ???? ???????? ?? ??? ???? ????? ?? ??? ????????? ???? ??. Microsoft ????-?????? ?????????? ?? ????? ?????? ?? ?? ??? ??.
?????????? ?? ??????? ????????? ??????? ??:217103  (http://support.microsoft.com/kb/217103/en-us/ )
????? ?? ???? ????