???? ID: 217103 - ????? ???????: 02 ?????? 2010 - ??????: 2.0

???? Robots.txt ????? ?????

?????? ??????This article applies to a different operating system than the one you are using. Article content that may not be relevant to you is disabled.
?? ?????? ??????? ???? ??? ?? ??? ???????????? ?? ??? Microsoft ??????? ?????????? ???????? (IIS) ??????? 7.0 ???????? ?? Microsoft Windows Server 2008 ??? ??? ??? IIS 7.0 ???????????? ??? ?????? ??????? ?????? ??? IIS ???????-?????? ?????? ?? ???? ??? ???? ??????? ?? ??? ????? Microsoft ??? ???? ?? ????:
HTTP://www.Microsoft.com/technet/Security/prodtech/IIS.mspx (http://www.microsoft.com/technet/security/prodtech/IIS.mspx)
IIS 7.0 ?? ???? ??? ???? ??????? ?? ??? ????? Microsoft ??? ???? ?? ????:
HTTP://www.IIS.NET/default.aspx?tabid=1 (http://www.iis.net/default.aspx?tabid=1)
??? ?? ??????? ???? | ??? ?? ??????? ????

??????

??? Spiders, Robots, ???? ????? ??? ???? ??, WWW ??? ????? ?? "?????" ??????? ?? ??????????? ??????? ??? ??? ????? ?? ???? ??? ??? ??????? ???? ???? ??? ?? ??????? ?????? ?? ??? ?????? ????? ??? ??? ???? ?? ??? ???????? ?? ??? ????? ?? ??????? WWW ???? ?? ?????? ?? "?? ????? shopping" ?????? ???? ?? ??????? ??????? ??????? ?? ????? ?? ??? ???????????? ?? ??? ???? ??? ??????? Robots prioritize ?? "scored" ?? ??? ?? ???? ???? ?? ?? ??? ?? ???? ??? ????????? ???? ?? ??? ??? ?????? ?? ??? ??, ??????? ?? ????????? ???

??? Robots.txt ????? ?? ????? ???? ??? ????? ?? ??? ??????????? ??? ????? ?? ?? ????? ??? ????? ??? ?? ????? ??? ?????? telling ???? ?????? ????? ?? ?????? ??, ??? Spiders ?? ??? ???????? ??? ???? ??? ???? ?? ????? ?? ??? ?? ??? Robots Robots.txt ??????? respect ???? ?? ??? ?????? ???? ??, ????? ???? well-written ??? Spiders ?? ????????? ?????? ?? ???? ?????

???? ???????

??? ????? ??????? ????? ?? ??? browses ???? ????, ?? "??????????-?????" ?? ??? ??? ???? ???? ?? ?? IIS ?? ??? ??? ??? ????? ??? ??????????, ?? ???? ??? ??????? crawls ???? ???? ??? ????? ?? ?????? ????? ?? ???? ???? ??:
  1. ????? ???? /robots.txt ????? ?? ??? ???? ??, ?? ????? ?? "?????????? ?????:" ?????? ?? ?? ???? ?? ??? ????? ??? ?? ??????? ???
    • ??? ?? ?????? ????????? ?? ??? ?????, ???? "?????????? ?????: WebRobot," ??? ??? ??? ???????? ?????? ?? ??? ??? pertain.
    • ??? ?? ????????? ???? ????? ?????, ?? ??? ??? ????? ?? ????, ??????? ??? ???? ?? ??? ???? "?????????? ?????: *," ?? ???? obeys ???
    • ??? ????? ????? ?? ??? ?? ????????? ??, ?? ?????? ??? ?? ?? ??????? ??? ?? ????? ??, ????? ????????? ?????? ?? ??????? ???? supersede ??????
  2. ??? ??????????-????? ?? ??? ???? ??? ??? ?? ??? ??? "Disallow:" ????? ????? ?? ????? ???? ?? ????? ???? ?? ????? ??? disallow ??? ????? ?? ?? ???? ?? ?????? ???? ??? ?? ?? ???? ?? ??? ???? ?? ??? ???? ??? ?????? ?? ???::
    • "????????: / ???????" /test/index.htm ?? ????? ? ???, ?? on ??? ???? ?? ??? ???? ??? ??????? ?? ???? ???? ???
    • "????????: /" ????? ????; ?? ????? ? ???? ?? ??? ???? ??? ??????? ?? ???? ???-??? ?? ?? desirable.
    • "????????:" ?? ??? ??????? ????? ???? ?? ????? ???? ?? ?????? ???? ???
  3. ????????? pound ?????? (#) ?? ??? ??????? denote ??????????, ?? ?????? ?? ???? ?? ?????? ?? ??? ???? ????? ????? ?????????

    • ?? ?????? disallows ???? ???? ?? ??? ??? ??? Spiders ??:
      # Make changes for all web spiders
      User-agent: *
      Disallow: / 
      						
    • ????? ?????? disallows ?? ??????? ?? "WebSpider" ?? ??? ?? ?? ????? "/ ?????" ?? "/ ??????":
      # Tell "WebSpider" where it can't go
      User-agent: WebSpider
      Disallow: /marketing
      Disallow: /sales
      
      # Allow all other robots to browse everywhere
      User-agent: *
      Disallow:
      						
    • ?? ?????? ???? ???? ??? ??????? ???? ??? Spiders denying ???? ??? ??? ???? ??? "SpiderOne" ?? ??? ?? ?????? ???? ??:
      # Allow "SpiderOne" in the site
      User-agent: SpiderOne
      Disallow:
      
      # Deny all other spiders
      User-agent: *
      Disallow: / 
      						
    • ?? ????? ?????? disallows ??? ???? ??? ???? ?? ??? FrontPage-??????? ?? ??:
      # Ignore FrontPage files
      User-agent: *
      Disallow: /_borders
      Disallow: /_derived
      Disallow: /_fpclass
      Disallow: /_overlay
      Disallow: /_private
      Disallow: /_themes
      Disallow: /_vti_bin
      Disallow: /_vti_cnf
      Disallow: /_vti_log
      Disallow: /_vti_map
      Disallow: /_vti_pvt
      Disallow: /_vti_txt
      						
Robots.txt ?????? ??? ??? ?? ???? ??????? ?? ??? ????? ??? ???? ?????:
HTTP://www.robotstxt.org/orig.HTML (http://www.robotstxt.org/orig.html)

HTTP://www.robotstxt.org/wc/FAQ.HTML (http://www.robotstxt.org/wc/faq.html)

HTTP://www.robotstxt.org/ (http://www.robotstxt.org/)

???? ???? ???? ??:
  • Microsoft Internet Information Services 6.0
  • Microsoft Internet Information Server 1.0
  • Microsoft Internet Information Server 2.0
  • Microsoft Internet Information Server 3.0
  • Microsoft Internet Information Services 5.0
  • Microsoft Internet Information Services 7.0
??????: 
kbhowto kbmt KB217103 KbMthi
???? ?????? ???????????? ?????? ????????
??????????: ?? ???? ?? ???? ??????? ?? ????? ?? Microsoft ????-?????? ?????????? ?????? ?????? ???? ??? ??. Microsoft ???? ??? ????-???????? ?? ????-???????? ????? ?????? ?? ???? ???????? ???? ?? ???? ????? ????? ??? ?? ??? ?????? ?? ???? ???? ???? ??? ????? ??. ???????, ????-???????? ???? ????? ???? ???? ???? ???. ?????, ????????, ?????-???? ?? ??????? ?? ???????? ?? ???? ???, ???? ?? ??? ?????? ???? ???? ??? ????? ??? ?? ???? ??. Microsoft ??????? ??? ???? ?? ?????? ?? ??????????, ????????? ?? ??? ?????? ?? ???? ????? ?? ???? ???????? ?? ??? ???? ????? ?? ??? ????????? ???? ??. Microsoft ????-?????? ?????????? ?? ????? ?????? ?? ?? ??? ??.
?????????? ?? ??????? ????????? ??????? ??:217103  (http://support.microsoft.com/kb/217103/en-us/ )