This article was previously published under Q170555
This article has been archived. It is offered "as is" and will no longer be updated.
For a Microsoft version of this article, see 194310.
Web Spiders or Robots are a great resource for people searching theInternet, but they present a problem to Web page designers who wanttheir pages to be seen and properly indexed. One popular solution to thisbehavior is to use the Hypertext Markup Language (HTML) META element.
The META element is used within the HEAD element to embed document meta-information that is not defined by other HEAD elements. This embeddedinformation can be extracted by servers and clients to identify, index, andcatalog specialized document meta-information. A META element can bewritten in one of two forms: META NAME and META HTTP-EQUIV. The NAMEattribute is returned in the HEAD of the document, while the HTTP-EQUIVattribute is converted into the HTTP response header which is parsed by theWeb server.
If you plan to use a Robot, you should use the META NAME form as shown inthese examples: