You try to crawl an Internet website that supports NTLM and anonymous authentication. In this situation, the crawl cannot complete successfully in Microsoft SharePoint Foundation 2010 or in Microsoft SharePoint Server 2010. Additionally, the crawl may fail with an access denied error message.
To resolve this issue, apply the following update:
2276336 Description of the SharePoint Server 2010 hotfix package (osrchwfe-x-none.msp): August 31, 2010
2266423 Description of the SharePoint Foundation 2010 hotfix package (Sts-x-none.msp): August 31, 2010
Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.