External content is not returned in the search results in SharePoint Portal Server 2003

In certain situations, Microsoft Office SharePoint Portal Server 2003 does not include external content in the content index. This issue occurs when the following conditions are true:

• The external content is retrieved from a complex link.
• The link points to a custom Web part in a different portal area.

In this scenario, you experience the following symptoms. When you click the link, the custom Web part retrieves external content, and the custom Web part displays that content in a Web page. However, if you try to search for that external content on the portal site, the content is not returned in the search results.

This issue occurs if the following are located in different portal areas:

• The Web page that contains the link
• The destination of the link

SharePoint Portal Server 2003 does not crawl external content that is retrieved from complex links. For example, you may experience this issue in the following scenario:

1. You configure custom Web parts in your SharePoint Portal Server 2003 portal site to retrieve data that is stored in Microsoft Content Management Server.
2. You use a query string parameter to pass the data that is retrieved from Content Management Server to certain Web pages in the portal areas.
3. A Web page in one area of the portal site contains a link to a Web page in a different area of the portal site. The destination of the link is the Web page that contains the custom Web part that retrieves data from Content Management Server.

In this scenario, when you view the gatherer log, the Web pages that contain the custom Web parts are crawled. However, the query string parameters are not crawled.

This problem was first fixed in a hotfix that is now contained in a service pack. If you installed the latest SharePoint Portal Server 2003 service pack, you do not have to install the hotfix.

Service pack information

This problem is corrected in Microsoft Office SharePoint Portal Server 2003 Service Pack 2.

To resolve this problem, obtain the latest service pack for SharePoint Portal Server 2003. For more information, click the following article number to view the article in the Microsoft Knowledge Base: After you install the service pack, follow the steps that are listed in the "More Information" section to set the FollowPortalLinks registry key and to enable the hotfix.

Hotfix information

How to obtain the hotfix

This issue is fixed in the SharePoint Portal Server 2003 post-Service Pack 1 Hotfix Package Hotfix Package September 30, 2004. For additional information, click the following article number to view the article in the Microsoft Knowledge Base:

Add the FollowPortalLinks registry entry, and then create a new site rule after you install the hotfix

After you install the hotfix, follow the steps in the "How to enable the hotfix" section.

The steps show you how to add the FollowPortalLinks registry entry to the following registry subkey and how to set the registry entry to a non-zero value: SharePoint Portal Server 2003 crawls complex links if the FollowPortalLinks registry entry is set to a non-zero value. If the FollowPortalLinks registry entry is set to 0 (zero) or if the FollowPortalLinks registry entry does not exist, SharePoint Portal Server 2003 does not crawl complex links.

Each portal site has its own GUID in the HKEY_LOCAL_MACHINE\Software\Microsoft\SPSSearch\CatalogNames registry subkey. The CatalogName is typically "Portal_Content" (without the quotation marks).

You must also create a new site rule and configure the new site rule to use the Include complex URLs option.

How to enable the hotfix

Warning If you use Registry Editor incorrectly, you may cause serious problems that may require you to reinstall your operating system. Microsoft cannot guarantee that you can solve problems that result from using Registry Editor incorrectly. Use Registry Editor at your own risk.

1. Stop the Microsoft SharePointPS Search service.
   1. Click Start, point to All Programs, point to Administrative Tools, and then click Services.
   2. In the list of services, right-click Microsoft SharePointPS Search, and then click Stop.
2. Add the FollowPortalLinks registry entry.
   1. Click Start, click Run, type regedit in the Open box, and then click OK.
   2. Locate and then click the following registry subkey:
      HKEY_LOCAL_MACHINE\Software\Microsoft\SPSSearch\CatalogNames\Site GUID\CatalogName
   3. On the Edit menu, point to New, and then click DWORD Value.
   4. Type FollowPortalLinks, and then press ENTER.
   5. Right-click FollowPortalLinks, and then click Modify.
   6. In the Edit DWORD Value dialog box, click Hexadecimal (if it is not already selected), type 1 in the Value data box, and then click OK.
   7. Quit Registry Editor.
3. Start the Microsoft SharePointPS Search service.
   1. Click Start, point to All Programs, point to Administrative Tools, and then click Services.
   2. In the list of services, right-click Microsoft SharePointPS Search, and then click Start.
4. Create a new site rule to include the appropriate site path for the complex link, and then configure the site rule to use the Follow complex links option. Move the new rule so that it is the first rule in the list of site rules.
   
   Note Make sure that existing site rules do not use the Follow complex links option.
5. Change the existing site rule that uses the http://ServerName/*.aspx path to use the Include all items in this path option.
   
6. Update the content index.

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

This problem was first corrected in SharePoint Portal Server 2003 Service Pack 2.

For more information about how to create site rules in SharePoint Portal Server 2003, see the "Managing Search Settings for the Portal" topic in the "Managing Search Settings" section of the "Administration" chapter of the Microsoft Office SharePoint Portal Server 2003 Administrator's Guide. The Microsoft Office SharePoint Portal Server 2003 Administrator's Guide (Administrator's Help.chm) is located in the Docs folder in the root folder of the SharePoint Portal Server 2003 CD.