"Content index state" remains stuck in "crawling" in an Exchange Server 2010 database

Applies to: Exchange Server 2010 StandardExchange Server 2010 Enterprise


On a Microsoft Exchange Server 2010 database, the Content Index state is displayed as "crawling," and it never appears to reach a state of "healthy." Additionally, even though the Number of Documents Successfully Indexed performance counter shows a progressive increase in the number of indexed documents, the value in the Number of Mailboxes Left to Crawl counter does not decrease as expected.

In this situation, the following events are logged in the Application log:
ID: 102
Level: Error
Source: MSExchange Search Indexer
Machine: Mailboxserver.contoso.com
Message: Exchange Search Indexer has failed to crawl the mailbox (8832ee80-47b8-4b0e-a98f-c4a239794707) in database (Database1) due to error: (Microsoft.Exchange.Search.MailboxCrawlFailedException: Failed to logon to mailbox). This mailbox will be retried later.
ID: 5617
Level: Error
Source: CI Troubleshooter
Machine: Mailboxserver.contoso.com
Message: CI troubleshooter exchange search service restart attempt 81.


This issue occurs when System Center Operations Manager (SCOM) is running the Toubleshoot-CI.ps1 script. This causes the Exchange Search service to restart before it completes the indexing process.


To resolve this issue, you must prevent the SCOM server from running the Troubleshoot-CI.ps1 script. To do this, follow these steps:
  1. Open System Center Operations Manager.
  2. Click Management Pack Objects, and then click Monitor.
  3. In the Look for box, type troubleshoot, and then click Find Now.
  4. Locate the item that corresponds to Troubleshoot-CI.ps1 script, right-click it, and then then click Properties.
  5. On the Override tab, click Override.
  6. Select the Enabled check box, set the Override value option to False, and then click OK.

More Information

Typically, SCOM runs the Troubleshoot-CI.ps1 script every two hours. However, if the Troubleshoot-CI.ps1 script detects one of the following symptoms, it triggers a restart of the Exchange search service.:
  • Deadlock: Exchange Search deadlocks while waiting on threads from MSSearch.
  • Corruption: One or more search indices are corrupted.
  • Stall: Resembles a deadlock in that the indices are not updated. 
  • Backlog: The Search catalog is backlogged, and therefore scheduled index searches don't run as expected.
The following is an example of an event that's logged when the Troubleshoot-CI.ps1 script runs:
ID: 5611
Level: Error
Source: CI Troubleshooter
Machine: Mailboxserver.contoso.com
Message: Indexing backlog reached a critical limit of 48 hours or the number of items in the retry queue is greater than 10000 for one or more databases: Database1 (0, 12572, 0)Database1 (0, 31859, 0)
If the databases have large mailboxes or mailboxes that have a high item count, it generally takes longer than two hours to complete the indexing process. And when the Exchange search service restarts, the mailboxes are put back into the list for re-indexing. In this situation, the crawling process may never get the chance to finish.