How to troubleshoot installation problems for Team Foundation Server 2010 SP1 and for later updates

This article describes how to troubleshoot installation problems for Team Foundation Server 2010 Service Pack 1 (SP1) and for later updates when the updates are installed on a Team Foundation Server application tier.

Initial troubleshooting steps

Note: You must be an administrator on the application tier machine and the SQL Server in order to install a patch.

Note: The patch will fail to install if IIS 6.0 compat is not installed. In this case, we will get a COMException with error code 0x80005000 (Unknown Error) when we try to stop the application pools. The error will show both in the configuration log and in the MSI log

Reapply the update

Iit is frequently easy to recover from errors that occur when you install updates: Just try rerunning the update. Before you rerun the update, stop all Team Foundation Server processes. If there is more than one application tier, the processes should be stopped on all application tiers. See the "Stopping Team Foundation Server processes" section for more information.

If the update reports that it was installed successfully and that the server is down or that some team project collections are not working, try running the TfsConfig updates /reapply command. By doing this, you may be able to recover from some common problems.

If neither of these methods resolves the issue, this troubleshooting guide can help you discover what caused the problem.

When the deployment has more than one application tier

If there is more than one application tier in a deployment, we recommend that any update installed on one of the application tiers is also installed on all other application tiers. In this manner, the service level of all application tiers will be the same and will match the service level of the configuration database.

If Service Pack 1 is installed on one of the application tiers in a deployment, the database is updated to Service Pack 1, and all other application tiers will stop working until they are also updated to Service Pack 1.

To identify mismatches in the service levels of application tiers, use the following methods:

• In a scenario in which an application tier does not work because it does not have Service Pack 1 installed, an error will be displayed in the Application log. The error details will include a “The requested schema property TFS_SCHEMA_VERSION did not match the expected value”.
• In a scenario in which other updates such as hotfixes are involved, the error is not generated. This behavior occurs because the application tiers can work even when their service levels differ from those of the database. To verify that the application tiers have the same updates that were applied to the database, check the “Application Tier” page on the Team Foundation Server Administration Console. The service level and the build number for the latest database update that is installed on the application tier will be shown under the “Application Tier Summary” section. The version of the database will be the same under “Data Tier Summary." The service level of all application tiers should match that of the data tier.

Types of failures

Team Foundation Server update installations may fail in two main ways:

• An update may fail while it is being installed and then report that it failed, displaying the message “Installation Did Not Succeed,” as in the following screen shot:
  
  Collapse this imageExpand this image

  
  
  
• An update reports that it was installed successfully, but the server does not work completely after the update is installed.

Case 1: The update reports “Installation Did Not Succeed”

To resolve this issue, follow these steps:

1. Try running the update again to recover from the problem.
2. If rerunning the update is unsuccessful, open the Windows Installer log and search for the "The configuration service host cannot stop because a running process is preventing it from stopping" message. See the "Finding the logs" section for information about how to find the Windows Installer log. If you find the message, it will point to some processes that are preventing the update from being installed. You must stop these processes, and then you can install the update again after waiting for at least 16 minutes.
3. If rerunning the update is unsuccessful, and you do not find the message that is mentioned in step 2 in the Windows Installer log, check the configuration logs to identify the problem. See the "Finding the logs" section for information about how to find the configuration logs.

Case 2: The update reports success, but the server does not work completely after the update is installed

Note If the server was not working completely before the update was installed, and the update is not intended to specifically fix the problem the server had, installing the update may not fix the problem.

Team Foundation Server is not reachable from clients after the update was installed

The Team Foundation Server application pools may have failed to start after the update was installed. You can try running the TfsServiceControl unquiesce command to start the application pools if they are stopped.

Team Foundation Server jobs do not run correctly after the update is installed

The Team Foundation Server Job Agent may have failed to start after the update was installed. You can find try running TfsServiceControl unquiesce to start the Job Agent.

If the Job Agent is already started, you can try restarting the Job Agent. To do this, click Start, click Run, start Services.msc, and then locate the Visual Studio Team Foundation Background Job Agent service. Right-click the service, and then click Restart.

The server is in the “Servicing” state after the update was installed

By running the TfsConfig diagnose /scope:updates command, you may be able to discover the location of the problem. After you have done this, you may be able to fix the problem by running TfsConfig updates /reapply.

If this does not work, check the configuration logs for information about the problem. See the "Finding the logs" section for information about how to find the configuration logs.

One of the collections is offline after the update was installed

You can expect the update to takes some time to apply to the collections after it is installed successfully on the application tier. If the collection is currently being updated, the state of the team project collection in the Team Foundation Administration Console will be displayed as “Offline (Servicing).” If the team project collection state is just “Offline,” the team project collection may have been offline before the process for applying the update was started. Or, the update installation may have failed on the collection.

If you expected the collection to be online but found it to be offline, you can try to start the collection by selecting it in the Team Foundation Administration Console and by then clicking Start Collection on the General tab at the bottom of the Team Project Collections page.

If the collection fails to start, the servicing job may have failed on that collection. You can verify the list of jobs that ran on the collection and the result of each job. To do this, select the team project collection on the Team Project Collections page in the Team Foundation Administration Console, and then click the Status tab. If there is a job of the “Servicing Collection” type and with a status “Failed,” you can double-click the job entry to see the log. The error in that log can typically be found at or near the end of the log. You can rerun the job by clicking Rerun Job.

Collections may be offline for other reasons. For example, if an attempt to create a collection originally failed, it will be displayed as “Offline.” Installing an update will not fix collections that are already in a failed state, as in this case.

The server is working, but there is a problem that did not show before the update was installed

Check the latest Team Foundation Server hotfixes for a fix. If no fix is available, you can also check the MSDN forums or contact customer support.

Failures that occur only when you uninstall an update

Uninstalling an update can fail in the same ways that installing an update can. Additionally, uninstalling an update does not work if one of the team project collections is not available. If one of the team project collection databases is not available, the database must become available before the update can be uninstalled. If one of the collections is in a bad state because it failed while it was being created, or because the collection is no longer required, the collection must be deleted before you uninstall the update. Deleting a team project collection is not reversible. Therefore you should do this only if none of the data in the team project collection will be needed at any time going forward.

Stopping Team Foundation Server processes

To maximize chances of an update installing successfully, we recommend that you stop all Team Foundation Server processes on the application tiers, and that you wait for at least 16 minutes after stopping the processes before you install the update.

If the Team Foundation Server deployment has multiple application tiers, the processes should be stopped on all application tiers before the update is installed.

The processes that must be stopped are the IIS application pools (the executable name for these processes is W3wp.exe), the TFS Job Agent service (TFSJobAgent.exe), and the Team Foundation Administration Console (TfsMgmt.exe).

To stop the application pools and the Team Foundation Server services, including the TFS Job Agent, you can run the TfsServiceControl quiesce command. To stop the Team Foundation Administration Console, close all instances of the Team Foundation Server Administration Console that are open in all user sessions on the computer.

Finding the logs

If there is a problem installing or uninstalling an update, the following types of logs may be relevant.

Configuration logs

To locate the configuration logs, open the Team Foundation Administration Console, and then clicking the Logs node. You can find the logs for the most recent operations by sorting the logs according to the creation date variable. Multiple operations are executed every time that an update is installed or uninstalled. Therefore, you may have to check more than one log file to find the error that caused the failure. You can quickly find the errors in these logs by searching on the following string:

Team project collection logs

These logs can be found by opening the Team Project Collections node in the Team Foundation Administration Console. You can check the list of jobs that ran on the team project collection and the result of each job by clicking the Status tab on the same page. If there is a job of the “Servicing Collection” type and with a status “Failed," this means that the update was not applied on the team project collection. You can double-click the entry for the job to see the log. The error in that log can typically be found at or near the end of the log.

Windows Installer Logs

In some cases, the Windows Installer logs for the patching operation can help you identify the problem. You can find these logs in the %TEMP% directory or in the directory immediately above that.

When you install an update, the log file name will be in the following format: For example, the log file name may resemble the following: When you uninstall an update, the log file name will be in the following format: For example, it may resemble the following: