FIX: Error message when you perform a rolling upgrade in a SQL Server 2008 cluster : "18401, Login failed for user SQLTEST\AgentService. Reason: Server is in script upgrade mode. Only administrator can connect at this time.[SQLState 42000]"

An error occurs when you perform a rolling upgrade to one of the following Microsoft SQL Server 2008 versions in a cluster environment:

• SQL Server 2008 Cumulative Update (CU) 7, build number from 10.00.1818 to 10.00.1822.
  973601

  (http://support.microsoft.com/kb/973601/ )

  Cumulative update package 7 for SQL Server 2008
• SQL Server 2008 Service Pack 1 (SP1) CU3 or CU4, build number from 10.00.2723 to 10.00.2739.
  971491

  (http://support.microsoft.com/kb/971491/ )

  Cumulative update package 3 for SQL Server 2008 Service Pack 1
  973602

  (http://support.microsoft.com/kb/973602/ )

  Cumulative update package 4 for SQL Server 2008 Service Pack 1

For example, consider the following scenario.

Note This example assumes that you are working with a 2 node SQL Server 2008 SP1 cluster. Node 1 is the active node (where the SQL Server cluster resource group is online) and Node 2 is the passive node.

1. The installation of SQL Server 2008 SP1 CU3 completes the upgrade on the passive node (Node 2).
2. An administrator initiates manual failover from active node (Node 1) to passive node (Node 2).
3. When SQL Server is started by the cluster resource manager on the currently upgraded node (Node2), SQL Server script upgrade process kicks off and it starts upgrading the msdb database to the SP1 CU3 version.
4. When the msdb database is being upgraded, SQL Server is started and is running. Meanwhile, SQL Server's cluster resource DLL reports healthy state to the cluster resource manager.
5. Then, cluster resource manager tries to bring SQL Server agent cluster resource online.
6. When SQLAgent.exe starts up, it tries to connect to SQL Server and runs Transact-SQL query to call the EnableAgentXPs function. This T-SQL call fails with the following error message:
   Error 18401, Login failed for user SQLTEST\AgentService. Reason: Server is in script upgrade mode. Only administrator can connect at this time.[SQLState 42000]
   
   
   Because of this failure, SQLAgent.exe terminates its startup process.
7. Cluster manager detects agent startup failure and fails over the SQL Server group to another node (Node1). During the failover, SQL Server is stopped and the msdb upgrade process is rolled back.
8. SQL Server and agent starts on the other node (Node 1) that is running SQL Server 2008 SP1.

In this scenario, if an administrator tries to fail over the SQL Server cluster group to Node 2 again, steps from (b) to (h) will repeat again and again.

Note In a multi-node scenario, when most of the nodes are upgraded, the setup program will fail over the active instance to one of the upgraded nodes. This causes the SQL resources to be moved between “upgraded” nodes since SQL Server agent does not start.

To resolve this problem, use the following table to find the appropriate method for your version of SQL Server.

Note If this problem occurs and you want to install an update that does not have this problem, follow the steps that are described in the "Installing SQL Server 2008 rolling updates on a failover cluster" section in KB 958734. For more information, visit the following Microsoft Knowledge Base article:

Cumulative update information

The release version of SQL Server 2008

The fix for this issue was first released in Cumulative Update 8. For more information about how to obtain this cumulative update package for SQL Server 2008, click the following article number to view the article in the Microsoft Knowledge Base:Note Because the builds are cumulative, each new fix release contains all the hotfixes and all the security fixes that were included with the previous SQL Server 2008 fix release. We recommend that you consider applying the most recent fix release that contains this hotfix. For more information, click the following article number to view the article in the Microsoft Knowledge Base:

SQL Server 2008 Service Pack 1

The fix for this issue was also released in Cumulative Update 5 for SQL Server 2008 Service Pack 1. For more information about this cumulative update package, click the following article number to view the article in the Microsoft Knowledge Base:Note Because the builds are cumulative, each new fix release contains all the hotfixes and all the security fixes that were included with the previous SQL Server 2008 fix release. Microsoft recommends that you consider applying the most recent fix release that contains this hotfix. For more information, click the following article number to view the article in the Microsoft Knowledge Base: Microsoft SQL Server 2008 hotfixes are created for specific SQL Server service packs. You must apply a SQL Server 2008 Service Pack 1 hotfix to an installation of SQL Server 2008 Service Pack 1. By default, any hotfix that is provided in a SQL Server service pack is included in the next SQL Server service pack.

Hotfix information

A supported hotfix is available from Microsoft. However, this hotfix is intended to correct only the problem that is described in this article. Apply this hotfix only to systems that are experiencing this specific problem. This hotfix might receive additional testing. Therefore, if you are not severely affected by this problem, we recommend that you wait for the next software update that contains this hotfix.

If the hotfix is available for download, there is a "Hotfix download available" section at the top of this Knowledge Base article. If this section does not appear, contact Microsoft Customer Service and Support to obtain the hotfix.

Note If additional issues occur or if any troubleshooting is required, you might have to create a separate service request. The usual support costs will apply to additional support questions and issues that do not qualify for this specific hotfix. For a complete list of Microsoft Customer Service and Support telephone numbers or to create a separate service request, visit the following Microsoft Web site: Note The "Hotfix download available" form displays the languages for which the hotfix is available. If you do not see your language, it is because a hotfix is not available for that language.

Prerequisites

You must have SQL Server 2008 Service Pack 1 installed before you apply this hotfix.

Restart information

You do not have to restart the computer after you apply this hotfix.

Registry information

You do not have to change the registry.

Hotfix file information

This hotfix contains only those files that are required to correct the issues that this article lists. This hotfix may not contain all the files that you must have to fully update a product to the latest build.

The English version of this hotfix has the file attributes (or later file attributes) that are listed in the following table. The dates and times for these files are listed in Coordinated Universal Time (UTC). When you view the file information, it is converted to local time. To find the difference between UTC and local time, use the Time Zone tab in the Date and Time item in Control Panel.

32-bit version

SQL Server 2008 Database Services Core Shared

SQL Server 2008 Database Services Core Instance

SQL Server 2008 Database Services Common Core

x64-based version

SQL Server 2008 Database Services Core Shared

SQL Server 2008 Database Services Core Instance

SQL Server 2008 Database Services Common Core

Itanium-architecture version

SQL Server 2008 Database Services Core Shared

SQL Server 2008 Database Services Core Instance

SQL Server 2008 Database Services Common Core

If the problem that is described in the "Symptoms" section occurs (steps from (b) to (h) repeat again and again), the administrator can set the SQL Server agent resource property to Do not restart group. If this option is set by using the Cluster Administration user interface (UI), the cluster resource manager brings the cluster resource group online and ignores the fact that SQL Server agent could not be brought online. Therefore, SQL Server service runs uninterruptedly and completes script upgrade on node 2. After script upgrade is completed on node 2, if the administrator manually brings agent resource online, the agent starts successfully.

Note The limitation of this method is that manually intervening during the upgrade process may not be acceptable if you are running a cluster that has many nodes. Therefore, we recommend that you wait for a hotfix solution.

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

Note This issue does not occur in SQL Server 2005. However, to keep a consistent behavior with SQL Server 2008. The changes in this fix were also included in Cumulative Update 7 for SQL Server 2005 Service Pack 3.

For more information about how to identify the SQL Server version that is installed on the computer, click the following article number to view the article in Microsoft Knowledge Base:



For more information about the same issue in a stand-alone environment, click the following article number to view the article in Microsoft Knowledge Base:



For more information about software update terminology, click the following article number to view the article in the Microsoft Knowledge Base: