- You install a Microsoft SQL Server 2012 failover cluster instance with two nodes (assume that the names of the nodes are Node 1 and Node 2).
- You install a new failover cluster instance of Microsoft SQL Server 2008 R2 or a new failover cluster instance of Microsoft SQL Server 2008 on Node 2.
- You uset he "Add Node" operation in order to add Node 1 to the failover cluster instance of SQL Server 2008 or SQL Server 2008 R2 that you installed on Node 2.
- You try to failover the failover cluster instance of SQL Server 2008 or SQL Server 2008 R2 from Node 2 to Node 1.
In this scenario, the failover cluster instance of SQL Server 2008 or SQL Server 2008 R2 stays in the "Pending Online" state on Node 1, and then it does not come online on Node 1. Finally, this failover cluster instance switches back to Node 2. Additionally, error messages that resemble the following are logged in the cluster logs:
Note The impact of this issue is that the instance of either SQL Server 2008 R2 or SQL Server 2008 only runs on the node that it is installed on, and the instance does not have high availability as it cannot fail over to other nodes.
- Stop and restart the cluster service.
- Restart the node.
- Either of these actions will cause all the active resources on the node to fail over to one of the other nodes on the cluster. Therefore, you must schedule the action during a downtime.
- If you have not yet installed the SQL Server 2008 R2 or the SQL Server 2008 instance, in order to prevent this issue, you can install the instance by using the Advanced cluster preparation and Advanced cluster completion procedures. For more information about these procedures, go and see the following article in SQL Server Books Online:
- To monitor the health of the resource, the cluster service instantiates an instance of Resource Host Subsystem (RHS.exe) for that instance on all the nodes of the cluster.
- The RHS.exe for SQL server uses the Sqlserver resource DLL file (sqlsrvres.dll) to monitor the health of the SQL instance. This program always uses the latest version of sqlsrvres.dll that is found on the node for this purpose.
- Sqlsrvres.dll queries the registry for information about SQL Server resources and uses it to check the health of the service.
Note These steps occur even before you run the "Add Node" operation.
If there was no instance of SQL Server on the passive nodes, there will be no copy of sqlsrvres.dll on the nodes. Therefore, the RHS.exe would have been exited as it has no work to do.
However, in the scenario that is covered in the "Symptoms" section, the following events occur on the remaining nodes of the cluster where you have not installed the instance for SQL Server 2008 R2 or SQL Server 2008:
- The instance of RHS.exe for the down-level version of SQL Server finds the SQL Server 2012 version of sqlsrvres.dll during instantiation. The SQL Server 2012 version of sqlsrvres.dll can detect both SQL Server 2012 and SQL Server 2008 R2 or SQL Server 2008 instances.
- For the instance of SQL Server 2012, it issues the sp_server_diagnostics command.
- For the instance of SQL Server 2008 or SQL Server 2008 R2, it issues the following command:
The default assumption is that the resource belongs to SQL Server 2012 version.
- The instance of RHS.exe then queries the private property of the SQL Server resource from the following cluster registry key in order to obtain the instance names of the SQL server:
HKEY_LOCAL_MACHINE\Cluster\Resources\< Resource GUID >\Parameters
- As soon as RHS.exe obtains the instance name, it looks up the following instance-specific registry key to obtain the version number:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\<MSSQL10_50.< Instance name >\Setup
- Because these registry keys for the down-level version are not yet populated on the nodes other than the one where the SQL Server 2008 R2 or SQL Server 2008 instance is installed, this instance of RHS.exe continues to run while assuming that the instance it has to monitor is a SQL Server 2012 instance (therefore, use sp_server_diagnostics for health checks).
- After you run the "Add Node" operation to add a new node to the SQL Server 2008 R2 or SQL Server 2008 instance, although the registry keys for the down level instance are now populated, because RHS.exe is not restarted, when the instance for SQL Server tries to come online on this newly added node, it continues to use the sp_server_diagnostics command for checking the state of the SQL Server 2008 R2 or SQL Server 2008 instance.
- Because this stored procedure is new in SQL Server 2012 and does not exist in the older versions of SQL Server, the query fails. Therefore, the resource cannot come online on these nodes, and you receive the error message that is mentioned in the "Symptoms" section.
Article ID: 2938136 - Last Review: 12 Mar 2014 - Revision: 1