The DPM service crashes with Event ID 917 after you change a protection group

Applies to: System Center 2012 Data Protection ManagerMicrosoft System Center 2012 Data Protection Manager Service Pack 1System Center 2012 R2 Data Protection Manager

Symptoms


After you change a protection group in Microsoft System Center 2012 Data Protection Manager (DPM 2012), the DPM service crashes with Event ID 917, and the console shuts down. This affects only protection groups that are configured for long-term protection (for example, groups that are configured for tape backup), especially if those groups contain data sources that are SQL AlwaysOn databases.

Cause


This is a known issue in DPM 2012. When you protect an SQL data source by using only tape backup and then the volume ID of the data source changes, DPM tries to invalidate the replica that has the old volume ID. Because there is no replica that does not have short-term protection, the service crashes with a NullReferenceException. The issue most frequently affects Microsoft SQL Server 2012 AlwaysOn availability groups because the underlying volumes frequently change.

Workaround


To work around this issue, create a protection group that contains all the SQL AlwaysOn databases. (If you have to adjust protection group settings, you should make the change when no jobs are running.) After the error occurs, restart the console, and then resolve the warning. At that point, the protection group will return to OK status, the new settings will take effect, and future backups will succeed.

More Information


Examples of MSDPM trace log messages follow.

1A04 02E8 04/05 11:29:06.121 27 FindPrincipalServerBlock.cs(245) 81291AE0-3056-4863-AAA8-AE8EF1761B51 NORMAL Found Principal server for DatasourceId = b4e3e890-9d40-4316-923b-bedd8e35450e Server= <SQL_Server>  1A04 1878 04/05 11:29:08.789 09 AppAssert.cs(126) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING ASSERT: (FileName:SqlWriterHelper.cs; LineNumber:250)  1A04 1878 04/05 11:29:08.789 09 AppAssert.cs(126) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING The parameter 'Not a valid replica for protection' is null.  1A04 2788 04/05 11:29:08.789 09 AppAssert.cs(126) CAEE15A7-CC9A-4590-9780-69F457C89CC8 WARNING ASSERT: (FileName:SqlWriterHelper.cs; LineNumber:250)  1A04 2788 04/05 11:29:08.789 09 AppAssert.cs(126) CAEE15A7-CC9A-4590-9780-69F457C89CC8 WARNING The parameter 'Not a valid replica for protection' is null.  1A04 1878 04/05 11:29:09.069 68 DpmThreadPool.cs(121) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING Caught unhandled exception : System.NullReferenceException: Object reference not set to an instance of an object.  1A04 1878 04/05 11:29:09.069 68 DpmThreadPool.cs(121) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING at Microsoft.Internal.EnterpriseStorage.Dls.WriterHelper.BaseWriterHelper.InvalidateReplicaAndForceHeavyWeightValidation(Message msg, Replica dbReplica, String traceFormatMessage)  1A04 1878 04/05 11:29:09.069 68 DpmThreadPool.cs(121) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING at Microsoft.Internal.EnterpriseStorage.Dls.WriterHelper.SqlWriterHelper.HandleUsnIdChangeForSqlType(Message msg)  1A04 1878 04/05 11:29:09.069 68 DpmThreadPool.cs(121) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING at Microsoft.Internal.EnterpriseStorage.Dls.WriterHelper.SqlWriterHelper.HandleUsnIdChangesOnConfigureProtectionSuccess(Message msg)  1A04 1878 04/05 11:29:09.069 68 DpmThreadPool.cs(121) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING at Microsoft.Internal.EnterpriseStorage.Dls.WriterHelper.BaseWriterHelper.OnConfigureDeltaTrackingSuccess(Message msg)  1A04 1878 04/05 11:29:09.069 68 DpmThreadPool.cs(121) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING at Microsoft.Internal.EnterpriseStorage.Dls.Prm.ConfigureProtectionMachine.ConfigureDeltaTrackingSucceeded(Message msg)  1A04 1878 04/05 11:29:09.069 68 DpmThreadPool.cs(121) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING at Microsoft.Internal.EnterpriseStorage.Dls.TaskExecutor.Fsm.Transition.Execute(Message msg)  1A04 1878 04/05 11:29:09.069 68 DpmThreadPool.cs(121) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING at Microsoft.Internal.EnterpriseStorage.Dls.TaskExecutor.Fsm.Engine.ChangeState(Message msg)  1A04 1878 04/05 11:29:09.069 68 DpmThreadPool.cs(121) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING at Microsoft.Internal.EnterpriseStorage.Dls.TaskExecutor.TaskInstance.Process(Object dummy)  1A04 1878 04/05 11:29:09.069 68 DpmThreadPool.cs(121) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING at Microsoft.Internal.EnterpriseStorage.Dls.TaskExecutor.FsmThreadFunction.Function(Object taskThreadContextObj)  1A04 1878 04/05 11:29:09.069 68 DpmThreadPool.cs(121) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)  1A04 1878 04/05 11:29:09.069 68 DpmThreadPool.cs(121) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)  1A04 1878 04/05 11:29:09.069 68 DpmThreadPool.cs(121) 81291AE0-3056-4863-AAA8-AE8EF1761B51 WARNING at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)